Method and apparatus for training a video feature extraction model and video recommendation

Abstract

The application discloses a kind of training video feature extraction model and video recommendation method and device, belong to computer technical field.The method includes: based on the attribute information of target sample video is extracted based on the video feature extraction model to be trained, obtains video feature information, based on the video type preference information corresponding to sample account is extracted based on the account feature extraction model to be trained, obtains account feature information;Video feature information, account feature information are input to the video similarity prediction model to be trained, and the predicted similarity is output;Based on the video feature extraction model to be trained, the account feature extraction model to be trained and the video similarity prediction model to be trained are regulated based on the predicted similarity and benchmark similarity.Parameter.Adopting the present application, it can be based on the attribute characteristics of target sample video itself and the video type preference information of sample account played this target sample video obtains relatively accurate video feature information.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computers, and in particular to a method and device for training a video feature extraction model and video recommendation. BACKGROUND

[0002] Currently, in some application programs with a video playing function, when a user finishes watching a video, the application program recommends other videos similar to the video to the user in the playing page of the video to improve the user's experience. A common method for determining similar videos is to first train a video feature extraction model, then use the trained video feature extraction model to extract features of each video to obtain video feature information corresponding to each video, calculate the similarity between each video and the latest playing video based on the video feature information, and finally recommend videos with high similarity to the user.

[0003] The current method for training a video feature extraction model is to obtain attribute information and a video type of a target sample video, input the attribute information of the target sample video into a video feature extraction model to be trained to obtain output video feature information corresponding to the target sample video, input the video feature information corresponding to the target sample video into a classification model to be trained to obtain output predicted video types corresponding to the target sample video, and finally train the video feature extraction model to be trained and the classification model to be trained based on the video type and the predicted video types of the target sample video to obtain a trained video feature extraction model.

[0004] However, the above training method is relatively rough, the video feature information obtained by using the above video feature extraction model is feature information irrelevant to the user, and the video feature information may be relatively inaccurate. After determining similar videos using the video feature information and recommending the similar videos to the user, the recommendation effect may be poor. SUMMARY

[0005] Embodiments of the present application provide a method for training a video feature extraction model and video recommendation, which can solve the problem that the training method in the prior art is relatively rough, resulting in relatively inaccurate video feature information.

[0006] In a first aspect, a method for training a video feature extraction model is provided, and the method comprises:

[0007] obtaining attribute information of a target sample video played by a sample account, a reference sample video corresponding to the target sample video, and a baseline similarity between the target sample video and the reference sample video;

[0008] perform feature extraction on the attribute information based on the video feature extraction model to be trained to obtain video feature information, and perform feature extraction on the video type preference information corresponding to the sample account based on the account feature extraction model to be trained to obtain account feature information;

[0009] input the video feature information and the account feature information into the video similarity prediction model to be trained, and output a predicted similarity between the target sample video and the reference sample video;

[0010] based on the predicted similarity and the reference similarity, perform parameter adjustment on the video feature extraction model to be trained, the account feature extraction model to be trained, and the video similarity prediction model to be trained;

[0011] if a preset end condition is met, the video feature extraction model after the parameter adjustment is determined as a trained video feature extraction model.

[0012] In a possible implementation, the reference sample video includes a positive sample video and a negative sample video, and the reference similarity between the target sample video and the reference sample video includes a first reference similarity between the target sample video and the positive sample video and a second reference similarity between the target sample video and the negative sample video.

[0013] obtaining a reference sample video corresponding to the target sample video and a reference similarity between the target sample video and the reference sample video includes:

[0014] obtaining a video played by the sample account and meeting a play time proximity condition with the target sample video as the positive sample video corresponding to the target sample video, where the first reference similarity is a first numerical value.

[0015] obtaining a video other than the target sample video and the positive sample video as the negative sample video corresponding to the target sample video, where the second reference similarity is a second numerical value, and the second numerical value is less than the first numerical value.

[0016] In a possible implementation, the obtaining of the video played by the sample account and meeting the play time proximity condition with the target sample video as the positive sample video corresponding to the target sample video includes:

[0017] obtaining at least one video played by the sample account and meeting a condition that a time difference between a play time point of the at least one video and a play time point of the target sample video is less than a preset time difference threshold as a pending positive sample video.

[0018] arranging the pending positive sample video in sequence based on the play time point to obtain a pending positive sample video sequence.

[0019] The pending positive sample video with a number of interval videos between the pending positive sample video and the target sample video less than a preset number is determined as a positive sample video corresponding to the target sample video.

[0020] In a possible implementation, the attribute information of the target sample video includes identification information of the target sample video and edge information of the target sample video, where the identification information of the target sample video is used to represent a unique mark of the target sample video, and the edge information of the target sample video is information associated with video content of the target sample video.

[0021] The attribute information of the target sample video played by the sample account includes:

[0022] The identification information of the target sample video and image data and audio data corresponding to the target sample video are obtained in a video library.

[0023] The image data corresponding to the target sample video is subjected to image recognition processing, and the audio data corresponding to the target sample video is subjected to audio recognition processing, to obtain the edge information of the target sample video.

[0024] In a possible implementation, the method further includes:

[0025] The historical playback data of the sample account is obtained, where the historical playback data includes historical videos played by the sample account in a historical period and a playback time proportion corresponding to each historical video, and the playback time proportion corresponding to each historical video is a ratio of a playback time of the historical video to a total time of the historical video.

[0026] The historical video with a corresponding playback time proportion greater than or equal to a playback time proportion threshold is determined as a historical preference video corresponding to the sample account.

[0027] Based on a video type corresponding to each historical preference video, video type preference information corresponding to the sample account is determined.

[0028] In a possible implementation, the video feature extraction model to be trained includes a first parameter matrix to be trained, and the account feature extraction model to be trained includes a second parameter matrix to be trained.

[0029] The attribute information is subjected to feature extraction based on the video feature extraction model to be trained, to obtain video feature information, which includes:

[0030] The attribute information is subjected to hash calculation, to obtain a first hash value corresponding to the attribute information.

[0031] The first feature information corresponding to the first hash value is obtained from the first parameter matrix to be trained, and is used as the video feature information;

[0032] The account feature extraction model based on the model to be trained extracts features from the video type preference information corresponding to the sample account to obtain account feature information, including:

[0033] The video type preference information is hashed to obtain a second hash value corresponding to the video type preference information;

[0034] The second feature information corresponding to the second hash value is obtained from the second parameter matrix to be trained, and is used as the account feature information.

[0035] In one possible implementation, the video similarity prediction model includes a hidden layer and a classification layer;

[0036] The step of determining the predicted similarity between the target sample video and the reference sample video based on the video feature information, the account feature information, and the video similarity prediction model to be trained includes:

[0037] The video feature information and the account feature information are input into the hidden layer to be trained to obtain combined feature information;

[0038] The combined feature information is input into the classification layer to be trained to obtain the predicted similarity between the target sample video and the reference sample video.

[0039] Secondly, a video recommendation method is provided, the method comprising:

[0040] Based on the video feature extraction model trained according to any of the above, feature extraction is performed on each video in the video library to obtain the video feature information corresponding to each video.

[0041] Receive a video recommendation request sent by a target terminal, wherein the video recommendation request carries identification information of the latest video played by the target terminal;

[0042] Determine the video feature information corresponding to the latest playing video;

[0043] Based on the video feature information corresponding to the latest playing video and the video feature information corresponding to each video, the similarity between each video and the latest playing video is determined;

[0044] Based on the similarity between each video and the latest playing video, a target recommended video corresponding to the latest playing video is determined among the videos, and the target recommended video is sent to the target terminal.

[0045] In a possible implementation, the determining the video feature information corresponding to the latest played video comprises:

[0046] If the video library stores the latest played video, the video feature information corresponding to the latest played video is acquired in the video library based on the identification information of the latest played video.

[0047] If the video library does not store the latest played video, attribute information of the latest played video is acquired, the attribute information of the latest played video is subjected to feature extraction based on the trained video feature extraction model, and the video feature information corresponding to the latest played video is obtained.

[0048] In a third aspect, a device for training a video feature extraction model is provided, and the device comprises:

[0049] An acquisition module is configured to acquire attribute information of a target sample video played by a sample account, a reference sample video corresponding to the target sample video, and a reference similarity between the target sample video and the reference sample video.

[0050] A first determination module is configured to perform feature extraction on the attribute information based on a video feature extraction model to be trained, to obtain video feature information, and perform feature extraction on video type preference information corresponding to the sample account based on an account feature extraction model to be trained, to obtain account feature information.

[0051] A second determination module is configured to input the video feature information and the account feature information into a video similarity prediction model to be trained, and output a predicted similarity between the target sample video and the reference sample video.

[0052] A parameter adjustment module is configured to perform parameter adjustment on the video feature extraction model to be trained, the account feature extraction model to be trained, and the video similarity prediction model to be trained based on the predicted similarity and the reference similarity.

[0053] A third determination module is configured to determine the video feature extraction model after the parameter adjustment as a trained video feature extraction model if a preset ending condition is met.

[0054] In a possible implementation, the reference sample video comprises a positive sample video and a negative sample video, and the reference similarity between the target sample video and the reference sample video comprises a first reference similarity between the target sample video and the positive sample video and a second reference similarity between the target sample video and the negative sample video.

[0055] The acquisition module is configured to:

[0056] obtain a video played by the sample account and meeting a play time proximity condition with the target sample video as a positive sample video corresponding to the target sample video, wherein the first reference similarity is a first value;

[0057] obtain a video other than the target sample video and the positive sample video as a negative sample video corresponding to the target sample video, wherein the second reference similarity is a second value, and the second value is less than the first value.

[0058] In a possible implementation, the obtaining module is configured to:

[0059] obtain at least one video played by the sample account and having a time difference between a play time point and a play time point of the target sample video less than a preset time difference threshold as a pending positive sample video;

[0060] arrange the pending positive sample video in sequence based on the play time point to obtain a pending positive sample video sequence;

[0061] determine a pending positive sample video in the pending positive sample video sequence and having a number of interval videos with the target sample video less than a preset number as a positive sample video corresponding to the target sample video.

[0062] In a possible implementation, the attribute information of the target sample video includes identification information of the target sample video and edge information of the target sample video, wherein the identification information of the target sample video is used to represent a unique mark of the target sample video, and the edge information of the target sample video is information associated with a video content of the target sample video.

[0063] The obtaining module is configured to:

[0064] obtain the identification information of the target sample video and image data and audio data corresponding to the target sample video in a video library;

[0065] perform image recognition processing on the image data corresponding to the target sample video and audio recognition processing on the audio data corresponding to the target sample video to obtain the edge information of the target sample video.

[0066] In a possible implementation, the obtaining module is further configured to:

[0067] obtain historical playback data of the sample account, the historical playback data comprising historical videos played by the sample account in a historical period and a playback time proportion corresponding to each historical video, the playback time proportion corresponding to each historical video being a ratio of a playback time of the historical video to a total time of the historical video;

[0068] determine, as a historical preference video corresponding to the sample account, a historical video corresponding to the playback time proportion being greater than or equal to a playback time proportion threshold;

[0069] determine, based on a video type corresponding to each historical preference video, video type preference information corresponding to the sample account.

[0070] In a possible implementation, the video feature extraction model to be trained comprises a first parameter matrix to be trained, and the account feature extraction model to be trained comprises a second parameter matrix to be trained.

[0071] The first determination module is configured to:

[0072] perform hash calculation on the attribute information to obtain a first hash value corresponding to the attribute information;

[0073] obtain, in the first parameter matrix to be trained, first feature information corresponding to the first hash value as the video feature information;

[0074] perform hash calculation on the video type preference information to obtain a second hash value corresponding to the video type preference information;

[0075] obtain, in the second parameter matrix to be trained, second feature information corresponding to the second hash value as the account feature information.

[0076] In a possible implementation, the video similarity prediction model comprises a hidden layer and a classification layer.

[0077] The second determination module is configured to:

[0078] input the video feature information and the account feature information into the hidden layer to be trained to obtain combined feature information;

[0079] input the combined feature information into the classification layer to be trained to obtain a predicted similarity between the target sample video and the reference sample video.

[0080] In a fourth aspect, a device for video recommendation is provided, and the device comprises:

[0081] The feature extraction module is configured to extract features of each video in the video library based on the trained video feature extraction model determined by the device according to any one of the preceding aspects, to obtain video feature information corresponding to each video.

[0082] The receiving module is configured to receive a video recommendation request sent by the target terminal, where the video recommendation request carries identification information of a latest played video of the target terminal.

[0083] The fourth determining module is configured to determine video feature information corresponding to the latest played video.

[0084] The fifth determining module is configured to determine a similarity between each video and the latest played video based on the video feature information corresponding to the latest played video and the video feature information corresponding to each video.

[0085] The sending module is configured to determine a target recommended video corresponding to the latest played video from the videos based on the similarity between each video and the latest played video, and send the target recommended video to the target terminal.

[0086] In a possible implementation, the fourth determining module is configured to:

[0087] If the video library stores the latest played video, the video feature information corresponding to the latest played video is obtained from the video library based on the identification information of the latest played video.

[0088] If the video library does not store the latest played video, attribute information of the latest played video is obtained, and the attribute information of the latest played video is extracted based on the trained video feature extraction model to obtain the video feature information corresponding to the latest played video.

[0089] In a fifth aspect, a computer device is provided, which includes a processor and a memory, and the memory stores at least one instruction, which is loaded and executed by the processor to implement the operations performed by the above method.

[0090] In a sixth aspect, a computer readable storage medium is provided, which stores at least one instruction, which is loaded and executed by the processor to implement the operations performed by the above method.

[0091] In a seventh aspect, a computer program product is provided, which includes at least one instruction, which is loaded and executed by the processor to implement the operations performed by the above method.

[0092] The technical scheme provided in the embodiments of the present application has the beneficial effects that: the scheme mentioned in the embodiments of the present application can perform feature extraction on attribute information of a target sample video based on a video feature extraction model to be trained to obtain video feature information, perform feature extraction on video type preference information corresponding to a sample account based on an account feature extraction model to be trained to obtain account feature information, and then determine a predicted similarity between the target sample video and a reference sample video based on the video feature information, the account feature information and a video similarity prediction model to be trained. Based on the predicted similarity and a benchmark similarity, the video feature extraction model to be trained, the account feature extraction model to be trained and the video similarity prediction model to be trained are adjusted, and when a preset condition is met, the video feature extraction model after adjustment is determined as a trained video feature extraction model. By using the present application, the attribute features of the target sample video itself and the video type preference information of the sample account that has played the target sample video can be comprehensively considered to jointly constrain the video feature information corresponding to the target sample video, so that more accurate video feature information is obtained, and when the video feature information is used to recommend similar videos to a user, better recommendation effect can be obtained. BRIEF DESCRIPTION OF DRAWINGS

[0093] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0094] Figure 1 is a method flowchart for training a video feature extraction model provided by the embodiments of the present application;

[0095] Figure 2 is a method flowchart for training a video feature extraction model provided by the embodiments of the present application;

[0096] Figure 3 is a feature extraction method flowchart provided by the embodiments of the present application;

[0097] Figure 4 is a schematic diagram of a to-be-determined positive sample video sequence provided by the embodiments of the present application;

[0098] Figure 5 is a feature extraction method flowchart provided by the embodiments of the present application;

[0099] Figure 6 is a method flowchart for training a video feature extraction model provided by the embodiments of the present application;

[0100] Figure 7is a method flowchart for training a video feature extraction model provided by an embodiment of the present application;

[0101] Figure 8 is a visual effect diagram of training a video feature extraction model provided by an embodiment of the present application;

[0102] Figure 9 is a comparative effect diagram of the number of plays per capita provided by an embodiment of the present application;

[0103] Figure 10 is a comparative effect diagram of the play time per capita provided by an embodiment of the present application;

[0104] Figure 11 is a method flowchart for video recommendation provided by an embodiment of the present application;

[0105] Figure 12 is a structural schematic diagram of a device for training a video feature extraction model provided by an embodiment of the present application;

[0106] Figure 13 is a structural schematic diagram of a device for video recommendation provided by an embodiment of the present application;

[0107] Figure 14 is a structural block diagram of a server provided by an embodiment of the present application. DETAILED DESCRIPTION

[0108] In order to make the purpose, technical scheme and advantages of the present application clearer, the embodiments of the present application will be further described in detail below with reference to the drawings.

[0109] The present application provides a method for training a video feature extraction model, which can be implemented by a server. The server can be a single server or a server cluster composed of multiple servers.

[0110] The server can include a processor, a memory, a communication component, etc., and the processor is connected with the memory and the communication component respectively.

[0111] The processor can be a CPU (Central Processing Unit). The processor can be configured to read instructions and process data, for example, obtaining attribute information of a target sample video played by a sample account, determining a reference sample video corresponding to the target sample video, determining a benchmark similarity between the target sample video and the reference sample video, performing feature extraction on the attribute information of the target sample video based on a video feature extraction model to be trained to obtain video feature information, performing feature extraction on video type preference information corresponding to the sample account based on an account feature extraction model to be trained to obtain account feature information, determining a predicted similarity between the target sample video and the reference sample video, and performing parameter adjustment on the video feature extraction model to be trained, the account feature extraction model to be trained, and the video similarity prediction model to be trained based on the predicted similarity and the benchmark similarity.

[0112] The memory can include a ROM (Read-Only Memory), a RAM (Random Access Memory), a CD-ROM (Compact Disc Read-Only Memory), a magnetic disk, an optical data storage device, and the like. The memory can be configured to store data, for example, data of the obtained attribute information of the target sample video, data of the determined reference sample video, data of the determined benchmark similarity between the target sample video and the reference sample video, data corresponding to the video feature extraction model to be trained, data corresponding to the account feature extraction model to be trained, and data corresponding to the video similarity prediction model to be trained, intermediate data generated in the process of performing parameter adjustment on the video feature extraction model to be trained, the account feature extraction model to be trained, and the video similarity prediction model to be trained based on the predicted similarity and the benchmark similarity, and the like.

[0113] The communication component can be a wired network connector, a wireless fidelity module, a Bluetooth module, a cellular network communication module, and the like. The communication component can be configured to receive and send signals, and the like.

[0114] Figure 1 And Figure 2 is a flowchart of a method for training a video feature extraction model provided by an embodiment of the present application. Referring to Figure 1 And Figure 2 The embodiment includes:

[0115] 101. Obtain attribute information of a target sample video played by a sample account, a reference sample video corresponding to the target sample video, and a benchmark similarity between the target sample video and the reference sample video.

[0116] In implementation, a training sample set can be acquired first, and the training sample set includes multiple sample input data and corresponding training targets.

[0117] When the sample input data is acquired, multiple accounts can be acquired as sample accounts respectively, and the following processing can be performed: based on historical behavior information of the sample accounts, a video played in a recent period of time can be determined as a target sample video, and attribute information of the target sample video can be acquired as part of the sample input data.

[0118] When the training target corresponding to the sample input data is acquired, a reference sample video corresponding to the target sample video can be acquired first, and a reference similarity between the target sample video and the reference sample video can be acquired as the training target.

[0119] 102、based on the video feature extraction model to be trained, the attribute information is subjected to feature extraction to obtain video feature information, and based on the account feature extraction model to be trained, the video type preference information corresponding to the sample account is subjected to feature extraction to obtain account feature information.

[0120] Optionally, the method for feature extraction of the video feature extraction model and the account feature extraction model can be as follows:

[0121] The video feature extraction model to be trained can include a first parameter matrix to be trained, and the account feature extraction model to be trained can include a second parameter matrix to be trained, wherein the first parameter matrix to be trained can include multiple first parameters, and the first parameters are parameters that need to be adjusted when the video feature extraction model to be trained is subjected to parameter adjustment; similarly, the second parameter matrix to be trained can include multiple second parameters, and the second parameters are parameters that need to be adjusted when the account feature extraction model to be trained is subjected to parameter adjustment.

[0122] As shown in Figure 3 When the video feature extraction model to be trained is used to perform feature extraction on the attribute information of the target sample video, the method for feature extraction can be as follows:

[0123] Hash calculation is performed on the attribute information input to obtain a first hash value corresponding to the attribute information, and first feature information corresponding to the first hash value is acquired from the first parameter matrix to be trained as the video feature information.

[0124] In implementation, the first parameter matrix includes multiple rows of first parameters, and each row of first parameters is a first feature information. The first hash value of the attribute information obtained by the hash algorithm can be in numerical form. After obtaining the first hash value, the first parameters in the row corresponding to the first hash value in the first parameter matrix can be extracted as the video feature information. For example, if the first hash value of the determined attribute information is 11, the first parameters in the 11th row of the first parameter matrix can be extracted as the video feature information.

[0125] Similarly, as shown in FIG. 6, when the video type preference information corresponding to the sample account is extracted by using the account feature extraction model to be trained, the feature extraction method can be as follows: Figure 3

[0126] The video type preference information is calculated by using the hash algorithm to obtain the second hash value corresponding to the video type preference information, and the second feature information corresponding to the second hash value in the second parameter matrix to be trained is obtained as the account feature information.

[0127] In implementation, the second parameter matrix includes multiple rows of second parameters, and each row of second parameters is a second feature information. The second hash value of the video type preference information obtained by the hash algorithm can be in numerical form. After obtaining the second hash value, the second parameters in the row corresponding to the second hash value in the second parameter matrix can be extracted as the account feature information. For example, if the second hash value of the video type preference information determined by the second hash algorithm is 50, the second parameters in the 50th row of the second parameter matrix can be extracted as the account feature information.

[0128] Optionally, when the attribute information of the target sample video includes multiple information, the first hash value corresponding to each attribute information can be calculated by using the hash algorithm, and the first feature information corresponding to each first hash value in the first parameter matrix is obtained, that is, the first feature information corresponding to each attribute information is obtained, and the video feature information includes the first feature information corresponding to the multiple attribute information.

[0129] Similarly, when the video type preference information corresponding to the sample account includes multiple information, the second hash value corresponding to each video type preference information can be calculated by using the hash algorithm, and the second feature information corresponding to each second hash value in the second parameter matrix is obtained, that is, the second feature information corresponding to each video type preference information is obtained, and the account feature information includes the second feature information corresponding to the multiple video type preference information.

[0130] ​It can be understood that the number of rows set by the first parameter matrix and the second parameter matrix needs to meet the requirements. When there are V types of attribute information of all videos or all sample videos in the application program, the first parameter matrix has at least V rows of data, that is, V rows of first feature information, as shown in Figure 3 When the first row of the first parameter matrix corresponds to a first hash value of 0, the last row of the first parameter matrix corresponds to a first hash value of at least V-1. Similarly, when there are N types of video type preference information corresponding to all accounts or all sample accounts in the application program, the second parameter matrix has at least N rows of data, that is, N rows of second feature information, as shown in Figure 3 When the first row of the second parameter matrix corresponds to a second hash value of 0, the last row of the second parameter matrix corresponds to a second hash value of at least N-1.

[0131] It can be understood that the hash algorithm for performing hash calculation on the attribute information corresponding to the target sample video is different from the hash algorithm for performing hash calculation on the video type preference information corresponding to the sample account, thereby ensuring the uniqueness of the first hash value corresponding to the attribute information and the second hash value corresponding to the video type preference information.

[0132] 103, input the video feature information and the account feature information into the video similarity prediction model to be trained, and output the predicted similarity between the target sample video and the reference sample video.

[0133] In implementation, each video in the video library of the application program can be numbered, and the identification information and the corresponding number of each video are stored.

[0134] After the video feature information and the account feature information are input into the video similarity prediction model to be trained, the predicted similarity corresponding to a plurality of numbers can be obtained, and the predicted similarity corresponding to each number indicates the predicted similarity between the video corresponding to the number and the target sample video.

[0135] Then, the predicted similarity between the reference sample video and the target sample video can be determined based on the number corresponding to the reference sample video.

[0136] 104, based on the predicted similarity and the benchmark similarity, the video feature extraction model to be trained, the account feature extraction model to be trained, and the video similarity prediction model to be trained are adjusted.

[0137] In implementation, the predicted similarity and the benchmark similarity can be input into a loss function, and a loss value can be calculated. The video feature extraction model to be trained, the account feature extraction model to be trained, and the video similarity prediction model to be trained can be adjusted based on the loss value, thereby completing one training.

[0138] 105、if the preset ending condition is met, the video feature extraction model after the parameter adjustment is determined as the trained video feature extraction model.

[0139] In implementation, based on the method of steps 101-104, each sample input data in the training sample set and the corresponding training target are used to respectively perform multiple parameter adjustments on the video feature extraction model to be trained, the account feature extraction model to be trained and the video similarity prediction model to be trained. After each parameter adjustment, the following judgment can be performed: whether the preset ending condition is met at present. If the preset ending condition is met, the video feature extraction model after the last parameter adjustment is determined as the trained video feature extraction model.

[0140] Optionally, the preset ending condition can be various, and the following are several listed conditions:

[0141] The first condition is that the number of times of performing parameter adjustment on the video feature extraction model to be trained using different target sample videos reaches a parameter adjustment number threshold. In implementation, the staff can pre-set the parameter adjustment number threshold, and when the number of parameter adjustments reaches the parameter adjustment number threshold, the training can be stopped, and the video feature extraction model obtained after the last parameter adjustment is determined as the trained video feature extraction model. The parameter adjustment number threshold can be any reasonable value, for example, can be 200, can also be 300, etc., and the embodiments of the present application are not limited thereto.

[0142] The second condition is that the loss values obtained by continuous preset number of times of training are all less than a preset loss value threshold. The preset number and the preset loss value threshold can be any reasonable value, for example, the preset number can be 3 or 5, etc., and the preset loss value threshold can be 0.05, etc., and the embodiments of the present application are not limited thereto.

[0143] The third condition is that the number of parameter adjustments reaches the parameter adjustment number threshold, and the loss values obtained by continuous preset number of times of training are all less than the preset loss value threshold.

[0144] The preset ending condition can be any one of the above three conditions, or other ending conditions, and the embodiments of the present application are not limited thereto.

[0145] In the embodiments of the present application, the reference sample video can include positive sample videos and negative sample videos, and the method of obtaining the positive sample video and the negative sample video corresponding to the target sample video can be various, and one of them will be introduced in detail as follows:

[0146] The method of obtaining the positive sample video corresponding to the target sample video is: obtaining the video played by the sample account and meeting the close playing time condition with the target sample video as the positive sample video corresponding to the target sample video.

[0147] In implementation, the multiple videos played successively in the same account can be determined as the videos with high similarity, and thus the video with a play time point close to the play time point of the target sample video can be obtained from the historical play data of the sample account as the positive sample video.

[0148] Optionally, the method for obtaining the positive sample video can have a more detailed process flow as follows:

[0149] At least one video played by the sample account and having a time difference between the play time point and the play time point of the target sample video less than a preset time difference threshold is obtained as a pending positive sample video. The pending positive sample videos are arranged in sequence based on the play time points to obtain a pending positive sample video sequence. The pending positive sample video in the pending positive sample video sequence and having a number of interval videos with the target sample video less than a preset number is determined as the positive sample video corresponding to the target sample video.

[0150] In implementation, the historical play data of the sample account in a recent period of time can be obtained, and the historical play data includes multiple pending positive sample videos played by the sample account. For example, the historical play data of a recent session can be obtained.

[0151] The pending positive sample videos in the historical play data are arranged in sequence based on the play time points to obtain a pending positive sample video sequence corresponding to the sample account, and the pending positive sample video having a number of interval videos with the target sample video less than a preset number is obtained as the positive sample video corresponding to the target sample video.

[0152] In the embodiments of the present application, the preset number can be any reasonable value, for example, 2, 3, or 4, etc., and the embodiments of the present application are not limited thereto.

[0153] Optionally, in obtaining the training sample set, the historical play data of a sample account in a recent period of time or a recent session can be obtained, and then the pending positive sample videos in the historical play data are arranged in sequence based on the play time points to obtain a pending positive sample video sequence corresponding to the sample account. Then, each pending positive sample video in the pending positive sample video sequence can be determined as a target sample video, and the positive sample video corresponding to each target sample video can be obtained in the pending positive sample video sequence based on a preset length sliding window.

[0154] The preset length can be the sum of the number of the pending positive sample videos before the target sample video and the number of the pending positive sample videos after the target sample video. For example, if the preset length is 8, four pending positive sample videos before the playback time point of the target sample video and four pending positive sample videos after the playback time point of the target sample video can be obtained.

[0155] It can be understood that when the number of the pending positive sample videos before the target sample video in the pending positive sample video sequence is less than "preset length / 2", all the reference sample videos before the target sample video and "preset length / 2" pending positive sample videos after the target sample video can be obtained as the positive sample videos corresponding to the target sample video.

[0156] When the number of the pending positive sample videos after the target sample video in the pending positive sample video sequence is less than "preset length / 2", all the reference sample videos after the target sample video and "preset length / 2" pending positive sample videos before the target sample video can be obtained as the positive sample videos corresponding to the target sample video.

[0157] For the case that the preset length of the sliding window is long, so that multiple positive sample videos corresponding to the target sample video are obtained, the multiple positive sample videos can be used as the positive sample videos for one-time parameter adjustment, for example, the preset length is 6, and six pending positive sample videos are obtained in the pending positive sample video sequence through the sliding window with the preset length. When the model is trained, the six pending positive sample videos can be determined as the positive sample videos corresponding to the target sample video in one-time training, so that six-time training parameter adjustment of the model is realized.

[0158] As shown in FIG. 1, Figure 4 As shown in FIG. 1, Figure 4 The pending positive sample video sequence shown in FIG. 1 includes six pending positive sample videos, which can be used as target sample videos respectively. If the preset length of the sliding window is 4, as shown in FIG. 1, Figure 4 (a) when the target sample video is w(t-2), the corresponding positive sample video can be any one of w(t-1) and w(t); as shown in FIG. 1, Figure 4 (b) when the target sample video is w(t-1), the corresponding positive sample video can be any one of w(t-2), w(t) and w(t+1); as shown in FIG. 1, Figure 4 (c) when the target sample video is w(t), the corresponding positive sample video can be any one of w(t-2), w(t-1), w(t+1) and w(t+2); as shown in FIG. 1, Figure 4(d) as shown, when the target sample video is w(t+1), the corresponding positive sample video can be any one of w(t-1), w(t) and w(t+2); as Figure 4 (e) as shown, when the target sample video is w(t+2), the corresponding positive sample video can be any one of w(t) and w(t+1).

[0159] The method of obtaining the negative sample video corresponding to the target sample video is: obtaining a video other than the target sample video and the positive sample video as the negative sample video corresponding to the target sample video.

[0160] In implementation, a video other than the target sample video and the positive sample video can be randomly obtained from all the videos stored in the application as the negative sample video corresponding to the target sample video.

[0161] It can be understood that the video other than the target sample video and the positive sample video herein can also be understood as a video other than the target sample video and the pending positive sample video.

[0162] For the case that the reference sample video includes the positive sample video and the negative sample video, correspondingly, the reference similarity of the target sample video to the reference sample video can include the first reference similarity of the target sample video to the positive sample video and the second reference similarity of the target sample video to the negative sample video.

[0163] When setting the first reference similarity and the second reference similarity, the first reference similarity can be set as a first value, the second reference similarity can be set as a second value, and the second value is less than the first value.

[0164] In the embodiments of the present application, the specific values of the first value and the second value can be set according to requirements, for example, the first value can be directly set as 1, the second value can be set as 0, or the first value can be set as 0.95, the second value can be set as 0.05, etc., and the embodiments of the present application are not limited thereto.

[0165] Optionally, the sample input data can have multiple settings, which can be as described above that the sample input data includes the attribute information of the target sample video, the reference similarity of the target sample video to the positive sample video, and the reference similarity of the target sample video to the negative sample video, or other forms, for example, the sample input data includes the attribute information of the target sample video, and the reference similarity of the target sample video to two positive sample videos, and the embodiments of the present application are not limited thereto.

[0166] Optionally, the attribute information of the target sample video can include various information. In the embodiment of the present application, the attribute information of the target sample video can include identification information (or ID) of the target sample video and side information of the target sample video. The identification information of the target sample video is used to represent the unique mark of the target sample video, and the side information of the target sample video is information associated with the video content of the target sample video. For example, in the case of feature extraction of audio and video, the target sample video is a music video, and the side information of the target sample video can include at least one of an associated singer and an associated song, wherein the associated singer is the singer of the song audio in the target sample video, and the associated song is the name of the song audio in the target sample video.

[0167] Optionally, when the identification information and the side information of the target sample video are obtained, the following processing can be performed: obtaining the identification information of the target sample video and the image data and the audio data corresponding to the target sample video in the video library, performing image recognition processing on the image data corresponding to the target sample video, and performing audio recognition processing on the audio data corresponding to the target sample video to obtain the side information of the target sample video.

[0168] For example, the target sample video is a music video, and the side information includes an associated singer and an associated song. When the identification information and the side information of the target sample video are obtained, the identification information, the corresponding image data and the audio data of the target sample video can be directly obtained in the video library. Then, the image data corresponding to the target sample video can be subjected to image recognition processing to determine the associated singer corresponding to the target sample video, and the audio data corresponding to the target sample video can be subjected to audio recognition processing to determine the associated song corresponding to the target sample video, thereby obtaining the side information of the target sample video.

[0169] Optionally, there can be various methods for performing image recognition processing on the image data corresponding to the target sample video. For example, a trained image recognition model can be used to perform image recognition processing on the image data corresponding to the target sample video. Of course, other image recognition methods can also be used, and the embodiment of the present application does not limit this.

[0170] Similarly, there can be various methods for performing audio recognition processing on the audio data corresponding to the target sample video. For example, a trained audio recognition model can be used to perform audio recognition processing on the audio data corresponding to the target sample video. Of course, other audio processing methods can also be used, and the embodiment of the present application does not limit this.

[0171] Corresponding to the case that the attribute information described above includes the identification information and the edge information, the video feature extraction model can further include an identification feature extraction sub-model and an edge information feature extraction sub-model, and the corresponding feature extraction process can be as follows:

[0172] The identification information of the target sample video is subjected to feature extraction based on the identification feature extraction sub-model to be trained, to obtain first video feature sub-information. The edge information of the target sample video is subjected to feature extraction based on the edge information feature extraction sub-model to be trained, to obtain second video feature sub-information.

[0173] It can be understood that the attribute information of the target sample video can be set according to actual needs, and the attribute information can further include other information, which is not limited in the embodiments of the present application.

[0174] When the attribute information corresponding to the target sample video includes the identification information and the edge information of the target sample video, as shown in Figure 5 The method for the video feature extraction model to be trained and the account feature extraction model to be trained to perform feature extraction in step 102 can be as follows:

[0175] The first parameter matrix to be trained in the video feature extraction model to be trained can include a first identification information sub-parameter matrix to be trained and a first edge information sub-parameter matrix to be trained, and the first identification information sub-parameter matrix to be trained and the first edge information sub-parameter matrix to be trained each include a plurality of first parameters.

[0176] When performing feature extraction, the first hash algorithm can be used to perform hash calculation on the identification information of the target sample video, to obtain a first hash value corresponding to the identification information of the target sample video, and then the first feature information corresponding to the first hash value of the identification information is obtained in the identification information sub-parameter matrix, that is, the first feature information corresponding to the identification information of the target sample video is obtained, which can be referred to as first video feature sub-information here.

[0177] In addition, the second hash algorithm is used to perform hash calculation on the edge information of the target sample video, to obtain a first hash value corresponding to the edge information of the target sample video, and then the first feature information corresponding to the first hash value of the edge information is obtained in the edge information sub-parameter matrix, that is, the first feature information corresponding to the edge information of the target sample video is obtained, which can be referred to as second video feature sub-information here.

[0178] The video feature information corresponding to the target sample video includes the first feature information corresponding to the identification information of the target sample video and the second feature information corresponding to the video type preference information of the target sample video, that is, the video feature information corresponding to the target sample video includes the first video feature sub-information and the second video feature sub-information.

[0179] It can be understood that the number of rows of the identification information sub-parameter matrix and the number of rows of the side information sub-parameter matrix need to meet the requirements. If the number of rows of the first parameter matrix is V rows, the sum of the number of rows of the identification information sub-parameter matrix and the number of rows of the side information sub-parameter matrix is equal to V. Wherein, when there are P kinds of identification information of all videos or all sample videos in the application program, the identification information sub-parameter matrix has at least P rows of data, that is, P rows of first characteristic information, such as shown in Figure 5 When the first hash value corresponding to the first row of the identification information sub-parameter matrix is 0, the last row of the identification information sub-parameter matrix corresponds to a first hash value of at least P-1. Similarly, when there are Q kinds of side information of all videos or all sample videos in the application program, the side information sub-parameter matrix has at least Q rows of data, that is, Q rows of second characteristic information, such as shown in Figure 5 When the second hash value corresponding to the first row of the side information sub-parameter matrix is 0, the last row of the side information sub-parameter matrix corresponds to a second hash value of at least Q-1.

[0180] Optionally, before performing feature extraction on the video type preference information corresponding to the sample account, the video type preference information corresponding to the sample account needs to be obtained first. There are many methods for obtaining the video type preference information, and one of the methods is as follows:

[0181] Obtain the historical playback data of the sample account, and the historical playback data includes historical videos played by the sample account in a historical period and a playback time length proportion corresponding to each historical video. The playback time length proportion corresponding to the historical video is a ratio of the playback time length of the historical video to the total time length of the historical video. The historical video with a corresponding playback time length proportion greater than or equal to a playback time length proportion threshold is determined as a historical preference video corresponding to the sample account. Based on the video type corresponding to each historical preference video, the video type preference information corresponding to the sample account is determined.

[0182] In implementation, the staff can pre-set the playback time length proportion threshold. When obtaining the video type preference information corresponding to the sample account, the historical playback data of the sample account in a historical period can be obtained first. The historical period can be any reasonable period, for example, the historical period can be the last week, or the last month, or the last half year, and so on. Of course, it can also be other periods, and the embodiments of the present application do not limit this.

[0183] The historical playback data includes historical videos played by the sample account in a historical period and a playback time proportion corresponding to each historical video, the playback time proportion corresponding to each historical video being a ratio of a playback time of the historical video to a total time of the historical video, for example, a total time of an A video is 10 minutes, the historical period is the last month, if the sample account plays the A video in the last month, the A video is a historical video corresponding to the sample account, if a playback time of the A video by the sample account is 2 minutes, the playback time proportion corresponding to the A video is 2 / 10, i.e. 0.2.

[0184] When the playback time proportion corresponding to each historical video is determined, the historical video with a playback time proportion greater than or equal to a playback time proportion threshold value can be determined as a historical preference video corresponding to the sample account, i.e. indicating that the user corresponding to the sample account likes the historical preference video.

[0185] When the historical preference video liked by the sample account is determined, the video type preference information corresponding to the sample account can be determined based on a video type corresponding to each historical preference video. There can be multiple methods for determining the video type preference information based on the video type corresponding to the historical preference video, two of which are as follows:

[0186] Firstly, a first number of video types can be selected from the video types corresponding to the multiple historical videos in an order from more to less in terms of the number of corresponding historical videos, and the first number of video types are determined as the video type preference information corresponding to the sample account, for example, the first number is 2, the historical videos played by the sample account are A historical video, B historical video and C historical video, the video types corresponding to the A historical video are a, b and c, the video types corresponding to the B historical video are b and c, and the video types corresponding to the C historical video are c and e, the number of historical videos corresponding to the a video type is 1, the number of historical videos corresponding to the b video type is 2, the number of historical videos corresponding to the c video type is 3, and the number of historical videos corresponding to the e video type is 1, and thus the video type preference information is the c video type and the b video type.

[0187] Secondly, the video types corresponding to the multiple historical videos can be arranged in an order from more to less in terms of the number of corresponding historical videos, and then the video types with a number of corresponding historical videos greater than or equal to a preset video number threshold value are determined as the video type preference information corresponding to the sample account.

[0188] The method for determining the video type preference information corresponding to the sample account can also be other reasonable methods, which are not limited in the embodiments of the present application.

[0189] In the embodiments of the present application, the structure of the video similarity prediction model can be any reasonable setting, and one of the possible settings is taken as an example for detailed introduction as follows:

[0190] The video similarity prediction model comprises a hidden layer and a classification layer, and a processing procedure of the corresponding video similarity prediction model can be as follows: inputting the video feature information and the account feature information into the hidden layer to be trained to obtain combined feature information; inputting the combined feature information into the classification layer to be trained to obtain a predicted similarity of the target sample video and the reference sample video.

[0191] Optionally, the video feature extraction model, the account feature extraction model and the video similarity prediction model in the embodiments of the present application can be multiple sub-models of the same model, or can be different models respectively.

[0192] Hereinafter, the video feature extraction model, the account feature extraction model and the video similarity prediction model are taken as an example of multiple sub-models of the same model for more detailed introduction:

[0193] The video feature extraction model, the account feature extraction model and the video similarity prediction model can be set based on an EGES (Enhanced Graph Embedding with Side Information) model. First, the EGES model comprises a feature extraction layer (only performing feature extraction on the item side), a hidden layer and a classification layer. Based on the EGES model, a model comprising the video feature extraction model (identifying a feature extraction sub-model and a side information feature extraction sub-model), the account feature extraction model and the video similarity prediction model (comprising the hidden layer and the classification layer) can be obtained, which can be named as an EIESG (Enhanced Item Embedding with Side Information and Global Context) model.

[0194] As shown in FIGS. 1, 2 and 3, the EIESG model comprises a feature extraction layer, a hidden layer and a classification layer. Figure 6 and Figure 7 As shown in FIGS. 1, 2 and 3, the EIESG model comprises a feature extraction layer, a hidden layer and a classification layer.

[0195] The identification information and the side information of the target sample video and the attribute information (in the embodiments of the present application, the video type preference information corresponding to the sample account) corresponding to the sample account are obtained as sample input data, and the reference similarity of the target sample video and the positive sample video and the reference similarity of the target sample video and the negative sample video are obtained as training targets.

[0196] The identification information of the target sample video is input into the identification feature extraction sub-model to be trained, so as to obtain first video feature sub-information; the edge information of the target sample video is input into the edge information feature extraction sub-model to be trained, so as to obtain second video feature sub-information; and the attribute information corresponding to the sample account is input into the account feature extraction model to be trained, so as to obtain account feature information. Based on the above feature extraction, the sparse features (i.e. Sparse Features shown in Figure 7 ) such as the identification information, the edge information and the video type preference information can be respectively converted into the dense feature information (i.e. Dense Embeddings shown in Figure 7 ) such as the first video feature sub-information, the second video feature sub-information and the account feature information, wherein, in the case that the target sample video has multiple edge information, the multiple edge information is input into the edge information feature extraction sub-model to obtain the second video feature sub-information corresponding to each edge information.

[0197] Then, the first video feature sub-information, the second video feature sub-information and the account feature information are input into the hidden layer to be trained, so as to obtain combined feature information. In the implementation, after the first video feature sub-information, the second video feature sub-information and the account feature information are input into the hidden layer, they can be multiplied by the corresponding weights, i.e. as shown in Figure 5 , the first video feature sub-information is multiplied by the weight a i , each second video feature sub-information is multiplied by a n , and the account feature information is multiplied by the corresponding weight a u , then the multiple feature information multiplied by the weights are combined, so as to obtain the combined feature information (i.e. Hidden Representation shown in Figure 7 ).

[0198] The combined feature information is further input into the classification layer (i.e. Sampled Softmax Classifier shown in Figure 7 ), in which the combined feature information is multiplied by the output matrix (i.e. Output Matrix shown in Figure 7 ), so as to obtain the similarity between the target sample video and each video in the video database of the application program, i.e. the similarity corresponding to the different numbered videos in the video database is output.

[0199] The similarity between the target sample video and the positive sample video is obtained based on the number of the positive sample video in the multiple similarities output by the classification layer, and the similarity between the target sample video and the negative sample video is obtained based on the number of the negative sample video in the multiple similarities output by the classification layer.

[0200] The similarity of the target sample video and the positive sample video, and the similarity of the target sample video and the negative sample video are input into a loss function to obtain a loss value, and the EIESG model is trained based on the loss value.

[0201] The EIESG model is trained multiple times based on the training method, and when a preset end condition is met, the trained EIESG model is obtained.

[0202] Then, the video feature extraction model in the EIESG model can be taken out, and the trained video feature extraction model is used to extract features of the videos in the application respectively to obtain video feature information corresponding to each video, and then the video feature information can be used in subsequent applications of the application.

[0203] Next, taking the video and the target sample video as music videos as an example, the trained EIESG model is evaluated.

[0204] The video types of audio videos can be divided into the following categories: Chinese style, European and American popular music, children's music, DJ dance music, music variety show, Cantonese popular music, movie soundtrack, electronic dance music, Chinese rap, and pure music.

[0205] Taking recalling 100 videos of each video type as an example, the evaluation is performed, and the visual results are shown in FIGS. 1-9. Figure 5 Figure 8 The classification between 100 videos of each two video types is shown in FIGS. 1-9, and a total of 81 classification pictures are included. The 36 classification pictures below the main diagonal line are mirror images of the 36 classification pictures above the main diagonal line, and therefore, the 36 classification pictures below the main diagonal line are not displayed.

[0206] Figure 8 The 9 classification pictures in the first row of FIGS. 1-9 show the classification between Chinese style and European and American popular music, children's music, DJ dance music, music variety show, Cantonese popular music, movie soundtrack, electronic dance music, Chinese rap, and pure music.

[0207] In FIGS. 1-9, Figure 8 The black video points and the gray video points in FIGS. 1-9 only distinguish two different video classifications in the classification picture, and do not specifically refer to a certain video type.

[0208] Figure 9 The average number of plays per person in the scenario of applying the video feature information obtained by the two different video feature extraction models to video recommendation is shown in FIG. 10. The solid line represents the average number of plays per person after using the video feature extraction model of the present scheme for feature extraction, and the dashed line represents the average number of plays per person after using the video feature extraction model of the prior art.​

[0209] Figure 10 The figure shows the average play time of each user in the scenario of video recommendation after the video is extracted by two different video feature extraction models, and the video feature information obtained is applied. The solid line represents the average play time of each user after the video feature extraction model of the present solution is used for feature extraction, and the dashed line represents the average play time of each user after the video feature extraction model of the prior art is used.

[0210] By Figure 9 And Figure 10 It can be seen that using the video feature extraction model in the present application embodiment can effectively improve the interest of users in the video, thereby effectively improving the average play times and the average play time.

[0211] In the present application embodiment, a video recommendation method is also provided, see Figure 11 The method can include:

[0212] 1101, based on the trained video feature extraction model, respectively extracting the features of each video in the video library to obtain the video feature information corresponding to each video.

[0213] Among them, the trained video feature extraction model can be any trained video feature extraction model described in the present application embodiment.

[0214] In implementation, after the video feature information corresponding to each video in the video library is obtained by extracting the features of each video in the video library based on the trained video feature extraction model, the video feature information can be stored correspondingly, that is, the video feature information corresponding to each video is stored correspondingly with the identification information of the video.

[0215] Corresponding to the setting of the video feature extraction model including the identification information feature extraction sub-model and the edge information feature extraction sub-model, the method flow of extracting the features of the video based on the video feature extraction model can be:

[0216] The identification information of the video is input into the identification information feature extraction sub-model, so as to obtain the first video feature sub-information corresponding to the video. The edge information of the video is input into the edge information feature extraction sub-model, so as to obtain the second video feature sub-information corresponding to the video.

[0217] Then, the first video feature sub-information and the second video feature sub-information can be added in position, so as to obtain the video feature information corresponding to the video. Or, the average value can be taken after the first video feature sub-information and the second video feature sub-information are added in position, so as to obtain the video feature information corresponding to the video. The present application embodiment does not limit the specific method.

[0218] For a newly released video, a video feature extraction model can be used to extract features of the video to obtain video feature information corresponding to the video. Only the side information feature extraction sub-model can be used to obtain second video feature sub-information corresponding to the video, and the second video sub-information is directly determined as the video feature information corresponding to the video.

[0219] 1102. Receive a video recommendation request sent by the target terminal.

[0220] The video recommendation request carries identification information of a latest played video of the target terminal.

[0221] In implementation, the latest played video can be a video just finished playing by the target terminal. After the target terminal finishes playing a video, the target terminal can send a video recommendation request to the background server of the application program.

[0222] The definition of the finished video can be any reasonable setting, for example, when the ratio of the playing time of the target terminal to the total time of the video is greater than a preset ratio threshold, the video can be determined as a finished video. Of course, other settings can also be used, and the embodiments of the application are not limited in this regard.

[0223] 1103. Determine video feature information corresponding to the latest played video.

[0224] In implementation, after receiving the video recommendation request sent by the target terminal, the background server can determine the video feature information corresponding to the latest played video based on the identification information of the latest played video carried in the video recommendation request and the corresponding relationship between the stored feature information and the identification information of the video.

[0225] Generally, not all videos played in the application program have copyrights. Some videos do not have copyrights, but can be applied for playing. For videos with copyrights, the video library stores the video and corresponding video feature information. For videos without copyrights, the video library does not store the video and corresponding video feature information.

[0226] Therefore, when determining the video feature information corresponding to the latest played video, the following processing can be performed:

[0227] If the video library stores the latest played video, the video feature information corresponding to the latest played video is obtained in the video library based on the identification information of the latest played video. If the video library does not store the latest played video, the attribute information of the latest played video is obtained, the attribute information of the latest played video is extracted based on the trained video feature extraction model to obtain the video feature information corresponding to the latest played video.

[0228] In implementation, first, it is determined whether the latest played video is stored in the video library through the identification information of the latest played video. If the latest played video is stored in the video library, it can be determined that the latest played video is a video with copyright. At this time, the video feature information corresponding to the identification information of the latest played video can be obtained in the video library according to the correspondence between the identification information and the video feature information in the video library, so as to obtain the video feature information corresponding to the latest played video.

[0229] If the latest played video is not stored in the video library, it can be determined that the latest played video is a video without copyright. At this time, the attribute information of the latest played video can be obtained first (the method for obtaining the attribute information has been introduced above, and will not be described here again), and then the video feature extraction model trained can be used to extract the features of the attribute information of the latest played video, so as to obtain the video feature information corresponding to the latest played video.

[0230] 1104、Based on the video feature information corresponding to the latest played video and the video feature information corresponding to each video, the similarity between each video and the latest played video is determined.

[0231] In implementation, there can be various methods for calculating the similarity between each video and the latest played video through the video feature information, for example, the similarity between two video feature information can be calculated through the Euclidean Distance method, the method of cosine of the included angle, etc. Of course, other calculation methods can also be used, and the embodiments of the present application are not limited thereto.

[0232] 1105、Based on the similarity between each video and the latest played video, the target recommended video corresponding to the latest played video is determined in each video, and the target recommended video is sent to the target terminal.

[0233] In implementation, there can be various methods for determining the target recommended video corresponding to the latest played video based on the similarity between each video and the latest played video. The following are two of them:

[0234] First, the staff can pre-set a similarity threshold. The video with a similarity greater than the similarity threshold to the latest played video is determined as the target recommended video corresponding to the latest played video.

[0235] Second, the staff can pre-set a video number threshold. After calculating the similarity between each video and the latest played video, the video number threshold videos can be selected in the order from large to small similarity as the target recommended video corresponding to the latest played video.

[0236] After determining the target recommended video corresponding to the latest played video, the display information of the target recommended video can be sent to the target terminal. The target terminal can display the display information of the target recommended video after playing the latest played video in the video playing interface of the application program. The user to which the target terminal belongs can play a target recommended video by clicking the display information of the target recommended video. Alternatively, after the target terminal plays the latest played video, a target recommended video can be automatically played. The target recommended videos can be played in the order of the similarity between each target recommended video and the latest played video from large to small.

[0237] Of course, after screening the videos based on the similarity between each video and the latest played video, the target recommended videos can be further screened based on other data, for example, the videos can be screened by the click volume, the play volume, and the like. The embodiments of the present application do not limit this.

[0238] All the optional technical solutions described above can be combined to form optional embodiments of the present application, which will not be described one by one here.

[0239] The scheme mentioned in the embodiments of the present application can be based on the video feature extraction model to be trained to extract the attribute information of the target sample video to obtain video feature information, based on the account feature extraction model to be trained to extract the video type preference information corresponding to the sample account to obtain account feature information, and based on the video feature information, the account feature information, and the video similarity prediction model to be trained to determine the predicted similarity between the target sample video and the reference sample video. Based on the predicted similarity and the reference similarity, the video feature extraction model to be trained, the account feature extraction model to be trained, and the video similarity prediction model to be trained are adjusted. When the preset condition is met, the video feature extraction model after adjustment is determined as the trained video feature extraction model. By using the present application, the attribute features of the target sample video itself and the video type preference information of the sample account that has played the target sample video can be considered together to constrain the video feature information corresponding to the target sample video, so as to obtain more accurate video feature information. When the video feature information is used to recommend similar videos to the user, better recommendation effect can be obtained.

[0240] The embodiments of the present application provide a device for training a video feature extraction model. The device can be a computer device in the above-mentioned embodiments, as shown in Figure 12 The device comprises:

[0241] The acquisition module 1210 is configured to acquire attribute information of a target sample video played by a sample account, a reference sample video corresponding to the target sample video, and a reference similarity between the target sample video and the reference sample video.

[0242] The first determination module 1220 is configured to perform feature extraction on the attribute information based on a video feature extraction model to be trained to obtain video feature information, and perform feature extraction on the video type preference information corresponding to the sample account based on an account feature extraction model to be trained to obtain account feature information.

[0243] The second determination module 1230 is configured to input the video feature information and the account feature information into a video similarity prediction model to be trained, and output a predicted similarity between the target sample video and the reference sample video.

[0244] The parameter adjustment module 1240 is configured to adjust the video feature extraction model to be trained, the account feature extraction model to be trained, and the video similarity prediction model to be trained based on the predicted similarity and the reference similarity.

[0245] The third determination module 1250 is configured to determine the video feature extraction model after the adjustment as a trained video feature extraction model if a preset end condition is met.

[0246] In a possible implementation, the reference sample video includes a positive sample video and a negative sample video, and the reference similarity between the target sample video and the reference sample video includes a first reference similarity between the target sample video and the positive sample video and a second reference similarity between the target sample video and the negative sample video.

[0247] The acquisition module 1210 is configured to:

[0248] acquire a video played by the sample account and meeting a play time proximity condition with the target sample video as the positive sample video corresponding to the target sample video, where the first reference similarity is a first numerical value.

[0249] acquire a video other than the target sample video and the positive sample video as the negative sample video corresponding to the target sample video, where the second reference similarity is a second numerical value, and the second numerical value is less than the first numerical value.

[0250] In a possible implementation, the acquisition module 1210 is configured to:

[0251] acquire at least one video played by the sample account and having a time difference between a play time point and a play time point of the target sample video less than a preset time difference threshold as a pending positive sample video.

[0252] arrange the pending positive sample videos in sequence based on the play time points to obtain a pending positive sample video sequence.

[0253] The pending positive sample video with a number of interval videos between the pending positive sample video and the target sample video less than a preset number is determined as a positive sample video corresponding to the target sample video.

[0254] In a possible implementation, the attribute information of the target sample video includes identification information of the target sample video and edge information of the target sample video, where the identification information of the target sample video is used to represent a unique mark of the target sample video, and the edge information of the target sample video is information associated with a video content of the target sample video.

[0255] The obtaining module 1210 is configured to:

[0256] obtain, in a video library, identification information of the target sample video, and image data and audio data corresponding to the target sample video;

[0257] perform image recognition processing on the image data corresponding to the target sample video, and perform audio recognition processing on the audio data corresponding to the target sample video, to obtain the edge information of the target sample video.

[0258] In a possible implementation, the obtaining module 1210 is further configured to:

[0259] obtain historical playback data of the sample account, where the historical playback data includes historical videos played by the sample account in a historical period and a playback time proportion corresponding to each historical video, and the playback time proportion corresponding to each historical video is a ratio of a playback time of the historical video to a total time of the historical video;

[0260] determine, as a historical preference video corresponding to the sample account, a historical video with a corresponding playback time proportion greater than or equal to a playback time proportion threshold;

[0261] determine, based on a video type corresponding to each historical preference video, video type preference information corresponding to the sample account.

[0262] In a possible implementation, the video feature extraction model to be trained includes a first parameter matrix to be trained, and the account feature extraction model to be trained includes a second parameter matrix to be trained.

[0263] The first determination module 1220 is configured to:

[0264] input the attribute information into a first hash algorithm to obtain a first hash value corresponding to the attribute information;

[0265] Obtaining first feature information corresponding to the first hash value in the first parameter matrix to be trained as the video feature information;

[0266] Inputting the video type preference information into a second hash algorithm to obtain a second hash value corresponding to the video type preference information;

[0267] Obtaining second feature information corresponding to the second hash value in the second parameter matrix to be trained as the account feature information.

[0268] In a possible implementation, the video similarity prediction model includes a hidden layer and a classification layer.

[0269] The second determination module 1230 is configured to:

[0270] Inputting the video feature information and the account feature information into a hidden layer to be trained to obtain combined feature information;

[0271] Inputting the combined feature information into a classification layer to be trained to obtain a predicted similarity between the target sample video and the reference sample video.

[0272] An embodiment of the present application provides a video recommendation device, which can be a computer device in the above-mentioned embodiments, as shown in the accompanying drawings, the device comprises: Figure 13

[0273] The feature extraction module 1310 is configured to perform feature extraction on each video in a video library based on a trained video feature extraction model determined by any one of the above-mentioned devices for training a video feature extraction model, to obtain video feature information corresponding to each video.

[0274] The receiving module 1320 is configured to receive a video recommendation request sent by a target terminal, wherein the video recommendation request carries identification information of a latest played video of the target terminal.

[0275] The fourth determination module 1330 is configured to determine video feature information corresponding to the latest played video.

[0276] The fifth determination module 1340 is configured to determine a similarity between each video and the latest played video based on the video feature information corresponding to the latest played video and the video feature information corresponding to each video.

[0277] The sending module 1350 is configured to determine a target recommended video corresponding to the latest played video from the videos based on the similarity between each video and the latest played video, and send the target recommended video to the target terminal.

[0278] ​In a possible implementation, the fourth determining module 1330 is configured to:

[0279] If the latest played video is stored in the video library, video feature information corresponding to the latest played video is obtained from the video library based on identification information of the latest played video.

[0280] If the latest played video is not stored in the video library, attribute information of the latest played video is obtained, and the attribute information of the latest played video is subjected to feature extraction based on the trained video feature extraction model, so as to obtain the video feature information corresponding to the latest played video.

[0281] It should be noted that the device for training a video feature extraction model provided in the above embodiments is only used as an example for the division of the above functional modules in training the video feature extraction model. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device for training a video feature extraction model and the method for training a video feature extraction model provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0282] The device for recommending a video provided in the above embodiments is only used as an example for the division of the above functional modules in recommending a video. In actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device for recommending a video and the method for recommending a video provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0283] Figure 14 FIG. 14 is a structural schematic diagram of a server provided in an embodiment of the present application. The server 1400 can have great differences due to different configurations or performances, and can include one or more central processors 1401 and one or more memories 1402. The memory 1402 stores at least one instruction, which is loaded and executed by the processor 1401 to implement the method provided in each method embodiment. Of course, the server can also have a wired or wireless network interface, a keyboard, an input and output interface, and other components for realizing the functions of the device, which will not be repeated here.

[0284] In an example embodiment, a computer readable storage medium, such as a memory including instructions executable by a processor in a terminal to perform the method of training a video feature extraction model and video recommendation in the above-described embodiments, is also provided. The computer readable storage medium can be non-transitory. For example, the computer readable storage medium can be a ROM, a RAM, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, and the like.

[0285] A person of ordinary skill in the art can understand that all or part of the steps of the above-described embodiments can be completed by hardware, or by a program instructing relevant hardware, and the program can be stored in a computer readable storage medium, such as a read-only memory, a magnetic disk or an optical disk.

[0286] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data for analysis, stored data, displayed data, etc.), and signals (including but not limited to signals transmitted between user terminals and other devices, etc.) involved in the present application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions. For example, the "video type preference information of the sample account" involved in the present application is obtained under full authorization.

[0287] The above-described is only an optional embodiment of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for training a video feature extraction model, characterized in that, The method comprises: obtaining identification information of a target sample video played by a sample account in a video library, and image data and audio data corresponding to the target sample video, wherein the identification information of the target sample video is used to represent a unique mark of the target sample video; performing image recognition processing on the image data corresponding to the target sample video, and performing audio recognition processing on the audio data corresponding to the target sample video, to obtain side information of the target sample video, wherein the side information of the target sample video is information associated with video content of the target sample video; obtaining attribute information of the target sample video, a reference sample video corresponding to the target sample video, and a reference similarity between the target sample video and the reference sample video, wherein the attribute information of the target sample video comprises the identification information of the target sample video and the side information of the target sample video; performing feature extraction on the attribute information based on a video feature extraction model to be trained, to obtain video feature information, and performing feature extraction on video type preference information corresponding to the sample account based on an account feature extraction model to be trained, to obtain account feature information; inputting the video feature information and the account feature information into a video similarity prediction model to be trained, to output a predicted similarity between the target sample video and the reference sample video; based on the predicted similarity and the reference similarity, adjusting parameters of the video feature extraction model to be trained, the account feature extraction model to be trained, and the video similarity prediction model to be trained; if a preset ending condition is met, determining the video feature extraction model after the adjustment as a trained video feature extraction model.

2. The method of claim 1, wherein, The reference sample video comprises a positive sample video and a negative sample video, and the reference similarity between the target sample video and the reference sample video comprises a first reference similarity between the target sample video and the positive sample video, and a second reference similarity between the target sample video and the negative sample video; obtaining the reference sample video corresponding to the target sample video, and the reference similarity between the target sample video and the reference sample video, comprises: obtaining a video played by the sample account and meeting a play time proximity condition with the target sample video as the positive sample video corresponding to the target sample video, wherein the first reference similarity is a first numerical value; obtaining a video other than the target sample video and the positive sample video as the negative sample video corresponding to the target sample video, wherein the second reference similarity is a second numerical value, and the second numerical value is less than the first numerical value.

3. The method of claim 2, wherein, The obtaining of the video played by the sample account and meeting the play time proximity condition with the target sample video as the positive sample video corresponding to the target sample video comprises: obtaining at least one video played by the sample account and having a time difference between a play time point and a play time point of the target sample video less than a preset time difference threshold as a pending positive sample video; arrange the pending positive sample videos in sequence based on the play time points to obtain a pending positive sample video sequence; determine, as a positive sample video corresponding to the target sample video, a pending positive sample video in the pending positive sample video sequence and having a number of interval videos with the target sample video less than a preset number.

4. The method of claim 1, wherein, The method further comprises: obtaining historical play data of the sample account, the historical play data comprising historical videos played by the sample account in a historical period and a play time proportion corresponding to each historical video, the play time proportion corresponding to each historical video being a ratio of a play time of the historical video to a total time of the historical video; determining, as a historical preference video corresponding to the sample account, a historical video having a corresponding play time proportion greater than or equal to a play time proportion threshold; determining video type preference information corresponding to the sample account based on a video type corresponding to each historical preference video.

5. The method of claim 1, wherein, The video feature extraction model to be trained comprises a first parameter matrix to be trained, and the account feature extraction model to be trained comprises a second parameter matrix to be trained; The feature extraction based on the video feature extraction model to be trained on the attribute information comprises: performing hash calculation on the attribute information to obtain a first hash value corresponding to the attribute information; obtaining, in the first parameter matrix to be trained, first feature information corresponding to the first hash value as the video feature information; The feature extraction based on the account feature extraction model to be trained on the video type preference information corresponding to the sample account comprises: performing hash calculation on the video type preference information to obtain a second hash value corresponding to the video type preference information; obtaining, in the second parameter matrix to be trained, second feature information corresponding to the second hash value as the account feature information.

6. The method of claim 1, wherein, The video similarity prediction model comprises a hidden layer and a classification layer; The determination of the predicted similarity between the target sample video and the reference sample video based on the video feature information, the account feature information and the video similarity prediction model to be trained comprises: inputting the video feature information and the account feature information into the hidden layer to be trained to obtain combined feature information; inputting the combined feature information into the classification layer to be trained to obtain the predicted similarity between the target sample video and the reference sample video.

7. A method for video recommendation, characterized in that, The method comprises: performing feature extraction on each video in a video library based on the trained video feature extraction model according to any one of claims 1-6 to obtain video feature information corresponding to each video; receiving a video recommendation request sent by a target terminal, wherein the video recommendation request carries identification information of a latest played video of the target terminal; determining video feature information corresponding to the latest played video; determining a similarity between each video and the latest played video based on the video feature information corresponding to the latest played video and the video feature information corresponding to each video; Based on the similarity between each video and the latest played video, a target recommended video corresponding to the latest played video is determined from the videos, and the target recommended video is sent to the target terminal.

8. The method of claim 7, wherein, The determining of the video feature information corresponding to the latest played video comprises: If the video library stores the latest played video, video feature information corresponding to the latest played video is obtained from the video library based on identification information of the latest played video; If the video library does not store the latest played video, attribute information of the latest played video is obtained, and the attribute information of the latest played video is subjected to feature extraction based on the trained video feature extraction model to obtain the video feature information corresponding to the latest played video.

9. A computer device, comprising: The computer device comprises a processor and a memory, the memory stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method according to any one of claims 1 to 8.

10. A computer-readable storage medium, characterized in that, The storage medium stores at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method according to any one of claims 1 to 8.

11. A computer program product, characterised in that, The computer program product comprises at least one instruction, and the at least one instruction is loaded and executed by the processor to implement the operations performed by the method according to any one of claims 1 to 8.

Images