A video recommendation method and device, computer device and storage medium

Abstract

Embodiments of the present application disclose a video recommendation method and device, computer equipment and a storage medium, which can obtain a historical browsing video and a to-be-recommended video of a target object; perform highlight video frame identification on video frames in the historical browsing video to obtain at least one target video segment; perform feature extraction on the video frames in the target video segment to obtain feature information corresponding to each target video segment; calculate a similarity between the target video segment and the to-be-recommended video according to the feature information of the target video segment and feature information of the to-be-recommended video; select a reference video segment from the target video segment according to the similarity; determine a target recommended video segment similar to the reference video segment from the to-be-recommended video, and recommend the target recommended video segment to the target object; and the embodiments of the present application can make full use of non-highlight video frames in the historical browsing video, and meet the demand that the user has at least one preference, thereby improving the accuracy of video recommendation.

Description

TECHNICAL FIELD

[0001] Embodiments of the present application relate to the technical field of computer technology, and specifically relate to a video recommendation method and device, a computer device, and a storage medium. BACKGROUND

[0002] Nowadays, people increasingly like to share personal videos recording daily life or personal preferences on social media platforms such as YouTube, Instagram, TikTok, and Kuaishou.

[0003] Among these personal videos, videos that are carefully edited and summarize the key points of a specific event are more attractive to the audience. However, in most cases, the original video of a personal video contains many irrelevant contents to the key points, and manually editing the key points in the original video is a tedious and time-consuming task. Therefore, how to use artificial intelligence technology to automatically detect video highlights and improve the efficiency of video editing to better serve people is the current development trend.

[0004] Currently, artificial intelligence technology uses video highlight detection (VHD) technology to extract the key points of the original video, that is, to extract the most interesting segments or frames from the original video. Video highlight detection technology is increasingly widely used in sports events, e-sports, and other fields.

[0005] However, when determining the key points of the original video, it is subjective and is determined based on the user's preferences, and the preferences of the same user may change. For the same original video, different users may have different highlight preferences. For example, for sports videos, some users prefer scoring video segments, and some users prefer team cooperation video segments, thus raising the problem of personalized video highlight detection (P-VHD). To solve this problem of personalized video highlight detection, it is necessary to extract highlight video segments and adapt to different user personalized preferences.

[0006] From the perspective of highlight video clips of a high-light video, the current video recommendation method only uses the highlight video clips in the historical browsing videos of a user to induce the preference of the user, does not use the non-highlight video clips in the historical browsing videos, and cannot comprehensively and objectively induce the preference of the user, so as to accurately determine the preference of the user. From the perspective of the preference, the current video recommendation method is generally based on the assumption that a user has only one preference, that is, the historical browsing videos of the user are regarded as the whole information to predict the preference of the user. However, since the interests of the same user can be multiple, for example, the interests of the same user are travel, swimming, pet raising and the like, it is unreasonable to regard the historical browsing videos of the user as the whole information to extract a whole preference to represent the multiple interests of the user, and this also cannot accurately determine the preference of the user. Thus, the accuracy of the video recommendation of the prior art is not high. SUMMARY

[0007] Embodiments of the present application provide a video recommendation method and device, computer equipment and a storage medium, which can improve the accuracy of video recommendation.

[0008] A video recommendation method comprises:

[0009] obtaining historical browsing videos and to-be-recommended videos of a target object, the historical browsing videos comprising at least two video frames;

[0010] performing highlight video frame identification on the video frames in the historical browsing videos to obtain at least one target video clip, the target video clip comprising a highlight video frame and a non-highlight video frame;

[0011] performing feature extraction on the video frames in the target video clip to obtain feature information corresponding to each target video clip;

[0012] calculating the similarity between the target video clip and the to-be-recommended videos according to the feature information of the target video clip and the feature information of the to-be-recommended videos;

[0013] selecting a reference video clip from the at least one target video clip according to the similarity;

[0014] determining a target recommended video clip similar to the reference video clip from the to-be-recommended videos, and recommending the target recommended video clip to the target object.

[0015] Correspondingly, embodiments of the present application provide a video recommendation device, comprising:

[0016] an obtaining unit configured to obtain historical browsing videos and to-be-recommended videos of a target object, the historical browsing videos comprising at least two video frames;

[0017] The identification unit is configured to perform highlight video frame identification on the video frames in the historical browsing video to obtain at least one target video segment, and the target video segment includes a highlight video frame and a non-highlight video frame.

[0018] The extraction unit is configured to perform feature extraction on the video frames in the target video segment to obtain feature information corresponding to each target video segment.

[0019] The calculation unit is configured to calculate a similarity between the target video segment and the to-be-recommended video according to the feature information of the target video segment and the feature information of the to-be-recommended video.

[0020] The selection unit is configured to select a reference video segment from the at least one target video segment according to the similarity.

[0021] The recommendation unit is configured to determine a target recommended video segment similar to the reference video segment from the to-be-recommended video, and recommend the target recommended video segment to the target object.

[0022] In some embodiments, the extraction unit can be configured to perform feature extraction on the video frames in the target video segment to obtain video features, and perform feature mapping processing on the video features to generate the feature information corresponding to the target video segment.

[0023] In some embodiments, the extraction unit can be specifically configured to perform video segment feature extraction on the target video segment to obtain initial video features, perform video frame feature extraction on the highlight video frames in the target video segment to obtain highlight video features, perform video frame feature extraction on the non-highlight video frames in the target video segment to obtain non-highlight video features, and generate the video features according to the initial video features, the highlight video features, and the non-highlight video features.

[0024] In some embodiments, the extraction unit can be specifically configured to perform feature mapping processing on the initial video features to obtain initial feature information, perform feature mapping processing on the highlight video features to obtain highlight feature information, perform feature mapping processing on the non-highlight video features to obtain non-highlight feature information, and fuse the initial feature information, the highlight feature information, and the non-highlight feature information to obtain the feature information corresponding to the target video segment.

[0025] In some embodiments, the extraction unit can be specifically configured to fuse the initial feature information and the highlight feature information to obtain fused highlight feature information, fuse the initial feature information and the non-highlight feature information to obtain fused non-highlight feature information, and fuse the fused highlight feature information and the fused non-highlight feature information to obtain the feature information corresponding to the target video segment.

[0026] In some embodiments, the identifying unit, in particular, can be configured to split the historical browsing video to obtain candidate video clips; and perform highlight video frame identification on the labels of the video frames in the candidate video clips to obtain at least one target video clip.

[0027] In some embodiments, the selecting unit, in particular, can be configured to calculate an initial similarity between the target video clip and the video to be recommended according to the initial feature information of the target video clip and the feature information of the video to be recommended; optimize the similarity according to the initial similarity to obtain an optimized similarity; and select a reference video clip from the at least one target video clip according to the optimized similarity.

[0028] The embodiments of the present application can obtain a historical browsing video of a target object and a video to be recommended, the historical browsing video comprising at least two video frames; perform highlight video frame identification on the video frames in the historical browsing video to obtain at least one target video clip, the target video clip comprising a highlight video frame and a non-highlight video frame; perform feature extraction on the video frames in the target video clip to obtain feature information corresponding to each target video clip; calculate a similarity between the target video clip and the video to be recommended according to the feature information of the target video clip and the feature information of the video to be recommended; select a reference video clip from the at least one target video clip according to the similarity; determine a target recommended video clip similar to the reference video clip from the video to be recommended, and recommend the target recommended video clip to the target object; since the embodiments of the present application can obtain at least one target video clip comprising a highlight video frame and a non-highlight video frame through highlight identification on the historical browsing video, on the one hand, the non-highlight video frames in the historical browsing video are fully utilized, and on the other hand, the at least one target video clip obtained can meet the demand of the user having at least one preference, thereby improving the accuracy of video recommendation. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative effort.

[0030] Figure 1 is a scene schematic diagram of the video recommendation method provided by the embodiments of the present application;

[0031] Figure 2 is a flowchart diagram of the video recommendation method provided by the embodiments of the present application;

[0032] Figure 3 is a flowchart diagram of obtaining at least one target video clip provided by the embodiments of the present application;

[0033] Figure 4 is a flowchart of a process for feature extraction of video frames in a target video segment provided by an embodiment of the present application;

[0034] Figure 5 is a flowchart of a process for obtaining video features provided by an embodiment of the present application;

[0035] Figure 6 is a flowchart of a process for obtaining feature information of a target video segment provided by an embodiment of the present application;

[0036] Figure 7 is a flowchart of a process for fusing initial feature information, highlight feature information and non-highlight feature information provided by an embodiment of the present application;

[0037] Figure 8 is a flowchart of a process for optimizing similarity provided by an embodiment of the present application;

[0038] Figure 9 is a PACNet algorithm framework diagram provided by an embodiment of the present application;

[0039] Figure 10 is a flowchart of a video recommendation method provided by an embodiment of the present application;

[0040] Figure 11 is a schematic diagram of feature mapping processing of video features to generate feature information corresponding to a target video segment provided by an embodiment of the present application;

[0041] Figure 12 is a schematic diagram of a computer device selecting a reference video segment from at least one target video segment according to similarity provided by an embodiment of the present application.

[0042] Figure 13 is a comparison diagram of a recommendation result of a PACNet algorithm and a recommendation result of a highlight detection baseline algorithm of prior art provided by an embodiment of the present application.

[0043] Figure 14 is a structural schematic diagram of a video recommendation device provided by an embodiment of the present application;

[0044] Figure 15 is a structural schematic diagram of a computer device provided by the present application. DETAILED DESCRIPTION

[0045] With reference to the drawings, the technical solutions in the embodiments of the present application will be described clearly and completely. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0046] The present application provides a video recommendation method and device, computer equipment and a storage medium. The video recommendation device can be integrated in the computer equipment, which can be a server or a terminal device.

[0047] The server can be a stand-alone physical server, a server cluster or a distributed system composed of multiple physical servers, a cloud server providing cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, content delivery network (CDN), and basic cloud computing services such as big data and artificial intelligence platforms. The terminal can be a smart phone, a tablet computer, a notebook computer, a desktop computer, a smart speaker, a smart watch, a smart television, etc., but is not limited thereto. The terminal and the server can be directly or indirectly connected through wired or wireless communication, and the present application is not limited in this regard.

[0048] The present application relates to artificial intelligence (AI). Artificial intelligence is the use of digital computers or computer-controlled machines to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology of computer science, which aims to understand the essence of intelligence and produce a new intelligent machine that can react in a similar way to human intelligence. Artificial intelligence is the design principle and implementation method of various intelligent machines, enabling machines to have perception, reasoning and decision-making functions.

[0049] Artificial intelligence technology is a comprehensive discipline, involving a wide range of fields, both hardware and software technologies. Artificial intelligence basic technologies generally include sensors, special artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operation / interaction systems, mechatronics, etc. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning, etc.

[0050] For example, see Figure 1For example, the video recommendation device integrated in the smart phone can obtain the historical browsing videos and the to-be-recommended videos of the target object through the video application installed on the smart phone, the historical browsing videos include at least two video frames; the highlight video frames in the historical browsing videos are identified to obtain at least one target video segment, the target video segment includes the highlight video frames and non-highlight video frames; the feature information corresponding to each target video segment is obtained by performing feature extraction on the video frames in the target video segment; the similarity between the target video segment and the to-be-recommended video is calculated according to the feature information of the target video segment and the feature information of the to-be-recommended video; the reference video segment is selected from the at least one target video segment according to the similarity; the target recommended video segment similar to the reference video segment is determined from the to-be-recommended video, and the target recommended video segment is recommended to the target object through the video application.

[0051] The target object refers to an object provided with a video recommendation service. For example, the target object can be a terminal or a communication account logged in the terminal.

[0052] The historical browsing video refers to a video browsed by the target object in historical time. The historical time can be a certain time period in the historical time, or all historical time. For example, when the historical time is a certain time period in the historical time, the historical browsing video is determined as a video browsed by the target object in the time period, such as a video browsed by the target object in the past month. For example, when the historical time is all historical time, the historical browsing video is determined as a video browsed by the target object in all historical time.

[0053] The historical browsing video carries attribute information, which represents the user's preference. For example, the attribute information includes suspense drama, science fiction drama, mythology drama, police and gangster drama, modern drama, costume drama, animation drama, comedy video, food video, fashion video, travel video, entertainment video, and life video. Different historical browsing videos can carry the same attribute information or different attribute information. For example, the video recommendation device receives multiple historical browsing videos, all of which can carry the same attribute information; different historical browsing videos can carry different attribute information; a part of the historical browsing videos can carry the same attribute information, and each of the other historical browsing videos can carry different attribute information.

[0054] The to-be-recommended video can be any video obtained by the video recommendation device. The to-be-recommended video can be a hot video on a certain platform, or a video with the same or similar attribute information as the historical browsing video of the target object.

[0055] For example,Figure 2 The following are described in detail, respectively. It should be noted that the order of the following embodiments is not limited as the preferred order of the embodiments.

[0056] 101、Obtain the historical browsing video of the target object and the video to be recommended.

[0057] The target object refers to an object provided with a video recommendation service. For example, the target object can be a terminal, a communication account logged in the terminal, or an object provided with a video recommendation service.

[0058] The historical browsing video includes at least two video frames. The historical browsing video refers to a video browsed by a user through a target object within a historical time. The historical time can be a certain time period within the historical time, or all historical time. For example, when the historical time is a certain time period within the historical time, the historical browsing video is determined to be a video browsed by the user through the target object within the time period. For example, the historical time is the last month, and the historical browsing video is determined to be a video browsed by the user through the target object within the last month. For example, when the historical time is all historical time, the historical browsing video is determined to be a video browsed by the user through the target object within all historical time.

[0059] The historical browsing video carries attribute information, which represents the user's preference. For example, the attribute information includes suspense drama, science fiction drama, mythology drama, police drama, modern drama, costume drama, animation drama, comedy video, food video, fashion video, travel video, entertainment video, and life video. Different historical browsing videos can carry the same attribute information or different attribute information.

[0060] The video to be recommended includes multiple video frames. The video to be recommended can be any video obtained by the video recommendation device. The video to be recommended can be a hot video on a certain platform, a video with the same or similar attribute information as the historical browsing video of the target object, a video of other users, or a video of the user himself / herself. These can be set according to actual needs.

[0061] 102、Identify the highlight video frame in the video frame of the historical browsing video to obtain at least one target video segment.

[0062] The user preference can be inferred according to the video frame in the historical browsing video. The target video segment includes a highlight video frame and a non-highlight video frame. The highlight video frame is the video frame preferred by the user. The non-highlight video frame is not the video frame preferred by the user.

[0063] For example, the historical browsing video includes a plurality of video frames, the plurality of video frames include highlight video frames and non-highlight video frames, the highlight video frames or the non-highlight video frames can be determined by setting the number of views, for example, when the number of views of a video frame is greater than or equal to a set threshold, the video frame is determined as a highlight video frame; when the number of views of the video frame is less than the set threshold, the video frame is determined as a non-highlight video frame. In addition, after the highlight video frames or the non-highlight video frames are determined, an identifier or a label can be set for the highlight video frames or the non-highlight video frames, so as to facilitate the video recommendation device to identify the highlight video frames and the non-highlight video frames.

[0064] The non-highlight video frames can be used as negative samples to enable the neural network used in the embodiment of the application to better extract feature information of the highlight video frames, because the highlight video frames exist relative to the non-highlight video frames, and the non-highlight video frames can better define the highlight video frames. If only the highlight video frames are used to summarize the user's preferences and the user's non-preferences are ignored, it is not objective and accurate enough. Therefore, the non-highlight video frames and the highlight video frames are used to jointly summarize the user's preferences, which can more accurately represent the user's preferences, thereby improving the accuracy of video recommendation and reducing the probability of false recommendation.

[0065] In the embodiment of the application, the number of target video segments includes at least one, so as to meet the demand of the user having at least one preference. In addition, if the target video segments are obtained by identifying the highlight video frames of different historical browsing videos, and the different historical browsing videos carry different attribute information, the obtained at least one target video segment also represents at least one preference of the user, that is, the demand of the user having at least one preference can be met.

[0066] Specifically, as shown in Figure 3 The process of obtaining at least one target video segment can be as follows:

[0067] A1, the historical browsing video is segmented to obtain a candidate video segment.

[0068] In the embodiment of the application, the historical browsing video is segmented by using a fixed length, so that at least one candidate video segment with a fixed length can be obtained. Of course, the historical browsing video can also be segmented by using any length, and at least one candidate video segment with a non-uniform length can be obtained.

[0069] In the embodiment of the application, the candidate video segment includes a video segment in which all video frames are highlight video frames, a video segment in which all video frames are non-highlight video frames, and a video segment in which video frames include highlight video frames and non-highlight video frames.

[0070] A2, high-light video frame identification is performed on the labels of the video frames in the candidate video clips to obtain at least one target video clip.

[0071] The target video clip includes high-light video frames and non-high-light video frames. The label of a high-light video frame can be 1, and the label of a non-high-light video frame can be 0. If the video recommendation device identifies that the label of a video frame is 1, the video recommendation device determines that the video frame is a high-light video frame. If the video recommendation device identifies that the label of a video frame is 0, the video recommendation device determines that the video frame is a non-high-light video frame.

[0072] 103, feature extraction is performed on the video frames in the target video clip to obtain feature information corresponding to each target video clip.

[0073] The feature extraction on the video frames in the target video clip can be performed by at least one feature extraction layer in the neural network. Each feature extraction layer extracts corresponding candidate feature information, which is the feature information corresponding to the target video clip. The feature extraction on the video frames in the target video clip can also be performed by the feature extraction layer and the output layer, so as to obtain the feature information corresponding to the target video clip. That is, the feature information of the target video clip can be candidate feature information obtained by any one of the feature extraction layers or the output layer. The feature information of the target video clip is preferably candidate feature information obtained by a deeper feature extraction layer, so as to improve the accuracy of video recommendation.

[0074] Specifically, as shown in FIG. 1, the feature extraction is performed on the video frames in the target video clip, and the specific process is as follows: Figure 4

[0075] B1, feature extraction is performed on the video frames in the target video clip to obtain video features.

[0076] The feature extraction layer of the neural network is used to preliminarily extract features of the video frames in the target video clip to obtain candidate video feature information of the target video clip, that is, video features. In this process, the video frames in the target video clip can only be subjected to feature extraction by one feature extraction layer, so that the video frames in the target video clip are generated as video features. Since the target video clip includes high-light video frames and non-high-light video frames, the obtained video features include high-light video features and non-high-light video features.

[0077] In addition to directly extracting features from the target video clip, the embodiment of the present application can also further process the target video clip. Specifically, as shown in FIG. 2, the target video clip is further processed to obtain the video features of the target video clip, and the specific process is as follows: Figure 5 ​As shown, the specific process of obtaining the video feature is as follows:

[0078] b11, video segment feature extraction is performed on the target video segment to obtain initial video features.

[0079] Among them, since the target video segment includes highlight video frames and non-highlight video frames, the obtained initial video features include highlight video features and non-highlight video features.

[0080] Among them, the embodiments of the present application can split the target video segment based on the identification or label of the video frame to obtain a set of highlight video frames and a set of non-highlight video frames, so as to facilitate subsequent video frame feature extraction on the highlight video frames and non-highlight video frame feature extraction on the non-highlight video frames. Of course, the embodiments of the present application can not need to split the target video segment, but directly perform video frame feature extraction on the highlight video frames and directly perform video frame feature extraction on the non-highlight video frames.

[0081] b12, video frame feature extraction is performed on the highlight video frames in the target video segment to obtain highlight video features.

[0082] Among them, the embodiments of the present application can extract highlight video frames from the target video segment based on the identification or label of the highlight video frames to form a set of highlight video frames; video frame feature extraction is performed on the set of highlight video frames to obtain highlight video features.

[0083] b13, video frame feature extraction is performed on the non-highlight video frames in the target video segment to obtain non-highlight video features.

[0084] Among them, the embodiments of the present application can extract non-highlight video frames from the target video segment based on the identification or label of the non-highlight video frames to form a set of non-highlight video frames; video frame feature extraction is performed on the set of non-highlight video frames to obtain non-highlight video features.

[0085] b14, generating video features according to initial video features, highlight video features and non-highlight video features.

[0086] Among them, the embodiments of the present application can fuse the initial video features, highlight video features and non-highlight video features to generate video features. The fusion method can splice the initial video features, highlight video features and non-highlight video features in the order of initial video features, highlight video features and non-highlight video features, or splice the initial video features, highlight video features and non-highlight video features in the order of highlight video features and non-highlight video features, initial video features.

[0087] In this embodiment of the application, the initial video features, highlight video features, and non-highlight video features may not be fused. The initial video features, highlight video features, and non-highlight video features are independent of each other. That is, the video features include the initial video features, highlight video features, and non-highlight video features.

[0088] B2. Perform feature mapping processing on the video features to generate feature information corresponding to the target video segment.

[0089] The video features are processed through the output layer of a neural network, which can be a fully connected layer, to obtain feature information. These video features can include initial video features, highlight video features, and non-highlight video features.

[0090] In addition to directly mapping video features, the embodiments of this application can also perform more in-depth processing on video features, specifically, such as... Figure 6 As shown, the specific process for obtaining the feature information of the target video segment is as follows:

[0091] b21. Perform feature mapping processing on the initial video features to obtain initial feature information.

[0092] The initial video features can be processed by mapping the output layer of the neural network to obtain initial feature information. Since the initial video features are obtained by feature extraction from the target video segment, which includes highlight and non-highlight video frames, the obtained initial video features include both highlight and non-highlight video features. Furthermore, since the initial feature information is obtained by feature mapping of the initial video features, the initial feature information includes both highlight and non-highlight feature information.

[0093] b22. Perform feature mapping processing on the highlight video features to obtain highlight feature information.

[0094] The highlight video feature can be processed by mapping the output layer of a neural network to obtain highlight feature information.

[0095] b23. Perform feature mapping processing on the non-highlight video features to obtain non-highlight feature information.

[0096] The non-highlight video feature can be processed by mapping the output layer of the neural network to obtain non-highlight feature information.

[0097] b24. The initial feature information, highlight feature information and non-highlight feature information are fused to obtain the feature information corresponding to the target video segment.

[0098] In this context, the initial feature information, highlight feature information, and non-highlight feature information are all feature vectors. There are several ways to fuse the initial feature information, highlight feature information, and non-highlight feature information. For example, the initial feature information, highlight feature information, and non-highlight feature information can be directly concatenated; or the highlight feature information and non-highlight feature information can be concatenated and then multiplied by the initial feature information to obtain the feature information.

[0099] In addition to the above, such as Figure 7 As shown, the specific method for fusing initial feature information, specular feature information, and non-spectral feature information can be as follows:

[0100] C1. The initial feature information and the specular feature information are fused to obtain the fused specular feature information.

[0101] There are several ways to fuse initial feature information and specular feature information. For example, the initial feature information and specular feature information can be added together to obtain fused specular feature information. Since the initial feature information includes both specular and non-spectral feature information, adding the initial and specular feature information can enhance the influence of the specular feature information in the fused specular feature information. For example, adding the initial feature information to twice the amount of specular feature information can also enhance the influence of the specular feature information in the fused specular feature information.

[0102] C2. The initial feature information and non-highlight feature information are fused to obtain the fused non-highlight feature information.

[0103] Similarly, there are multiple ways to fuse initial feature information and non-highlight feature information. For example, the initial feature information and non-highlight feature information can be added together to obtain fused non-highlight feature information. Since the initial feature information includes both highlight and non-highlight feature information, adding the initial and non-highlight feature information can enhance the influence of the non-highlight feature information in the fused non-highlight feature information. For example, adding the initial feature information to twice the amount of non-highlight feature information can also enhance the influence of the non-highlight feature information in the fused non-highlight feature information.

[0104] C3. The fused highlight feature information and the fused non-highlight feature information are fused together to obtain the feature information corresponding to the target video segment.

[0105] There are several ways to fuse the fused highlight feature information and the fused non-highlight feature information. For example, the fused highlight feature information and the fused non-highlight feature information can be spliced ​​together, such as vertically spliced, to obtain the feature information. Alternatively, the fused highlight feature information and the fused non-highlight feature information can be added together to obtain the feature information.

[0106] In this case, the feature information can also be obtained through a multimodal feature extractor. The feature information of the target video segment obtained is the fusion feature of the target video segment. The fusion feature can be audio features, facial features, or action features, etc.

[0107] 104. Calculate the similarity between the target video segment and the video to be recommended based on the feature information of the target video segment and the feature information of the video to be recommended.

[0108] The feature information of the video to be recommended can be obtained by feature extraction through at least one feature extraction layer of a neural network.

[0109] In this application embodiment, the similarity between the target video segment and the video to be recommended can be obtained using a Gaussian kernel function based on the feature information of the target video segment and the feature information of the video to be recommended; the similarity between the target video segment and the video to be recommended can also be obtained using cosine similarity based on the feature information of the target video segment and the feature information of the video to be recommended; or the similarity between the target video segment and the video to be recommended can also be obtained using a Transformer neural network based on the feature information of the target video segment and the feature information of the video to be recommended.

[0110] 105. Select a reference video segment from at least one target video segment based on similarity.

[0111] There is at least one reference video segment. Since each target video segment has a similarity score to the video to be recommended, the similarity scores are sorted from highest to lowest; a higher similarity score indicates a greater similarity between the target video segment and the video to be recommended. Following this descending sorting, at least one high-ranking similarity score is selected from the multiple similarity scores, and the reference video segment corresponding to this high-ranking similarity score is obtained from the target video segment.

[0112] For example, there are three target video segments: a first target video segment, a second target video segment, and a third target video segment. Each target video segment has a corresponding similarity score with the video to be recommended. The similarity score between the first target video segment and the video to be recommended is the first similarity score, the similarity score between the second target video segment and the video to be recommended is the second similarity score, and the similarity score between the third target video segment and the video to be recommended is the third similarity score. The similarity scores are sorted from highest to lowest, with the first similarity score higher than the second similarity score, and the second similarity score higher than the third similarity score. In this embodiment, a preset threshold can be set for the similarity score. When the similarity score reaches the preset threshold, it becomes the target similarity score. For example, if both the first and second similarities score reach the preset threshold, then both the first and second similarities score are considered target similarities. This allows the first and second similarities scores to be selected. Based on the first similarity score, reference video segments corresponding to the first similarity score are selected from the target video segments; similarly, based on the second similarity score, reference video segments corresponding to the second similarity score are selected from the target video segments. The embodiments of this application also preset the number of reference video segments. For example, the preset number of reference video segments is one, but it is not limited to one. Then, the target video segment corresponding to the highest similarity in the ranking is determined as the parameter video segment.

[0113] Among them, such as Figure 8 As shown in the embodiments of this application, the similarity can be optimized, as follows:

[0114] D1. Calculate the initial similarity between the target video segment and the video to be recommended based on the initial feature information of the target video segment and the feature information of the video to be recommended.

[0115] In this process, feature extraction of video frames within the target video segment can be performed through at least one feature extraction layer in a neural network. Each feature extraction layer extracts corresponding candidate feature information, and the feature information of the target video segment can be candidate feature information obtained from any feature extraction layer or output layer.

[0116] In the case of optimizing similarity in this application embodiment, the calculation of similarity and initial similarity uses candidate feature information of the target video segment obtained through different neural network layers, respectively. For example, similarity uses a deeper neural network layer, such as a fully connected layer, to obtain candidate feature information of the target video segment as the feature information of the target video segment, and the feature information of the target video segment and the feature information of the video to be recommended are calculated together; initial similarity uses a shallower neural network layer, such as the first feature extraction layer, to obtain candidate feature information of the target video segment as the initial feature information of the target video segment, and the initial feature information of the target video segment and the feature information of the video to be recommended are calculated together.

[0117] Furthermore, in the case of optimizing similarity in the embodiments of this application, the calculation of similarity and initial similarity can be performed using different methods. For example, the similarity is calculated using a Gaussian kernel function based on the feature information of the target video segment and the feature information of the video to be recommended; the initial similarity is calculated using cosine similarity based on the initial feature information of the target video segment and the feature information of the video to be recommended. The feature information of the target video segment and the initial feature information of the target video segment can be the same or different.

[0118] Furthermore, in the case of optimizing similarity in the embodiments of this application, the calculation of similarity and initial similarity can be achieved by using candidate feature information of the video to be recommended obtained from different neural network layers as the feature information of the video to be recommended, and then calculating the feature information of the target video segment and the feature information of the video to be recommended. For example, the similarity can be calculated by using candidate feature information of the video to be recommended obtained from a shallower feature extraction layer, and then calculating the feature information of the target video segment and the feature information of the video to be recommended; the initial similarity can be calculated by using candidate feature information of the video to be recommended obtained from a deeper feature extraction layer, and then calculating the initial feature information of the target video segment and the feature information of the video to be recommended. The feature information of the target video segment and the initial feature information of the target video segment can be the same or different.

[0119] D2. Optimize the similarity based on the initial similarity to obtain the optimized similarity.

[0120] In this embodiment, the initial similarity and the similarity score can be fused to optimize the similarity score, thereby obtaining an optimized similarity score. For example, the initial similarity score and the similarity score can be added or multiplied to optimize the similarity score, which can improve the accuracy of video recommendations.

[0121] D3. Based on the optimized similarity, select a reference video segment from at least one target video segment.

[0122] 106. Identify target recommended video clips that are similar to reference video clips from the videos to be recommended, and recommend the target recommended video clips to the target audience.

[0123] In this process, based on the similarity between a reference video segment and the video to be recommended, the positions of video frames similar to the reference video segment within the video to be recommended are determined. This embodiment of the application can use a fixed length to segment the video to be recommended, thereby obtaining target recommended video segments with video frames similar to the reference video segment; alternatively, this embodiment of the application can directly segment and extract video frames similar to the reference video segment to generate target recommended video segments. There is at least one target recommended video segment.

[0124] Specifically, after recommending the target video clip to the target audience, when the target audience is a terminal, the user opens the terminal and the terminal can receive the target video clip; when the target audience is a logged-in account, the user logs in to the logged-in account and the logged-in account can receive the target video clip.

[0125] This application embodiment can obtain the historical browsing videos and videos to be recommended for a target object. The historical browsing videos include at least two video frames. Highlight video frame recognition is performed on the video frames in the historical browsing videos to obtain at least one target video segment, which includes highlight video frames and non-highlight video frames. Feature extraction is performed on the video frames within the target video segments to obtain feature information corresponding to each target video segment. Based on the feature information of the target video segments and the feature information of the videos to be recommended, the similarity between the target video segments and the videos to be recommended is calculated. A reference video segment is selected from the at least one target video segment based on the similarity. A target recommended video segment similar to the reference video segment is determined from the videos to be recommended, and the target recommended video segment is recommended to the target object. Since this application embodiment can obtain at least one target video segment including highlight video frames and non-highlight video frames based on highlight recognition in historical browsing videos, it fully utilizes the non-highlight video frames in the historical browsing videos. Furthermore, the obtained at least one target video segment can satisfy the user's need for at least one preference, thereby improving the accuracy of video recommendations.

[0126] Based on the method described in the above application embodiments, the following will provide further detailed explanations with examples.

[0127] In this embodiment, the example of a video device specifically integrated into a computer device will be used for illustration. First, it should be noted that this embodiment defines an algorithm called PACNet to implement this embodiment. The PACNet algorithm framework diagram is shown below.Figure 9 As shown, the PACNet algorithm includes a Decision Boundary Customizer (DBC) module and a Mini History Record (Mi-Hi) module. This PACNet algorithm can be applied to video recommendation in areas such as game streaming, sports events, natural scenery, and food preparation. This embodiment of the application uses the PACNet algorithm to provide end-to-end real-time video recommendations to users.

[0128] like Figure 10 As shown, a video recommendation method has the following specific process:

[0129] 201. Computer devices acquire the target object's historical browsing videos and videos to be recommended.

[0130] The target object can be a terminal, a communication account logged into the terminal, or any other object that is provided with video recommendation services.

[0131] The viewed videos history includes at least two video frames. Viewed videos are those videos that a user has viewed within a specific timeframe related to the target object. Since user preferences may change, this specific timeframe can be defined as the last three months. This way, even if user preferences change, the fluctuations won't be too significant, thus improving the accuracy of video recommendations. Figure 9 As shown, there can be at least one historical video, which is represented by the symbol H, for example, H1, H2...H M These represent different videos viewed in the past.

[0132] The video to be recommended includes multiple video frames. The video to be recommended can be a trending video on a certain platform obtained by the computer device. The platform can be such as Tencent Video, WeChat's video account platform, or other WeChat mini-program platforms. The video to be recommended can also be a video that the user wants to edit.

[0133] 202. The computer device segments the historically viewed videos to obtain candidate video segments.

[0134] The process of computer devices segmenting historical browsing videos to obtain candidate video segments can be understood as achieving this by using the Mini History Recording (Mi-Hi) module in the PACNet algorithm to segment the historical browsing videos with a fixed length. In this way, at least one candidate video segment with a fixed length is obtained, which is a more granular and refined video compared to the historical browsing videos.

[0135] The candidate video segments include video segments where all video frames are highlight video frames, video segments where all video frames are non-highlight video frames, and video segments where the video frames include both highlight and non-highlight video frames.

[0136] 203. The computer equipment performs highlight video frame recognition on the tags of video frames in the candidate video segments to obtain at least one target video segment.

[0137] The target video segment includes highlight video frames and non-highlight video frames. Highlight video frames can be labeled 1, and non-highlight video frames can be labeled 0. If the computer device identifies a video frame with a label of 1, it determines that the video frame is a highlight video frame; if the computer device identifies a video frame with a label of 0, it determines that the video frame is a non-highlight video frame. The computer device identifies highlight video frames by analyzing the labels of video frames in the candidate video segments, thus obtaining at least one target video segment that includes both highlight and non-highlight video frames.

[0138] 204. Extract features from the video frames within the target video segment to obtain feature information corresponding to each target video segment.

[0139] In this process, feature extraction of video frames within the target video segment can be performed by passing the video frames within the target video segment through a feature extraction layer and an output layer to obtain the feature information corresponding to the target video segment.

[0140] Specifically, feature extraction is performed on video frames within the target video segment, and the specific process is as follows:

[0141] Feature extraction is performed on video frames within the target video segment to obtain video features; feature mapping is then performed on the video features to generate feature information corresponding to the target video segment.

[0142] This application embodiment can be understood as follows: a computer device inputs at least one target video segment into a feature encoder. The feature encoder can be understood as a feature extraction layer in a neural network. The feature encoder extracts features from the target video segment to obtain video features of the target video segment. The video features are represented by the symbol c. Different target video segments correspond to different video features, for example, c1, c2, ... c n These represent different video features. The video to be recommended shares the same feature encoder as the target video segment.

[0143] This embodiment can be understood as introducing a decision boundary customizer (DBC) into the output layer of the neural network. The decision boundary customizer learns to generate target parameters for the classifier, which are the feature information of the target video segment. The feature information of the target video segment is represented by the symbol θ, and the feature information of different target video segments is different, for example, θ1, θ2, ... θ n These represent the feature information of different target video segments. There is a one-to-one correspondence between the feature information of each target video segment and its video features.

[0144] Specifically, the process of extracting features from video frames within the target video segment to obtain video features is as follows:

[0145] Video segment features are extracted from the target video segment to obtain initial video features; highlight video frames in the target video segment are extracted to obtain highlight video features; non-highlight video frames in the target video segment are extracted to obtain non-highlight video features; and video features are generated based on the initial video features, highlight video features, and non-highlight video features.

[0146] Among them, the initial video features, highlight video features, and non-highlight video features are independent of each other. That is, the video features include the initial video features, highlight video features, and non-highlight video features.

[0147] Based on this, specifically, feature mapping processing is performed on the video features to generate feature information corresponding to the target video segment, such as... Figure 11 As shown, the specific process is as follows:

[0148] For example, in embodiments of this application, the symbol c' can be used to represent the initial video features, and the symbol c pos To represent the characteristics of specular video, the symbol c is used. neg Non-spectral features are represented by the symbol fc to denote a fully connected layer, and initial feature information is represented by the symbol φ. pos Highlight feature information is represented by the symbol φ. neg Non-spectral feature information is represented by the symbol θ. pos The fused specular features are represented by the symbol θ. neg The non-highlight feature information after fusion is represented by the symbol θ, which represents the feature information of the target video segment.

[0149] The initial video features c' are processed by feature mapping, that is, they can be processed by a fully connected layer fc to obtain the initial feature information φ; the specular video features c pos Feature mapping is performed to obtain the specular feature information φ pos For non-highlight video features c neg Feature mapping is performed to obtain non-spectral feature information φ neg ;The initial feature information φ and the specular feature information φ pos Non-highlight feature information φ neg The feature information θ of the target video segment is obtained by fusion, which is also the target parameter.

[0150] Specifically, the initial feature information φ and the specular feature information φ pos Non-highlight feature information φ negThe specific process of fusing the features to obtain the feature information θ can be as follows:

[0151] The initial feature information φ and the specular feature information φ pos The specular features θ are then fused together to obtain the fused specular feature information. pos The fusion method can be addition, for example, θ pos =φ+φ pos .

[0152] The initial feature information φ and the non-spectral feature information φ neg The fusion process is performed to obtain the fused non-spectral feature information θ. neg The fusion method can be addition, for example, θ neg =φ+φ neg .

[0153] The fused specular feature information θ pos and the fused non-spectral feature information θ neg The fusion process yields the feature information corresponding to the target video segment, i.e., the target parameter θ. The fusion method can be concatenation, for example, θ = [θ...]. pos ;θ neg ].

[0154] As can be understood from the embodiments of this application, the Decision Boundary Customizer (DBC) processes three types of data: initial video features, specular video features, and non-spectral video features. All three types of data pass through a non-shared fully connected layer to obtain the target parameter θ of the final mini-spectral classifier, i.e., the feature information of the target video segment. The mini-spectral classifier is an adaptive classifier.

[0155] This application embodiment uses a mini-highlight classifier to characterize user preferences. This mini-highlight classifier draws the decision boundary of highlight video frames between highlight and non-highlight video frames. Furthermore, to encourage the Decision Boundary Customizer (DBC) to generate the mini-highlight classifier, this application embodiment equips the DBC with a regularization module. The regularization module ensures that the mini-highlight classifier only focuses on the decision boundary of the highlight video frames in the video to be recommended, thereby preventing classifier degradation and inaccurate video recommendation results. It can be understood that the regularization module is equivalent to optimizing the parameters in the neural network layers.

[0156] 205. The computer equipment calculates the similarity between the target video segment and the video to be recommended based on the feature information of the target video segment and the feature information of the video to be recommended.

[0157] The decision boundary customizer learns the target parameters θ of the generated classifier to create a personalized mini-highlight classifier for the user, making the mini-highlight classifier adaptive. The feature information of this target video segment does not require any retraining; it can be obtained and stored after a single training iteration. Figure 9 As shown, the mini specular classifier uses the symbol T. θ It means that T θ1 T θ2 T θ2 ......T θn This represents different mini specular classifiers.

[0158] Among them, the feature information f of the recommended video in Features can be extracted through at least one feature extraction layer in a neural network, and can be obtained through methods such as... Figure 9 Features can be extracted from the feature encoder in the model, or they can be extracted from the visual geometric group network (VGG) or deep residual network (ResNet), which have better expressive power and greater depth.

[0159] It is understandable that, such as Figure 9 As shown, the mini specular classifier uses the feature information of the target video segment, namely the target parameter θ and the feature information f of the video to be recommended. in The similarity between the target video segment and the video to be recommended is calculated, and the similarity score is represented by s. Different scores between different target video segments and the video to be recommended are represented by different symbols, such as s1, s2, s3, ... s n .

[0160] In this application embodiment, the similarity between the target video segment and the video to be recommended can be obtained using a Gaussian kernel function based on the feature information of the target video segment and the feature information of the video to be recommended; the similarity between the target video segment and the video to be recommended can also be obtained using cosine similarity based on the feature information of the target video segment and the feature information of the video to be recommended; or the similarity between the target video segment and the video to be recommended can also be obtained using a Transformer neural network based on the feature information of the target video segment and the feature information of the video to be recommended.

[0161] The embodiments of this application can optimize the similarity. The specific process is as follows: calculate the initial similarity between the target video segment and the recommended video based on the initial feature information of the target video segment and the feature information of the video to be recommended; optimize the similarity based on the initial similarity to obtain the optimized similarity; select a reference video segment from at least one target video segment based on the optimized similarity.

[0162] As mentioned above, similarity is the target parameter θ obtained by the Decision Boundary Customizer (DBC), which is the feature information of the target video segment and the feature information f of the video to be recommended. in The initial feature information of the target video segment used for the initial similarity calculation is obtained through calculation. In this embodiment of the application, the initial feature information of the target video segment used for the initial similarity calculation is obtained from... Figure 9 The video features extracted by the feature encoder in the code, such as c1, c2...c n The initial similarity is calculated by comparing the initial feature information with the feature information f of the video to be recommended. in The calculations yielded the results.

[0163] Then, in this embodiment of the application, the initial similarity between the target video segment and the video to be recommended can be obtained using a Gaussian kernel function based on the initial feature information of the target video segment and the feature information of the video to be recommended; the initial similarity between the target video segment and the video to be recommended can also be obtained using cosine similarity based on the initial feature information of the target video segment and the feature information of the video to be recommended; or the initial similarity between the target video segment and the video to be recommended can also be obtained using a Transformer neural network based on the initial feature information of the target video segment and the feature information of the video to be recommended. The initial similarity can be represented by weight w, and different symbols are used to represent the initial similarity between different target video segments and the video to be recommended, for example, w1, w2, w3...w n .

[0164] This involves optimizing the initial similarity to obtain the optimized similarity. This can be achieved by multiplying the initial similarity by the current similarity, for example, multiplying w1 by s1, w2 by s2, and so on. n With s n Multiply.

[0165] 206. A computer device selects a reference video segment from at least one target video segment based on similarity.

[0166] For example, such as Figure 12As shown, the historical browsing videos include a first historical browsing video, a second historical browsing video, a third historical browsing video, and a fourth historical browsing video. The target video segment in the first historical browsing video is a food close-up video segment, the target video segment in the second historical browsing video is a forest walk video segment, the target video segment in the third historical browsing video is a singing video segment, and the target video segment in the fourth historical browsing video is a Big Ben video segment. The similarity between the food close-up video segment and the video to be recommended is the first similarity, the similarity between the forest walk video segment and the video to be recommended is the second similarity, the similarity between the singing video segment and the video to be recommended is the third similarity, and the similarity between the Big Ben video segment and the video to be recommended is the fourth similarity. Among them, the second similarity is ranked higher than the first, third, and fourth similarity; the higher the similarity, the more similar the target video segment is to the video to be recommended. In this embodiment, a reference video segment is preset, and the forest walk video segment corresponding to the second similarity is determined as the reference video segment.

[0167] 207. The computer device determines a target recommended video segment that is similar to a reference video segment from the video to be recommended, and recommends the target recommended video segment to the target object.

[0168] For example, as described above, in this embodiment of the application, the forest walk video segment corresponding to the second similarity is determined as the reference video segment. Then, based on the similarity between the forest walk video segment and the video to be recommended, the positions of video frames similar to the reference video segment in the video to be recommended are determined. These video frames are then segmented and extracted to generate the target recommended video segment. For example... Figure 12 As shown, the target recommended video clip is the forest walk video clip from the recommended videos.

[0169] Specifically, after recommending the target video clip to the target audience, when the target audience is a terminal, the user opens the terminal and the terminal can receive the target video clip; when the target audience is a logged-in account, the user logs in to the logged-in account and the logged-in account can receive the target video clip.

[0170] This application embodiment can obtain the historical browsing videos and videos to be recommended for a target object. The historical browsing videos include at least two video frames. Highlight video frame recognition is performed on the video frames in the historical browsing videos to obtain at least one target video segment, which includes highlight video frames and non-highlight video frames. Feature extraction is performed on the video frames within the target video segments to obtain feature information corresponding to each target video segment. Based on the feature information of the target video segments and the feature information of the videos to be recommended, the similarity between the target video segments and the videos to be recommended is calculated. A reference video segment is selected from the at least one target video segment based on the similarity. A target recommended video segment similar to the reference video segment is determined from the videos to be recommended, and the target recommended video segment is recommended to the target object. Since this application embodiment can obtain at least one target video segment including highlight video frames and non-highlight video frames based on highlight recognition in historical browsing videos, it fully utilizes the non-highlight video frames in the historical browsing videos. Furthermore, the obtained at least one target video segment can satisfy the user's need for at least one preference, thereby improving the accuracy of video recommendations.

[0171] For example, embodiments of this application employ the PACNet algorithm to select multiple target video segments from multiple historically viewed videos. Based on the feature information of the target video segments and the feature information of the videos to be recommended, the similarity between the target video segments and the videos to be recommended is calculated. Based on this similarity, three reference video segments are selected from the multiple target video segments. Based on the reference video segments, three target recommended video segments are selected from the videos to be recommended. Existing technologies employ a specular detection baseline algorithm to select three target recommended video segments from the videos to be recommended based on multiple historically viewed videos.

[0172] The recommendation results of the PACNet algorithm used in this application embodiment are compared with the recommendation results of the specular detection baseline algorithm in the prior art. Figure 13 As shown. The videos to be recommended include a first video to be recommended and a second video to be recommended, from... Figure 13 As can be seen, based on the first video to be recommended, this embodiment of the application successfully extracted a close-up video clip of a firefly; based on the second video to be recommended, this embodiment of the application successfully extracted a video clip of cake decoration, while the general highlight detection baseline algorithm cannot achieve this effect.

[0173] To facilitate better implementation of the video recommendation method provided in the embodiments of this application, the embodiments of this application also provide a video recommendation device based on the above-mentioned video recommendation method. The video recommendation device can be integrated into a computer device, such as a terminal, which may include a tablet computer, a laptop computer, and / or a personal computer.

[0174] For example, such as Figure 14As shown, the video recommendation device may include an acquisition unit 301, a recognition unit 302, an extraction unit 303, a calculation unit 304, a selection unit 305, and a recommendation unit 306, as follows:

[0175] (1) Obtain unit 301;

[0176] The acquisition unit 301 can be used to acquire the historical browsing videos and videos to be recommended for the target object. The historical browsing videos include at least two video frames.

[0177] (2) Identification unit 302;

[0178] The identification unit 302 can be used to identify highlight video frames in historically viewed videos to obtain at least one target video segment, which includes highlight video frames and non-highlight video frames.

[0179] In some embodiments, the identification unit may be used to segment historically viewed videos to obtain candidate video segments; and to identify highlight video frames by identifying the tags of video frames in the candidate video segments to obtain at least one target video segment.

[0180] (3) Extraction unit 303;

[0181] The extraction unit 303 can be used to extract features from video frames within a target video segment to obtain feature information corresponding to each target video segment.

[0182] In some embodiments, the extraction unit 303 can be used to extract features from video frames within a target video segment to obtain video features; and to perform feature mapping processing on the video features to generate feature information corresponding to the target video segment.

[0183] In some embodiments, the extraction unit 303 may be specifically used to extract video segment features from the target video segment to obtain initial video features; extract video frame features from the highlight video frames in the target video segment to obtain highlight video features; extract video frame features from the non-highlight video frames in the target video segment to obtain non-highlight video features; and generate video features based on the initial video features, highlight video features, and non-highlight video features.

[0184] In some embodiments, the extraction unit 303 may be specifically used to perform feature mapping processing on the initial video features to obtain initial feature information; perform feature mapping processing on the highlight video features to obtain highlight feature information; perform feature mapping processing on the non-highlight video features to obtain non-highlight feature information; and fuse the initial feature information, highlight feature information and non-highlight feature information to obtain feature information corresponding to the target video segment.

[0185] In some embodiments, the extraction unit 303 may be specifically used to fuse the initial feature information and the highlight feature information to obtain the fused highlight feature information; fuse the initial feature information and the non-highlight feature information to obtain the fused non-highlight feature information; and fuse the fused highlight feature information and the fused non-highlight feature information to obtain the feature information corresponding to the target video segment.

[0186] (4) Calculation unit 304;

[0187] The calculation unit 304 can be used to calculate the similarity between the target video segment and the video to be recommended based on the feature information of the target video segment and the feature information of the video to be recommended.

[0188] (5) Select unit 305;

[0189] Selection unit 305 is used to select a reference video segment from at least one target video segment based on similarity.

[0190] In some embodiments, the selection unit 305 may be specifically used to calculate the initial similarity between the target video segment and the video to be recommended based on the feature information of the target video segment and the feature information of the video to be recommended; optimize the similarity based on the initial similarity to obtain the optimized similarity; and select a reference video segment from at least one target video segment based on the optimized similarity.

[0191] (6) Recommended Unit 306;

[0192] Recommendation unit 306 can be used to determine target recommended video segments that are similar to reference video segments from the video to be recommended, and recommend the target recommended video segments to the target object.

[0193] The acquisition unit 301 of this application embodiment can acquire the historical browsing videos and videos to be recommended for the target object. The historical browsing videos include at least two video frames. The identification unit 302 can perform highlight video frame recognition on the video frames in the historical browsing videos to obtain at least one target video segment. The target video segment includes highlight video frames and non-highlight video frames. The extraction unit 303 can perform feature extraction on the video frames in the target video segment to obtain feature information corresponding to each target video segment. The calculation unit 304 can calculate the similarity between the target video segment and the video to be recommended based on the feature information of the target video segment and the feature information of the video to be recommended. The selection unit 305 can select a reference video segment from at least one target video segment based on the similarity. The recommendation unit 306 can determine a target recommended video segment similar to the reference video segment from the video to be recommended and recommend the target recommended video segment to the target object. Since this application embodiment can obtain at least one target video segment including highlight video frames and non-highlight video frames based on highlight recognition on the historical browsing videos, it makes full use of the non-highlight video frames in the historical browsing videos. On the other hand, the obtained at least one target video segment can meet the user's need for at least one preference, thereby improving the accuracy of video recommendation.

[0194] This application also provides a computer device, which may be a server, such as... Figure 15 As shown, it illustrates a schematic diagram of the server structure involved in an embodiment of this application. Specifically:

[0195] The server may include components such as a processor 401 with one or more processing cores, a memory 402 with one or more computer-readable storage media, a power supply 403, and an input unit 404. Those skilled in the art will understand that... Figure 15 The server structure shown does not constitute a limitation on the server and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0196] in:

[0197] The processor 401 is the control center of the server, connecting various parts of the server through various interfaces and lines. It performs various server functions and processes data by running or executing software programs and / or modules stored in the memory 402, and by calling data stored in the memory 402, thereby enabling overall monitoring of the server. Optionally, the processor 401 may include one or more processing cores; preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly handles the operating system, user interface, and applications, and the modem processor mainly handles wireless communication. It is understood that the modem processor may not be integrated into the processor 401.

[0198] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback function, image playback function, etc.), etc.; the data storage area may store data created according to the use of the server, etc. In addition, the memory 402 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0199] The server also includes a power supply 403 that supplies power to the various components. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. The power supply 403 may also include one or more DC or AC power supplies, recharging systems, power fault detection circuits, power converters or inverters, power status indicators, and other arbitrary components.

[0200] The server may also include an input unit 404, which can be used to receive input digital or character information communication, and to generate keyboard, mouse, joystick, optical or trackball signal inputs related to user settings and function control.

[0201] Although not shown, the server may also include a display unit, etc., which will not be described in detail here. Specifically, in this embodiment, the processor 401 in the server loads the executable files corresponding to the processes of one or more applications into the memory 402 according to the following instructions, and the processor 401 runs the applications stored in the memory 402 to realize various functions, as follows:

[0202] The process involves: acquiring the target object's historical browsing videos and videos to be recommended, with each historical browsing video consisting of at least two video frames; identifying highlight video frames in the historical browsing videos to obtain at least one target video segment, which includes highlight and non-highlight video frames; extracting features from the video frames within the target video segment to obtain feature information corresponding to each target video segment; calculating the similarity between the target video segment and the video to be recommended based on the feature information of the target video segment and the video to be recommended; selecting a reference video segment from the at least one target video segment based on the similarity; determining target recommended video segments similar to the reference video segment from the video to be recommended, and recommending the target recommended video segment to the target object.

[0203] The specific implementation of each of the above operations can be found in the preceding embodiments, and will not be repeated here. It should be noted that the computer device provided in this application embodiment and the video recommendation method in the above embodiments belong to the same concept, and its specific implementation process can be found in the above method embodiments, and will not be repeated here.

[0204] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be performed by instructions, or by instructions controlling related hardware. These instructions can be stored in a computer-readable storage medium and loaded and executed by a processor.

[0205] Therefore, embodiments of this application provide a storage medium storing a computer program adapted for loading by a processor to execute the steps of any of the video recommendation methods provided in embodiments of this application. For example, the instructions can execute the following steps:

[0206] The process involves: acquiring the target object's historical browsing videos and videos to be recommended, with each historical browsing video consisting of at least two video frames; identifying highlight video frames in the historical browsing videos to obtain at least one target video segment, which includes highlight and non-highlight video frames; extracting features from the video frames within the target video segment to obtain feature information corresponding to each target video segment; calculating the similarity between the target video segment and the video to be recommended based on the feature information of the target video segment and the video to be recommended; selecting a reference video segment from the at least one target video segment based on the similarity; determining target recommended video segments similar to the reference video segment from the video to be recommended, and recommending the target recommended video segment to the target object.

[0207] According to one aspect of this application, a computer program product or computer program is provided, comprising a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to perform the methods provided in the various optional implementations of the above embodiments.

[0208] For details on the implementation of each of the above operations, please refer to the previous examples, which will not be repeated here.

[0209] The storage medium may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0210] Since the program stored in the storage medium can execute the steps of any of the video recommendation methods provided in the embodiments of this application, it can achieve the beneficial effects that any of the video recommendation methods provided in the embodiments of this application can achieve, as detailed in the preceding embodiments, and will not be repeated here.

[0211] The foregoing has provided a detailed description of a video recommendation method, apparatus, computer device, and storage medium provided in the embodiments of this application. Specific examples have been used to illustrate the principles and implementation methods of the embodiments of this application. The descriptions of the embodiments above are only for the purpose of helping to understand the methods and core ideas of the embodiments of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the embodiments of this application. Therefore, the content of this specification should not be construed as a limitation on the embodiments of this application.

Claims

1. A video recommendation method, characterized by, The method comprises the following steps: obtaining a historical browsing video and a to-be-recommended video of a target object, the historical browsing video comprising at least two video frames; performing highlight video frame identification on the video frames in the historical browsing video to obtain at least one target video segment, the target video segment comprising a highlight video frame and a non-highlight video frame; performing feature extraction on the video frames in the target video segment to obtain feature information corresponding to each target video segment; calculating the similarity between the target video segment and the to-be-recommended video according to the feature information of the target video segment and the feature information of the to-be-recommended video; selecting a reference video segment from the at least one target video segment according to the similarity; determining a target recommended video segment similar to the reference video segment from the to-be-recommended video, and recommending the target recommended video segment to the target object; wherein the feature extraction on the video frames in the target video segment comprises: performing video segment feature extraction on the target video segment to obtain initial video features, performing video frame feature extraction on the highlight video frame in the target video segment to obtain highlight video features, performing video frame feature extraction on the non-highlight video frame in the target video segment to obtain non-highlight video features, and generating video features according to the initial video features, the highlight video features, and the non-highlight video features; performing feature mapping processing on the video features to generate the feature information corresponding to the target video segment; the highlight video frame identification on the video frames in the historical browsing video comprises: when the viewing times of a video frame are greater than or equal to a set threshold, determining that the video frame is a highlight video frame; when the viewing times of a video frame are less than a set threshold, determining that the video frame is a non-highlight video frame; the selection of the reference video segment from the at least one target video segment according to the similarity comprises: calculating the initial similarity between the target video segment and the to-be-recommended video according to the initial feature information of the target video segment and the feature information of the to-be-recommended video; optimizing the similarity according to the initial similarity to obtain an optimized similarity; selecting the reference video segment from the at least one target video segment according to the optimized similarity.

2. The video recommendation method of claim 1, wherein, The video features comprise the initial video features, the highlight video features, and the non-highlight video features; the feature mapping processing on the video features to generate the feature information corresponding to the target video segment comprises: performing feature mapping processing on the initial video features to obtain initial feature information; performing feature mapping processing on the highlight video features to obtain highlight feature information; performing feature mapping processing on the non-highlight video features to obtain non-highlight feature information; fusing the initial feature information, the highlight feature information, and the non-highlight feature information to obtain the feature information corresponding to the target video segment.

3. The video recommendation method of claim 2, wherein, the fusion of the initial feature information, the highlight feature information, and the non-highlight feature information to obtain the feature information corresponding to the target video segment comprises: fuse the initial feature information and the highlight feature information to obtain fused highlight feature information; fuse the initial feature information and the non-highlight feature information to obtain fused non-highlight feature information; fuse the fused highlight feature information and the fused non-highlight feature information to obtain feature information corresponding to the target video segment.

4. The video recommendation method of claim 1, wherein, The highlight video frame identification on the video frames in the historical browsing video obtains at least one target video segment, and the method comprises the steps of: splitting the historical browsing video to obtain a candidate video segment; performing highlight video frame identification on the labels of the video frames in the candidate video segment to obtain at least one target video segment.

5. A video recommendation apparatus, characterized by comprising: comprise: an acquisition unit configured to acquire a historical browsing video of a target object and a video to be recommended, the historical browsing video comprising at least two video frames; an identification unit configured to perform highlight video frame identification on the video frames in the historical browsing video to obtain at least one target video segment, the target video segment comprising a highlight video frame and a non-highlight video frame; the identification unit is configured to: when the number of views of a video frame is greater than or equal to a set threshold, determine that the video frame is a highlight video frame; and when the number of views of a video frame is less than the set threshold, determine that the video frame is a non-highlight video frame; an extraction unit configured to perform feature extraction on the video frames in the target video segment to obtain feature information corresponding to each target video segment; the extraction unit is specifically configured to perform video segment feature extraction on the target video segment to obtain initial video features; perform video frame feature extraction on the highlight video frames in the target video segment to obtain highlight video features; perform video frame feature extraction on the non-highlight video frames in the target video segment to obtain non-highlight video features; and generate video features according to the initial video features, the highlight video features, and the non-highlight video features; perform feature mapping processing on the video features to generate feature information corresponding to the target video segment; a calculation unit configured to calculate a similarity between the target video segment and the video to be recommended according to the feature information of the target video segment and the feature information of the video to be recommended; a selection unit configured to select a reference video segment from at least one target video segment according to the similarity; a recommendation unit configured to determine a target recommended video segment similar to the reference video segment from the video to be recommended, and recommend the target recommended video segment to the target object; the selection unit is configured to calculate an initial similarity between the target video segment and the video to be recommended according to initial feature information of the target video segment and feature information of the video to be recommended; optimize the similarity according to the initial similarity to obtain an optimized similarity; select a reference video segment from at least one target video segment according to the optimized similarity.

6. A computer device, comprising: comprise a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the video recommendation method in any one of claims 1 to 4. comprise a memory and a processor; the memory stores a computer program, and the processor is used to run the computer program in the memory to execute the video recommendation method in any one of claims 1 to 4.

7. A storage medium, characterized by The storage medium stores a computer program, and the computer program is suitable for being loaded by the processor to execute the video recommendation method in any one of claims 1 to 4.

Images