Video recommendation method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides a video recommendation method and device, and relates to the technical field of computers, in particular to the technical field of intelligent recommendation. According to the specific implementation scheme, at least one first video clip corresponding to at least one user recommendation behavior of a user is acquired; the user recommendation behaviors comprise at least one of the following behaviors: liking, forwarding, commenting, barrage and collection; acquiring a second video matched with the at least one user recommendation behavior and the at least one first video clip; the second video is from a to-be-recommended video; recommending the second video to the user; the video clip expressed by the user emotion is accurately recognized, the video content which the user is really interested in is efficiently focused, and the user experience of video recommendation is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computers, specifically to the field of intelligent recommendation technology, and in particular to a video recommendation method and device, an electronic device, a computer-readable storage medium, and a computer program product. Background Art

[0002] In the current video recommendation technology field, especially in the recommendation system of short video platforms, although there are many methods to try to improve the accuracy of recommendations and user satisfaction, there are generally problems such as large recommendation granularity and inability to accurately focus on the segments that users are really interested in. Existing technologies mainly rely on interest tags, user portraits, and multimodal information to describe and match users with video content. Although these methods have improved the personalization of recommendations to a certain extent, they often ignore the differences in the attractiveness of different segments within the video to users, resulting in low recommendation efficiency and the need to improve user experience. Summary of the invention

[0003] The present disclosure provides a video recommendation method and apparatus, an electronic device, a computer-readable storage medium, and a computer program product.

[0004] According to a first aspect, a video recommendation method is provided, the method comprising: obtaining at least one first video clip corresponding to at least one user recommendation behavior of a user; the user recommendation behavior comprising at least one of the following: like, forward, comment, barrage, favorite; obtaining a second video matching the at least one user recommendation behavior and the at least one first video clip; the second video comes from the video to be recommended; and recommending the second video to the user.

[0005] According to a second aspect, a video recommendation device is provided, which includes: a first acquisition unit, which acquires at least one first video clip corresponding to at least one user recommendation behavior of a user; the user recommendation behavior includes at least one of the following: like, forward, comment, barrage, and collection; a second acquisition unit, which acquires a second video matching the at least one user recommendation behavior and the at least one first video clip; the second video comes from the video to be recommended; and a recommendation unit, which recommends the second video to the user.

[0006] According to a third aspect, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor, wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the method described in any implementation manner of the first aspect.

[0007] According to a fourth aspect, a non-transitory computer-readable storage medium storing computer instructions is provided, where the computer instructions are used to cause a computer to execute the method described in any implementation of the first aspect.

[0008] According to a fifth aspect, a computer program product is provided, comprising a computer program, which implements the method described in any implementation manner of the first aspect when executed by a processor.

[0009] The video recommendation method and device provided by the embodiments of the present disclosure, firstly, improve the accuracy of recommendation. The present disclosure can realize a finer granularity division and understanding of video content, so as to accurately locate the segments that users are really interested in for recommendation. This improvement solves the problem that the recommendation granularity is large and the user's interest points cannot be accurately focused in the prior art, and improves the accuracy and personalization level of recommendation. Secondly, the user experience is optimized. By learning the patterns contained in the user's multiple recommendation behaviors, it is possible to recommend videos or video segments that are more in line with the user's personalized needs. Users will encounter less uninteresting content during browsing or viewing, and reduce the time waste and emotional exhaustion caused by frequent switching or skipping of uninteresting content. This not only improves the user's viewing efficiency and satisfaction, but also enhances the user's stickiness and loyalty to the platform. Thirdly, the recommendation efficiency is improved. Compared with the traditional strategy of recommendation based on the overall video, the present disclosure focuses on the recommendation of video segments, which effectively avoids the waste of recommendation resources caused by the partial matching of the overall video with the user's interest. Finally, content distribution is promoted and authors are encouraged to create. More fine-grained recommendations can more quickly capture the user's immediate interest changes. By learning the changing patterns of users' multiple recommendation behaviors in real time, the recommendation strategy can be adjusted in a timely manner, further improving the response speed and efficiency of the recommendation system. After recommending users' favorite videos or video clips to users, it will further stimulate users' enthusiasm for likes, forwarding, comments, and bullet comments. This will also encourage creators to produce more high-quality and attractive content clips.

[0010] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0012] Figure 1 is a schematic diagram of a video recommendation scenario provided by an embodiment of the present disclosure;

[0013] Figure 2 is a flowchart of a video recommendation method provided by an embodiment of the present disclosure;

[0014] Figure 3 is a flow chart of a video clip acquisition step provided by an embodiment of the present disclosure;

[0015] Figure 4 is a schematic diagram of a video clip acquisition scenario provided by an embodiment of the present disclosure;

[0016] Figure 5 is a schematic diagram of another video clip acquisition scenario provided by an embodiment of the present disclosure;

[0017] Figure 6 is a schematic diagram of a scene for determining the start time of a video clip provided by an embodiment of the present disclosure;

[0018] Figure 7 is a schematic diagram of another scenario for determining the start time of a video clip provided by an embodiment of the present disclosure;

[0019] Figure 8 is a flowchart of a recommended video acquisition step provided by an embodiment of the present disclosure;

[0020] Fig. 9 is a structural diagram of a video recommendation device provided by an embodiment of the present disclosure;

[0021] Fig.10 The block diagram is a block diagram of an electronic device for implementing the video recommendation method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0022] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0023] A specific problem faced by current video recommendation systems, especially in short video platforms, is that the granularity of recommended content is too large, and it is impossible to accurately capture and recommend video clips that users are really interested in. Although recommendation systems have used advanced technologies such as interest tags, music portraits, and multimodal information fusion to enhance the personalization of recommendations, most of these methods focus on the overall description and matching of the entire video content, while ignoring the significant differences in user appeal between individual clips within the video. This extensive recommendation method is likely to push complete videos that users are not interested in or are only partially interested in to users, thereby reducing the effectiveness of the recommendation and the user's viewing experience. Therefore, how to achieve fine-grained analysis of video content and accurately identify and recommend the clips that users are most interested in has become a key issue that needs to be urgently solved in current recommendation systems.

[0024] In order to solve the technical problem of how to recommend videos based on specific clips in the overall video that resonate with users, the present disclosure proposes a video recommendation method.

[0025] It is understandable that the method can be integrated into various computing devices, including but not limited to terminal devices with text processing capabilities (such as smart phones, tablets, personal computers, etc.) and servers (such as application servers, Web servers, whether single servers or cluster-deployed server systems). The method does not rely on a specific hardware platform or software architecture. During the execution of the method, both independently running terminal devices and terminal-server architectures that work collaboratively through a network can effectively utilize the method disclosed in the present invention. When the terminal device is executed independently, the method can be independent of the external network. In scenarios where higher processing performance or wider data resource support is required, the method of the present invention also supports communication between the terminal device and the server, utilizing the powerful computing power and rich data resources of the server to jointly complete the method. The method can adapt to different operating systems and platform environments, including mobile operating systems such as iOS, Android, and Hongmeng, desktop operating systems such as Windows and macOS, and server operating systems such as Linux and Unix.

[0026] It can be understood that the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solution of the present disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0027] Figure 1 The following is a video recommendation process for facilitating understanding of the technical solution provided in the embodiment of the present application. In this actual application scenario, there is no distinction between a server providing client access, a server providing video playback, and a server providing video recommendation, and the term server is used uniformly.

[0028] In a typical video recommendation application scenario, users access a server with video recommendation capabilities through a client application on their smart devices (such as mobile phones, tablets, or computers). The user first starts the client and connects to the Internet to access the video content service. Once the connection is successful, the user begins to browse the video list or search for specific video content on the client interface. At this point, the server not only serves as a storage and distribution center for video content, but also collects and analyzes user behavior information. The server captures the user's interest preferences and viewing habits in real time or periodically through the user's interaction with the client, such as clicks, viewing time, skips, likes, comments, and other behaviors.

[0029] At the same time, the server is responsible for efficiently transmitting the video stream to the user client according to the user's request to ensure smooth playback of the video content. During this process, the video playback server will also record the user's playback information, such as viewing progress, network status, etc., which can be used to subsequently optimize the recommendation algorithm and improve the user experience. When the user finishes watching a video or actively triggers a recommendation request, the server will also record the above information. The information recorded or obtained by the server can be integrated with the user's historical viewing history, interest tags, behavior patterns, and current contextual information (such as time, location, social relationships, etc.), and analyzed using the recommendation algorithm model to generate a personalized video recommendation list.

[0030] Finally, these recommendation results will be immediately returned to the user's client and presented to the user in various forms such as cards, lists or video previews. Users can choose to continue watching based on the recommended content, or further explore other videos of interest.

[0031] like Figure 1 In the exemplary application scenario shown, it is assumed that there is video 1 in the user's viewing history, in which the user likes video 1 when the video screen "smoke screen" and the background music "country music" are played. The user is watching video 2, in which the user forwards video 2 after the video screen "fire screen" and the background music "country music" are played.

[0032] Based on the user's like behavior and like time record for video 1, and the forwarding behavior and forwarding time record for video 2, the server obtains two video clips related to the user's recommendation behavior and the playback time of the video clips based on the method disclosed in the present invention. The video screen highlight analysis and background music climax clip analysis (or video interval division) of video 1 before the like moment are performed to obtain a video clip referred to as "cooking smoke picture country music". The video screen highlight analysis and background music climax clip analysis (or video interval division) of video 2 before the forwarding moment are performed to obtain a video clip referred to as "fire picture country music".

[0033] The server performs recommendation index analysis based on the recent video clip "fire scene country music", and extracts recommendation index 1 (such as the "country music" tag) and recommendation index 2 (such as the "fire scene" tag).

[0034] Considering that the video clip “Fireside Country Music” was played recently and the user recommendation behavior is “forward” (“forward” means a higher recommendation degree than “like”), the indicator weights of the two recommendation indicators in the video clip “Fireside Country Music” are set to a relatively high 90.

[0035] Table 1 Video clips

[0036] Video clips Recommended indicators Indicator weight "Heatfire Country Music" #1 "Country Music" Tag 90 "Heatfire Country Music" #2"Fire Image" tag 90 "Smoke Scene Country Music" #1 "Country Music" Tag 50 "Smoke Scene Country Music" #3"Smoke from cooking" tag 50

[0037] Similarly, the server also performs recommendation index analysis on the previous video clip "smoke from cooking, country music", and extracts recommendation index 1 (such as the "country music" tag) and recommendation index 3 (such as the "fire picture" tag).

[0038] Considering that the video clip "Smoke from the Kitchen, Country Music" was played earlier and the user recommendation behavior was "Like" ("Forward" means a higher recommendation than "Like"), the indicator weights of the two recommendation indicators in the video clip "Smoke from the Kitchen, Country Music" are set to a relatively low 50. The video clip table after setting is shown in Table 1.

[0039] The server extracts the relevant recommendation index and relevance for the most recent video clip "fire scene country music" and the previous video clip "smoke scene country music". As can be seen from the above table, both video clips have a recommendation index "country music" label, assuming that the relevance of the two country music is 0.9 (assuming that the relevance value ranges from 0 to 1).

[0040] When the recommendation indicators are merged, the recommendation weight of the "country music" label will become: 90+0.9×50=135. The updated recommendation indicator table is shown in Table 2.

[0041] Table 2 Recommended indicators

[0042] Recommended indicators Indicator weight Indicator weight #1 "Country Music" Tag #1 "Country Music" Tag 135 #2"Fire Image" tag #2"Fire Image" tag 90 #3"Smoke from cooking" tag #1 "Country Music" Tag 50

[0043] The server matches video 3 in the recommended video library that the user has not watched yet. According to the recommendation index, feature extraction is performed on video 3 to obtain the "country music" label and the "field scene" label. Assuming that there is also video 4 (not shown in the figure), feature extraction obtains the "Chinese style music" label and the "fire scene".

[0044] The server ranks video 3 and video 4 for recommendation to the user. The index weight of the "country music" tag hit by video 3 is 135, while the index weight of the "fireplace image" tag hit by video 4 is 90. Assuming that the above two tags are equally important to the user, but the index weight value of video 3 is larger, the weighted score of video 3 will be higher than that of video 4.

[0045] In other words, the server's recommendation index system, through intelligent analysis of users' past recommendation behavior records, successfully captured the user's emotional accumulation process towards "country music" while watching, thereby achieving the purpose of accurate recommendation.

[0046] Furthermore, if the specific video segment position of the "country music" tag in video 3 can also be located, the server can directly recommend the user to jump to the video segment position of video 3 (such as Figure 1 At this time, during the viewing process, the user can directly jump from the "fire scene country music" video clip of video 2 to the "field scene country music" video clip of recommended video 3, thereby achieving the purpose of more accurate recommendation.

[0047] In short, in the current video recommendation system, although the video content itself contains rich and multi-dimensional semantic information, traditional technologies often find it difficult to accurately capture these subtle differences, resulting in recommendation results that tend to be broad and general. Specifically, they tend to evaluate the entire video as a single unit, ignoring the different emotions, themes, or points of interest that may be carried by different clips within the video. This "one-size-fits-all" approach limits the accuracy and personalization of the recommendation system. At the same time, faced with the numerous video records accumulated in the user's viewing history, existing technologies lack effective means to deeply analyze and integrate the accumulated emotional changes generated during the viewing of these videos, making it difficult to accurately depict the user's dynamic interest preferences.

[0048] In order to overcome the above limitations, the present disclosure proposes an innovative video recommendation method, the core of which lies in the refined processing of video content and the in-depth mining of user viewing emotions. On the one hand, through advanced content analysis technology, this method can accurately locate and extract key clips in the video, which often carry the core information or specific emotions of the video, thereby significantly reducing the recommendation granularity, making the recommended content closer to the user's immediate interests and needs, and effectively reducing the interference and processing costs of invalid information. On the other hand, in response to user viewing behavior, this method designs a set of efficient cumulative viewing emotion extraction and accumulation mechanisms, which can capture and analyze the accumulated emotions generated by users during the viewing process in the form of "indicator weights of recommendation indicators", thereby constructing a more accurate and efficient video recommendation method.

[0049] Figure 2 A process 100 of an embodiment of a video recommendation method according to the present disclosure is shown. The video recommendation method comprises the following steps:

[0050] Step 101, obtaining at least one first video clip corresponding to at least one user recommendation behavior of a user; the user recommendation behavior includes at least one of the following: like, forward, comment, bullet screen, and collection.

[0051] Step 101 includes at least two core purposes, namely, obtaining the user's recommended behavior and obtaining the first video segment corresponding to the recommended behavior.

[0052] First, for the core purpose of obtaining user recommendation behavior, the video playback system will obtain video playback records, as well as user recommendation behaviors such as likes, forwarding, comments, bullet screens, and collections during the video playback process.

[0053] The first way to use the above information is to store it in the form of a data table or other data records. First, the user table is used to store the basic information of the user, including the user's unique identifier and other user attribute fields. Secondly, the video table is used to store the basic information of the video, including the video attribute fields such as the video's unique identifier, and also contains key information such as the author, title, and upload time of the video. Thirdly, the video play table is used to record the details of the video playback, including the playback attribute fields such as the unique identifier of the playback record, the unique identifier of the associated user and the unique identifier of the video, and also contains fields such as the playback start time and the playback end time. Finally, in order to store user recommendation behaviors such as likes, forwarding, comments, barrages, and collections, a flexible data table design can be adopted. One strategy is to design a unified behavior record table, which contains common fields such as behavior unique identifier, user unique identifier, video unique identifier, behavior type (such as likes, forwarding, comments, etc.), behavior time, and an optional additional information field for storing additional data required for specific behaviors (such as comment content, barrage text, etc.). This design facilitates centralized management and query of user behavior data. Another strategy is to design independent data tables for each behavior type, such as the likes table, forwarding table, comment table, barrage table, and collection table. Each table contains fields specific to the behavior, such as the like time in the likes table, the comment content in the comment table, etc.

[0054] The second way to use the above information is that the system can choose not to store it, but directly use the user's playback records, user recommendation behavior records and other data for the calculation of the present disclosure. This method can reduce unnecessary database write operations. This method can be used in all embodiments of the present disclosure.

[0055] The beneficial effect of the above steps is that it fully captures the diverse interactions of users during video playback, including viewing behaviors and rich recommendation behaviors such as likes, forwarding, comments, barrage, and collections. The collection of this comprehensive data helps to deeply analyze users' interests and preferences. Based on this, the system can build a more accurate and personalized video recommendation strategy to ensure that each user can receive content push tailored to their needs and even resonate with them, thereby improving the sophistication and satisfaction of the user experience.

[0056] Second, for the second core purpose, obtain the first video clip corresponding to the recommended behavior. In the video viewing process, the user's recommendation behavior, such as likes, forwarding, positive comments, barrage and collection, is a direct reflection of the user's emotions and interest preferences. These behaviors often occur immediately after the user encounters the interest point or emotional resonance point in the video content.

[0057] Take barrage as an example. When a user watches a video clip with a beautiful background music, which is deeply integrated with the emotions in the video, triggering the user's emotional resonance, the user may quickly send a barrage as a positive evaluation and recognition of the music and emotional expression in the video clip. This process not only demonstrates the user's instant feedback mechanism for video content, but also reveals the key role of background music in enhancing the user's viewing experience and promoting emotional resonance. The real-time interactive nature of barrage allows users to instantly share their feelings and discoveries during the viewing process.

[0058] As for forwarding, positive comments, likes, and collections, although they may occur after the video is played, they may also occur during the video playback. Users may comment, forward, like, or collect a wonderful scene, a thought-provoking subtitle, or an engaging voice narration in the video. Therefore, the recommended behaviors triggered by users during video playback, such as likes, forwarding, comments, sending barrages, and collections, deeply reflect the user's immediate interest in the video content and the urgent desire to share, rather than just waiting for feedback after the video is played.

[0059] Optionally, the emotional color of the comments is determined. The purpose of the determination is to capture the user's emotional attitude towards the video content and to screen out positive comments that can express the user's recommendation attitude and even emotional resonance. A common and effective method is to determine based on the emotional dictionary. By constructing a detailed emotional dictionary, the words in the comment text can be matched with the emotional tendencies in the dictionary. In addition, the emotional color determination methods based on machine learning models are becoming increasingly mature. These methods use labeled comment data sets to train the model so that it can automatically learn and identify the emotional features in the comments.

[0060] In the process of video recommendation, user recommendation behaviors such as forwarding, commenting, barrage and collection constitute the key data foundation. These behaviors are not limited to background music, but also widely involve multiple dimensions such as video images, subtitles and voice. By analyzing these data, the system can understand which video clips touch users in terms of background music, vision, text or hearing, and then highlight these elements in the recommendation strategy to improve the user experience of watching videos.

[0061] The beneficial effect of the above steps is that by deeply analyzing the user's instant recommendation behavior during video playback, the user's emotions and interest preferences are accurately captured. The above steps not only significantly improve the personalization of the video recommendation system, but also greatly enhance the user's viewing experience and satisfaction. By comprehensively considering multiple dimensions of video content, including background music, pictures, subtitles and voice, it is possible to comprehensively evaluate the factors that determine the user's preference for video clips. The above steps help to more accurately obtain user interests and emotional resonance content, optimize the recommendation algorithm, and thus recommend more personalized and high-quality video content to users.

[0062] Step 102: Obtain a second video that matches the at least one user recommendation behavior and the at least one first video clip; the second video comes from the video to be recommended.

[0063] Step 102 includes at least three core purposes: obtaining features of at least one first video segment, obtaining a relationship between at least one first video segment, and calculating a recommendation index for recalling a second video based on the relationship.

[0064] First, for core purpose one, obtain the features of at least one first video clip. Obtaining the features of at least one first video clip is the basis of step 102. Only by efficiently and accurately extracting key information in the video clip, such as content theme, emotional tendency, visual elements, etc., can we accurately identify the video clips expressing user emotions. This requires the use of advanced video analysis technologies, such as image recognition, natural language processing, and deep learning algorithms to accurately parse the video content. The technical effect is that through precise feature extraction, we can better understand the essence of each video clip and provide a solid foundation for subsequent video matching.

[0065] Second, for the second core purpose, obtain the relationship between at least one first video clip. Obtaining the relationship between at least one first video clip is the key to building the recommendation logic. The technical problem here is how to identify the intrinsic connection between video clips, such as whether they belong to the same series, whether they tell the same or related storylines, whether they have similar visual styles or emotional expressions, etc. This requires the system to have strong data association and pattern recognition capabilities, and to be able to mine potential association rules from massive video data. In terms of technical effects, accurately obtaining the relationship between the first video clips can significantly improve the personalization and relevance of video recommendations. When a user shows interest in a specific video clip, the system can predict other content that the user may be interested in based on the intrinsic connections between these clips, thereby recommending a second video that is more in line with the user's preferences. This kind of content-based in-depth understanding and association analysis not only enhances the user experience, but also improves the overall effectiveness and satisfaction of the recommendation system.

[0066] Third, for the third core purpose, the recommendation index for recalling the second video is calculated based on the above relationship. The user's historical behavior, current interests and the correlation between video clips are comprehensively considered to calculate the recommendation index that best meets the user's needs. The key to the technical problem lies in how to design a scientific and reasonable recommendation algorithm to convert the relationship between the video clips that the user has watched continuously into a quantifiable recommendation index, reflecting the accumulation of user emotions and the continuous focus of user interest areas. Collaborative filtering, content-based recommendation or hybrid recommendation strategies can be adopted. The recommendation index is continuously calculated and accumulated by mining the similarity between users or the similarity between videos. When calculating the recommendation index, technical means such as weight adjustment and normalization are introduced to ensure the fairness and accuracy of the recommendation results. In terms of technical effects, by scientifically calculating the recommendation index and recalling the second video, the system can provide users with more accurate and personalized recommendation services. This can not only improve user satisfaction and loyalty, but also promote the effective dissemination and distribution of video content, bringing greater commercial value to content creators and platforms. At the same time, with the continuous optimization and iteration of the recommendation algorithm, the system can continuously learn the changes in user preferences and achieve more dynamic and intelligent recommendation services.

[0067] Step 103: recommend the second video to the user.

[0068] Recommending the carefully selected second video to users is the final step in the entire recommendation process.

[0069] This step can enhance the user experience in a variety of ways. When a video or video clip is identified and recommended to a user, it can be displayed in the form of thumbnails, short video previews, or dynamic posters, so that users can understand the content of the clip at a glance, thereby stimulating their desire to click. For example, using smart prompts and pop-ups, relevant videos or clips can be instantly recommended when users browse or search, reducing the user's active search costs. In addition, social functions can also be used to support users to share the results of video recommendations on social platforms. For example, buttons such as "Share with friends" can be set on the recommendation interface.

[0070] Figure 3 Step 101 of the video recommendation method according to one embodiment of the present disclosure is shown. Step 101 may include the following steps:

[0071] Step 1011, obtaining a first video corresponding to the user recommended behavior;

[0072] Step 1012, obtaining the time when the user's recommended behavior occurs; the time when the behavior occurs is before the end time when the user watches the first video;

[0073] Step 1013, according to the first video feature of the first video, obtain the start time and end time of the first video segment corresponding to the user recommended behavior; the first video feature includes at least one of the following: video background music feature, video screen feature, video subtitle feature, video voice feature; the start time of the first video segment is before the time when the behavior occurs;

[0074] Step 1014: Obtain a first video segment from the first video according to the start time of the first video segment and the end time of the first video segment.

[0075] Optionally, an implementation of step 1011 can perform cross-table retrieval between data tables of the system. Assuming that the user's recommendation behavior is recorded in a behavior record table, the table contains common fields such as behavior unique identifier, user unique identifier, video unique identifier, behavior type (such as like, forward, comment, etc.), behavior time, and an optional additional information field for storing additional data required for specific behaviors (such as comment content, barrage text, etc.). By querying the above behavior record table, the video unique identifier (such as video ID or unique code) corresponding to the user's recommended behavior is extracted. Subsequently, this video identifier is used as a query condition to further access the video table storing detailed video information. Assuming that the video table is used to store basic information of the video, including video attribute fields such as the video's unique identifier, and also contains key information such as the video's author, title, and upload time. The system quickly locates and extracts video resources that match the video's unique identifier through an efficient data retrieval mechanism. This process ensures that the recommendation system can accurately retrieve the video content recommended by the user.

[0076] Optionally, an implementation of step 1012 is similar to step 1011, and is also performed by searching from the system data table. Assuming that there is a behavior time field in the behavior record table, the behavior occurrence time corresponding to the user recommended behavior can be quickly retrieved through an efficient data retrieval mechanism.

[0077] In step 1012, it is particularly emphasized that the time when the user's recommendation behavior occurs must be before the end time of watching the first video. This requirement is based on the in-depth understanding and analysis of user behavior patterns. The reason is that the recommendation behavior after the end of the video is often difficult to directly associate with a specific segment within the video due to the time lag, so it is impossible to accurately analyze which part of the video the user is more interested in. Its technical effect is that such a time constraint helps the recommendation system to more accurately capture the user's immediate feedback and preferences, thereby optimizing the recommendation strategy and improving the pertinence and effectiveness of the recommendation.

[0078] At the same time, it is also necessary to realize that the timing of video recommendation behavior is not always absolutely accurate. In actual scenarios, there are indeed cases where users may like, forward, and other recommendation behaviors after a period of time after watching the content that really attracts them. However, through statistical analysis of a large amount of user data, we can find that in most cases, users' recommendation behaviors tend to occur before or shortly after the moment they discover the content of interest. That is, the recommendation logic based on time sequence proposed in this disclosure is valid.

[0079] Optionally, in one implementation of step 1013, the system uses the rich features displayed by the first video, such as the background music features, picture features, subtitle features, and voice features of the video, to accurately determine the start time and end time of the video clip corresponding to the user's recommendation behavior. According to the general rules of user behavior, the start time of the video clip that can touch the user's emotions and trigger the recommendation behavior is naturally earlier than the user's recommendation behavior time, because this is the premise and basis for the user to generate the recommendation intention. However, for the end time of the video clip, the situation may be more complicated. Although the user's recommendation behavior often follows the moment when he finds the content of interest, it does not mean that the playback of the clip must have ended. The user may have been deeply attracted during the playback of the clip, thereby generating the recommendation intention and performing the recommendation operation shortly thereafter. Therefore, the end time of the video clip is not necessarily earlier than the user's recommendation behavior time.

[0080] A situation such as Figure 4 As shown, when the user is watching the first video, the first video clip that resonates with the user emotionally has a start time of t2 and an end time of t3. The user clicks "like" at time t1 after t3 (when the video has not finished playing).

[0081] Another situation is Figure 5 As shown in the figure, when the user was watching the first video, the first video clip that resonated with him emotionally had a start time of t2. The user then made a user recommendation behavior of "like" at the subsequent time t1 (when the end time t3 of the first video clip had not yet arrived). It is worth noting that although the video and the clip had not yet reached the end at time t3, the user decisively took the action of "like" at time t1 and made an active user recommendation. This behavior shows the user's immediate liking for the clip and also reveals the user's positive expectations for the subsequent content.

[0082] It is worth noting that no matter the user chooses to continue watching until the end of the video after the recommendation moment (moment t1) or closes the first video in advance, this behavioral choice does not affect the emotional resonance that the user has already generated at the recommendation moment.

[0083] Optionally, in an implementation of step 1014, the first video segment can be directly extracted from the first video using the start time and end time of the video segment.

[0084] There are two ways to store the above video clips. The first storage method is to store the information of the video clip in the system's video clip table. The video clip table is used to store detailed information of the video clip, including the video clip unique identifier, title, description, upload time, etc. The video clip table has two options: directly storing the actual video clip and only retaining the video index. The method of directly storing the video clip means saving the actual content of the video file on the data table, which will take up a lot of storage space, especially when the video library is large and the video resolution is high. The second storage method is to retain the index of the video clip to the video, which is a more flexible and efficient storage method. Under this strategy, we only save the unique identifier of the video (such as the video ID or URL), and the start time and end time of each video clip relative to the entire video. In this way, when a user requests a video clip, the server can quickly locate the original video file based on the index information and extract the time period required by the user for playback without storing multiple duplicate video clips. This method greatly saves storage space and is also convenient for dynamic update and management of video content.

[0085] The various times of users watching videos involved in the above steps 1012, 1013, and 1014, including the start time and end time of users watching videos, can be obtained through the video playback system. One implementation method may be that the video playback server will record and store a series of log information related to user behavior in real time during the process of providing video services to users. Whenever a user starts watching a video, the server will generate a unique session ID and record the start time of the session, which is the user's viewing start time. At the same time, the session information may also include the user's device information, IP address, etc., to facilitate subsequent data analysis and tracking. During the video playback process, the server will continuously record various playback events, such as buffering start, buffering end, playback pause, playback continuation, playback end, etc. By parsing the timestamps of these events, the continuity and interruption of the user's video viewing can be accurately inferred, and then the end time of the user's video viewing can be calculated.

[0086] The various times of the video clips involved in the above steps 1012, 1013, and 1014 may need to be achieved through a combination of multiple technologies such as video detection. These technologies usually combine algorithms from multiple fields such as computer video, audio analysis, and natural language processing to achieve in-depth understanding and accurate segmentation of video content. Using multiple features in the video (such as picture changes, color distribution, sound features, music features, subtitle content, etc.) as the basis for segmentation, the algorithm can automatically identify the key turning points or emotional climaxes in the video. User behavior data (such as viewing time, pause points, likes, forwarding, fast forward and rewind operations, etc.) can be used to assist and optimize the extraction process of these video clips.

[0087] In summary, the technical effects of the above steps 1011 to 1014 include: by strictly defining the time of occurrence of the user's recommendation behavior to be before the end time of watching the video, the recommendation deviation caused by time lag is effectively avoided, making the recommendation closer to the user's immediate interests and preferences. By utilizing the multi-dimensional features of the video (such as background music, picture, subtitles, voice, etc.) combined with user behavior data, the start and end time of the video clip corresponding to the user's recommendation behavior is accurately located, which is not only conducive to capturing the user's immediate emotional resonance point, but also takes into account the possible lag of user behavior, and improves the flexibility and accuracy of video clip extraction. The above steps support the capture and presentation of clips that the user is really interested in from the original video, enhancing the user experience. It provides strong support for the personalized recommendation strategy of the video platform.

[0088] Figure 6 to Figure 7 As shown, the subordinate extension of the key step 1013 in the video recommendation method according to the present disclosure is further elaborated in detail, that is, the subordinate extension of the feature "According to the first video feature of the first video, the start time and the end time of the first video segment corresponding to the user recommendation behavior are obtained".

[0089] For the lower extension of step 1013, optionally, one of the extension methods is to obtain a video interval before the behavior occurs according to the first video feature, and obtain the start time of the first video segment according to the video interval;

[0090] A second video interval after the start time of the first video segment is obtained, and an end time of the first video segment is obtained according to the second video interval.

[0091] For the above implementation, Figure 6 As shown, the identification and utilization of video intervals is a key step. It is known that the user performed a user recommendation behavior at time t1, and it is necessary to extract the first video segment that started before time t1 and triggered the user recommendation behavior. That is, it is necessary to determine the start time t2 and the end time t3 of the first video segment.

[0092] This task can be accomplished by extracting the video intervals in the video. This task can be performed based on a variety of features within the video, such as rapid changes in the content of the picture, significant differences in color distribution, ups and downs in the sound, changes in the rhythm of the music, and the emotional orientation of the subtitle content.

[0093] The technical effect of the above optional implementation is that video interval extraction is simple and reliable. By intelligently analyzing various features inside the video (such as picture changes, sound features, etc.) and user behavior data (such as viewing time, pause points, etc.), the system can automatically and accurately identify the key turning points or emotional climaxes in the video, thereby dividing the video interval. This automated processing method not only improves efficiency, but also reduces the subjectivity and errors caused by human intervention, ensuring the reliability of video interval extraction. Secondly, the method shows a high degree of flexibility in the extraction of video clips. By determining the user's recommended behavior time, and tracing back to the previous video interval to determine the start time of the clip, and then further analyzing the subsequent video interval to define the end time of the clip, the system can flexibly divide the clip according to the actual situation of the video content. In addition, it is worth noting that while this method pursues the fine division of video clips, it may also face the problem of dividing the clips finely.

[0094] Specifically, in the process of extracting video intervals, features such as video background music, pictures, subtitles, and voice can be used. For video background music, audio analysis technology can be used to identify rhythm changes, volume fluctuations, and the appearance and disappearance of specific melodies in the music, all of which can serve as potential signs of video intervals. For example, when the background music suddenly increases or changes style, it often indicates a major turning point in the video content, thus becoming a key point in dividing video intervals.

[0095] Changes in video images can also be used. Using computer vision technology, pixel differences between consecutive frames, changes in color distribution, and trajectories of moving objects can be analyzed to detect significant changes in the image. These changes may include scene switching, sudden changes in the actions of video characters, or shifts in visual focus, which together constitute natural separators for video intervals.

[0096] As an important supplement to the content, video subtitles also provide rich information to assist in the extraction of video intervals. By analyzing the appearance and disappearance of subtitles, the emotional color of the content, and the synchronization with the screen content, we can identify those subtitle segments that are closely related to the key plot of the video. These subtitle segments are often accompanied by major developments in the video content, and therefore become an important basis for dividing video intervals.

[0097] The changes in voice tone, speaking speed, and the use of specific words can reflect the speaker's emotional state and intention, and thus reveal the direction of the video content. Through speech recognition and natural language processing technology, the speech content can be automatically analyzed, and the key dialogues or monologues in the video can be identified by combining the changes in speech features. These are often the key moments that divide the video interval.

[0098] For the lower-level extension of step 1013, optionally, another extension method is to obtain a first candidate segment before the time when the behavior occurs according to the first video feature;

[0099] The first video segment start time and the first video segment end time are acquired according to the candidate segment start time and the candidate segment end time of the first candidate segment.

[0100] For the above implementation, Figure 7 As shown, the first video feature is extracted to directly identify candidate segments (such as climax segments) in the video. It is known that the user performed a user recommendation behavior at time t1, and it is necessary to extract the first video segment that started before the user recommendation behavior and triggered the user recommendation behavior. That is, it is necessary to determine the start time t2 and the end time t3 of the first video segment.

[0101] like Figure 7 As shown, the climax segment of the background music can be extracted by the climax feature of the video background music. The climax segment of the video screen can also be extracted by the highlight feature of the video screen. Other video features can also be used to extract candidate segments in the video, and the details can be seen in the implementation method below.

[0102] The above optional implementation has the beneficial effect of improving the accuracy of identifying candidate video segments. These video features, such as the climax features of background music and the highlight features of the video screen, are all very expressive and attractive parts of the video content, and they are often closely related to the user's emotional resonance and recommendation intention. Therefore, using these features for identification can more directly capture the "highlight moments" in the user's mind, making the extracted first video segment closer to the user's real feelings and needs.

[0103] This method adopts a candidate segment-based strategy when determining the start and end time of a video segment. This means that after identifying a possible candidate segment, the system does not immediately determine its boundaries, but further analyzes the specific content of the candidate segment to ensure that the selected segment can fully contain the key parts that trigger the user's recommendation behavior. This strategy not only improves the accuracy of recognition, but also ensures the integrity and coherence of the video segment, allowing users to have a better viewing experience.

[0104] Optionally, acquiring the first video segment start time and the first video segment end time according to the candidate segment start time and the candidate segment end time of the first candidate segment includes:

[0105] In response to the first video feature including at least two features, obtaining a candidate segment start time and a candidate segment end time corresponding to each feature;

[0106] According to the candidate segment start times of the at least two characteristics, selecting the earliest candidate segment start time as the first video segment start time;

[0107] According to the candidate segment end time of the at least two characteristics, the latest candidate segment end time is selected as the first video segment end time.

[0108] like Figure 7 As shown, the system uses two features to identify the first video clip. The first feature is the video background music climax feature. Using the background music feature, the system identifies the "music climax" candidate clip (such as Figure 7 The second feature is the video highlight feature. Using the highlight feature, the system identifies candidate segments of “highlight” (such as Figure 7 ). At this time, the start time of the "Music Climax" candidate segment is slightly earlier than the "Screen Highlight" candidate segment, and the end time of the "Music Climax" candidate segment is slightly earlier than the "Screen Highlight" candidate segment. Since the "Music Climax" candidate segment and the "Screen Highlight" candidate segment have a large overlap, it can be determined that these two segments describe the same first video segment. From the perspective of video production, although the multiple basic elements that make up the video, such as pictures and music, support and complement each other in emotional expression and story narration, their creation and editing processes are often relatively independent, and their rhythm and focus are sometimes not completely consistent. This leads to the possibility that video elements such as "Screen Highlight" and "Music Climax" that belong to the same video segment may deviate in time points.

[0109] like Figure 7 As shown, after determining that multiple features describe the same video segment, the candidate segments detected by the multiple features can be merged. For example, the start time of the "music climax" candidate segment is slightly earlier than the "screen highlight" candidate segment, and the end time of the "music climax" candidate segment is slightly earlier than the "screen highlight" candidate segment, so the start time of the "music climax" candidate segment is taken as the start time of the first video segment, and the end time of the "screen highlight" candidate segment is taken as the end time of the first video segment.

[0110] Optionally, before the above implementation, it is determined whether the at least one feature points to the same video clip, and the time overlap between the features is calculated. The overlap ratio of the candidate clips detected by different features on the time axis is calculated, that is, their start time and end time overlap each other. If the overlap ratio exceeds a predefined threshold, it can be determined that these features are describing the same video clip. For example, in the background music climax and picture highlight features, if the climax or highlight time periods of the two overlap for the most part, it can be considered that they are describing the wonderful part of the same video content.

[0111] The beneficial effect of the above optional implementation is that the system can more comprehensively capture the start and end time of the video segment by combining multiple features in the video (such as the climax of the background music and the highlights of the video screen). When the candidate segments indicated by different features have slight differences in time, this method ensures the integrity and coherence of the video segment by selecting the earliest start time and the latest end time as the boundary of the final video segment.

[0112] Optionally, acquiring the first video segment start time and the first video segment end time according to the candidate segment start time and the candidate segment end time of the first candidate segment includes:

[0113] Obtain a video interval of the first video before and after the start time of the candidate segment and the end time of the candidate segment;

[0114] According to the video interval, the candidate segment start time and the candidate segment end time, the first video segment start time and the first video segment end time are acquired.

[0115] like Figure 7 As shown, when various video features are used to directly detect the video candidate segments in the first video, there may be errors. For example, when the background music representing the climax begins, the character in the video may be singing a complete song. For example, the starting time of the background music climax is t2', but the time when the character in the video starts singing is t2. Therefore, using the moment t2' as the video candidate segment will cause unreasonable division of video segments. The use of video interval detection technology can detect the presence of a video interval at moment t2. At this time, the start time t2' of the music climax is combined with the previous video interval t2 to obtain a more reasonable video segment boundary. For another example, the end time of the highlight of the picture is t3', and the use of video interval detection technology can detect the presence of a video interval at moment t3. At this time, the end time t2' of the music climax is combined with the subsequent video interval t2 to obtain a more reasonable video segment boundary.

[0116] Optionally, the preliminary start time of the candidate segment is determined by video feature analysis, which is recorded as T1. Using video interval detection technology, a video interval is searched in the time range before T1. This video interval should be a significant interval related to the content transition or emotional change of the candidate segment, which is recorded as T2. T2 is used as a more reasonable start time for the candidate segment.

[0117] Optionally, the preliminary end time of the candidate segment is determined by video feature analysis, which is recorded as T3. Using video interval detection technology, a video interval is searched in the time range after T3. This video interval should be a significant interval related to the content transition or emotional change of the candidate segment, which is recorded as T4. T4 is used as a more reasonable end time for the candidate segment.

[0118] The beneficial effect of the above optional implementation is that by combining video feature analysis and video interval detection technology, the accuracy of video segment division is improved. Traditional video segment division may only rely on direct detection of video features, such as background music climax or screen highlights, which may cause the division result to be not completely consistent with the emotion or plot transition of the actual content of the video. By introducing video interval detection technology, significant intervals in the video content can be identified, and these intervals are often closely related to the content transition or emotional change. Therefore, combining video features with video intervals can more accurately determine the boundaries of video segments. Effectively reduce the unreasonable segment division problem caused by video feature detection errors. For example, at the beginning of the background music climax, if the start time of the segment is determined only based on the music features, the actual start time of the character activity (such as singing) in the video may be ignored, resulting in premature segment division. By combining video interval detection technology, the video interval before the start of the character activity can be identified, so as to adjust the start time of the segment to make it more consistent with the actual situation of the video content.

[0119] The lower-level extension of step 1013 also includes extracting candidate video segments based on various video features.

[0120] Optionally, the acquisition of the features of the candidate video background music segments includes:

[0121] Extracting basic music features of background music; the basic music features include at least one of the following: volume, pitch, and rhythm;

[0122] Calculating a characteristic curve of the background music according to the basic music characteristics;

[0123] Based on the feature curve, the features of the candidate video background music segments are calculated.

[0124] Specifically, first, the basic music features of the background music, such as volume, pitch, rhythm, etc., are extracted through audio analysis techniques, such as fast Fourier transform (FFT) or Mel-frequency cepstral coefficients (MFCC). These features can reflect the physical properties and auditory perception of the music. Then, statistical or machine learning methods, such as sliding window average or time series analysis, are used to calculate the characteristic curve of the background music based on the basic music features. The curve can show the changing trend of the music features over time. Finally, based on the characteristic curve, through feature extraction algorithms (such as principal component analysis PCA) or machine learning models (such as support vector machines SVM or neural networks), the features that can characterize the candidate clips of the video background music, such as the emotional tendency of the music, the climax part, etc., are calculated and extracted.

[0125] The above optional implementation has the beneficial effect of improving the accuracy of emotion recognition: by extracting the basic music features of background music (such as volume, pitch, rhythm), we can deeply understand the emotional expression of music, and then more accurately identify the emotional climax or turning point in the video. Enhance the rationality of segment division: The characteristic curve calculated based on the basic music features can reflect the ups and downs of music emotions, which helps to more reasonably divide video segments, especially those closely related to music emotions.

[0126] Optionally, the acquisition of the highlight feature of the video picture includes:

[0127] Extracting basic picture features of the video picture; the basic picture features include at least one of the following: picture scene, picture object, picture emotional state, and picture color;

[0128] According to the basic picture features, the highlight semantic features of the video picture are calculated as the highlight features of the video picture.

[0129] Specifically, first, use image processing techniques, such as image segmentation and object recognition, to extract basic video features, including scene (such as indoor, outdoor), objects (such as people, animals, buildings), emotional state (such as joy, sadness), color, etc. These features can be automatically identified and classified by pre-trained models or algorithms. Then, based on these basic picture features, natural language processing (NLP) or deep learning techniques, such as convolutional neural network (CNN) combined with recurrent neural network (RNN), are used to calculate the highlight semantic features of the video picture. These features can reflect the key information or emotional climax points in the picture, thus serving as the highlight features of the video picture.

[0130] The beneficial effect of the above optional implementation is to improve the understanding of visual content: by extracting basic picture features such as scene, object, emotional state and color, the visual content of the video picture can be fully understood, providing a rich information basis for video analysis and processing. Enhanced highlight segment recognition: The highlight semantic features calculated based on basic picture features can accurately identify visual highlights in the video, such as wonderful moments, emotional outbursts, etc., which helps to improve the accuracy of video summaries or recommendations.

[0131] Optionally, the acquisition of the video subtitle feature includes: extracting text features of the subtitle; extracting features representing the candidate video segment from the text features as the video subtitle feature;

[0132] The text features include at least one of the following: text keyword features, text semantic features, and text sentiment features;

[0133] The acquisition of the video speech feature includes: converting the video speech into text and extracting text features of the text; and extracting features representing the candidate video segments from the text features as the video speech feature.

[0134] Specifically, regarding the acquisition of video subtitle features, first, the subtitle text is extracted from the video using optical character recognition (OCR) technology, and preprocessed, such as word segmentation, stop word removal, etc. Then, text features are extracted from the processed text, including text keyword features (such as word frequency statistics), text semantic features (such as word vector representation), and text sentiment features (such as sentiment dictionary matching). Finally, based on these text features, through feature selection or dimensionality reduction technology, features that can characterize the candidate video clips, such as topic keywords, sentiment tendencies, etc., are extracted as video subtitle features.

[0135] Specifically, regarding the acquisition of video speech features, first, speech recognition technology (such as automatic speech recognition ASR) is used to convert the speech in the video into text. Then, similar to subtitle feature extraction, feature extraction is performed on the converted text, including text keyword features, text semantic features, and text emotional features. These features can reflect the key information and emotional color of the speech content. Finally, from the extracted text features, features that can characterize the candidate video clips, such as key information points in the speech, emotional changes, etc., are selected or calculated as video speech features.

[0136] The above optional implementation has the beneficial effect of enhancing the understanding of text content: by extracting the text features of subtitles (such as keywords, semantics, and emotions), the text information in the video can be deeply understood, providing important text clues for video analysis and processing. Assisting in video segmentation: Subtitle features can reflect the theme and focus of the video content. Combined with other video features, video segments can be more accurately divided, especially those closely related to the text content.

[0137] Optionally, the first video feature further includes at least one of the following: a comment feature, a bullet screen feature;

[0138] The acquisition of the comment feature includes: extracting text features of the comment; using features representing the candidate video segments in the text features as the comment feature; or extracting distribution features of the comment as the comment feature;

[0139] The text features include at least one of the following: text keyword features, text semantic features, and text sentiment features;

[0140] The acquisition of the barrage feature includes: extracting text features of the barrage; using features of the candidate video segments represented by the text features as the barrage feature; or extracting distribution features of the barrage feature as the barrage feature.

[0141] Specifically, for comment features, the text features of the comments can be extracted, including text keywords, semantic and emotional features, and based on these features, features that characterize candidate video clips can be calculated, such as user evaluation of video content, emotional tendencies, etc. At the same time, the distribution features of comments, such as time distribution, user activity, etc., can also be considered as supplementary information. For barrage features, the text features of barrage are also extracted, and based on these features, features that characterize candidate video clips can be calculated, such as barrage hot spots, user interaction intensity, etc. In addition, the distribution features of barrage, such as density, location distribution, etc., can also provide valuable information.

[0142] The above optional implementation has the beneficial effect of improving speech content recognition: after converting the video speech into text, the extracted text features can accurately reflect the speech content, which helps to identify key information or dialogues in the video. Enhanced sentiment analysis: The text sentiment features in the speech features can reflect the speaker's emotional state, providing strong support for video sentiment analysis, thereby improving the emotional rationality of video segment division.

[0143] Figure 8 A flow chart 102 of an embodiment of step 102 of the video recommendation method according to the present disclosure is shown. Step 102 may include the following steps:

[0144] Step 1021, obtaining a first video segment feature of the at least one first video segment as a recommendation indicator for recalling the second video; the recommendation indicator also has an indicator weight; the first video segment feature includes at least one of the following: a video segment label, a background music name, a background music label, a background music feature vector, a video scene semantic label, a video object semantic label, a video screen feature vector, a subtitle label, a subtitle feature vector, a comment label, and a comment feature vector;

[0145] Step 1022: In response to the first video segment and the previous first video segment having related recommendation indicators and correlation, the related recommendation indicators are merged, and the recommendation weights of the related recommendation indicators are weighted and merged using the correlation to obtain updated indicator weights; the previous first video segment is another first video segment watched by the user before the first video segment;

[0146] Step 1023: recall the video to be recommended according to the recommendation index and the index weight to obtain a second video.

[0147] Optionally, in one implementation of step 1021, feature extraction is first performed on the first video clip corresponding to the user's recommended behavior. The system automatically extracts key features, such as music tags ("country music") and visual tags ("fire scene"), through video content recognition technology, and uses these features as recommendation indicators.

[0148] Optionally, the importance of the video clip is evaluated based on the user's recommendation behavior (e.g., "forward" is more recommended than "like"). Based on the above evaluation, these recommendation indicators are assigned weights to describe the cumulative effect of continuous recommendation behavior on related recommendation indicators. This process is dynamic and automated, ensuring that the recommendation system can capture the user's interest and preference in continuously recommending videos in real time.

[0149] An implementation method of the above optional embodiment requires cross-table retrieval between data tables of the system. Recommendation index table: stores recommendation indexes and their weights extracted from video clips, and associates them with specific video clips. It includes fields such as index unique identifier, index name, and index weight. The server determines and updates the index name and index weight in the recommendation index table based on the video content in the video clip table.

[0150] The technical beneficial effect of the above step 1021 is that through the above technical implementation method, the recommendation system can more accurately capture the changes in user interests, whether based on the latest watched video clips or related clips in historical data. By setting the weights of the recommendation indicators, the system can automatically optimize the recommendation strategy based on the user's specific behavior (such as forwarding, liking, etc.) and the timeliness of the content. This personalized recommendation method not only improves user satisfaction and engagement, but also enhances the interactivity between users and the platform, and promotes wider dissemination and discovery of content. At the same time, the automated feature extraction and weight allocation mechanism also reduces the need for manual intervention and improves the efficiency and scalability of the recommendation system.

[0151] Optionally, in one implementation of step 1022, the system's recommendation index table records the recommendation index and its weight, and the recommendation index table supports the merging operation of the recommendation index therein. When the user watches a new first video segment (which we call the current segment), the system checks whether there is a related recommendation index between the segment and the previous first video segment watched before. This correlation can be determined by comparing the name, type or semantic similarity of the recommendation index.

[0152] Once the relevant recommendation indicators are identified, the system will then perform a merge operation. The merge operation not only involves merging the same recommendation indicators into one entry, but also involves weighted merging of the weights of these recommendation indicators. The weighted merging of weights can be based on a variety of factors, such as the time interval between two video clips (closer time intervals may be given higher weights), users' recommendation behaviors for different video clips (such as forwarding is given a higher weight than liking), and the importance of the recommendation indicator itself (some indicators may naturally have higher priorities).

[0153] Specifically, a basic weight is first assigned to each relevant recommendation indicator, and then these weights are adjusted according to the above factors. For example, a time decay function can be used to reduce the weight of the recommendation indicator in the video clips watched earlier. The time screening function can be linear decay or nonlinear decay; the decay means that the longer the distance from the current moment, the greater the decay. At the same time, additional weights are added to the recommendation indicators in the current clip according to the user's recommendation behavior (such as the number of forwardings).

[0154] The beneficial effect of the above step 1022 is that by constructing a recommendation index that supports merging operations, the intelligence and personalization level of video recommendation is improved. When a user watches and recommends a video clip multiple times, the system can intelligently identify and merge the recommendation indexes related to the previously watched clips, which not only simplifies the recommendation logic, but also comprehensively considers the time interval, user behavior and the importance of the recommendation index itself through a weighted merging mechanism, and realizes the dynamic adjustment of the index weight. This design effectively improves the timeliness and accuracy of the recommendation results, making the recommended content closer to the user's immediate interests and long-term preferences, thereby enhancing the user experience and promoting the dissemination and sharing of content.

[0155] Optionally, in one implementation of step 1023, the system has a table of videos to be recommended and an index table of videos to be recommended. The table of videos to be recommended contains basic information of each video, such as a video unique identifier, title, description, upload time, etc. The index table to be recommended includes one or more recommendation index fields for subsequent recall and sorting (these fields can be indexes or foreign keys, pointing to detailed records of the recommendation index table). The index table of videos to be recommended needs to record in detail the hits of each video to be recommended on each recommendation index.

[0156] During the recall process, the system will match and filter out videos that meet the user's preferences from the index table of videos to be recommended based on the recommendation index table (including recommendation weights) calculated based on the user's multiple recommendation behaviors established in the previous steps. This usually involves comparing and sorting the recommendation indexes. In order to achieve efficient recall, the system can use a variety of technical means, such as index optimization, caching mechanism, distributed storage, etc., to reduce query time and improve system response speed.

[0157] The beneficial effect of the above step 1023 is that, through the structured video table to be recommended and the index table of the video to be recommended, the comprehensive recording of video information and the accurate tracking of recommendation indicators are achieved. Through the video table to be recommended, the system can efficiently store the basic attributes of each video, providing a rich data foundation for the subsequent recommendation process. The index table of the video to be recommended further refines the performance of the video in multiple recommendation dimensions, ensuring that each video can be accurately evaluated and matched.

[0158] In summary, the beneficial effects of the above steps 1021 to 1023 include: jointly building an efficient, intelligent and user-friendly video recommendation system. The system can accurately capture the dynamic changes of user interests, automatically optimize the recommendation strategy by using user behavior and the timeliness of content, realize personalized recommendations, significantly improve user satisfaction and engagement, and promote active interaction between users and the platform and the wide dissemination of content. By building recommendation indicators that support merging operations, the intelligence level of recommendations is further improved, and relevant recommendation indicators are intelligently identified and merged, making the recommendation results more timely and accurate, closer to the user's immediate and long-term interests, and enhancing the user experience and the dissemination of content.

[0159] For the lower-level extension of step 1021, optionally, obtaining the first video segment feature of the at least one first video segment as a recommendation indicator for recalling the second video includes:

[0160] According to the play time of the first video clip and / or the recommendation behavior weight of the relevant user recommendation behavior, the indicator weight is weighted to serve as the updated indicator weight;

[0161] The recommended behavior weight is calculated according to the type of the recommended behavior and / or the user's recommended words.

[0162] Reference Figure 1 In the example in , the server performs a recommendation index analysis based on the most recent video clip "fire scene country music". It extracts recommendation index 1 (such as the "country music" tag) and recommendation index 2 (such as the "fire scene" tag). Similarly, the server also performs a recommendation index analysis on the previous video clip "smoke scene country music". It extracts recommendation index 1 (such as the "country music" tag) and recommendation index 3 (such as the "fire scene" tag).

[0163] Considering that the video clip "fire scene country music" was played recently and the user recommendation behavior was "forward" ("forward" means a higher recommendation than "like"), the indicator weights of the two recommendation indicators in the video clip "fire scene country music" are set to a relatively high 90. Considering that the video clip "smoke scene country music" was played earlier and the user recommendation behavior was "like" ("like" means a lower recommendation than "forward"), the indicator weights of the two recommendation indicators in the video clip "smoke scene country music" are set to a relatively low 50.

[0164] The beneficial effect of the above optional implementation is that by comprehensively considering the length of the playback time of the first video clip and the diversity of the user's recommendation behavior, the system can more accurately evaluate the user's interest in different content and preference changes. In particular, the recommendation behavior weight is introduced and calculated based on the type of recommendation behavior (such as likes, shares, comments, etc.) and the specific content of the user's recommendation, so that the weight distribution is closer to the user's real feedback and behavioral intentions. This weighting method not only strengthens the impact of user behavior on the recommendation results, but also promotes the system's in-depth understanding and prediction of user preferences. Therefore, this technology effectively improves the accuracy and relevance of the recommendation results, allowing users to find video content that suits their interests more quickly, enhancing user experience and satisfaction.

[0165] Optionally, obtaining the first video segment feature of the at least one first video segment as a recommendation indicator for recalling the second video includes:

[0166] The video features of the first video to which the first video clip belongs are obtained as recommendation indicators for recalling the second video; the video features include at least one of the following: video tags, video names, video description keywords, video authors, video author tags, user tags, and authors followed by users.

[0167] Reference Figure 1 In the example above, the server performs a recommendation index analysis based on the recent video clip "fire scene country music". It extracts recommendation index 1 (such as the "country music" tag) and recommendation index 2 (such as the "fire scene" tag). Suppose that a tag of an author that the user follows is also extracted as index 4 (such as the "Chinese style painter" tag, Figure 1 (not shown)

[0168] Indicator 4 (the label "Chinese-style painter") is actually more related to the author characteristics that users pay attention to, rather than directly targeting the content characteristics of the specific video clip "fire scene country music". In order to improve the accuracy and relevance of the recommendation, the indicator weight of indicator 4 is adjusted accordingly to reduce its contribution in recalling video clips such as "fire scene country music" as the second video. This adjustment is intended to enable the recommendation system to more finely distinguish the impact of features of different dimensions on the recommendation results, ensuring that those indicators directly related to the content of the video clip (such as the label of the video clip, etc.) occupy a more important position in the recommendation process, while indirect indicators such as author characteristics, user characteristics, and characteristics of authors that users pay attention to are appropriately considered according to specific circumstances.

[0169] The beneficial effect of the above optional implementation is that by introducing video features based on video clip features, the recommendation dimension is further expanded, enabling the system to comprehensively consider users' personal preferences, social relationships and the influence of content creators to achieve more comprehensive and in-depth recommendations.

[0170] For the lower-level extension in step 1022, optionally, the previous first video segment is iteratively processed to obtain an updated index weight.

[0171] like Figure 1 As shown in the figure, the server extracts the relevant recommendation index and relevance for the most recent video clip "fire scene country music" and the previous video clip "smoke scene country music". As can be seen from the table above, both video clips have a recommendation index "country music" label, assuming that the relevance of the two country music is 0.9 (assuming that the relevance value ranges from 0 to 1). When the recommendation indexes are merged, the recommendation weight of the "country music" label will become: 90+0.9×50=135.

[0172] Assume that for the previous video clip "smoke scene country music", there is another video clip "flowing water scene country music" ( Figure 1 (Not shown in the figure, assuming a recommendation weight of 30). The recommendation indicators extracted from it are: indicator 5 "flowing water" label and indicator 1 "country music" label. The video clip "flowing water country music" also has a related recommendation indicator "country music" label with the recent video clip "fireplace country music". Assuming that the relevance of this country music is also 0.9, the recommendation weight of the "country music" label will become: 90+0.9×50+0.9×30=162.

[0173] Optionally, during the iterative processing, introducing a processing flag is an effective strategy. The flag may be a Boolean value and is used to avoid repeated processing of video segments that have been processed or merged.

[0174] The beneficial effect of the above optional implementation is that, through iterative processing, the system can continuously and dynamically adjust the weight of the recommendation index related to each video clip according to the user's latest behavior and preference changes.

[0175] Optionally, in response to the existence of cross-modality related recommendation indicators, the weighted combination is performed on the cross-modality related recommendation indicators to obtain updated indicator weights.

[0176] like Figure 1As shown in the figure, the most recent video clip "fire scene country music" and the previous video clip "smoke scene country music", both of which have a recommendation indicator "country music" label, assuming that the correlation between the two country music is 0.9 (assuming that the correlation value range is 0 to 1). When the recommendation indicators are merged, the recommendation weight of the "country music" label will become: 90+0.9×50=135.

[0177] Assume that for the previous video clip "smoke scene country music", there is another video clip "country scene suona music" ( Figure 1 (not shown in the figure, assuming a recommendation weight of 30). The recommendation indicators extracted from it are: indicator 6 "country scene" label and indicator 7 "Suona music" label. Then the video clip "country scene suona music" also has related cross-modal recommendation indicators with the recent video clip "fire scene country music": "country scene" label and "country music" label. Assuming that the correlation between these two cross-modal labels is 0.5 and the "country scene" label can be merged into "country music", the recommendation weight of the "country music" label will become: 90+0.9×50+0.5×30=150.

[0178] The beneficial effect of the above optional implementation is that it effectively integrates the correlation information between different modalities (such as text, images, audio, etc.), so that the recommendation system can more comprehensively evaluate the user's preference for content, thereby providing more accurate and personalized recommendations.

[0179] Optionally, a relevance threshold of the recommendation indicators is set, and the weighted combination is performed on the recommendation indicators whose relevance is higher than the relevance threshold.

[0180] The beneficial effect of the above optional implementation is that those recommendation indicators that are truly significantly relevant are screened out for weighted merging, which avoids incorporating a large number of low-relevance or irrelevant indicators into the calculation, thereby reducing computing power consumption, reducing noise interference, and improving recommendation efficiency.

[0181] For the lower-level extension of step 1023, optionally, the step of recalling the video to be recommended according to the recommendation index and the index weight to obtain a second video includes:

[0182] Obtaining a second video feature of each video to be recommended, where the second video feature corresponds to the recommendation indicator;

[0183] For each video to be recommended, matching is performed with the second video feature according to the recommendation index to obtain a matching score of each recommendation index; weighting is performed on the matching score according to the index weight of the recommendation index to obtain a weighted score of each video to be recommended;

[0184] All the videos to be recommended are sorted in descending order according to their weighted scores, and the second video is obtained by screening.

[0185] like Figure 1 As shown in the figure, the server matches video 3 in the recommended video library that the user has not watched yet. According to the recommendation index, feature extraction is performed on video 3 to obtain the "country music" label and the "field scene" label. Assuming that there is also video 4 (not shown in the figure), feature extraction obtains the "Chinese style music" label and the "fire scene".

[0186] The server ranks video 3 and video 4 for recommendation to the user. The index weight of the "country music" tag hit by video 3 is 135, while the index weight of the "fireplace image" tag hit by video 4 is 90. Assuming that the above two tags are equally important to the user, but the index weight value of video 3 is larger, the weighted score of video 3 will be higher than that of video 4.

[0187] The beneficial effect of the above optional implementation is that the server's recommendation index system successfully captures the user's emotional accumulation process during viewing through intelligent analysis of the user's past video viewing history records (sorted from earliest to latest in terms of viewing time are video 2, video 1), thereby achieving the purpose of accurate recommendation.

[0188] For the lower-level extension of step 1023, optionally, the second video is divided into video segments to obtain candidate video segments;

[0189] Matching the candidate video segments according to the recommendation index and the recommendation weight to obtain a second video segment;

[0190] The second video segment is recommended to the user.

[0191] like Figure 1 As shown in FIG. 1 , if the specific video segment position of the “country music” tag in video 3 can also be located, the server can directly recommend the user to jump to the video segment position of video 3 (eg, Figure 1 At this time, during the viewing process, the user can be directly recommended to watch the "field scene country music" video clip of video 3 from the "fire scene country music" video clip of video 2, thereby obtaining a more accurate recommendation.

[0192] The beneficial effect of the above optional implementation is that it improves the accuracy of recommendations, and by targeting specific video clips rather than the entire video, the recommended content is more in line with the user's immediate interests and preferences. Users do not need to find the parts of interest in the complete video themselves, which saves time and improves the viewing experience. Secondly, this segment-level recommendation method enhances the flexibility and personalization of recommendations, allowing the recommendation system to more carefully capture subtle changes in user needs and make corresponding adjustments. Furthermore, by directly recommending users to jump to specific video clips, a seamless viewing experience is achieved. Users can smoothly transition from one video clip to another related clip during the viewing process, avoiding the sense of interruption caused by switching videos.

[0193] For the lower-level extension of step 1023, optionally, the second video is divided into video segments to obtain candidate video segments;

[0194] Acquire a second video segment feature of each candidate video segment, where the second video segment feature corresponds to the recommendation index;

[0195] For each of the candidate video clips, matching is performed with the second video clip feature according to the recommendation index to obtain a matching score of each recommendation index; and the matching scores are weighted according to the index weight of the recommendation index to obtain a weighted score of each of the candidate video clips;

[0196] Sorting all the candidate video clips in descending order according to their weighted scores, and screening out the second video clip;

[0197] The second video segment is recommended to the user.

[0198] by Figure 1 Taking the recommendation index of as an example, instead of matching video 3 or video 4, video 3 and video 4 are first divided into video segments, and then the divided video segments are directly matched.

[0199] The beneficial effect of the above optional implementation is that it significantly improves the accuracy of recommendations because it allows the system to match in-depth to the video clip level of the video content. In some cases, although the entire video may not fully meet the recommendation indicators, the specific video clips within it may highly match the user's interests. This is because video clips, as a subset of video content, can often more accurately reflect a specific scene, emotion or theme in the video, and these elements are exactly what users are concerned about. In contrast, if matching is based only on the entire video, those clips that are highly relevant to the user's interests may be ignored due to the diverse content contained in the video, resulting in omissions in the recommendation process. The segment-level matching method can effectively avoid this situation. By analyzing each segment in the video one by one, it ensures that all content related to the user's interests is taken into consideration, thereby reducing errors and omissions in the recommendation process.

[0200] Further references Figure 8 As an implementation of the methods shown in the above figures, the present disclosure provides an embodiment of an information processing device, which is similar to Figure 1 Corresponding to the method embodiment shown, the device can be specifically implemented in various electronic devices.

[0201] like Figure 8 As shown, the video recommendation device 200 provided in this embodiment includes: a first acquisition unit 201, a second acquisition unit 202, and a recommendation unit 203.

[0202] The first acquisition unit 201 acquires at least one first video clip corresponding to at least one user recommendation behavior of the user; the user recommendation behavior includes at least one of the following: like, forward, comment, bullet screen, and collection;

[0203] A second acquisition unit 202 acquires a second video that matches the at least one user recommendation behavior and the at least one first video segment; the second video comes from the video to be recommended;

[0204] The recommendation unit 203 recommends the second video to the user.

[0205] In this embodiment, the specific processing of the first acquisition unit 201, the second acquisition unit 202, and the recommendation unit 203 and the technical effects thereof can be referred to in Figure 1 The relevant descriptions of step 101, step 102, and step 103 in the corresponding embodiment are not repeated here.

[0206] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0207] Obtaining a first video corresponding to the user recommended behavior;

[0208] Obtaining the time when the behavior recommended by the user occurs; the time when the behavior occurs is before the end time when the user watches the first video;

[0209] According to the first video feature of the first video, the start time and the end time of the first video segment corresponding to the user recommended behavior are obtained; the first video feature includes at least one of the following: video background music feature, video screen feature, video subtitle feature, and video voice feature; the start time of the first video segment is before the time when the behavior occurs;

[0210] Obtain a first video segment from the first video according to the start time of the first video segment and the end time of the first video segment.

[0211] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0212] Acquire a video interval before the behavior occurs according to the first video feature, and acquire a start time of the first video segment according to the video interval;

[0213] A second video interval after the start time of the first video segment is obtained, and an end time of the first video segment is obtained according to the second video interval.

[0214] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0215] Acquire a first candidate segment before the behavior occurs according to the first video feature;

[0216] The first video segment start time and the first video segment end time are acquired according to the candidate segment start time and the candidate segment end time of the first candidate segment.

[0217] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0218] In response to the first video feature including at least two features, obtaining a candidate segment start time and a candidate segment end time corresponding to each feature;

[0219] According to the candidate segment start times of the at least two characteristics, selecting the earliest candidate segment start time as the first video segment start time;

[0220] According to the candidate segment end time of the at least two characteristics, the latest candidate segment end time is selected as the first video segment end time.

[0221] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0222] Obtain a video interval of the first video before and after the start time of the candidate segment and the end time of the candidate segment;

[0223] According to the video interval, the candidate segment start time and the candidate segment end time, the first video segment start time and the first video segment end time are acquired.

[0224] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0225] Extracting basic music features of background music; the basic music features include at least one of the following: volume, pitch, and rhythm;

[0226] Calculating a characteristic curve of the background music according to the basic music characteristics;

[0227] Based on the feature curve, the features of the candidate video background music segments are calculated.

[0228] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0229] Extracting basic picture features of the video picture; the basic picture features include at least one of the following: picture scene, picture object, picture emotional state, and picture color;

[0230] According to the basic picture features, the highlight semantic features of the video picture are calculated as the highlight features of the video picture.

[0231] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0232] The acquisition of the video subtitle features includes: extracting text features of the subtitles; extracting features representing the candidate video segments from the text features as the video subtitle features;

[0233] The text features include at least one of the following: text keyword features, text semantic features, and text sentiment features;

[0234] The acquisition of the video speech feature includes: converting the video speech into text and extracting text features of the text; and extracting features representing the candidate video segments from the text features as the video speech feature.

[0235] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows: the first video feature further includes at least one of the following: a comment feature, a bullet screen feature;

[0236] The acquisition of the comment feature includes: extracting text features of the comment; using features representing the candidate video segments in the text features as the comment feature; or extracting distribution features of the comment as the comment feature;

[0237] The text features include at least one of the following: text keyword features, text semantic features, and text sentiment features;

[0238] The acquisition of the barrage feature includes: extracting text features of the barrage; using features of the candidate video segments represented by the text features as the barrage feature; or extracting distribution features of the barrage feature as the barrage feature.

[0239] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0240] Acquire a first video segment feature of the at least one first video segment as a recommendation indicator for recalling the second video; the recommendation indicator also has an indicator weight;

[0241] The first video clip feature includes at least one of the following: a video clip label, a background music name, a background music label, a background music feature vector, a video scene semantic label, a video object semantic label, a video picture feature vector, a subtitle label, a subtitle feature vector, a comment label, and a comment feature vector;

[0242] In response to the first video segment and the previous first video segment having related recommendation indicators and correlation, the related recommendation indicators are merged, and the recommendation weights of the related recommendation indicators are weighted and merged using the correlation to serve as updated indicator weights; the previous first video segment is another first video segment watched by the user before the first video segment;

[0243] The video to be recommended is recalled according to the recommendation index and the index weight to obtain a second video.

[0244] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0245] According to the play time of the first video clip and / or the recommendation behavior weight of the relevant user recommendation behavior, the indicator weight is weighted to serve as the updated indicator weight;

[0246] The recommended behavior weight is calculated according to the type of the recommended behavior and / or the user's recommended words.

[0247] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0248] The video features of the first video to which the first video clip belongs are obtained as recommendation indicators for recalling the second video; the video features include at least one of the following: video tags, video names, video description keywords, video authors, video author tags, user tags, and authors followed by users.

[0249] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0250] The previous first video segment is iteratively processed to obtain an updated indicator weight.

[0251] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0252] In response to the existence of cross-modality related recommendation indicators, the weighted combination is performed on the cross-modality related recommendation indicators to obtain updated indicator weights.

[0253] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0254] A relevance threshold of the recommendation indicators is set, and the weighted combination is performed on the recommendation indicators whose relevance is higher than the relevance threshold.

[0255] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0256] Obtaining a second video feature of each video to be recommended, where the second video feature corresponds to the recommendation indicator;

[0257] For each video to be recommended, matching is performed with the second video feature according to the recommendation index to obtain a matching score of each recommendation index; weighting is performed on the matching score according to the index weight of the recommendation index to obtain a weighted score of each video to be recommended;

[0258] All the videos to be recommended are sorted in descending order according to their weighted scores, and the second video is obtained by screening.

[0259] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0260] Dividing the second video into video segments to obtain candidate video segments;

[0261] Matching the candidate video segments according to the recommendation index and the recommendation weight to obtain a second video segment;

[0262] The second video segment is recommended to the user.

[0263] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0264] Dividing the second video into video segments to obtain candidate video segments;

[0265] Acquire a second video segment feature of each candidate video segment, where the second video segment feature corresponds to the recommendation index;

[0266] For each of the candidate video clips, matching is performed with the second video clip feature according to the recommendation index to obtain a matching score of each recommendation index; and the matching scores are weighted according to the index weight of the recommendation index to obtain a weighted score of each of the candidate video clips;

[0267] Sorting all the candidate video clips in descending order according to their weighted scores, and screening out the second video clip;

[0268] The second video segment is recommended to the user.

[0269] In some optional implementations of this embodiment, the apparatus 200 is further configured as follows:

[0270] A first acquisition unit is configured to acquire at least one first video clip corresponding to at least one user recommendation behavior of a user; the user recommendation behavior includes at least one of the following: like, forward, comment, bullet screen, and favorite;

[0271] A second acquisition unit is configured to acquire a second video that matches the at least one user recommendation behavior and the at least one first video segment; the second video is from the video to be recommended;

[0272] A recommendation unit is configured to recommend the second video to the user.

[0273] In the technical solution of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0274] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0275] Figure 8A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0276] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0277] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0278] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the video recommendation method. For example, in some embodiments, the video recommendation method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the video recommendation method described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to perform the video recommendation method in any other appropriate manner (e.g., by means of firmware).

[0279] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0280] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable information processing device, so that the program code, when executed by the processor or controller, implements the functions / operations specified in the flow chart and / or block diagram. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0281] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0282] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0283] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an information server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communications network). Examples of communications networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0284] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0285] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0286] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A video recommendation method, characterized in that: The method comprises: Obtain at least one first video clip corresponding to at least one user recommendation behavior of the user; the user recommendation behavior includes at least one of the following: like, forward, comment, bullet screen, and collection; Acquire a second video that matches the at least one user recommendation behavior and the at least one first video segment; the second video is from the video to be recommended; The second video is recommended to the user.

2. The method according to claim 1, characterized in that The obtaining of at least one first video clip corresponding to at least one user recommendation behavior of the user includes: Obtaining a first video corresponding to the user recommended behavior; Obtaining the time when the behavior recommended by the user occurs; the time when the behavior occurs is before the end time when the user watches the first video; According to the first video feature of the first video, the start time and the end time of the first video segment corresponding to the user recommended behavior are obtained; the first video feature includes at least one of the following: video background music feature, video screen feature, video subtitle feature, and video voice feature; the start time of the first video segment is before the time when the behavior occurs; Obtain a first video segment from the first video according to the start time of the first video segment and the end time of the first video segment.

3. The method according to claim 2, characterized in that The acquiring, according to the first video feature of the first video, the first video segment start time and the first video segment end time corresponding to the user recommended behavior includes: Acquire a video interval before the behavior occurs according to the first video feature, and acquire a start time of the first video segment according to the video interval; A second video interval after the start time of the first video segment is obtained, and an end time of the first video segment is obtained according to the second video interval.

4. The method according to claim 2, characterized in that: The acquiring, according to the first video feature of the first video, the first video segment start time and the first video segment end time corresponding to the user recommended behavior includes: Acquire a first candidate segment before the behavior occurs according to the first video feature; According to the candidate segment start time and the candidate segment end time of the first candidate segment, the first video segment start time and the first video segment end time are acquired.

5. The method according to claim 4, characterized in that The acquiring the first video segment start time and the first video segment end time according to the candidate segment start time and the candidate segment end time of the first candidate segment includes: In response to the first video feature including at least two features, obtaining a candidate segment start time and a candidate segment end time corresponding to each feature; According to the candidate segment start times of the at least two characteristics, selecting the earliest candidate segment start time as the first video segment start time; According to the candidate segment end time of the at least two characteristics, the latest candidate segment end time is selected as the first video segment end time.

6. The method according to any one of claims 4-5, characterized in that: The acquiring the first video segment start time and the first video segment end time according to the candidate segment start time and the candidate segment end time of the first candidate segment includes: Obtain a video interval of the first video before and after the candidate segment start time and the candidate segment end time; According to the video interval, the candidate segment start time and the candidate segment end time, the first video segment start time and the first video segment end time are acquired.

7. The method according to claim 2, characterized in that The acquisition of the characteristics of the candidate video background music segments includes: Extracting basic music features of background music; the basic music features include at least one of the following: volume, pitch, and rhythm; Calculating a characteristic curve of the background music according to the basic music characteristics; Based on the feature curve, the features of the candidate video background music segments are calculated.

8. The method according to claim 2, characterized in that: The acquisition of the highlight feature of the video picture includes: Extracting basic picture features of the video picture; the basic picture features include at least one of the following: picture scene, picture object, picture emotional state, and picture color; According to the basic picture features, the highlight semantic features of the video picture are calculated as the highlight features of the video picture.

9. The method according to claim 2, characterized in that: The acquisition of the video subtitle features includes: extracting text features of the subtitles; extracting features representing the candidate video segments from the text features as the video subtitle features; The text features include at least one of the following: text keyword features, text semantic features, and text sentiment features; The acquisition of the video speech feature includes: converting the video speech into text and extracting text features of the text; and extracting features representing the candidate video segments from the text features as the video speech feature.

10. The method according to claim 2, characterized in that The first video feature also includes at least one of the following: a comment feature, a bullet screen feature; The acquisition of the comment feature includes: extracting text features of the comment; using features representing the candidate video segments in the text features as the comment feature; or extracting distribution features of the comment as the comment feature; The text features include at least one of the following: text keyword features, text semantic features, and text sentiment features; The acquisition of the barrage feature includes: extracting text features of the barrage; using features of the candidate video segments represented by the text features as the barrage feature; or extracting distribution features of the barrage feature as the barrage feature.

11. The method according to claim 1, characterized in that The obtaining of a second video matching the at least one user recommended behavior and the at least one first video segment includes: Acquire a first video segment feature of the at least one first video segment as a recommendation indicator for recalling the second video; the recommendation indicator also has an indicator weight; The first video clip feature includes at least one of the following: a video clip label, a background music name, a background music label, a background music feature vector, a video scene semantic label, a video object semantic label, a video picture feature vector, a subtitle label, a subtitle feature vector, a comment label, and a comment feature vector; In response to the first video segment and the previous first video segment having related recommendation indicators and correlation, the related recommendation indicators are merged, and the recommendation weights of the related recommendation indicators are weighted and merged using the correlation to serve as updated indicator weights; the previous first video segment is another first video segment watched by the user before the first video segment; The video to be recommended is recalled according to the recommendation index and the index weight to obtain a second video.

12. The method according to claim 11, characterized in that The obtaining of the first video segment feature of the at least one first video segment as a recommendation indicator for recalling the second video includes: According to the play time of the first video clip and / or the recommendation behavior weight of the relevant user recommendation behavior, the indicator weight is weighted to serve as the updated indicator weight; The recommended behavior weight is calculated according to the type of the recommended behavior and / or the user's recommended words.

13. The method according to claim 11, characterized in that The obtaining of the first video segment feature of the at least one first video segment as a recommendation indicator for recalling the second video includes: The video features of the first video to which the first video clip belongs are obtained as recommendation indicators for recalling the second video; the video features include at least one of the following: video tags, video names, video description keywords, video authors, video author tags, user tags, and authors followed by users.

14. The method according to claim 11, characterized in that Also includes: The previous first video segment is iteratively processed to obtain an updated indicator weight.

15. The method according to claim 11, characterized in that Also includes: In response to the existence of cross-modality related recommendation indicators, the weighted combination is performed on the cross-modality related recommendation indicators to obtain updated indicator weights.

16. The method according to claim 11, characterized in that Also includes: A relevance threshold of the recommendation indicators is set, and the weighted combination is performed on the recommendation indicators whose relevance is higher than the relevance threshold.

17. The method according to claim 11, characterized in that The step of recalling the video to be recommended according to the recommendation index and the index weight to obtain a second video includes: Obtaining a second video feature of each video to be recommended, where the second video feature corresponds to the recommendation indicator; For each video to be recommended, matching is performed with the second video feature according to the recommendation index to obtain a matching score of each recommendation index; weighting is performed on the matching score according to the index weight of the recommendation index to obtain a weighted score of each video to be recommended; All the videos to be recommended are sorted in descending order according to their weighted scores, and the second video is obtained by screening.

18. The method according to claim 17, characterized in that Also includes: Dividing the second video into video segments to obtain candidate video segments; Matching the candidate video segments according to the recommendation index and the recommendation weight to obtain a second video segment; The second video segment is recommended to the user.

19. The method according to claim 17, characterized in that Also includes: Dividing the second video into video segments to obtain candidate video segments; Acquire a second video segment feature of each candidate video segment, where the second video segment feature corresponds to the recommendation index; For each of the candidate video clips, matching is performed with the second video clip feature according to the recommendation index to obtain a matching score of each recommendation index; and the matching scores are weighted according to the index weight of the recommendation index to obtain a weighted score of each of the candidate video clips; Sorting all the candidate video clips in descending order according to their weighted scores, and screening out the second video clip; The second video segment is recommended to the user.

20. A video recommendation device, comprising: A first acquisition unit, acquiring at least one first video clip corresponding to at least one user recommendation behavior of the user; The user recommendation behavior includes at least one of the following: like, forward, comment, bullet screen, and collection; A second acquisition unit, acquiring a second video matching the at least one user recommended behavior and the at least one first video segment; The second video is from the video to be recommended; A recommendation unit is configured to recommend the second video to the user.

21. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 19.

22. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 19.

23. A computer program product, comprising a computer program, which, when executed by a processor, implements the method of any one of claims 1 to 19.

Images