Short video recommendation method and system based on big data

Abstract

The invention provides a short video recommendation method and system based on big data. A short video recommendation system based on big data comprises an operation information acquisition module, an operation information matching module, an information data acquisition module, an information data processing module, a video pushing module and an interested video processing module. According to the invention, the information data acquisition module acquires the related data of the video G, the information data processing module obtains the push video set and the push sequence of the internal videos thereof according to the acquired related data of the video G, and the video push module pushes the internal videos thereof according to the obtained push sequence of the internal videos of the push video set. Therefore, the video recommendation strategy is optimized, more personalized recommendation strategies are formulated for different users, and the satisfaction degree and retention rate of the users can be well improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video recommendation, and in particular to a short video recommendation method and system based on big data. Background Art

[0002] In existing technologies, short video recommendations are mainly based on users' browsing history, likes, comments and other behavioral data, as well as the content features of the video. Since most existing recommendation systems are based on traditional machine learning algorithms and use a relatively single video recommendation method, they are unable to formulate more personalized recommendation strategies for different users, and cannot adapt well to changes in users' interests and needs. It is difficult to provide high-quality recommended content that meets user needs and interests, and it is difficult to improve user satisfaction and retention rates. Summary of the invention

[0003] In response to the above problems, the present application provides a short video recommendation method and system based on big data, which obtains push information according to the acquired video G related data, pushes videos according to the push information, formulates more personalized recommendation strategies for different users, provides high-quality recommended content that meets user needs and interests, and updates the recommendation model in real time to improve user satisfaction and retention rate.

[0004] As one aspect of the present application, a short video recommendation method based on big data is provided, comprising: S1: Obtain the user's operation information on the currently watched video and match it with the interest operation list; S2: If the video successfully matches any one or more operations in the interest operation list, the video is recorded as video G and the process goes to S3; otherwise, the process goes back to S1; S3: Obtain the text information in the video G, where the text information includes the video copy and the comment text, and segment the video copy and the comment text according to the custom dictionary, match the obtained words with the stop word list, and record the unmatched words as feature words d a , by the characteristic word d a Form a feature word set D, and obtain a dictionary vector based on the feature word set D; S4: Based on the dictionary vector, calculate each feature word d through the TF-IDF algorithm a TF-IDF value, and the feature word d whose TF-IDF value is greater than the first threshold β a Record it in the temporary storage set E of feature words, and store the feature words d in the temporary storage set E. a Match with the label database to obtain the label set F of video G 1 {H 1 , H 2 ,⋯H c ⋯、He}, where H e The subscript e indicates that video G has a total of e tags, H c Represents the label set F 1 Any tag in it; S5: According to the feature word temporary storage set E, the feature word d a The matching result with the tag database will be the feature word d that successfully matches the tag database a Record as keywords and classify them. One label corresponds to multiple keywords, and the label H is obtained. c Corresponding keyword set , c = 1, 2, ⋯, e, calculate the label H c The corresponding correlation value J c , J c For each keyword set The average value of the TF-IDF value of the internal keywords, according to the relevance value J c Get the correlation value set K 1 {L e , L e-1 ,⋯L c ⋯、L 1}, L c is the correlation value set K 1 Any correlation value within, correlation value set K 1 By the correlation value J c Sort them from large to small, and get them by the correlation value set K 1 Get the label set corresponding to the arrangement order and update it as label set F 1 {H e , H e-1 ,⋯H c ⋯、H 1}; S6: Select a tag set F from the unwatched video library p {H e , H e-1 ,⋯H c ⋯、H 1 Each tag H in} c Representative video collection M h , h = 1, 2, ⋯, e, the initial value of p is 1, p is used as a number to select the label set, according to the label set F p {H e , H e-1 ,⋯H c ⋯、H 1 The order of labels inside} is from left to right, starting from the corresponding representative video set M h Randomly select a video from the list and record the selected videos in the push video set N in sequence p{ξ e , e-1 ,⋯ξ c ⋯、ξ 1}, delete the pushed video collection N from the unwatched video library p Internal video; S7: Push video set N p {ξ e , e-1 ,⋯ξ c ⋯、ξ 1}The video sequence inside will push video set N from left to right p Push the video in the content to the user; S8: Determine the push video set N p Check whether all internal videos have been pushed. If so, push the video. e The update is recorded as video G, and the process returns to S3; otherwise, the process goes to S9; S9: Continue to push video collection N p If the video is not pushed internally, return to S8.

[0005] More preferably, the step of obtaining the dictionary vector based on the feature word set D specifically includes: The dictionary vector exists in the form of: ; B is the word frequency vector corresponding to the feature word set D, i a The characteristic word d a The number of times it appears in the text information. The subscript A indicates that there are a total of A feature words in the feature word set D.

[0006] More preferably, for the feature word d whose TF-IDF value is greater than the first threshold β in S4 a The steps recorded in the temporary feature word set E specifically include: S4.1: Get the number of all videos in the video library and record it as C, let j= , j is the feature word d a Total number of occurrences; S4.2: Let k = 1, k is used as a number to select feature words; S4.3: Select feature word d k and the number of times it appears in the text information i k , get the video library containing the feature word d k The number of all videos is recorded as ; S4.4: Let d k，TF-IDF = ; S4.5: Determine “d k，TF-IDF >β" is true, if "d k，TF-IDF>β” holds true, then the feature word d k Record in the temporary storage set E of feature words and go to S4.6; if it is "d k，TF-IDF >β” does not hold, go to S4.6; S4.6: Determine whether "k≥A" holds. If so, start the label matching process in S4; if not, assign k+1 to k and return to S4.3.

[0007] More preferably, for S5, the label H is calculated c The corresponding correlation value J c , according to the correlation value J c Get the correlation value set K 1 , the specific steps include: S5.1: Let m = 1, where m is used as a number to select a keyword set; S5.2: Get keyword set The number of internal keywords is recorded as , and the TF-IDF value of the keywords inside it is recorded as P m，n，TF-IDF , n=1, 2, ⋯, , , n is the keyword set The label of the TF-IDF value of the internal keyword, Q m = , Q m That is the label H m The corresponding correlation value, Q m Recorded in collection Inside; S5.3: Determine whether "m≥e" is true. If so, proceed to S5.4. If not, assign m+1 to m and return to S5.2. S5.4: Based on the collection Internal correlation value Q m Sort from large to small to get the correlation value set K 1 {L e , L e-1 ,⋯L c ⋯、L 1}.

[0008] More preferably, for S8 and S9, further optimization of video recommendation is included, and the specific steps are as follows: S8.1: Set the correlation value set K 1 Denoted as the correlation value set W 1 ; S8.2: Pushing video set N pDuring the push process of the video, the user's operation information on the currently watched video is obtained in real time and matched with the interest operation list. If it successfully matches any one or more operations in the interest operation list, the video is placed in the interest video list; S8.3: Determine the set of videos to be pushed N p Check whether all internal videos have been pushed. If so, proceed to S8.4; otherwise, continue to push video set N. p For videos that are not pushed internally, return to S8.2; S8.4: Determine whether the number of videos in the interested video list is zero. If the number of videos in the interested video list is zero, execute steps S6-S7 and return to S8.2; if the number of videos in the interested video list is not zero, proceed to S8.5; S8.5: Update all videos in the list of interested videos to video G, execute steps S3-S5, and obtain the relevance value set K 1 Note: X p , the obtained label set F 1 Update the label set T p {U r , U r-1 ,⋯,U 1}, the letter r represents the tag set T p The total number of internal tags is r, proceed to S8.6; S8.6: Determine whether "p=1" is true. If "p=1" is true, then the correlation value set W 1 Get the corresponding label set F 1 ; If "p=1" does not hold, go to S8.7; S8.7: Select a label set F p and T p , let the label set s p =F p ∩T p , t p =s p ∩F p ,u p =s p ∩T p , respectively from the label set t p and u p Extract the relevance value set W corresponding to its internal label p，1 {V v 、V v-1 ,⋯V w ⋯、V 1} and X p，1 {Y v , Y v-1 ,⋯Yw ⋯、Y 1}, where V w and Y w It represents different relevance values ​​corresponding to the same label, where v represents the label set F p and T p There are v identical labels in total; S8.8: Let Z = v, Z is used as the number to select the correlation value V w and Y w ; S8.9: Select the correlation value V Z and Y Z ,make = ,Will Recorded in the correlation value set W p，3 middle, That is the correlation value V Z and Y Z The average value of S8.10: Determine whether "Z ≤ 1" is true. If "Z ≤ 1" is not true, assign Z-1 to Z and return to S8.9; if "Z ≤ 1" is true, let the label set x p =C Fp t p ,y p =C Tp u p , label set x p The label set F p Remove the tag set t from p Other elements of the internal element, label set y p is the label set T p Remove the tag set u from p The other elements of the internal elements are selected from the label set x p and p Extract the relevance value set W corresponding to its internal label p，2 With X p，2 , let the correlation value set W p，4 =W p，2 ∪X p，2 ∪W p，3 , the correlation value set W p，4 Each correlation value in is sorted from large to small, and the resulting set is recorded as the correlation value set W p+1 {α z , α z-1 ,⋯α δ ⋯、α 1}, the letter z represents the set of correlation values ​​W p+1 There are a total of z correlation values, α δRepresents the set of correlation values ​​W p+1 Any correlation value within, enter S8.11; S8.11: Determine whether "z ≥ θ" is true. If "z ≥ θ" is true, then according to the correlation value α δ Select the correlation value set W from large to small p+1 The first θ elements are updated and recorded as the relevance value set W p+1 {α z , α z-1 ,⋯α μ ⋯、α z-θ}, α μ Represents the set of correlation values ​​W p+1 Any correlation value within, from the correlation value set W p+1 Get the corresponding label set and record it as F p+1 {H z , H z-1 ,⋯H μ ⋯、H z-θ}, go to S8.13; if “z ≥ θ” does not hold, go to S8.12; S8.12: From the correlation value set W p+1 Get its corresponding label set and record it as F p+1 {H z , H z-1 ,⋯H δ ⋯、H 1}, enter S8.13; S8.13: Assign p+1 to p, execute steps S6-S7, delete the video records in the video list of interest, and return to S8.2.

[0009] More preferably, the stop word list is the stop word list of Harbin Institute of Technology, the stop word list of Fudan University, Any one of the stop words.

[0010] As another aspect of the present application, a short video recommendation system based on big data is provided, comprising: An operation information acquisition module is used to obtain the user's operation information on the currently watched video; An operation information matching module, which is used to match the user's operation information on the currently watched video with the interest operation list; An information data acquisition module, which is used to acquire information data; An information data processing module, which is used to process information data; Video push module, the video push module is used to push videos.

[0011] More preferably, it further comprises an interested video processing module, and the interested video processing module is used for processing the interested video.

[0012] The present invention has the following advantages: 1. The present invention obtains relevant data of video G through the information data acquisition module. The information data processing module obtains the push video set and the push order of the videos therein according to the obtained video G relevant data. The video pushing module pushes the videos therein according to the obtained push order of the videos therein, thereby optimizing the video recommendation strategy and formulating more personalized recommendation strategies for different users, which can greatly improve user satisfaction and retention rate.

[0013] 2. The present invention, through the setting of the interested video processing module, can formulate more personalized recommendation strategies for different users, provide high-quality recommended content that meets the needs and interests of users, and update the recommendation model in real time, thereby improving user satisfaction and retention rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0015] Figure 1 This is a flow chart of a short video recommendation method based on big data in Example 1 of the present invention.

[0016] Figure 2 This is a structural diagram of a short video recommendation system based on big data in Example 2 of the present invention. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical scheme and advantages of the present application clearer, some embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. However, it will be appreciated by those skilled in the art that in the various embodiments of the present application, many technical details are proposed in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical scheme claimed in the present application can also be implemented.

[0018] Example 1: See Figure 1 Embodiment 1 of the present invention provides a short video recommendation method based on big data, comprising: S1: Obtain the user's operation information on the currently watched video and match it with the interest operation list. It should be further explained that the user, i.e. the analysis object, can be a new user or an old user; S2: If the video successfully matches any one or more operations in the interest operation list, the video is recorded as video G and the process goes to S3; otherwise, the process goes back to S1; It should be further explained that, the operation information refers to the user's operation browsing history, that is, the user's likes, comments, shares and other operations; among them, the interest operation list includes likes, forwarding, favorites, sharing and following operations.

[0019] By obtaining the user's operation information on the video in real time and analyzing the user's operation information in a timely manner, we can understand the user's feedback on the video content, thereby optimizing the video content and improving the video quality. For example, if a user likes or comments on a video, the push frequency of such videos will be increased and the push strategy will be improved.

[0020] S3: Obtain the text information in the video G, where the text information includes the video copy and the comment text, and segment the video copy and the comment text according to the custom dictionary, match the obtained words with the stop word list, and record the unmatched words as feature words d a , by the characteristic word d a Form a feature word set D, and obtain a dictionary vector based on the feature word set D; Regarding step S3, it is necessary to further explain that the video copy usually attracts the audience with a concise, vivid and interesting language style. The video copy can appear at the beginning, end or middle of the video, or can be published together with the video as an independent text content. It can be a simple text description or a more complex text script or script. The video copy here refers to the text in the video G used to describe, publicize or promote the video content, which is a brief introduction to the theme of the video G, a description of the plot, a comment on the characters or events in the video, etc. The comment text refers to the text description in the comment area of ​​the video G, which is the text content that comments and evaluates the video content. It can be comments on the theme of the video, comments on the characters or events in the video, or comments on the quality and creativity of the video production. The type of custom dictionary is selected manually according to the actual situation of video G. The following are several custom dictionaries to choose from: Huicong Dictionary, Dictionary, stuttering word segmentation dictionary and Dictionary: Huicong Dictionary contains a large number of common words and phrases, which is suitable for general Chinese text segmentation needs; dictionary, is a The algorithmic Chinese word segmentation tool provides an extensible word segmentation dictionary, and users can add, modify and delete words in the dictionary as needed; Jieba word segmentation dictionary, Jieba word segmentation is a Chinese word segmentation tool based on word graph, which provides a customizable word segmentation dictionary, and users can add, modify and delete words in the dictionary; The dictionary provides an expandable word segmentation dictionary. Users can add, modify and delete words in the dictionary as needed. As mentioned above, it is better to use a custom dictionary that can add, modify and delete words in the dictionary as needed, so the selection object of the custom dictionary can be Dictionary, stuttering word segmentation dictionary and Any one in the dictionary; Among them, the stop word lists are the stop word list of Harbin Institute of Technology, the stop word list of Fudan University, Any one in the stop word list; using the stop word list can remove stop words and reduce text noise; due to the large number of stop words and high frequency of occurrence, removing stop words can greatly reduce the number of words that need to be processed, making important words more prominent, thereby saving storage space and computing resources and improving the speed of natural language processing.

[0021] S4: Based on the dictionary vector, calculate each feature word d through the TF-IDF algorithm a TF-IDF value, and the feature word d whose TF-IDF value is greater than the first threshold β a Record it in the temporary storage set E of feature words, and store the feature words d in the temporary storage set E. a Match with the label database to obtain the label set F of video G 1 {H 1 , H 2 ,⋯H c ⋯、H e}, where H e The subscript e indicates that video G has a total of e tags, H c Represents the label set F 1 Any tag in it; For step S4, it is necessary to further explain that the TF-IDF algorithm is a commonly used weighted technology for information retrieval and text mining. The principle of the TF-IDF algorithm is: if a word appears in an article with a high frequency TF and rarely appears in other articles, it is considered that this word or phrase has a good ability to distinguish categories and is suitable for classification. The TF-IDF algorithm consists of two parts, TF and IDF, where TF refers to word frequency, which refers to the number of times a word appears in an article, and IDF refers to inverse document frequency, which refers to the total number of documents containing the word divided by the number of documents containing the word; here we use the TF-IDF algorithm to distinguish the categories of different videos; The first threshold β is set manually according to the actual situation of the video G; Among them, the label database can perform custom annotations on the data to facilitate subsequent data query and classification; the label database is implemented by aggregating data with the same label to form a label pool, each of which contains one or more labels, and each label corresponds to one or more keywords.

[0022] By using the tag database, it is easier to match feature words a The corresponding labels provide a flexible and effective way to retrieve, match and classify data; a In the process of matching with corresponding tags, relevant data can be quickly located and found, which helps to improve the convenience of data utilization.

[0023] S5: According to the feature word temporary storage set E, the feature word d a The matching result with the tag database will be the feature word d that successfully matches the tag database a Record as keywords and classify them. One label corresponds to multiple keywords, and the label H is obtained. c Corresponding keyword set , c = 1, 2, ⋯, e, calculate the label H c The corresponding correlation value J c , J c For each keyword set The average value of the TF-IDF value of the internal keywords, according to the relevance value J c Get the correlation value set K 1 {L e , L e-1 ,⋯L c ⋯、L 1}, L c is the correlation value set K 1 Any correlation value within, correlation value set K 1 By the correlation value J c Sort them from large to small, and get them by the correlation value set K 1 Get the label set corresponding to the arrangement order and update it as label set F 1 {H e , H e-1 ,⋯H c ⋯、H 1}; For step S5, it is necessary to further explain that by setting keyword conditions and keyword classification steps, key information related to the tag can be quickly and effectively extracted, which is convenient for quickly finding the required data when processing the data later, improving the accuracy of information retrieval and the efficiency of analysis, promoting information management and search engine optimization, and providing feature words in natural language processing. a Classification and characteristic words a Analysis and other functions.

[0024] S6: Select a tag set F from the unwatched video library p {H e , H e-1 ,⋯H c ⋯、H 1 Each tag H in} c Representative video collection M h , h = 1, 2, ⋯, e, the initial value of p is 1, p is used as a number to select the label set, according to the label set F p {H e , H e-1 ,⋯H c ⋯、H 1 The order of labels inside} is from left to right, starting from the corresponding representative video set M h Randomly select a video from the list and record the selected videos in the push video set N in sequence p {ξ e , e-1 ,⋯ξ c ⋯、ξ 1}, delete the pushed video collection N from the unwatched video library p Internal video; S7: Push video set N p {ξ e , e-1 ,⋯ξ c ⋯、ξ 1}The video sequence inside will push video set N from left to right p Push the video in the content to the user; As for the order of video push, videos randomly selected from the video set representing the corresponding tags are pushed in sequence according to the relevance of the tags. This can improve users' acceptance and interest in pushed videos. Videos with high relevance to tags that users are interested in are pushed first, which can meet users' interest needs, improve users' acceptance and satisfaction with pushed content, and increase users' interest and curiosity in pushed content, thereby increasing click-through rate and viewing rate.

[0025] S8: Determine the push video set N p Check whether all internal videos have been pushed. If so, push the video. eUpdates are recorded as videos. It needs to be further explained that the video will be pushed e The update is recorded as video G, that is, the selected video corresponding to the label with the maximum correlation value is updated as video G, and return to S3; otherwise, enter S9; The selection of video G can improve the accuracy and pertinence of the analyzed video, ensure that the analyzed video is more relevant to the specific tag, thereby improving the accuracy and pertinence of the analysis, facilitating in-depth analysis of the topic, and improving the quality and depth of the analyzed video. By continuously updating the analyzed video and selecting the video with the greatest relevance to the latest tag, it can be ensured that the quality and depth of the analyzed video are continuously improved, providing more comprehensive and in-depth analysis results.

[0026] S9: Continue to push video collection N p If the video is not pushed internally, return to S8.

[0027] For new users, adopting a new video push method and continuously pushing videos with topics or tags that new users are interested in can help us better understand the interests of new users, thereby more accurately pushing videos that new users are interested in, increasing new users' trust in the video push platform, and improving new user retention rates.

[0028] For old users, the new video push method can recommend more videos related to their interests, which will help them discover new content, more interesting content, and guide them to further explore the video push platform. By more accurately recommending videos that match their interests, we can attract old users and increase their engagement. At the same time, recommending videos related to their interests can also promote interaction and communication between old users and the community, increase their stickiness to the platform, and improve their satisfaction. At the same time, recommending videos related to their interests can also help them find what they want faster and improve their experience. We can better adapt to changes in the interests and needs of old users, provide better services, and improve their retention rate.

[0029] The steps of obtaining the dictionary vector based on the feature word set D specifically include: The dictionary vector exists in the form of: ; B is the word frequency vector corresponding to the feature word set D, i a The characteristic word d a The number of times it appears in the text information, the subscript A indicates that there are a total of A feature words in the feature word set D. It should be further explained that the dictionary vector is composed of the feature word set D in the video G and the word frequency vector B corresponding to the feature word set D.

[0030] The word frequency vector B can be used in multiple tasks and can represent the feature word d aThe connections and differences between them can help us better "read" words and sentences; the word frequency vector B is a fixed-length continuous dense vector, which takes up less resources and has a fast calculation speed; if these high-dimensional words are reduced in dimension and projected onto the coordinates, we can see similar feature words d a Will tend to be grouped together, irrelevant feature words d a will be far away, reflecting the characteristic word d in the word frequency vector B a The connection and difference between the characteristic word d a The distribution is more intuitive.

[0031] For the feature word d whose TF-IDF value is greater than the first threshold β in S4 a The steps recorded in the temporary feature word set E specifically include: S4.1: Get the number of all videos in the video library and record it as C, let j= , j is the feature word d a Total number of occurrences; S4.2: Let k = 1, k is used as a number to select feature words; S4.3: Select feature word d k and the number of times it appears in the text information i k , get the video library containing the feature word d k The number of all videos is recorded as ; S4.4: Let d k，TF-IDF = ; S4.5: Determine “d k，TF-IDF >β" is true, if "d k，TF-IDF >β” holds true, then the feature word d k Record in the temporary storage set E of feature words and go to S4.6; if it is "d k，TF-IDF >β” does not hold, go to S4.6; S4.6: Determine whether "k≥A" holds. If so, start the label matching process in S4; if not, assign k+1 to k and return to S4.3.

[0032] By setting the first threshold β to filter out feature words with a large word frequency, the screening accuracy can be improved. The TF-IDF algorithm is a commonly used word weight calculation method. By calculating the frequency and importance of words in the document, words related to the topic can be more accurately filtered out; some words with high frequency but irrelevant to the topic can be filtered out to reduce text noise interference; by filtering out words related to the topic, the amount of data for subsequent processing can be reduced and processing efficiency can be improved; the user experience can be improved, and filtering out words related to the topic can provide more accurate results and improve the user experience; the consumption of computing resources can be reduced, and filtering out words related to the topic can reduce unnecessary computing resources and time consumption.

[0033] For S5, calculate the label H c The corresponding correlation value J c , according to the correlation value J c Get the correlation value set K 1 , the specific steps include: S5.1: Let m = 1, where m is used as a number to select a keyword set; S5.2: Get keyword set The number of internal keywords is recorded as , and the TF-IDF value of the keywords inside it is recorded as P m，n，TF-IDF , n=1, 2, ⋯, , , n is the keyword set The label of the TF-IDF value of the internal keyword, Q m = , Q m That is the label H m The corresponding correlation value, Q m Recorded in collection It needs to be further explained that Q m The calculation method is as follows: for the keyword set , calculate the sum of the TF-IDF values ​​of the keywords inside it Divide by the keyword set The number of internal keywords ; S5.3: Determine whether "m≥e" is true. If so, proceed to S5.4. If not, assign m+1 to m and return to S5.2. S5.4: Based on the collection Internal correlation value Q m Sort from large to small to get the correlation value set K 1 {L e , L e-1 ,⋯L c⋯、L 1}.

[0034] By calculating the relevance value of a tag, we can more accurately measure the degree of association between the tag and the target information, thereby optimizing the accuracy and efficiency of information classification; we can enhance the accuracy of the recommendation system and more accurately measure the user's preference for different items, thereby enhancing the accuracy of the recommendation system; in natural language processing tasks, calculating the relevance value of a tag can help the model understand the meaning and context of the text more accurately, thereby improving the results of natural language processing tasks.

[0035] For S8 and S9, video recommendations are further optimized. The specific steps are as follows: It should be further explained that after the first execution of operations S1-S7 is completed, operations S8.1-S8.13 are started.

[0036] S8.1: Set the correlation value set K 1 Denoted as the correlation value set W 1 , it needs to be further explained that the correlation value set K 1 The relevance value set data generated after the first execution of operations S1-S7 is completed, and its corresponding tag set and its relevance value set data are stored together; S8.2: Pushing video set N p During the push process of the video, the user's operation information on the currently watched video is obtained in real time and matched with the interest operation list. If it successfully matches any one or more operations in the interest operation list, the video is placed in the interest video list; By analyzing the collected videos of interest, formulating more personalized recommendation strategies for different users, providing high-quality recommended content that meets user needs and interests, updating the recommendation model in real time, and improving the short video recommendation strategy, we can more accurately discover video content of interest, thereby accurately recommending short videos that users are interested in and increasing user stickiness to the platform; in the process of short video recommendation, some videos related to topics of interest may be recommended, so that more related topics and trends can be discovered, thereby improving user satisfaction and retention rate.

[0037] S8.3: Determine the set of videos to be pushed N p Check whether all internal videos have been pushed. If so, proceed to S8.4; otherwise, continue to push video set N. p If the video is not pushed internally, return to S8.2; S8.4: Determine whether the number of videos in the interested video list is zero. If the number of videos in the interested video list is zero, execute steps S6-S7 and return to S8.2; if the number of videos in the interested video list is not zero, proceed to S8.5; It should be further explained that the method for handling the number of videos in the list of interested videos is to continue to push videos based on the current latest tag set. Since the tag relevance of each video is very different, as the videos are pushed continuously, the video types that users like can always be found, thereby updating the tag relevance set.

[0038] The way of handling the number of videos in the list of interested videos can enhance user engagement. For example, if a user is interested in videos with a specific tag, and as the number of videos watched by the user gradually increases, the user's engagement with the tag may increase, so that the user continues to pay attention to more videos with similar tags, and may interact and share more content with the tag; it can expand the user's horizons. By pushing more videos with similar tags, it can help users understand all aspects of the tag more comprehensively, which helps users broaden their horizons, discover new points of interest and knowledge, and enhance their understanding of related fields; it can improve user satisfaction, so that users will be more willing to accept more video recommendations with similar tags, and are more willing to spend time watching and exploring; it can promote interaction and communication between users and the community, so that users will have a deeper interest in the tag and participate in community discussions, so that users can exchange ideas and opinions with other users and obtain more information and knowledge.

[0039] S8.5: Update all videos in the list of interested videos to video G, execute steps S3-S5, and obtain the relevance value set K 1 Note: X p , the obtained label set F 1 Update the label set T p {U r , U r-1 ,⋯,U 1}, the letter r represents the tag set T p The total number of internal tags is r, proceed to S8.6; Regarding step S8.5, it needs to be further explained that when executing steps S3-S5, all text information of the videos G is obtained at the same time and then analyzed and processed.

[0040] S8.6: Determine whether "p=1" is true. If "p=1" is true, then the correlation value set W 1 Get the corresponding label set F 1If "p=1" is not true, then go to S8.7. It should be further explained that the label set F is obtained here. 1 This is to facilitate finding the corresponding label set F when step S8.7 is executed for the first time 1 ; S8.7: Select a label set F p and T p , let the label set s p =F p ∩T p , t p =s p ∩F p ,u p =s p ∩T p , respectively from the label set t p and u p Extract the relevance value set W corresponding to its internal label p，1 {V v 、V v-1 ,⋯V w ⋯、V 1} and X p，1 {Y v , Y v-1 ,⋯Y w ⋯、Y 1}, where V w and Y w It represents different relevance values ​​corresponding to the same label, where v represents the label set F p and T p There are v identical labels in total; It should be further explained that the label set s p The label set F p and T p The common elements within p For the tag set s p and F p The common element inside, u p is the tag set s p and T p Common elements of the interior.

[0041] S8.8: Let Z = v, Z is used as the number to select the correlation value V w and Y w ; S8.9: Select the correlation value V Z and Y Z ,make = ,Will Recorded in the correlation value set W p，3 middle, That is the correlation value V Z and Y Z The average value of S8.10: Determine whether "Z ≤ 1" is true. If "Z ≤ 1" is not true, assign Z-1 to Z and return to S8.9; if "Z ≤ 1" is true, let the label set x p =C Fp t p ,y p =C Tp u p , label set x p The label set F p Remove the tag set t from p Other elements of the internal element, label set y p is the label set T p Remove the tag set u from p The other elements of the internal elements are selected from the label set x p and p Extract the relevance value set W corresponding to its internal label p，2 With X p，2 , let the correlation value set W p，4 =W p，2 ∪X p，2 ∪W p，3 , the correlation value set W p，4 Each correlation value in is sorted from large to small, and the resulting set is recorded as the correlation value set W p+1 {α z , α z-1 ,⋯α δ ⋯、α 1}, the letter z represents the set of correlation values ​​W p+1 There are a total of z correlation values, α δ Represents the set of correlation values ​​W p+1 Any correlation value within, enter S8.11; Regarding step S8.10, it is necessary to further explain that for the newly obtained correlation value set W p，3 and the label set F p With label set T p The set of correlation values ​​of different elements in W p，2 and X p，2 , for the set of correlation values ​​W p，3 , W p，2 and X p，2 Find the union and get the correlation value set W p+1 .

[0042] S8.11: Determine whether "z ≥ θ" is true. If "z ≥ θ" is true, then according to the correlation value α δSelect the correlation value set W from large to small p+1 The first θ elements are updated and recorded as the relevance value set W p+1 {α z , α z-1 ,⋯α μ ⋯、α z-θ}, α μ Represents the set of correlation values ​​W p+1 Any correlation value within, from the correlation value set W p+1 Get the corresponding label set and record it as F p+1 {H z , H z-1 ,⋯H μ ⋯、H z-θ}, go to S8.13; if “z ≥ θ” does not hold, go to S8.12; According to the correlation value α δ Select the correlation value set W from large to small p+1 The first θ elements can effectively prevent the situation where too many tags lead to an increase in the number of videos pushed to users. By selecting tags that are more relevant to user interests, we can more accurately predict user interests and provide content that better meets user needs, thereby improving recommendation accuracy; it can reduce redundant recommendations, thereby improving user satisfaction; by reducing the recommendation of videos that do not match user interests, the user experience can be improved; and it reduces system resource consumption. Although users may watch more videos, if these videos are not relevant, it will waste the system's computing resources and network bandwidth. By selecting tags with larger relevance values, we can use system resources more effectively.

[0043] S8.12: From the correlation value set W p+1 Get its corresponding label set and record it as F p+1 {H z , H z-1 ,⋯H δ ⋯、H 1}, enter S8.13; S8.13: Assign p+1 to p, execute steps S6-S7, delete the video records in the video list of interest, and return to S8.2.

[0044] The present invention obtains the relevant data of video G through the information data acquisition module, and the information data processing module obtains the push video set and the push order of the videos therein according to the obtained video G relevant data. The video pushing module pushes the internal videos of the push video set according to the obtained push order of the videos therein, thereby optimizing the video recommendation strategy and formulating more personalized recommendation strategies for different users, which can well improve user satisfaction and retention rate.

[0045] The present invention, through the provision of an interested video processing module, can formulate more personalized recommendation strategies for different users, provide high-quality recommended content that meets user needs and interests, and update the recommendation model in real time, thereby improving user satisfaction and retention rate.

[0046] Example 2: See Figure 2 Embodiment 2 of the present invention provides a short video recommendation system based on big data, including: An operation information acquisition module is used to obtain the user's operation information on the currently watched video; An operation information matching module, which is used to match the user's operation information on the currently watched video with the interest operation list; An information data acquisition module, which is used to acquire information data; An information data processing module, which is used to process information data; Video push module, the video push module is used to push videos.

[0047] It should be further explained that, an interested video processing module is also included, and the interested video processing module is used to process the interested video.

[0048] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention. Parts not described in detail in this specification belong to the prior art known to those skilled in the art.

Claims

1. A short video recommendation method based on big data, characterized in that: include: S1: Obtain the user's operation information on the currently watched video and match it with the interest operation list; S2: If the video successfully matches any one or more operations in the interest operation list, the video is recorded as video G and the process goes to S3; otherwise, the process goes back to S1; S3: Obtain the text information in the video G, where the text information includes the video copy and the comment text, and segment the video copy and the comment text according to the custom dictionary, match the obtained words with the stop word list, and record the unmatched words as feature words d a , by the characteristic word d a Form a feature word set D, and obtain a dictionary vector based on the feature word set D; S4: Based on the dictionary vector, calculate each feature word d through the TF-IDF algorithm a TF-IDF value, and the feature word d whose TF-IDF value is greater than the first threshold β a Record it in the temporary storage set E of feature words, and store the feature words d in the temporary storage set E. a Match with the label database to obtain the label set F1{H1, H2, ⋯H c ⋯、H e }, where H e The subscript e indicates that video G has a total of e tags, H c Represents any label in the label set F1; S5: According to the feature word temporary storage set E, the feature word d a The matching result with the tag database will be the feature word d that successfully matches the tag database a Record as keywords and classify them. One label corresponds to multiple keywords, and the label H is obtained. c Corresponding keyword set , c = 1, 2, ⋯, e, calculate the label H c The corresponding correlation value J c , J c For each keyword set The average value of the TF-IDF value of the internal keywords, according to the relevance value J c Get the correlation value set K1{L e , L e-1 ,⋯L c ⋯、L1},L c is any correlation value in the correlation value set K1, and the correlation value set K1 consists of the correlation value J c Sort them in descending order, and get the label set corresponding to the arrangement order from the relevance value set K1, and update it as the label set F1{H e , H e-1 ,⋯H c ⋯、H1}; S6: Select a tag set F from the unwatched video library p {H e , H e-1 ,⋯H c Each label H in ⋯、H1} c Representative video collection M h , h = 1, 2, ⋯, e, the initial value of p is 1, p is used as a number to select the label set, according to the label set F p {H e , H e-1 ,⋯H c The order of labels in ⋯、H1} is from left to right, starting from the corresponding representative video set M h Randomly select a video from the list and record the selected videos in the push video set N in sequence p {ξ e , e-1 ,⋯ξ c ⋯,ξ1}, delete the pushed video set N from the unwatched video library p Internal video; S7: Push video set N p {ξ e , e-1 ,⋯ξ c The order of the videos in ⋯,ξ1} is from left to right, and the video set N is pushed. p Push the video in the content to the user; S8: Determine the set of videos to be pushed N p Check whether all internal videos have been pushed. If so, push the video. e The update is recorded as video G and returned to S3; Otherwise, go to S9; S9: Continue to push video collection N p If the video is not pushed internally, return to S8.

2. A short video recommendation method based on big data according to claim 1, characterized in that: The steps of obtaining the dictionary vector based on the feature word set D specifically include: The dictionary vector exists in the form of: ; B is the word frequency vector corresponding to the feature word set D, i a The characteristic word d a The number of times it appears in the text information. The subscript A indicates that there are a total of A feature words in the feature word set D.

3. The short video recommendation method based on big data according to claim 2, characterized in that: For the feature word d whose TF-IDF value is greater than the first threshold β in S4 a The steps recorded in the temporary feature word set E specifically include: S4.1: Get the number of all videos in the video library and record it as C, let j= , j is the feature word d a Total number of occurrences; S4.2: Let k = 1, k is used as a number to select feature words; S4.3: Select feature word d k and the number of times it appears in the text information i k , get the video library containing the feature word d k The number of all videos is recorded as ; S4.4: Order k，TF-IDF = ; S4.5: Determine "d k，TF-IDF >β" is true, if "d k，TF-IDF >β” holds true, then the feature word d k Record in the temporary storage set E of feature words and go to S4.6; if "d k，TF-IDF >β” does not hold, go to S4.6; S4.6: Determine whether "k≥A" holds. If so, start the label matching process in S4; if not, assign k+1 to k and return to S4.

3.

4. The short video recommendation method based on big data according to claim 3, characterized in that: For S5, calculate the label H c The corresponding correlation value J c , according to the correlation value J c The correlation value set K1 is obtained, and the specific steps include: S5.1: Let m = 1, where m is used as a number to select a keyword set; S5.2: Get keyword set The number of internal keywords is recorded as , and the TF-IDF value of the keywords inside it is recorded as P m，n，TF-IDF , n=1, 2, ⋯, , , n is the keyword set The label of the TF-IDF value of the internal keyword, Q m = , Q m That is the label H m The corresponding correlation value, Q m Recorded in collection Inside; S5.3: Determine whether "m≥e" is true. If so, proceed to S5.4; if not, assign m+1 to m and return to S5.

2. S5.4: Based on the collection Internal correlation value Q m Sort them from large to small to get the correlation value set K1{L e , L e-1 ,⋯L c ⋯, L1}.

5. The short video recommendation method based on big data according to claim 4, characterized in that: For S8 and S9, video recommendations are further optimized. The specific steps are as follows: S8.1: record the correlation value set K1 as the correlation value set W1; S8.2: Pushing video set N p During the push process of the video, the user's operation information on the currently watched video is obtained in real time and matched with the interest operation list. If it successfully matches any one or more operations in the interest operation list, the video is placed in the interest video list; S8.3: Determine the set of videos to be pushed N p Check whether all internal videos have been pushed. If so, proceed to S8.4; otherwise, continue to push video set N. p If the video is not pushed internally, return to S8.2; S8.4: Determine whether the number of videos in the interested video list is zero. If the number of videos in the interested video list is zero, execute steps S6-S7 and return to S8.2; if the number of videos in the interested video list is not zero, proceed to S8.5; S8.5: Update all videos in the list of interested videos as video G, execute steps S3-S5, and record the obtained relevance value set K1 as X p , update the obtained label set F1 to label set T p {U r , U r-1 ,⋯,U1}, the letter r represents the label set T p The total number of internal tags is r, proceed to S8.6; S8.6: Determine whether "p=1" is true. If "p=1" is true, obtain the corresponding label set F1 from the correlation value set W1; if "p=1" is not true, enter S8.7; S8.7: Select a label set F p and T p , let the label set s p =F p ∩T p , t p =s p ∩F p ,u p =s p ∩T p , respectively from the label set t p and u p Extract the relevance value set W corresponding to its internal label p，1 {V v 、V v-1 ,⋯V w ⋯、V1} and X p，1 {Y v , Y v-1 ,⋯Y w ⋯, Y1}, where V w and Y w It represents different relevance values ​​corresponding to the same label, where v represents the label set F p and T p There are v identical labels in total; S8.8: Let Z = v, Z is used as the number to select the correlation value V w and Y w ; S8.9: Select the correlation value V Z and Y Z ,make = ,Will Recorded in the correlation value set W p，3 middle, That is the correlation value V Z and Y Z The average value of S8.10: Determine whether "Z ≤ 1" holds. If "Z ≤ 1" does not hold, assign Z-1 to Z and return to S8.9; if "Z ≤ 1" holds, set the label set x p =C Fp t p ,y p =C Tp u p , label set x p The label set F p Remove the tag set t from p Other elements of the internal element, label set y p is the label set T p Remove the tag set u from p The other elements of the internal elements are selected from the label set x p and p Extract the relevance value set W corresponding to its internal label p，2 With X p，2 , let the correlation value set W p，4 =W p，2 ∪X p，2 ∪W p，3 , the correlation value set W p，4 Each correlation value in is sorted from large to small, and the resulting set is recorded as the correlation value set W p+1 {α z , α z-1 ,⋯α δ ⋯、α1}, the letter z represents the set of correlation values ​​W p+1 There are a total of z correlation values, α δ Represents the set of correlation values ​​W p+1 Any correlation value within, enter S8.11; S8.11: Determine whether "z≥θ" is true. If "z≥θ" is true, then according to the correlation value α δ Select the correlation value set W from large to small p+1 The first θ elements are updated and recorded as the relevance value set W p+1 {α z , α z-1 ,⋯α μ ⋯、α z-θ }, α μ Represents the set of correlation values ​​W p+1 Any correlation value within, from the correlation value set W p+1 Get the corresponding label set and record it as F p+1 {H z , H z-1 ,⋯H μ ⋯、H z-θ }, go to S8.13; if "z≥θ" does not hold, go to S8.12; S8.12: From the correlation value set W p+1 Get its corresponding label set and record it as F p+1 {H z , H z-1 ,⋯H δ ⋯, H1}, enter S8.13; S8.13: Assign p+1 to p, execute steps S6-S7, delete the video records in the video list of interest, and return to S8.

2.

6. The short video recommendation method based on big data according to claim 5, characterized in that: include: The stop word lists are the stop word list of Harbin Institute of Technology, the stop word list of Fudan University, Any one of the stop words.

7. A short video recommendation system based on big data, characterized in that: The system applies a short video recommendation method based on big data as described in any one of claims 1 to 6, including: An operation information acquisition module is used to obtain the user's operation information on the currently watched video; An operation information matching module, which is used to match the user's operation information on the currently watched video with the interest operation list; An information data acquisition module, which is used to acquire information data; An information data processing module, which is used to process information data; Video push module, the video push module is used to push videos.

8. The short video recommendation system based on big data according to claim 7, characterized in that: Also includes: The interested video processing module is used to process the interested video.

Images