A short video recommendation method and system based on big data

Abstract

The present invention provides a short video recommendation method and system based on big data. A short video recommendation system based on big data includes: an operation information acquisition module, an operation information matching module, an information data acquisition module, an information data processing module, a video push module, and an interested video processing module. The present invention obtains relevant data of video G through the information data acquisition module. The information data processing module obtains a push video set and the order in which the videos within the set are pushed based on the obtained data related to video G. The video push module pushes the videos within the set based on the obtained order in which the videos are pushed. This optimizes the video recommendation strategy and develops more personalized recommendation strategies for different users, which can significantly improve user satisfaction and retention rates.

Description

Technical Field

[0001] The present invention relates to the technical field of video recommendation, and in particular to a short video recommendation method and system based on big data. Background Art

[0002] In existing technologies, short video recommendations are mainly based on users' browsing history, likes, comments and other behavioral data, as well as the content features of the video. Since most existing recommendation systems are based on traditional machine learning algorithms and adopt a relatively single video recommendation method, they are unable to formulate more personalized recommendation strategies for different users, cannot adapt well to changes in users' interests and needs, and are difficult to provide high-quality recommended content that meets users' needs and interests, making it difficult to improve user satisfaction and retention rates. Summary of the Invention

[0003] In response to the above problems, the present application provides a short video recommendation method and system based on big data, which obtains push information based on the acquired video G related data, pushes videos based on the push information, formulates more personalized recommendation strategies for different users, provides high-quality recommended content that meets user needs and interests, and updates the recommendation model in real time to improve user satisfaction and retention rate.

[0004] As one aspect of the present application, a short video recommendation method based on big data is provided, comprising:

[0005] S1: Obtain the user's operation information on the currently watched video and match it with the list of interested operations;

[0006] S2: If the video successfully matches any one or more operations in the interest operation list, the video is recorded as video G and proceeds to S3; otherwise, return to S1;

[0007] S3: Get the text information in the video G, which includes the video copy and comment text, and segment the video copy and comment text according to the custom dictionary, match the obtained words with the stop word list, and record the unmatched words as feature words d a , by the feature word d a Form a feature word set D, and obtain a dictionary vector based on the feature word set D;

[0008] S4: Based on the dictionary vector, calculate each feature word d using the TF-IDF algorithm a TF-IDF value, the feature word d whose TF-IDF value is greater than the first threshold β a Record it in the temporary storage set E of feature words, and store the feature words d in the temporary storage set E. a Match with the label database to obtain the label set F1{H1, H2, ⋯Hc ⋯、H e}, where H e The subscript e indicates that the video G has a total of e labels, H c Represents any label in the label set F1;

[0009] S5: According to the feature word temporary storage set E, the feature word d a The matching result with the tag database will be the feature word d that successfully matches the tag database a Record as keywords and classify them. One label corresponds to multiple keywords, and the label H is obtained. c Corresponding keyword set , c=1,2,⋯,e, calculate the label H c The corresponding correlation value J c , J c For each keyword set The average value of the TF-IDF value of the internal keywords, according to the relevance value J c Get the correlation value set K1{L e , L e-1 、⋯L c ⋯、L1},L c is any correlation value in the correlation value set K1, and the correlation value set K1 is composed of the correlation value J c Sort them in descending order, and get the tag set corresponding to the order from the relevance value set K1, and update it as the tag set F1{H e 、H e-1 、⋯H c ⋯、H1};

[0010] S6: Select tag set F from unwatched video library p {H e 、H e-1 、⋯H c Each tag H in ⋯、H1} c Representative video collection M h , h=1, 2, ⋯, e, the initial value of p is 1, p is used as a number to select the label set, according to the label set F p {H e 、H e-1 、⋯H c The order of labels inside ⋯、H1} is from left to right, starting from the corresponding representative video set M h Randomly select a video and record the selected video in the push video set N p {ξ e ,ξ e-1 、⋯ξ c ⋯,ξ1}, delete the pushed video set N from the unwatched video libraryp Internal video;

[0011] S7: Push video set N p {ξ e ,ξ e-1 、⋯ξ c The order of the videos in ⋯、ξ1} will push the video set N from left to right. p Push the video to the user;

[0012] S8: Determine the push video set N p Check if all internal videos have been pushed. If so, push the video. e The update is recorded as video G and returns to S3; otherwise, enter S9;

[0013] S9: Continue to push video collection N p If the video is not pushed internally, return to S8.

[0014] More preferably, the step of obtaining the dictionary vector based on the feature word set D specifically includes:

[0015] The dictionary vector exists in the form of: ; B is the word frequency vector corresponding to the feature word set D, i a is the feature word d a The number of times it appears in the text information. The subscript A indicates that there are a total of A feature words in the feature word set D.

[0016] More preferably, for the feature word d whose TF-IDF value is greater than the first threshold β in S4, a The steps of recording in the temporary feature word set E specifically include:

[0017] S4.1: Get the number of all videos in the video library and record it as C, let j= , j is the feature word d a Total number of occurrences;

[0018] S4.2: Let k = 1, k is used as a number to select feature words;

[0019] S4.3: Select feature word d k and its occurrence times in the text information i k , get the video library containing the feature word d k The number of all videos is recorded as ;

[0020] S4.4: Let d k，TF-IDF = ;

[0021] S4.5: Determine “d k，TF-IDF>β" is true, if "d k，TF-IDF >β” holds true, then the feature word d k Record in the temporary feature word set E and go to S4.6; if "d k，TF-IDF >β” does not hold, go to S4.6;

[0022] S4.6: Determine whether "k ≥ A" holds. If so, start the label matching process in S4; if not, assign k + 1 to k and return to S4.3.

[0023] More preferably, for S5, calculate the label H c The corresponding correlation value J c , according to the correlation value J c Obtaining the correlation value set K1, the specific steps include:

[0024] S5.1: Let m = 1, where m is used as the number to select the keyword set;

[0025] S5.2: Get keyword set The number of internal keywords is recorded as , the TF-IDF value of the keywords inside it is recorded as P m，n，TF-IDF , n=1, 2, ⋯, 、 , n is the keyword set The label of the TF-IDF value of the internal keyword, Q m = , Q m That is the label H m The corresponding correlation value, Q m Recorded in collection Inside;

[0026] S5.3: Determine whether "m ≥ e" holds. If so, proceed to S5.4. If not, assign m + 1 to m and return to S5.2.

[0027] S5.4: Based on the collection Intracorrelation value Q m Sort from large to small and get the correlation value set K1{L e , L e-1 、⋯L c ⋯、L1}.

[0028] More preferably, for S8 and S9, further optimization of video recommendation is included, and the specific steps are as follows:

[0029] S8.1: Record the correlation value set K1 as the correlation value set W1;

[0030] S8.2: Pushing video set N p During the push process of the video, the user's operation information on the currently watched video is obtained in real time and matched with the interest operation list. If it successfully matches any one or more operations in the interest operation list, the video will be placed in the interest video list;

[0031] S8.3: Determine the push video set N p Check whether all internal videos have been pushed. If so, proceed to S8.4. Otherwise, continue pushing video set N. p If the video is not pushed internally, return to S8.2;

[0032] S8.4: Determine whether the number of videos in the interested video list is zero. If so, execute steps S6-S7 and return to S8.2. If not, proceed to S8.5.

[0033] S8.5: Update all videos in the list of interested videos to video G, execute steps S3-S5, and record the obtained correlation value set K1 as X. p , update the obtained label set F1 to be the label set T p {U r 、U r-1 ,⋯,U1}, the letter r represents the label set T p The total number of internal tags is r, proceed to S8.6;

[0034] S8.6: Determine whether "p=1" holds. If so, obtain the corresponding label set F1 from the correlation value set W1. If not, proceed to S8.7.

[0035] S8.7: Select a label set F p and T p , let the label set s p =F p ∩T p , t p =s p ∩F p ,u p =s p ∩T p , respectively from the label set t p and u p Extract the relevance value set W corresponding to its internal label p，1 {V v 、V v-1 、⋯V w ⋯、V1} and X p，1 {Yv 、Y v-1 、⋯Y w ⋯、Y1}, where V w and Y w It represents different relevance values corresponding to the same tag, where v represents the tag set F p and T p There are v identical labels in total;

[0036] S8.8: Let Z = v, Z is used as the number to select the correlation value V w and Y w ;

[0037] S8.9: Select the correlation value V Z and Y Z ,make = ,Will Recorded in the correlation value set W p，3 middle, That is the correlation value V Z and Y Z The average value of

[0038] S8.10: Determine whether "Z ≤ 1" is true. If "Z ≤ 1" is not true, assign Z-1 to Z and return to S8.9; if "Z ≤ 1" is true, let the label set x p =C Fp t p ,y p =C Tp u p , label set x p is the label set F p Remove the tag set t p Other elements of the internal element, tag set y p is the label set T p Remove the tag set u p The other elements of the internal element are selected from the label set x p and y p Extract the relevance value set W corresponding to its internal label p，2 With X p，2 , let the correlation value set W p，4 =W p，2 ∪X p，2 ∪W p，3 , the correlation value set W p，4 Each correlation value in is sorted from large to small, and the resulting set is recorded as the correlation value set W p+1 {α z , α z-1 ,⋯α δ⋯、α1}, the letter z represents the set of correlation values W p+1 There are a total of z correlation values, α δ Represents the set of correlation values W p+1 Any correlation value within, go to S8.11;

[0039] S8.11: Determine whether "z≥θ" is true. If "z≥θ" is true, then according to the correlation value α δ Select the correlation value set W from large to small p+1 The first θ elements are updated and recorded as the correlation value set W p+1 {α z , α z-1 ,⋯α μ ⋯、α z-θ}, α μ Represents the set of correlation values W p+1 Any correlation value within, by the correlation value set W p+1 Get the corresponding label set and record it as F p+1 {H z 、H z-1 、⋯H μ ⋯、H z-θ}, go to S8.13; if "z ≥ θ" does not hold, go to S8.12;

[0040] S8.12: From the correlation value set W p+1 Get its corresponding label set and record it as F p+1 {H z 、H z-1 、⋯H δ ⋯, H1}, enter S8.13;

[0041] S8.13: Assign p+1 to p, execute steps S6-S7, delete the video record in the video list of interest, and return to S8.2.

[0042] More preferably, the stop word list is the stop word list of Harbin Institute of Technology, the stop word list of Fudan University, Any one of the stop words.

[0043] As another aspect of the present application, a short video recommendation system based on big data is provided, comprising:

[0044] An operation information acquisition module is used to obtain the user's operation information on the currently watched video;

[0045] An operation information matching module is used to match the user's operation information on the currently watched video with the list of interested operations;

[0046] An information data acquisition module, which is used to acquire information data;

[0047] An information data processing module, which is used to process information data;

[0048] Video push module, the video push module is used to push videos.

[0049] More preferably, the device further includes an interesting video processing module, which is used to process interesting videos.

[0050] The present invention has the following advantages:

[0051] 1. The present invention obtains relevant data of video G through the information data acquisition module. The information data processing module obtains the push video set and the push order of the videos therein according to the obtained video G relevant data. The video push module pushes the videos therein according to the obtained push order of the videos therein, thereby optimizing the video recommendation strategy and formulating more personalized recommendation strategies for different users, which can greatly improve user satisfaction and retention rate.

[0052] 2. The present invention, through the provision of an interest video processing module, can formulate more personalized recommendation strategies for different users, provide high-quality recommended content that meets user needs and interests, and update the recommendation model in real time, thereby improving user satisfaction and retention rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying any creative work.

[0054] Figure 1 This is a flow chart of a short video recommendation method based on big data in Example 1 of the present invention.

[0055] Figure 2 This is a structural diagram of a short video recommendation system based on big data in Example 2 of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solutions and advantages of the present application more clearly understood, some embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and Examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not intended to limit the present application. However, it will be appreciated by those skilled in the art that in each embodiment of the present application, many technical details are provided in order to enable the reader to better understand the present application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in the present application can still be implemented.

[0057] Example 1: See Figure 1 , Embodiment 1 of the present invention provides a short video recommendation method based on big data, comprising:

[0058] S1: Obtain the user's operation information on the currently watched video and match it with the list of interested operations. It should be further explained that the user, i.e., the analysis object, can be a new user or an old user;

[0059] S2: If the video successfully matches any one or more operations in the interest operation list, the video is recorded as video G and proceeds to S3; otherwise, return to S1;

[0060] It should be further explained that the operation information refers to the user's operation browsing history, that is, the user's likes, comments, shares and other operations; among them, the interest operation list includes likes, forwarding, collection, sharing and follow operations.

[0061] By acquiring and analyzing user action information on videos in real time, we can understand user feedback on video content, thereby optimizing video content and improving video quality. For example, if a user likes or comments on a video, we can increase the push frequency of such videos and improve the push strategy.

[0062] S3: Get the text information in the video G, which includes the video copy and comment text, and segment the video copy and comment text according to the custom dictionary, match the obtained words with the stop word list, and record the unmatched words as feature words d a , by the feature word d a Form a feature word set D, and obtain a dictionary vector based on the feature word set D;

[0063] Regarding step S3, it should be further explained that the video copy usually attracts the audience with a concise, clear, vivid and interesting language style. The video copy can appear at the beginning, end or middle of the video, or can be published as independent text content together with the video. It can be a simple text description or a more complex text script or script. The video copy here refers to the text in the video G used to describe, promote or advertise the video content, such as a brief introduction to the theme of the video G, a description of the plot, or a commentary on the characters or events in the video.

[0064] Comment text refers to the text description within the comment area of video G, which is the text content that comments and evaluates the video content. It can be comments on the theme of the video, comments on the characters or events in the video, or comments on the production quality, creativity, etc. of the video;

[0065] Among them, the type of custom dictionary is selected manually according to the actual situation of video G. The following are several custom dictionaries to choose from: HC Dictionary, Dictionary, Jieba word segmentation dictionary and Dictionary: Huicong Dictionary contains a large number of common words and phrases, suitable for general Chinese text segmentation needs; dictionary, It is a The algorithmic Chinese word segmentation tool provides an extensible word segmentation dictionary, and users can add, modify, and delete words in the dictionary as needed; Jieba word segmentation dictionary, Jieba word segmentation is a Chinese word segmentation tool based on word graphs, which provides a customizable word segmentation dictionary, and users can add, modify, and delete words in the dictionary; The dictionary provides an extensible word segmentation dictionary. Users can add, modify and delete words in the dictionary as needed. As mentioned above, it is better to use a custom dictionary that can add, modify and delete words in the dictionary as needed. Therefore, the selection object of the custom dictionary can be Dictionary, Jieba word segmentation dictionary and Any one in the dictionary;

[0066] Among them, the stop word list is the stop word list of Harbin Institute of Technology, the stop word list of Fudan University, Any one in the stop word list; using the stop word list can remove stop words and reduce text noise; since stop words are numerous and appear frequently, removing stop words can greatly reduce the number of words that need to be processed, making important words more prominent, thereby saving storage space and computing resources and improving the speed of natural language processing.

[0067] S4: Based on the dictionary vector, calculate each feature word d using the TF-IDF algorithm aTF-IDF value, the feature word d whose TF-IDF value is greater than the first threshold β a Record it in the temporary storage set E of feature words, and store the feature words d in the temporary storage set E. a Match with the label database to obtain the label set F1{H1, H2, ⋯H c ⋯、H e}, where H e The subscript e indicates that the video G has a total of e labels, H c Represents any label in the label set F1;

[0068] Regarding step S4, it should be further explained that the TF-IDF algorithm is a commonly used weighted technology for information retrieval and text mining. The principle of the TF-IDF algorithm is: if a word appears with a high frequency TF in an article and rarely appears in other articles, it is considered that this word or phrase has good category discrimination ability and is suitable for classification. The TF-IDF algorithm consists of two parts, TF and IDF, where TF refers to term frequency, which refers to the number of times a word appears in an article, and IDF refers to inverse document frequency, which refers to the total number of documents containing the word divided by the number of documents containing the word. Here we use the TF-IDF algorithm to distinguish the categories of different videos;

[0069] The first threshold β is set manually based on the actual situation of the video G;

[0070] Among them, the label database can perform custom annotations on the data to facilitate subsequent data query and classification; the label database is implemented by aggregating data with the same label to form a label pool, each label pool contains one or more labels, and each label corresponds to one or more keywords.

[0071] By using the tag database, it is easier to match feature words d a The corresponding label provides a flexible and effective way to retrieve, match and classify data; a In the process of matching with corresponding tags, relevant data can be quickly located and found, which helps to improve the convenience of data utilization.

[0072] S5: According to the feature word temporary storage set E, the feature word d a The matching result with the tag database will be the feature word d that successfully matches the tag database a Record as keywords and classify them. One label corresponds to multiple keywords, and the label H is obtained. c Corresponding keyword set , c=1,2,⋯,e, calculate the label H c The corresponding correlation value Jc , J c For each keyword set The average value of the TF-IDF value of the internal keywords, according to the relevance value J c Get the correlation value set K1{L e , L e-1 、⋯L c ⋯、L1},L c is any correlation value in the correlation value set K1, and the correlation value set K1 is composed of the correlation value J c Sort them in descending order, and get the tag set corresponding to the order from the relevance value set K1, and update it as the tag set F1{H e 、H e-1 、⋯H c ⋯、H1};

[0073] Regarding step S5, it is necessary to further explain that by setting keyword conditions and keyword classification steps, key information related to the tag can be quickly and effectively extracted, which facilitates the rapid retrieval of required data during subsequent data processing, improves information retrieval accuracy and analysis efficiency, promotes information management and search engine optimization, and provides feature words in natural language processing. a Classification and feature words d a Analysis and other functions.

[0074] S6: Select tag set F from unwatched video library p {H e 、H e-1 、⋯H c Each tag H in ⋯、H1} c Representative video collection M h , h=1, 2, ⋯, e, the initial value of p is 1, p is used as a number to select the label set, according to the label set F p {H e 、H e-1 、⋯H c The order of labels inside ⋯、H1} is from left to right, starting from the corresponding representative video set M h Randomly select a video and record the selected video in the push video set N p {ξ e ,ξ e-1 、⋯ξ c ⋯,ξ1}, delete the pushed video set N from the unwatched video library p Internal video;

[0075] S7: Push video set N p {ξ e ,ξ e-1 、⋯ξ cThe order of the videos in ⋯、ξ1} will push the video set N from left to right. p Push the video to the user;

[0076] As for the order of video push, videos randomly selected from the video set representing the corresponding tags are pushed in sequence according to the relevance of the tags. This can improve users' acceptance and interest in pushed videos; videos with high relevance to tags that users are interested in are pushed first, which can meet users' interest needs, improve users' acceptance and satisfaction with pushed content, and increase users' interest and curiosity in pushed content, thereby improving click-through rate and viewing rate.

[0077] S8: Determine the push video set N p Check if all internal videos have been pushed. If so, push the video. e Updates are recorded as videos. It needs to be further explained that the video will be pushed e The update is recorded as video G, that is, the selected video corresponding to the label with the largest correlation value is updated and recorded as video G, and the process returns to S3; otherwise, it goes to S9;

[0078] The selection of video G can improve the accuracy and pertinence of the analyzed video, ensure that the analyzed video is more relevant to the specific tag, thereby improving the accuracy and pertinence of the analysis, facilitating in-depth analysis of the topic, and improving the quality and depth of the analyzed video. By continuously updating the analyzed video and selecting the video with the greatest relevance to the latest tag, the quality and depth of the analyzed video can be ensured to continuously improve, providing more comprehensive and in-depth analysis results.

[0079] S9: Continue to push video collection N p If the video is not pushed internally, return to S8.

[0080] For new users, adopting a new video push method and continuously pushing videos with topics or tags that new users are interested in can help us better understand the interests of new users, thereby pushing videos that new users are interested in more accurately, increasing new users' trust in the video push platform, and improving new user retention rates.

[0081] For old users, the new video push method can recommend more videos related to their interests, help them discover new content, let them discover more interesting content, and guide them to further explore the video push platform; by more accurately recommending videos that match the interests of old users, it can attract old users and increase their participation; at the same time, recommending videos related to the interests of old users can also promote the interaction and communication between old users and the community, increase the stickiness of old users to the platform, and improve the satisfaction of old users; at the same time, recommending videos related to the interests of old users can also help old users find the content they want faster and improve the experience of old users; it can better adapt to the changes in interests and needs of old users, provide better services, and improve the retention rate of old users.

[0082] The steps of obtaining the dictionary vector based on the feature word set D include:

[0083] The dictionary vector exists in the form of: ; B is the word frequency vector corresponding to the feature word set D, i a is the feature word d a The number of times it appears in the text information. The subscript A indicates that there are a total of A feature words in the feature word set D. It should be further explained that the dictionary vector is composed of the feature word set D in the video G and the word frequency vector B corresponding to the feature word set D.

[0084] The word frequency vector B can be used in multiple tasks and can represent the feature word d a The connections and differences between them can help us better "read" words and sentences; the word frequency vector B is a fixed-length continuous dense vector that takes up less resources and has a fast calculation speed; if these high-dimensional words are reduced in dimension and projected onto the coordinates, we can see similar feature words d a Will tend to be grouped together, irrelevant feature words d a Will be far away, reflecting the characteristic word d in the word frequency vector B a The connection and difference between the feature word d a The distribution is more intuitive.

[0085] For the feature word d whose TF-IDF value is greater than the first threshold β in S4 a The steps of recording in the temporary feature word set E specifically include:

[0086] S4.1: Get the number of all videos in the video library and record it as C, let j= , j is the feature word d a Total number of occurrences;

[0087] S4.2: Let k = 1, k is used as a number to select feature words;

[0088] S4.3: Select feature word d k and its occurrence times in the text information i k , get the video library containing the feature word d k The number of all videos is recorded as ;

[0089] S4.4: Let d k，TF-IDF = ;

[0090] S4.5: Determine “d k，TF-IDF >β" is true, if "d k，TF-IDF >β” holds true, then the feature word d k Record in the temporary feature word set E and go to S4.6; if "d k，TF-IDF >β” does not hold, go to S4.6;

[0091] S4.6: Determine whether "k ≥ A" holds. If so, start the label matching process in S4; if not, assign k + 1 to k and return to S4.3.

[0092] By setting the first threshold β to filter out feature words with a higher word frequency, the screening accuracy can be improved. The TF-IDF algorithm is a commonly used word weight calculation method. By calculating the frequency and importance of words in a document, words related to the topic can be more accurately filtered out; some words with high frequency but irrelevant to the topic can be filtered out to reduce text noise interference; by filtering out words related to the topic, the amount of data for subsequent processing can be reduced, and processing efficiency can be improved; the user experience can be improved, and filtering out words related to the topic can provide more accurate results and improve the user experience; the consumption of computing resources can be reduced, and filtering out words related to the topic can reduce unnecessary computing resources and time consumption.

[0093] For S5, calculate the label H c The corresponding correlation value J c , according to the correlation value J c Obtaining the correlation value set K1, the specific steps include:

[0094] S5.1: Let m = 1, where m is used as the number to select the keyword set;

[0095] S5.2: Get keyword set The number of internal keywords is recorded as , the TF-IDF value of the keywords inside it is recorded as P m，n，TF-IDF , n=1, 2, ⋯, 、 , n is the keyword set The label of the TF-IDF value of the internal keyword, Q m = , Q m That is the label H m The corresponding correlation value, Q m Recorded in collection It needs to be further explained that Q m The calculation method is as follows: for the keyword set , calculate the sum of the TF-IDF values of the keywords inside it Divide by the keyword set Number of internal keywords ;

[0096] S5.3: Determine whether "m ≥ e" holds. If so, proceed to S5.4. If not, assign m + 1 to m and return to S5.2.

[0097] S5.4: Based on the collection Intracorrelation value Q m Sort from large to small and get the correlation value set K1{L e , L e-1 、⋯L c ⋯、L1}.

[0098] By calculating the relevance value of a tag, we can more accurately measure the degree of association between the tag and the target information, thereby optimizing the accuracy and efficiency of information classification; we can enhance the accuracy of the recommendation system and more accurately measure the user's preference for different items, thereby enhancing the accuracy of the recommendation system; in natural language processing tasks, calculating the relevance value of a tag can help the model more accurately understand the meaning and context of the text, thereby improving the effectiveness of natural language processing tasks.

[0099] For S8 and S9, we also need to further optimize video recommendations. The specific steps are as follows:

[0100] It should be further explained that after operations S1-S7 are completed for the first time, operations S8.1-S8.13 are started.

[0101] S8.1: Record the relevance value set K1 as relevance value set W1. It should be further explained that relevance value set K1 is the relevance value set data generated after the first execution of operations S1-S7. The corresponding tag set is stored together with its relevance value set data.

[0102] S8.2: Pushing video set N pDuring the push process of the video, the user's operation information on the currently watched video is obtained in real time and matched with the interest operation list. If it successfully matches any one or more operations in the interest operation list, the video will be placed in the interest video list;

[0103] By analyzing the collected videos of interest, formulating more personalized recommendation strategies for different users, providing high-quality recommended content that meets user needs and interests, updating the recommendation model in real time, and improving the short video recommendation strategy, we can more accurately discover video content of interest, thereby accurately recommending short videos of interest to users and increasing user stickiness to the platform; in the process of short video recommendation, we may recommend some videos related to topics of interest to users, so that we can discover more related topics and trends, thereby improving user satisfaction and retention.

[0104] S8.3: Determine the push video set N p Check whether all internal videos have been pushed. If so, proceed to S8.4. Otherwise, continue pushing video set N. p If the video is not pushed internally, return to S8.2;

[0105] S8.4: Determine whether the number of videos in the interested video list is zero. If so, execute steps S6-S7 and return to S8.2. If not, proceed to S8.5.

[0106] It should be further explained that the method for handling the number of videos in the list of interested videos is to continue to push videos based on the current latest tag set. Since the tag relevance of each video is very different, as videos are pushed continuously, the type of videos that users like can always be found, thereby updating the tag relevance set.

[0107] The way of handling the number of videos in the list of videos of interest can enhance user engagement. For example, if a user is interested in videos with a specific tag, and as the number of videos watched by the user gradually increases, the user's engagement with the tag may increase, and the user may continue to pay attention to more videos with similar tags, and may interact and share more content with the tag; it can broaden the user's horizons. By pushing more videos with similar tags, it can help users understand all aspects of the tag more comprehensively, which helps users broaden their horizons, discover new points of interest and knowledge, and enhance their understanding of related fields; it can improve user satisfaction, so that users will be more willing to accept more video recommendations with similar tags, and more willing to spend time watching and exploring; it can promote interaction and communication between users and the community, so that users will have a deeper interest in the tag and participate in community discussions, so that users can exchange ideas and opinions with other users and obtain more information and knowledge.

[0108] S8.5: Update all videos in the list of interested videos to video G, execute steps S3-S5, and record the obtained correlation value set K1 as X. p , update the obtained label set F1 to be the label set T p {U r 、U r-1 ,⋯,U1}, the letter r represents the label set T p The total number of internal tags is r, proceed to S8.6;

[0109] Regarding step S8.5, it should be further explained that when executing steps S3-S5, all text information of the videos G is obtained at the same time and then analyzed and processed.

[0110] S8.6: Determine whether "p = 1" holds. If so, obtain the corresponding tag set F1 from the correlation value set W1. If not, proceed to S8.7. It should be noted that the tag set F1 is obtained here to facilitate finding the corresponding tag set F1 when step S8.7 is executed for the first time.

[0111] S8.7: Select a label set F p and T p , let the label set s p =F p ∩T p , t p =s p ∩F p ,u p =s p ∩T p , respectively from the label set t p and up Extract the relevance value set W corresponding to its internal label p，1 {V v 、V v-1 、⋯V w ⋯、V1} and X p，1 {Y v 、Y v-1 、⋯Y w ⋯、Y1}, where V w and Y w It represents different relevance values corresponding to the same tag, where v represents the tag set F p and T p There are v identical labels in total;

[0112] It should be further explained that the label set s p is the label set F p and T p Common elements within, t p For the tag set s p and F p The common element inside, u p For the tag set s p and T p Common elements of the interior.

[0113] S8.8: Let Z = v, Z is used as the number to select the correlation value V w and Y w ;

[0114] S8.9: Select the correlation value V Z and Y Z ,make = ,Will Recorded in the correlation value set W p，3 middle, That is the correlation value V Z and Y Z The average value of

[0115] S8.10: Determine whether "Z ≤ 1" is true. If "Z ≤ 1" is not true, assign Z-1 to Z and return to S8.9; if "Z ≤ 1" is true, let the label set x p =C Fp t p ,y p =C Tp u p , label set x p is the label set F p Remove the tag set t p Other elements of the internal element, tag set y p is the label set Tp Remove the tag set u p The other elements of the internal element are selected from the label set x p and y p Extract the relevance value set W corresponding to its internal label p，2 With X p，2 , let the correlation value set W p，4 =W p，2 ∪X p，2 ∪W p，3 , the correlation value set W p，4 Each correlation value in is sorted from large to small, and the resulting set is recorded as the correlation value set W p+1 {α z , α z-1 ,⋯α δ ⋯、α1}, the letter z represents the set of correlation values W p+1 There are a total of z correlation values, α δ Represents the set of correlation values W p+1 Any correlation value within, go to S8.11;

[0116] Regarding step S8.10, it is necessary to further explain that for the newly obtained correlation value set W p，3 and label set F p With label set T p The set of correlation values of different elements in W p，2 and X p，2 , for the correlation value set W p，3 、W p，2 and X p，2 Find the union to get the correlation value set W p+1 .

[0117] S8.11: Determine whether "z≥θ" is true. If "z≥θ" is true, then according to the correlation value α δ Select the correlation value set W from large to small p+1 The first θ elements are updated and recorded as the correlation value set W p+1 {α z , α z-1 ,⋯α μ ⋯、α z-θ}, α μ Represents the set of correlation values W p+1 Any correlation value within, by the correlation value set W p+1 Get the corresponding label set and record it as F p+1 {H z 、H z-1 、⋯H μ ⋯、H z-θ}, go to S8.13; if "z ≥ θ" does not hold, go to S8.12;

[0118] According to the correlation value α δ Select the correlation value set W from large to small p+1 The first θ elements can effectively prevent the situation where too many tags lead to an increase in the number of videos pushed to users. By selecting tags that are more relevant to user interests, we can more accurately predict user interests and provide content that better meets user needs, thereby improving recommendation accuracy; it can reduce redundant recommendations, thereby improving user satisfaction; by reducing the recommendation of videos that do not match user interests, it can improve user experience; and it can reduce system resource consumption. Although users may watch more videos, if these videos are irrelevant, it will waste the system's computing resources and network bandwidth. By selecting tags with larger relevance values, we can use system resources more effectively.

[0119] S8.12: From the correlation value set W p+1 Get its corresponding label set and record it as F p+1 {H z 、H z-1 、⋯H δ ⋯, H1}, enter S8.13;

[0120] S8.13: Assign p+1 to p, execute steps S6-S7, delete the video record in the video list of interest, and return to S8.2.

[0121] The present invention obtains the relevant data of video G through the information data acquisition module, and the information data processing module obtains the push video set and the push order of the videos therein according to the obtained video G relevant data. The video pushing module pushes the videos therein according to the obtained push order of the videos therein, thereby optimizing the video recommendation strategy and formulating more personalized recommendation strategies for different users, which can greatly improve user satisfaction and retention rate.

[0122] The present invention, through the provision of an interested video processing module, can formulate more personalized recommendation strategies for different users, provide high-quality recommended content that meets user needs and interests, and update the recommendation model in real time, thereby improving user satisfaction and retention rate.

[0123] Example 2: See Figure 2 , Embodiment 2 of the present invention provides a short video recommendation system based on big data, including:

[0124] An operation information acquisition module is used to obtain the user's operation information on the currently watched video;

[0125] An operation information matching module is used to match the user's operation information on the currently watched video with the list of interested operations;

[0126] An information data acquisition module, which is used to acquire information data;

[0127] An information data processing module, which is used to process information data;

[0128] Video push module, the video push module is used to push videos.

[0129] It should be further explained that the system further includes an interested video processing module, which is used to process interested videos.

[0130] It should be understood that those skilled in the art may make improvements or modifications based on the above description, and all such improvements and modifications shall fall within the scope of protection of the appended claims. Any portion of this specification not described in detail is prior art known to those skilled in the art.

Claims

1. A short video recommendation method based on big data, characterized in that: include: S1: Obtain the user's operation information on the currently watched video and match it with the list of interested operations; S2: If the video successfully matches any one or more operations in the interest operation list, the video is recorded as video G and proceeds to S3; otherwise, return to S1; S3: Get the text information in the video G, which includes the video copy and comment text, and segment the video copy and comment text according to the custom dictionary, match the obtained words with the stop word list, and record the unmatched words as feature words d a , by the feature word d a Form a feature word set D, and obtain a dictionary vector based on the feature word set D; S4: Based on the dictionary vector, calculate each feature word d using the TF-IDF algorithm a TF-IDF value, the feature word d whose TF-IDF value is greater than the first threshold β a Record it in the temporary storage set E of feature words, and store the feature words d in the temporary storage set E. a Match with the label database to obtain the label set F1{H1, H2, ⋯H c ⋯、H e }, where H e The subscript e indicates that the video G has a total of e labels, H c Represents any label in the label set F1; S5: According to the feature word temporary storage set E, the feature word d a The matching result with the tag database will be the feature word d that successfully matches the tag database a Record as keywords and classify them. One label corresponds to multiple keywords, and the label H is obtained. c Corresponding keyword set , c=1,2,⋯,e, calculate the label H c The corresponding correlation value J c , J c For each keyword set The average value of the TF-IDF value of the internal keywords, according to the relevance value J c Get the correlation value set K1{L e , L e-1 、⋯L c ⋯、L1},L c is any correlation value in the correlation value set K1, and the correlation value set K1 is composed of the correlation value J c Sort them in descending order, and get the tag set corresponding to the order from the relevance value set K1, and update it as the tag set F1{H e 、H e-1 、⋯H c ⋯、H1}; S6: Select tag set F from unwatched video library p {H e 、H e-1 、⋯H c Each tag H in ⋯、H1} c Representative video collection M h , h=1, 2, ⋯, e, the initial value of p is 1, p is used as a number to select the label set, according to the label set F p {H e 、H e-1 、⋯H c The order of labels inside ⋯、H1} is from left to right, starting from the corresponding representative video set M h Randomly select a video and record the selected video in the push video set N p {ξ e ,ξ e-1 、⋯ξ c ⋯,ξ1}, delete the pushed video set N from the unwatched video library p Internal video; S7: Push video set N p {ξ e ,ξ e-1 、⋯ξ c The order of the videos in ⋯、ξ1} will push the video set N from left to right. p Push the video to the user; S8: Determine the push video set N p Check if all internal videos have been pushed. If so, push the video. e The update is recorded as video G and returned to S3; Otherwise, go to S9; S9: Continue to push video collection N p If the video is not pushed internally, return to S8.

2. The short video recommendation method based on big data according to claim 1, characterized in that: The steps of obtaining the dictionary vector based on the feature word set D include: The dictionary vector exists in the form of: ; B is the word frequency vector corresponding to the feature word set D, i a is the feature word d a The number of times it appears in the text information. The subscript A indicates that there are a total of A feature words in the feature word set D.

3. The short video recommendation method based on big data according to claim 2, characterized in that: For the feature word d whose TF-IDF value is greater than the first threshold β in S4 a The steps of recording in the temporary feature word set E specifically include: S4.1: Get the number of all videos in the video library and record it as C, let j= , j is the feature word d a Total number of occurrences; S4.2: Let k = 1, k is used as a number to select feature words; S4.3: Select feature word d k and its occurrence times in the text information i k , get the video library containing the feature word d k The number of all videos is recorded as ; S4.4: Order k，TF-IDF = ; S4.5: Determine "d k，TF-IDF >β" is true, if "d k，TF-IDF >β” holds true, then the feature word d k Record in the temporary feature word set E and go to S4.6; if "d k，TF-IDF >β” does not hold, go to S4.6; S4.6: Determine whether "k ≥ A" holds. If so, start the label matching process in S4. If not, assign k + 1 to k and return to S4.

3.

4. The short video recommendation method based on big data according to claim 3, characterized in that: For S5, calculate the label H c The corresponding correlation value J c , according to the correlation value J c Obtaining the correlation value set K1, the specific steps include: S5.1: Let m = 1, where m is used as the number to select the keyword set; S5.2: Get keyword set The number of internal keywords is recorded as , the TF-IDF value of the keywords inside it is recorded as P m，n，TF-IDF , n=1, 2, ⋯, 、 , n is the keyword set The label of the TF-IDF value of the internal keyword, Q m = , Q m That is the label H m The corresponding correlation value, Q m Recorded in collection Inside; S5.3: Determine whether "m ≥ e" holds. If so, proceed to S5.

4. If not, assign m + 1 to m and return to S5.

2. S5.4: Based on the collection Intracorrelation value Q m Sort from large to small and get the correlation value set K1{L e , L e-1 、⋯L c ⋯、L1}.

5. The short video recommendation method based on big data according to claim 4, characterized in that: For S8 and S9, we also need to further optimize video recommendations. The specific steps are as follows: S8.1: Record the correlation value set K1 as the correlation value set W1; S8.2: Pushing video set N p During the push process of the video, the user's operation information on the currently watched video is obtained in real time and matched with the interest operation list. If it successfully matches any one or more operations in the interest operation list, the video will be placed in the interest video list; S8.3: Determine the push video set N p Check whether all internal videos have been pushed. If so, proceed to S8.

4. Otherwise, continue pushing video set N. p If the video is not pushed internally, return to S8.2; S8.4: Determine whether the number of videos in the interested video list is zero. If so, execute steps S6-S7 and return to S8.

2. If not, proceed to S8.

5. S8.5: Update all videos in the list of interested videos to video G, execute steps S3-S5, and record the obtained correlation value set K1 as X. p , update the obtained label set F1 to be the label set T p {U r 、U r-1 ,⋯,U1}, the letter r represents the label set T p The total number of internal tags is r, proceed to S8.6; S8.6: Determine whether "p=1" holds. If so, obtain the corresponding label set F1 from the correlation value set W1. If not, proceed to S8.

7. S8.7: Select a label set F p and T p , let the label set s p =F p ∩T p , t p =s p ∩F p ,u p =s p ∩T p , respectively from the label set t p and u p Extract the relevance value set W corresponding to its internal label p，1 {V v 、V v-1 、⋯V w ⋯、V1} and X p，1 {Y v 、Y v-1 、⋯Y w ⋯、Y1}, where V w and Y w It represents different relevance values corresponding to the same tag, where v represents the tag set F p and T p There are v identical labels in total; S8.8: Let Z = v, Z is used as the number to select the correlation value V w and Y w ; S8.9: Select the correlation value V Z and Y Z ,make = ,Will Recorded in the correlation value set W p，3 middle, That is the correlation value V Z and Y Z The average value of S8.10: Determine whether "Z≤1" holds. If "Z≤1" does not hold, assign Z-1 to Z and return to S8.

9. If "Z≤1" holds, set the label set x p =C Fp t p ,y p =C Tp u p , label set x p is the label set F p Remove the tag set t p Other elements of the internal element, tag set y p is the label set T p Remove the tag set u p The other elements of the internal element are selected from the label set x p and y p Extract the relevance value set W corresponding to its internal label p，2 With X p，2 , let the correlation value set W p，4 =W p，2 ∪X p，2 ∪W p，3 , the correlation value set W p，4 Each correlation value in is sorted from large to small, and the resulting set is recorded as the correlation value set W p+1 {α z , α z-1 ,⋯α δ ⋯、α1}, the letter z represents the set of correlation values W p+1 There are a total of z correlation values, α δ Represents the set of correlation values W p+1 Any correlation value within, go to S8.11; S8.11: Determine whether "z≥θ" is true. If "z≥θ" is true, then according to the correlation value α δ Select the correlation value set W from large to small p+1 The first θ elements are updated and recorded as the correlation value set W p+1 {α z , α z-1 ,⋯α μ ⋯、α z-θ }, α μ Represents the set of correlation values W p+1 Any correlation value within, by the correlation value set W p+1 Get the corresponding label set and record it as F p+1 {H z 、H z-1 、⋯H μ ⋯、H z-θ }, go to S8.13; if "z ≥ θ" does not hold, go to S8.12; S8.12: From the correlation value set W p+1 Get its corresponding label set and record it as F p+1 {H z 、H z-1 、⋯H δ ⋯, H1}, enter S8.13; S8.13: Assign p+1 to p, execute steps S6-S7, delete the video record in the video list of interest, and return to S8.

2.

6. The short video recommendation method based on big data according to claim 5, characterized in that: include: The stop word list is the stop word list of Harbin Institute of Technology, the stop word list of Fudan University, Any one of the stop words.

7. A short video recommendation system based on big data, characterized in that: The system applies the short video recommendation method based on big data according to any one of claims 1 to 6, including: An operation information acquisition module is used to obtain the user's operation information on the currently watched video; An operation information matching module is used to match the user's operation information on the currently watched video with the list of interested operations; An information data acquisition module, which is used to acquire information data; An information data processing module, which is used to process information data; Video push module, the video push module is used to push videos.

8. The short video recommendation system based on big data according to claim 7, characterized in that: Also includes: The video-of-interest processing module is used to process the video-of-interest.

Images