An implicit video recommendation algorithm

Abstract

An implicit video recommendation algorithm, by calculating the proportion of the time length of the video watched by the user to the total time length of the video, then the data dispersion situation data is counted, the data is taken as the parameter, the data table of the user and the video is established, then the cosine similarity is calculated, after sorting, the video with the highest similarity is recommended, according to the accuracy, recall rate of the training result, the proportion of data division is dynamically adjusted, so as to achieve the purpose of optimizing the final recommendation result. By deeply mining the relationship between users and videos, this can solve the problem of long tail commodity recommendation and data sparsity, and the time complexity of this algorithm is low, and the operation speed is fast.

Description

Technical Field

[0001] The present invention relates to the field of video-on-demand recommendation, and in particular to an implicit video recommendation algorithm. Background Art

[0002] With the development of the internet and information technology, people have gradually moved from an era of information scarcity to an era of information overload. In this era, both information producers and consumers face significant challenges. For consumers, it's extremely difficult to sift through the vast amount of information available to them. For producers, it's equally challenging to make their information stand out and attract the attention of a wide range of users. Recommender systems are tools for resolving this dilemma.

[0003] In recent years, personalized recommendation systems have been widely used in the film and video fields by analyzing large amounts of user behavior logs to provide different users with different personalized page displays to increase the click-through rate and conversion rate of websites.

[0004] Existing recommendation algorithms, such as collaborative filtering, have poor generalization capabilities and are unable to generalize the similarity between two items to other items. This leads to a serious problem: popular items have a strong head effect, easily generating similarities with a large number of items. However, long-tail items, due to their sparse feature vectors, rarely share similarities with other items, making them less likely to be recommended. However, discovering long-tail items and increasing their likelihood of being recommended is one of the most important goals of recommendation systems. This can actually be seen as a natural flaw in collaborative filtering. Whether based on users or items, collaborative filtering is inherently weak at processing sparse vectors, which is why the head effect is so pronounced.

[0005] Data sparsity occurs when users only rate a small fraction of the items available in the database. Larger datasets generally have sparser data, as computation typically scales linearly with the number of users and items. Recommendation techniques are effective when the dataset is limited, but as the dataset grows, the volume of recommendations generated becomes less effective. The SVD algorithm is computationally expensive, with a cubic time complexity and a quadratic space complexity, and data transformations are difficult to understand. Summary of the Invention

[0006] In order to overcome the deficiencies of the above technologies, the present invention provides an implicit video recommendation algorithm with low algorithm time complexity and fast operation speed.

[0007] The technical solution adopted by the present invention to overcome the technical problems is:

[0008] An implicit video recommendation algorithm includes the following steps:

[0009] a) Calculate the ratio of the time users spend watching a video to the total time of the video;

[0010] b) Summarize the proportion of video viewing time of all users to the total video duration, sort all the proportions from largest to smallest, and divide them into four equal parts;

[0011] c) Create a pivot table of users and videos;

[0012] d) Calculate the cosine similarity between videos;

[0013] e) Set a similarity coefficient and put videos whose cosine similarity is greater than the similarity coefficient into the recommendation list;

[0014] f) After recommending the recommendation list to the user, obtain the real data of the user's feedback and calculate the accuracy and recall rate;

[0015] g) Adjust the ratio of the ratio value division and the similarity coefficient, and repeat steps b) to f) O Second-rate.

[0016] Furthermore, in step a), the ratio value is calculated using the formula: ratio = duration of the video watched by the user / total duration of the video.

[0017] Furthermore, when all the proportion values ​​in step b) are arranged from largest to smallest and divided into four equal parts, the value at the first cutoff point is defined as the upper quartile Q3, the value at the second cutoff point is defined as the median, and the value at the third cutoff point is defined as the lower quartile Q1. The interquartile range is IQR, where IQR = Q3 - Q1. The upper inner limit is Q3 + 1.5 IQR, the lower inner limit is Q1 - 1.5 IQR, the upper outer limit is Q3 + 3 IQR, and the lower outer limit is Q1 - 3 IQR. Values ​​between the upper inner limit and the upper outer limit, and between the lower inner limit and the lower outer limit, are mild outliers, while values ​​outside the upper and lower outer limits are extreme outliers.

[0018] Furthermore, in step c), the leftmost end of each row of the pivot table of users and videos is the video ID, the top of each column is the user ID, and the values ​​in the pivot table of users and videos are the user's preference values ​​for the video, the preference values ​​are 1, 2, 3, 4, 5, the value from the upper inner limit to the upper quartile is defined as a preference value of 5, the value from the upper quartile to the median is defined as a preference value of 4, the value from the median to the lower quartile is defined as a preference value of 3, the value from the lower quartile to the lower inner limit is defined as a preference value of 2, and the value from the lower inner limit to the lower outer limit is defined as a preference value of 1.

[0019] Furthermore, in step d), the formula Calculate the cosine similarity between videos , where For the User to The likeability value of the video, is the total number of users, , is the total number of videos, For the User to The likeability value of the video, , all cosine similarities Construct a video similarity matrix.

[0020] Furthermore, in step e), the similarity coefficient is 0.8, and videos with cosine similarity greater than the similarity coefficient are put into the recommendation list, and then the recommendation list is deduplicated to delete videos that the user has already watched.

[0021] Furthermore, in step f), the accuracy is the ratio of the data in the recommendation list that is identical to the real data to the recommended data, and the recall is the ratio of the data that is identical to the real data to the real data.

[0022] Further, O The value of is 5.

[0023] The beneficial effects of the present invention are as follows: by calculating the proportion of the user's viewing time to the total video length, then statistically analyzing the data dispersion, using this data as a parameter, creating a pivot table of users and videos, and then calculating cosine similarity. After sorting, the videos with the highest similarity are recommended. Based on the accuracy and recall rates of the training results, the data partitioning ratio is dynamically adjusted to achieve the purpose of optimizing the final recommendation results. By deeply exploring the relationship between users and videos to make recommendations, this can solve the problem of long-tail products not being recommended and the problem of data sparsity. In addition, this algorithm has low time complexity and fast operation speed. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0025] The following is combined with Figure 1 The present invention is further described.

[0026] When users do not like / dislike or rate a video, collaborative filtering based on items is performed on the video. The SVD matrix decomposition recommendation algorithm cannot be implemented due to the lack of corresponding values. In this case, a new algorithm model needs to be established - an implicit video recommendation algorithm. Specifically, an implicit video recommendation algorithm of the present invention includes the following steps:

[0027] a) Calculate the ratio of the time users spend watching a video to the total time of the video;

[0028] b) Summarize the proportion of video viewing time of all users to the total video duration, sort all the proportions from largest to smallest, and divide them into four equal parts;

[0029] c) Create a pivot table of users and videos;

[0030] d) Calculate the cosine similarity between videos;

[0031] e) Set a similarity coefficient and put videos whose cosine similarity is greater than the similarity coefficient into the recommendation list;

[0032] f) After recommending the recommendation list to the user, obtain the real data of the user's feedback and calculate the accuracy and recall rate;

[0033] g) Adjust the ratio of the ratio value division and the similarity coefficient, and repeat steps b) to f) O Second-rate.

[0034] By calculating the proportion of user viewing time to total video time, we find the upper edge of the data (the upper quartile Q3), the median, the lower quartile Q1, the lower edge, and one outlier. The interquartile range (IQR) is calculated as Q3-Q1. Two lines, identical to the median, are drawn at Q3+1.5IQR and Q1-1.5IQR. These two lines are the outlier cutoff points, known as the inner limits. Two lines are drawn at Q3+3IQR and Q1-3IQR, known as the outer limits. Points outside the inner limits represent outliers. Outliers between the inner and outer limits are considered mild, while those outside the outer limits are considered extreme. Using this data as a parameter, we create a pivot table of user and video data. We then calculate cosine similarity, sort the data, and recommend videos with the highest similarity. Based on the training results, we dynamically adjust the data partitioning ratio to optimize the final recommendation results based on the precision and recall rates. By deeply exploring the relationship between users and videos to make recommendations, we can solve the problem of long-tail products not being recommended and the problem of sparse data. In addition, this algorithm has low time complexity and fast computing speed.

[0035] Example 1:

[0036] In step a), the ratio value is calculated using the formula: ratio = duration of video watched by the user / total duration of the video.

[0037] Example 2:

[0038] When all proportion values ​​in step b) are arranged from largest to smallest and divided into four equal parts, the value at the first cutoff point is defined as the upper quartile Q3, the value at the second cutoff point is defined as the median, and the value at the third cutoff point is defined as the lower quartile Q1. The interquartile range is the IQR, where IQR = Q3 - Q1. The upper inner limit is Q3 + 1.5 IQR, the lower inner limit is Q1 - 1.5 IQR, the upper outer limit is Q3 + 3 IQR, and the lower outer limit is Q1 - 3 IQR. Values ​​between the upper inner limit and the upper outer limit, and between the lower inner limit and the lower outer limit, are mild outliers; values ​​outside the upper and lower outer limits are extreme outliers.

[0039] Example 3:

[0040] The leftmost end of each row of the pivot table of users and videos in step c) is the video ID, and the top of each column is the user ID. The values ​​in the pivot table of users and videos are the user's preference values ​​for the video, and the preference values ​​are 1, 2, 3, 4, and 5. The value from the upper inner limit to the upper quartile is defined as a preference value of 5, indicating very much like, the value from the upper quartile to the median is defined as a preference value of 4, indicating like, the value from the median to the lower quartile is defined as a preference value of 3, indicating average, the value from the lower quartile to the lower inner limit is defined as a preference value of 2, indicating dislike, and the value from the lower inner limit to the lower outer limit is defined as a preference value of 1, indicating disgust.

[0041] For example:

[0042]

[0043] Example 4:

[0044] In step d), the formula Calculate the cosine similarity between videos , where For the User to The likeability value of the video, is the total number of users, , is the total number of videos, For the User to The likeability value of the video, , all cosine similarities Construct a video similarity matrix.

[0045] For example, the cosine similarity of videos 001 and 002 is .

[0046] Example 5:

[0047] In step e), if the similarity coefficient is 0.8, videos with a cosine similarity greater than the similarity coefficient are placed in the recommendation list. Then, duplicates are removed from the recommendation list, and videos that the user has already watched are deleted. For example, for every video that user A1 likes and likes, find videos with a similarity greater than the similarity coefficient, place them in the recommendation list, delete duplicates from the list, and delete videos that user A1 has already watched. This list serves as the recommendation list for user A1.

[0048] Example 6:

[0049] In step f), the precision is the ratio of the correct data (identical to the real data) in the recommendation list to the recommended data, and the recall is the ratio of the correct data (identical to the real data) in the recommendation list to the real data. After the recommendation list is recommended to user A1, real data will be fed back. If some of the recommended list items are highly liked or favored by the user, the precision and recall at this point will be recorded.

[0050] Example 7:

[0051] Preferably, O The value of is 5. By adjusting the similarity coefficient, for example, adding or subtracting 0.05, and adjusting the standards for the inner and outer limits, for example, the upper inner limit is Q3 + 1.2IQR, and then returning to step b) to recalculate, after 5 iterations, compare the precision and recall rate of each time, and select the recommendation with the highest precision and recall rate as the final recommendation.

[0052] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art will be able to modify the technical solutions described in the aforementioned embodiments or substitute equivalents for some of the technical features. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. An implicit video recommendation algorithm, characterized in that: The steps include: a) Calculate the ratio of the time users spend watching a video to the total time of the video; b) Summarize the proportion of video viewing time of all users to the total video duration, sort all the proportions from largest to smallest, and divide them into four equal parts; c) Create a pivot table of users and videos; d) Calculate the cosine similarity between videos; e) Set a similarity coefficient and put videos whose cosine similarity is greater than the similarity coefficient into the recommendation list; f) After recommending the recommendation list to the user, obtain the real data of the user's feedback and calculate the accuracy and recall rate; g) Adjust the ratio of the ratio value division and the similarity coefficient, and repeat steps b) to f) O Second-rate; When all the proportion values ​​in step b) are arranged from large to small and divided into four equal parts, the value at the first division point is defined as the upper quartile Q3, the value at the second division point is defined as the median, and the value at the third division point is defined as the lower quartile Q1. The interquartile range is IQR, IQR=Q3-Q1, the upper inner limit is Q3+1.5IQR, the lower inner limit is Q1-1.5IQR, the upper outer limit is Q3+3IQR, and the lower outer limit is Q1-3IQR; In step c), the leftmost end of each row of the pivot table of users and videos is the video ID, and the top of each column is the user ID. The values ​​in the pivot table of users and videos are the user's preference values ​​for the video, and the preference values ​​are 1, 2, 3, 4, and 5. The value from the upper inner limit to the upper quartile is defined as a preference value of 5, the value from the upper quartile to the median is defined as a preference value of 4, the value from the median to the lower quartile is defined as a preference value of 3, the value from the lower quartile to the lower inner limit is defined as a preference value of 2, and the value from the lower inner limit to the lower outer limit is defined as a preference value of 1; In step d), the formula Calculate the cosine similarity between videos , where For the User to The likeability value of the video, is the total number of users, , is the total number of videos, For the User to The likeability value of the video, , all cosine similarities Construct a video similarity matrix.

2. The implicit video recommendation algorithm according to claim 1, characterized in that In step a), the ratio value is calculated using the formula: ratio = duration of video watched by the user / total duration of the video.

3. The implicit video recommendation algorithm according to claim 1, characterized in that: In step e), the similarity coefficient is 0.

8. Videos with cosine similarity greater than the similarity coefficient are put into the recommendation list, and then the recommendation list is deduplicated to delete videos that the user has already watched.

4. The implicit video recommendation algorithm according to claim 1, characterized in that: In step f), the accuracy is the ratio of the data in the recommendation list that is identical to the real data to the recommended data, and the recall is the ratio of the data that is identical to the real data to the real data.

5. The implicit video recommendation algorithm according to claim 1, characterized in that: O The value of is 5.

Images