Personalized video course recommendation method and system

Abstract

The invention provides a personalized video course recommendation method and system, and relates to the technical field of data processing, and the method comprises the steps: constructing a hierarchical reinforcement learning network which comprises a high-level reinforcement learning network and a low-level reinforcement learning network; acquiring course learning data of the user; according to the course learning data, whether the user interest summary needs to be updated or not is decided through a high-level reinforcement learning network, if yes, whether part of interest courses in the user interest summary are deleted or not is decided through a low-level reinforcement learning network, and the user interest summary is updated through course learning data after part of historical courses are deleted; determining a recommendation index of each video course through a video course recommendation model according to the updated user interest summary; in combination with the quality and the favorable comment rate of each video course, performing quality correction on the recommendation index of each video course; and recommending the video courses to the user according to the corrected recommendation indexes of the video courses.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a personalized video course recommendation method and system. Background Art

[0002] Among the massive amount of video courses, personalized recommendations can help users quickly find courses that meet their needs, saving users time in searching and screening and improving learning efficiency.

[0003] Traditional personalized recommendation methods mainly use rule-based recommendation algorithms, collaborative filtering recommendation algorithms, and deep learning algorithms. Traditional personalized recommendation methods often assume that users' interests are static. However, in the actual course learning process, users tend to change their interests as their cognition improves. Traditional personalized recommendation methods cannot capture users' dynamic interests in continuous learning behaviors, resulting in difficulty in adapting to changes in users' interests as the learning process deepens, and the accuracy of personalized recommendations decreases. Summary of the invention

[0004] In order to solve the technical problem that the traditional personalized recommendation method in the prior art cannot capture the user's dynamic interest in continuous learning behavior, resulting in difficulty in adapting to changes in user interests as the learning process deepens, and the accuracy of personalized recommendations decreases, the present invention provides a personalized video course recommendation method and system.

[0005] The technical solution provided by the embodiment of the present invention is as follows:

[0006] First aspect:

[0007] An embodiment of the present invention provides a personalized video course recommendation method, comprising:

[0008] S1: Constructing a hierarchical reinforcement learning network and a video course recommendation model, wherein the hierarchical reinforcement learning network includes a high-level reinforcement learning network and a low-level reinforcement learning network that are interconnected;

[0009] S2: Obtain the user's course learning data;

[0010] S3: Based on the course learning data, the high-level reinforcement learning network is used to decide whether the user interest profile needs to be updated. If so, the low-level reinforcement learning network is used to decide whether to delete some of the interest courses in the user interest profile, and the user interest profile is updated by the course learning data after deleting some of the historical courses.

[0011] S4: Determine the recommendation index of each video course through the video course recommendation model according to the updated user interest profile;

[0012] S5: Based on the quality and praise rate of each video course, the recommendation index of each video course is corrected;

[0013] S6: Recommending video courses to users based on the revised recommendation index of each video course.

[0014] Second aspect:

[0015] An embodiment of the present invention provides a personalized video course recommendation system, including:

[0016] processor;

[0017] A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the personalized video course recommendation method as described in the first aspect is implemented.

[0018] The third aspect:

[0019] An embodiment of the present invention provides a computer-readable storage medium, on which a computer program is stored. When the program is executed by a processor, the personalized video course recommendation method as described in the first aspect is implemented.

[0020] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0021] (1) In the present invention, a hierarchical reinforcement learning network is used to maintain the user interest profile. The high-level reinforcement learning network can decide whether to update the user interest profile. If an update is required, the low-level reinforcement learning network can decide whether to delete some interest courses in the user interest profile to capture the user's dynamic interest in continuous learning behavior. This can better adapt to complex learning processes and interest changes. The accuracy of personalized recommendations will decrease, and the accuracy of personalized recommendations will be improved.

[0022] (2) In the present invention, after determining the recommendation index of a video course, the system makes corrections based on the quality and favorable rating of each video course. This ensures that the recommended video courses not only meet the user's interests, but also have high quality and user ratings, thereby increasing the user's trust and satisfaction with the course. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1A flowchart of a personalized video course recommendation method provided by an embodiment of the present invention;

[0025] Figure 2 A structural diagram of a personalized video course recommendation system provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The technical solution of the present invention is described below in conjunction with the accompanying drawings.

[0027] In the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "example" is intended to present the concept in a specific way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.

[0028] In the embodiments of the present invention, "image" and "picture" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same. "of", "corresponding, relevant" and "corresponding" can sometimes be used interchangeably. It should be noted that when the difference between them is not emphasized, the meanings they intend to express are the same.

[0029] In the embodiments of the present invention, sometimes a subscript such as W1 may be mistakenly written as a non-subscript such as W1. When the difference is not emphasized, the meanings to be expressed are consistent.

[0030] In order to make the technical problems, technical solutions and advantages to be solved by the present invention more clear, a detailed description will be given below with reference to the accompanying drawings and specific embodiments.

[0031] Reference Manual Attached Figure 1 , which shows a flow chart of a personalized video course recommendation method provided by an embodiment of the present invention.

[0032] The embodiment of the present invention provides a personalized video course recommendation method, which can be implemented by a personalized video course recommendation device, which can be a terminal or a server. The processing flow of the personalized video course recommendation method can include the following steps:

[0033] S1: Build a hierarchical reinforcement learning network and video course recommendation model.

[0034] Among them, the hierarchical reinforcement learning network includes a high-level reinforcement learning network and a low-level reinforcement learning network that are interconnected.

[0035] It should be noted that the high-level reinforcement learning network is used to decide whether to update the user interest profile. The low-level reinforcement learning network decides whether to delete the current interest course. The high-level reinforcement learning network and the low-level reinforcement learning network jointly maintain the user interest profile, which contains multiple interest courses that the user has learned and can represent the user's interests.

[0036] Optionally, the environment space of the high-level reinforcement learning network is specifically: S1 = (s1, s2), s1 represents the average value of the cosine similarity between the feature vector of each interest course in the user's interest profile and the feature vector of the target video course, s2 represents the average value of the element-by-element product between the feature vector of each interest course in the user's interest profile and the feature vector of the target video course, wherein the target video course is the video course that the user has most recently learned in the course learning data.

[0037] The action space of the high-level reinforcement learning network is specifically: A1 = {0, 1}, 0 means that the user interest profile does not need to be updated, and 1 means that the user interest profile needs to be updated.

[0038] The reward function of the high-level reinforcement learning network is used to provide rewards or penalties for the high-level reinforcement learning network based on the results of the action execution. Specifically:

[0039] R h (a h ,s h )=logP(θ|s h )-logP(θ'|s h )

[0040] Among them, R h () represents the reward function of the high-level reinforcement learning network, a h represents the action of the high-level reinforcement learning network, s h represents the state of the high-level reinforcement learning network, log represents the logarithmic function with base 10, θ represents the user interest summary before update, θ' represents the user interest summary before update, logP(θ|s) represents the log-likelihood estimate of the user interest summary before update, and logP(θ'|s) represents the log-likelihood estimate of the user interest summary after update.

[0041] It should be noted that if the user interest profile before and after the update is more consistent with the current state, the reward will be higher. By maximizing the difference in log-likelihood estimates, the reward function encourages the high-level reinforcement learning network to better understand the user's behavior pattern. If the user's interest pattern changes, the log-likelihood estimates before and after the update will also be different, and the network will be able to make corresponding adjustments in a timely manner.

[0042] Optionally, the environment space of the low-level reinforcement learning network is specifically: S2=(s3), where s3 represents the cosine similarity between the feature vectors of each interest course in the user interest profile and the feature vector of the target video course.

[0043] The action space of the low-level reinforcement learning network is specifically: A2 = {0, 1}, 0 means that the current interest course does not need to be deleted, and 1 means that the current interest course needs to be deleted.

[0044] The reward function of the underlying reinforcement learning network is used to provide rewards or penalties to the lower-level reinforcement learning network based on the results of the action execution. Specifically:

[0045] R l (a l ,s l )=Sim avg (x,x * )-Sim' avg (x,x * )

[0046] Among them, R l () represents the reward function of the low-level reinforcement learning network, a l represents the action of the low-level reinforcement learning network, s l represents the state of the low-level reinforcement learning network, x represents the course of interest, and x * Indicates the target video course, Sim avg (x,x * ) represents the average cosine similarity between each interest course and the target video course, Sim' avg (x,x * ) represents the average value of the cosine similarity between each interest course and the target video course after deleting some interest courses.

[0047] Among them, by calculating the average cosine similarity between the embedding vector of the historical course and the target course, the similarity between the course that the user has studied in the past and the target course can be measured. If this similarity is high, it means that the course that the user has studied in the past is similar to the target course, and the user's interest may not have changed. If the similarity is low, it means that the user's interest has changed, and the interest profile may need to be updated. Therefore, these features, as a feature set of high-level tasks, can help the system dynamically determine whether the user's interests have changed, and adjust and update the user's interest profile accordingly, thereby improving the accuracy and personalization of recommendations.

[0048] It should be noted that through the mechanism of rewards and penalties, the low-level reinforcement learning network can better maintain the user interest profile. Retaining interest courses related to the target video course and deleting irrelevant interest courses helps to ensure the accuracy and timeliness of the user interest profile. At the same time, by optimizing the reward function, the low-level reinforcement learning network can adjust the user interest profile more accurately, thereby improving the accuracy of personalized video course recommendations. The recommended courses are more in line with the user's actual interests, which enhances the user's trust and satisfaction with the recommendation system.

[0049] In one possible implementation, the high-level reinforcement learning network and / or the low-level reinforcement learning network is trained by a gradient descent method.

[0050] Among them, Gradient Descent is a commonly used optimization algorithm for minimizing the value of a function, and is usually used in the model training process in machine learning and deep learning. It updates the parameters in the opposite direction of the function gradient so that the function value gradually approaches the minimum or maximum value (depending on the optimization goal).

[0051] Specifically, with the goal of maximizing rewards, find the optimal network parameters:

[0052]

[0053] Among them, Θ represents the network parameters of the high-level reinforcement learning network or the low-level reinforcement learning network, arg max represents the value of Θ when the maximum value is taken, a represents the action, s represents the state, P(a|s,Θ) represents the sampling probability, and R(a,s) represents the reward function.

[0054] It should be noted that the goal is to find the optimal network parameters by maximizing the reward, which means that the network will be trained to maximize the reward the system receives when recommending video courses, which can make the recommendation system more accurately adapt to the interests and needs of users.

[0055] During the training of a high-level reinforcement learning network, the gradient of the high-level policy function is calculated:

[0056]

[0057] in, represents the gradient of the high-level policy function, N represents the total number of decisions, represents the action of the high-level reinforcement learning network at the i-th decision, represents the state of the high-level reinforcement learning network at the i-th decision, Indicates that the i-th decision is in state Take The action probability, Indicates that the i-th decision is in state Take The reward is , log represents the logarithmic function with base 10, represents the gradient calculation with respect to Θ.

[0058] Furthermore, the parameters are updated by the gradient descent method:

[0059]

[0060] Among them, Θ t+1 represents the parameters at t+1 iterations, Θ t represents the parameter at the tth iteration, α h Represents the learning rate of the high-level policy function.

[0061] In the present invention, the high-level reinforcement learning network and / or the low-level reinforcement learning network are trained by the gradient descent method, which can effectively optimize the recommendation system and improve the accuracy of recommendations and user satisfaction.

[0062] Similarly, the training of the high-level reinforcement learning network can be completed according to the same or similar method as that of the high-level reinforcement learning network. To avoid repetition, the present invention will not go into details.

[0063] S2: Obtain the user's course learning data.

[0064] Among them, course learning data includes users’ course learning records and viewing time of each course, etc.

[0065] S3: Based on the course learning data, the high-level reinforcement learning network decides whether the user interest profile needs to be updated. If so, the low-level reinforcement learning network decides whether to delete some of the interest courses in the user interest profile. The user interest profile is updated by the course learning data after deleting some historical courses.

[0066] It should be noted that as time goes by and the user's learning behavior changes, the user's interests may change, and the original interest profile may be outdated. By regularly updating the user's interest profile, we can avoid recommending content that no longer matches their interests to the user, thereby reducing information overload and user dissatisfaction with the recommendation system. By using the high-level reinforcement learning network to decide whether to update the user's interest profile, the system can dynamically adjust the user's interest profile based on the user's latest course learning data. This helps capture the changes and evolution of user interests, thereby better reflecting the user's current interests and needs. The low-level reinforcement learning network decides whether to delete some of the interest courses in the user's interest profile, which can help the system filter out courses that the user is not interested in or has already mastered, thereby reducing the amount of data that the recommendation system needs to process, saving computing resources and storage space, and improving the efficiency of the recommendation system.

[0067] S4: According to the updated user interest profile, the recommendation index of each video course is determined through the video course recommendation model.

[0068] In a possible implementation, S4 specifically includes:

[0069] By introducing the heat correction weight, the feature vectors of each interest course in the updated user interest profile are fused through the video course recommendation model to obtain the user's interest fusion feature vector:

[0070]

[0071] Among them, Y represents the interest fusion feature vector, h i represents the heat correction weight coefficient of the i-th interest course, Y i represents the feature vector of the i-th interest course, and n represents the total number of interest courses in the updated user interest profile.

[0072] It should be noted that the more popular the video courses are, the wider their appeal is, and the content is generally applicable to most users, making it more difficult to distinguish the interests of each user. Therefore, when interests are fused, the weight should be reduced accordingly, so that the interest fusion feature vector can more accurately reflect the user's true interests.

[0073] Optionally, the heat correction weight coefficient is calculated as follows:

[0074]

[0075] Among them, h i represents the heat correction weight coefficient of the i-th interest course, C represents the total number of registered users on the platform, and C i represents the total number of people who have watched the i-th interest course, and exp represents the exponential function with the natural constant e as the base.

[0076] It should be noted that the popularity of an interest course can be To perform the calculation, as mentioned above, the higher the heat, the smaller the heat correction weight coefficient should be, so the inverse is taken, and then normalized, and the value is adjusted between 0 and 1 to form the final heat correction weight coefficient.

[0077] In the present invention, the calculation method of the heat correction weight coefficient utilizes the total number of platform registrations and the total number of course viewers. Through nonlinear mapping and standardization processing, it more effectively reflects the true popularity of the course, which is conducive to personalized recommendations and improving the recommendation quality of the recommendation system.

[0078] Calculate the cosine similarity between the feature vector of the target video course and the user's interest fusion feature vector as the recommendation index of the target video course:

[0079]

[0080] Among them, σ k represents the recommendation index of the kth target video course, Y k represents the feature vector of the kth target video course, represents the transpose of the feature vector of the kth target video course, Y represents the interest fusion feature vector, and ∥∥ represents the modulo operation of the vector.

[0081] In the present invention, by calculating the cosine similarity between the target video course feature vector and the user interest fusion feature vector, the matching degree between the user interest and the course can be quantitatively measured. Courses with high recommendation indexes mean that they are more similar to the user's interests in the feature space and are more likely to meet the user's preferences, thereby achieving personalized recommendations.

[0082] In a possible implementation, heat correction weights and attention weights are introduced, and through the video course recommendation model, feature fusion is performed on the feature vectors of each interest course in the updated user interest profile to obtain the user's interest fusion feature vector:

[0083]

[0084] Among them, Y represents the interest fusion feature vector, λ i represents the attention weight coefficient of the i-th interest course, h i represents the heat correction weight coefficient of the i-th interest course, Y i represents the feature vector of the i-th interest course, and n represents the total number of interest courses in the updated user interest profile.

[0085] In the present invention, the attention weight can be adjusted according to the user's attention to different interest courses, so as to achieve personalized recommendations. For courses that users pay more attention to, higher attention weights are assigned, which have a greater impact on the final interest fusion feature vector, making the recommendation more in line with the user's personalized interests and preferences. The combination of attention weight and heat correction weight can effectively balance the relationship between the user's personalized interests and the popularity of the course. If a course is very popular but the user is not very interested in it, the weight of the course in the recommendation can be reduced by the joint use of attention weight and heat correction weight, so as to better meet the personalized needs of users.

[0086] Optionally, the attention weight coefficient is calculated as follows:

[0087] λ=hT ReLU[W(Y * ⊙Y i )+b]

[0088] Among them, λ represents the attention weight coefficient vector, which includes the attention weight coefficients of each interest course, h represents the projection vector, and h T represents the transpose of the projection vector, which is used to decide how to adjust the attention weight according to the interest courses and target courses in the updated user interest profile. ReLU() represents the activation function. Y * The feature vector representing the target video course, Y i represents the feature vector of the i-th interest course, ⊙ represents the element-by-element product operation, W represents the weight matrix, and b represents the bias term.

[0089] In the present invention, the weight of each interest course can be adaptively adjusted according to the similarity and correlation between the user's interest and the target course, so that better attention can be paid to the interest courses that are relevant and important to the target course, thereby improving the relevance and accuracy of the recommendation.

[0090] Calculate the cosine similarity between the feature vector of the target video course and the user's interest fusion feature vector as the recommendation index of the target video course:

[0091]

[0092] Among them, σ k represents the recommendation index of the kth target video course, Y k represents the feature vector of the kth target video course, represents the transpose of the feature vector of the kth target video course, Y represents the interest fusion feature vector, and ∥∥ represents the modulo operation of the vector.

[0093] In the present invention, by calculating the cosine similarity between the target video course feature vector and the user interest fusion feature vector, the matching degree between the user interest and the course can be quantitatively measured. Courses with high recommendation indexes mean that they are more similar to the user's interests in the feature space and are more likely to meet the user's preferences, thereby achieving personalized recommendations.

[0094] S5: Based on the quality and praise rate of each video course, make quality corrections to the recommendation index of each video course.

[0095] In a possible implementation, S5 specifically includes: performing quality correction on the recommendation index of each video course according to the following formula:

[0096]

[0097] in, represents the revised recommendation index of the kth target video course, σ k represents the recommendation index of the kth target video course, represents the quality correction factor of the kth target video course, Represents the positive review correction factor of the k-th target video course.

[0098] In the present invention, after determining the recommendation index of the video course, the system makes corrections based on the quality and favorable rating of each video course. This ensures that the recommended video courses not only meet the user's interests, but also have high quality and user evaluation, thereby improving the user's trust and satisfaction with the course.

[0099] Optionally, the quality correction factor for the kth target video course Specifically:

[0100]

[0101] in, represents the average browsing time of the kth target video course, l k Represents the total duration of the kth target video course.

[0102] In the present invention, by considering the ratio of the actual viewing time of the user to the total duration of the video course, the actual preferences and viewing behavior of the user can be better reflected. If the average viewing time of a video course is much longer than the total duration, it means that most users will watch it in its entirety after viewing it, which may mean that the video quality is high or the content is more attractive. Therefore, the quality correction factor will more prominently recommend such popular high-quality video courses.

[0103] Optionally, the positive rating correction factor of the kth target video course Specifically:

[0104]

[0105] Among them, b k represents the total number of people who give positive comments on the kth target video course, B k Represents the total number of people who evaluated the k-th target video course.

[0106] In the present invention, the positive review correction factor takes into account the user's evaluation of the video course. The higher the proportion of positive reviews of a video course by users, the more popular the video course is among users and the more likely it is to be a high-quality and interesting course. By calculating the positive review correction factor, the system will increase the recommendation index of popular video courses, making it easier for users to see and choose these well-received courses.

[0107] S6: Recommending video courses to users based on the revised recommendation index of each video course.

[0108] Optionally, the top 10 video courses may be displayed to the user in descending order of the revised recommendation index to recommend the video courses to the user.

[0109] In a possible implementation, the training method of the video course recommendation model specifically includes:

[0110] With the goal of maximizing customer praise rate, the video course recommendation model is trained using an improved particle swarm optimization algorithm.

[0111] It should be noted that training the video course recommendation model with the improved particle swarm optimization algorithm is expected to improve the performance and effect of the recommendation model, so that the recommended courses are more in line with customer preferences and needs, thereby increasing customer praise for the recommended courses.

[0112] Among them, the particle swarm optimization algorithm (PSO) is a group intelligence optimization algorithm inspired by the group behavior of biological groups such as bird flocks or fish schools. The PSO algorithm simulates the behavior of bird flocks when foraging. Each individual (particle) adjusts its position and speed based on its own experience and the experience of neighboring individuals to find the optimal solution in the search space.

[0113] Specifically, the traditional particle swarm optimization algorithm has a "clustering effect" during the search process, which causes particles to tend to move towards the local optimal solution, which may miss the global optimal solution and easily fall into the local optimal solution. The improved particle swarm optimization algorithm proposed in the present invention introduces the crossover and mutation operations of the genetic algorithm to solve the problems of the traditional particle swarm optimization algorithm. The specific implementation method is:

[0114] Initialize the particle swarm, which contains multiple particles. The position of each particle represents a feasible model parameter. The model parameters include: projection vector h, weight matrix W, bias term b and other model parameters.

[0115] The customer praise rate is used as the fitness function of the particle swarm.

[0116] Calculate the fitness value of each particle in the particle swarm.

[0117] According to the fitness value of each particle, the individual optimal position and the global optimal position are determined.

[0118] Generate a random number and compare it with the crossover probability p e Compare the size, if the random number is less than the crossover probability p e, then the position of the current particle is cross-operated with the individual optimal position to update the position of the current particle:

[0119] Z new =rZ+(1-r)Z best

[0120] Among them, Z new represents the updated position of the current particle, Z represents the original position of the current particle, r represents a random number between 0 and 1, and Z best represents the optimal position of an individual.

[0121] It should be noted that the cross operation between the current particle position and the individual optimal position can utilize the individual optimal information, increase the diversity of the search, accelerate the convergence speed, maintain the population diversity, and increase the exploratory nature of the search space, thereby improving the efficiency and performance of the particle swarm optimization algorithm.

[0122] If the fitness value of the updated current particle is greater than the individual optimal position, the individual optimal position is replaced by the current particle position.

[0123] Among them, the crossover probability P e Specifically:

[0124]

[0125] Among them, P e represents the crossover probability, P e,max represents the maximum crossover probability, P e,min represents the minimum crossover probability, δ represents the fitness value of the current particle, δ avg represents the average fitness value of the population, δ max Represents the maximum fitness value among the particles.

[0126] It should be noted that the crossover probability is adjusted according to the fitness value of the current particle, so that when the fitness is high, the crossover probability is large, which is conducive to better exploration of the search space and finding a better solution; when the fitness is low, the crossover probability is reduced, which helps to reduce unnecessary crossover operations and prevent falling into the local optimal solution.

[0127] Continue to generate a new random number and compare it with the crossover probability P e Compare the size, if the random number is less than the crossover probability P e , then the position of the current particle is cross-operated with the global optimal position to update the position of the current particle:

[0128] Z new =rZ+(1-r)P best

[0129] Among them, Znew represents the updated position of the current particle, Z represents the original position of the current particle, r represents a random number between 0 and 1, and P best represents the global optimal position.

[0130] It should be noted that the cross operation between the current particle position and the global optimal position can utilize the global optimal information, increase the diversity of the search, accelerate the convergence speed, maintain the population diversity, and increase the exploratory nature of the search space, thereby improving the efficiency and performance of the particle swarm optimization algorithm.

[0131] If the fitness value of the updated current particle is greater than the individual optimal position, the individual optimal position is replaced by the current particle position.

[0132] Continue to generate a new random number and compare it with the mutation probability P m Compare the size. If the random number is less than the mutation probability P, m , then the position of the current particle is mutated and updated:

[0133]

[0134] Among them, Z new represents the updated position of the current particle, Z represents the original position of the current particle, r represents a random number between 0 and 1, and Z best represents the optimal position of an individual, Z worst Indicates the worst position of an individual.

[0135] It should be noted that the mutation operation method is conducive to increasing the diversity of the population, avoiding falling into the local optimal solution, exploring new solution space, and increasing the adaptability of individuals, thereby helping the improved particle swarm optimization algorithm to better search for high-quality optimal solutions.

[0136] Among them, the mutation probability P m Specifically:

[0137]

[0138] Among them, P m represents the mutation probability, P m,max represents the maximum mutation probability, P m,min represents the minimum mutation probability, δ represents the fitness value of the current particle, and δ avg represents the average fitness value of the population, δ max Represents the maximum fitness value among the particles.

[0139] It should be noted that the mutation probability is adjusted according to the fitness value of the current particle, so that when the fitness is high, the mutation probability is large, which is conducive to better exploration of the search space and finding a better solution; when the fitness is low, the mutation probability is reduced, which helps to reduce unnecessary mutation operations and prevent falling into the local optimal solution.

[0140] If the fitness value of the updated current particle is greater than the individual optimal position, the individual optimal position is replaced by the current particle position.

[0141] Update the global optimal position.

[0142] Determine whether the number of iterations has reached the maximum number of iterations. If so, output the model parameters represented by the position of the particle with the highest fitness value. Otherwise, return to continue iterating.

[0143] In the present invention, the improved particle swarm optimization algorithm combines the advantages of the two algorithms by introducing the crossover and mutation operations of the genetic algorithm, improves the search diversity, search efficiency and global convergence, and makes the algorithm more robust and applicable to different types of optimization problems.

[0144] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0145] (1) In the present invention, a hierarchical reinforcement learning network is used to maintain the user interest profile. The high-level reinforcement learning network can decide whether to update the user interest profile. If an update is required, the low-level reinforcement learning network can decide whether to delete some interest courses in the user interest profile to capture the user's dynamic interest in continuous learning behavior. This can better adapt to complex learning processes and interest changes, and the accuracy of personalized recommendations will decrease, thereby improving the accuracy of personalized recommendations.

[0146] (2) In the present invention, after determining the recommendation index of a video course, the system makes corrections based on the quality and favorable rating of each video course. This ensures that the recommended video courses not only meet the user's interests, but also have high quality and user ratings, thereby increasing the user's trust and satisfaction with the course.

[0147] Reference Manual Attached Figure 2 , showing a structural schematic diagram of a personalized video course recommendation system provided by the present invention.

[0148] The present invention also provides a personalized video course recommendation system 20, which is applied to the above-mentioned personalized video course recommendation method, comprising:

[0149] Processor 201;

[0150] The memory 202 stores computer-readable instructions, and when the computer-readable instructions are executed by the processor 201, the personalized video course recommendation method as described in the method embodiment is implemented.

[0151] The personalized video course recommendation system 20 provided by the present invention can execute the above-mentioned personalized video course recommendation method and achieve the same or similar technical effects. To avoid repetition, the present invention will not go into details.

[0152] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0153] (1) In the present invention, a hierarchical reinforcement learning network is used to maintain the user interest profile. The high-level reinforcement learning network can decide whether to update the user interest profile. If an update is required, the low-level reinforcement learning network can decide whether to delete some interest courses in the user interest profile to capture the user's dynamic interest in continuous learning behavior. This can better adapt to complex learning processes and interest changes. The accuracy of personalized recommendations will decrease, and the accuracy of personalized recommendations will be improved.

[0154] (2) In the present invention, after determining the recommendation index of a video course, the system makes corrections based on the quality and favorable rating of each video course. This ensures that the recommended video courses not only meet the user's interests, but also have high quality and user ratings, thereby increasing the user's trust and satisfaction with the course.

[0155] It should be understood that the processor in the embodiment of the present invention may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0156] It should also be understood that the memory in the embodiments of the present invention may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memories. Among them, the non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of random access memory (RAM) are available, such as static RAM (SRAM), dynamic random access memory (DRAM), synchronous DRAM (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link DRAM (SLDRAM), and direct rambus RAM (DR RAM).

[0157] The above embodiments can be implemented in whole or in part by software, hardware (such as circuits), firmware or any other combination. When implemented by software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that contains one or more available media sets. The available medium can be a magnetic medium (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state hard disk.

[0158] It should be understood that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the associated objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding.

[0159] In the present invention, "at least one" means one or more, and "more than one" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or multiple.

[0160] It should be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0161] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0162] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described equipment, devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0163] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0164] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0165] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0166] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0167] An embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the personalized video course recommendation method as described in the method embodiment is implemented.

[0168] A computer-readable storage medium provided by the present invention can implement the steps and effects of the personalized video course recommendation method of the above method embodiment. To avoid repetition, the present invention will not go into details.

[0169] The beneficial effects brought about by the technical solution provided by the embodiment of the present invention include at least:

[0170] (1) In the present invention, wavelet transform is used to extract multiple feature graphs from the voltage prediction signal, which can more comprehensively reflect the changes in the power grid state. The disturbance parameters are calculated based on the feature graphs. The disturbance parameters may include frequency disturbance, voltage fluctuation, harmonic distortion, etc., which can more accurately quantify the disturbance in the power grid. Then, the power grid operation quality can be detected in real time based on the disturbance parameters. It can more accurately reflect the power grid operation status when a large number of wind power systems are connected, adapt to the intermittent and uncertain nature of the wind power system, reduce false alarms, and improve detection accuracy.

[0171] (2) In the present invention, the voltage signal is predicted by the improved GRU-RNN model, which can more accurately predict the voltage change trend in the future period, carry out status warning, and respond to the changes in the grid operation status in a timely manner, which helps to make corresponding adjustments and measures in advance before changes occur in the grid.

[0172] (3) In the present invention, when the grid operation quality is poor, the improved GRU-RNN is used for intelligent diagnosis, which can help determine the possible problems or faults in the grid, help to quickly locate the root cause of the problem, and guide the operation and maintenance personnel to perform effective maintenance and processing, which can effectively monitor and manage the grid operation status and improve the reliability, safety and economy of the grid.

[0173] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed by the present invention, which should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

[0174] There are a few points to note:

[0175] (1) The drawings of the embodiments of the present invention only relate to the structures related to the embodiments of the present invention, and other structures may refer to the general design.

[0176] (2) For the sake of clarity, in the drawings used to describe the embodiments of the present invention, the thickness of the layers or regions is exaggerated or reduced, that is, these drawings are not drawn according to the actual scale. It is understood that when an element such as a layer, film, region or substrate is referred to as being "on" or "under" another element, the element may be "directly" "on" or "under" the other element or there may be intermediate elements.

[0177] (3) In the absence of conflict, the embodiments of the present invention and the features therein may be combined with each other to obtain new embodiments.

[0178] The above are only specific embodiments of the present invention, but the protection scope of the present invention is not limited thereto. The protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A personalized video course recommendation method, characterized in that: include: S1: Constructing a hierarchical reinforcement learning network and a video course recommendation model, wherein the hierarchical reinforcement learning network includes a high-level reinforcement learning network and a low-level reinforcement learning network that are interconnected; S2: Obtain the user's course learning data; S3: Based on the course learning data, the high-level reinforcement learning network is used to decide whether the user interest profile needs to be updated. If so, the low-level reinforcement learning network is used to decide whether to delete some of the interest courses in the user interest profile, and the user interest profile is updated by the course learning data after deleting some of the historical courses. S4: Determine the recommendation index of each video course through the video course recommendation model according to the updated user interest profile; S5: Based on the quality and praise rate of each video course, the recommendation index of each video course is corrected; S6: Recommending video courses to users based on the revised recommendation index of each video course.

2. The personalized video course recommendation method according to claim 1, characterized in that: The environment space of the high-level reinforcement learning network is specifically: S1 = (s1, s2), s1 represents the average value of the cosine similarity between the feature vector of each interest course in the user interest profile and the feature vector of the target video course, s2 represents the average value of the element-by-element product between the feature vector of each interest course in the user interest profile and the feature vector of the target video course, wherein the target video course is the video course that the user has learned most recently in the course learning data; The action space of the high-level reinforcement learning network is specifically: A1 = {0, 1}, 0 means that the user interest profile does not need to be updated, and 1 means that the user interest profile needs to be updated; The reward function of the high-level reinforcement learning network is used to provide rewards or penalties for the high-level reinforcement learning network according to the action execution results, specifically: R h (a h ,s h )=logP(θ|s h )-logP(θ'|s h ) Among them, R h () represents the reward function of the high-level reinforcement learning network, a h represents the action of the high-level reinforcement learning network, s h represents the state of the high-level reinforcement learning network, log represents the logarithmic function with base 10, θ represents the user interest summary before update, θ' represents the user interest summary before update, logO(θ|s) represents the log-likelihood estimate of the user interest summary before update, and logP(θ'|s) represents the log-likelihood estimate of the user interest summary after update.

3. The personalized video course recommendation method according to claim 1, characterized in that: The environment space of the low-level reinforcement learning network is specifically: S2=(s3), where s3 represents the cosine similarity between the feature vectors of each interest course in the user interest profile and the feature vector of the target video course; The action space of the low-level reinforcement learning network is specifically: A2 = {0, 1}, 0 means that the current interest course does not need to be deleted, and 1 means that the current interest course needs to be deleted; The reward function of the low-level reinforcement learning network is used to provide rewards or penalties for the low-level reinforcement learning network according to the action execution results, specifically: R l (the l ,s l )=Yes avg (x,x * )-Yes' avg (x,x * ) Among them, R l () represents the reward function of the low-level reinforcement learning network, a l represents the action of the low-level reinforcement learning network, s l represents the state of the low-level reinforcement learning network, x represents the course of interest, and x * Indicates the target video course, Sim avg (x,x * ) represents the average cosine similarity between each interest course and the target video course, Sim' avg (x,x * ) represents the average value of the cosine similarity between each interest course and the target video course after deleting some interest courses.

4. The personalized video course recommendation method according to claim 1, characterized in that: The high-level reinforcement learning network and / or the low-level reinforcement learning network are trained by a gradient descent method.

5. The personalized video course recommendation method according to claim 1, characterized in that: The S4 specifically includes: The heat correction weight is introduced, and the feature vectors of each interest course in the updated user interest profile are fused through the video course recommendation model to obtain the user's interest fusion feature vector: Among them, Y represents the interest fusion feature vector, h i represents the heat correction weight coefficient of the i-th interest course, Y i represents the feature vector of the i-th interest course, and n represents the total number of interest courses in the updated user interest profile; The cosine similarity between the feature vector of the target video course and the interest fusion feature vector of the user is calculated as the recommendation index of the target video course: Among them, σ k represents the recommendation index of the kth target video course, Y k represents the feature vector of the kth target video course, represents the transpose of the feature vector of the kth target video course, Y represents the interest fusion feature vector, and ∥∥ represents the modulo operation of the vector.

6. The personalized video course recommendation method according to claim 1, characterized in that: The S4 specifically includes: The heat correction weight and attention weight are introduced, and the feature vectors of each interest course in the updated user interest profile are fused through the video course recommendation model to obtain the user's interest fusion feature vector: Among them, Y represents the interest fusion feature vector, λ i represents the attention weight coefficient of the i-th interest course, h i represents the heat correction weight coefficient of the i-th interest course, Y i represents the feature vector of the i-th interest course, and n represents the total number of interest courses in the updated user interest profile; The cosine similarity between the feature vector of the target video course and the interest fusion feature vector of the user is calculated as the recommendation index of the target video course: Among them, σ k represents the recommendation index of the kth target video course, Y k represents the feature vector of the kth target video course, represents the transpose of the feature vector of the kth target video course, Y represents the interest fusion feature vector, and ∥∥ represents the modulo operation of the vector.

7. The personalized video course recommendation method according to claim 6, characterized in that: The calculation method of the attention weight coefficient is specifically as follows: λ=h T ReLU[W(Y * ⊙Y i )+b] Among them, λ represents the attention weight coefficient vector, which includes the attention weight coefficients of each interest course, h represents the projection vector, and h T represents the transpose of the projection vector, which is used to determine how to adjust the attention weight according to the interest courses and target courses in the updated user interest profile, ReLU () represents the activation function, and Y * The feature vector representing the target video course, Y i represents the feature vector of the i-th interest course, ⊙ represents the element-by-element product operation, W represents the weight matrix, and b represents the bias term.

8. The personalized video course recommendation method according to claim 7, characterized in that: The training method of the video course recommendation model specifically includes: With the goal of maximizing customer praise rate, the video course recommendation model is trained by using an improved particle swarm optimization algorithm.

9. The personalized video course recommendation method according to claim 1, characterized in that: The S5 is specifically: The quality of the recommendation index of each video course is corrected according to the following formula: in, represents the revised recommendation index of the kth target video course, σ k represents the recommendation index of the kth target video course, represents the quality correction factor of the kth target video course, Represents the positive review correction factor of the k-th target video course.

10. A personalized video course recommendation system, characterized in that: include: processor; A memory having computer-readable instructions stored thereon, wherein when the computer-readable instructions are executed by the processor, the personalized video course recommendation method according to any one of claims 1 to 9 is implemented.

Images