Course recommendation method and related equipment based on text similarity
By constructing a course content corpus and generating course vectors using multiple models, determining the optimal weights, and constructing a similarity matrix, the problem of dependence on user behavior records in existing technologies is solved, achieving efficient and accurate course recommendation.
Patent Information
- Application Number
- CN202010355262.5
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2020-04-29
- Publication Date
- 2024-11-05
- Estimated Expiration
- 2040-04-29
AI Technical Summary
Existing course recommendation methods rely too heavily on user behavior records, resulting in low recommendation efficiency and accuracy.
The text similarity-based course recommendation method obtains the content features of courses, constructs a course content corpus, generates course vectors using TF-IDF model, topic model, and Word2vec model, determines the optimal weights using an ensemble tree model, constructs a similarity matrix, and recommends similar courses.
It reduces reliance on user behavior records, improves the efficiency and accuracy of course recommendations, reduces computational complexity, and avoids the subjectivity and tediousness of manually setting weights.
Smart Images

Figure CN111639485B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of data processing, and in particular to a course recommendation method, device, computer device and storage medium based on text similarity. Background Art
[0002] In the era of widespread Internet access, people are increasingly inclined to acquire knowledge on the Internet. Online courses are popular among users due to their convenience, variety of formats, and repeatable learning. There are many types of online course videos, and there are many sub-fields in different educational fields. How to find courses that users like is an urgent problem to be solved in the online education industry.
[0003] Traditional solutions require course recommendations based on a large number of user behavior records. For example, one method is to simply perform statistical sorting based on the number of course clicks, viewing time, etc., evaluate the popularity of the course, and then form a popularity ranking list with the top N courses. However, this method only focuses on courses for which users have behavior. The more user operations, the greater the chance of being recommended. Courses without user behavior will not be recommended. Another method is collaborative filtering, whose basic idea is to recommend courses that have been watched by groups similar to the user or to recommend courses similar to courses that the user has watched. This method also requires the use of user behavior information or user operation information on the course.
[0004] Therefore, it is necessary to provide a course recommendation method that can reduce the dependence on user behavior records and improve the efficiency and accuracy of course recommendations. Summary of the invention
[0005] In view of this, it is necessary to provide a course recommendation method based on text similarity, a course recommendation device based on text similarity, a computer device and a storage medium, which can solve the technical problem that the current course recommendation method relies too much on user behavior records, resulting in low course recommendation efficiency and accuracy.
[0006] A first aspect of an embodiment of the present invention provides a course recommendation method based on text similarity, and the course recommendation method based on text similarity includes:
[0007] Obtaining content features of a course in a preset database, wherein the content features include a course title, a course theme, and a course introduction;
[0008] constructing a course content corpus according to the content characteristics;
[0009] Calling the TF-IDF model to identify the course content corpus to obtain a first vector, calling the topic model to identify the course corpus to obtain a second vector, and calling the Word2vec model to identify the course content corpus to obtain a third vector;
[0010] Determine a first optimal weight corresponding to the first vector, determine a second optimal weight corresponding to the second vector, determine a third optimal weight corresponding to the third vector, and construct a final vector of the course according to the first optimal weight, the second optimal weight, and the third optimal weight;
[0011] Calculating the similarity between any two courses according to the final vectors of the courses, and obtaining a similarity matrix of all courses in the preset database according to the similarity between the any two courses;
[0012] The historically viewed courses corresponding to the user identity information in the preset database are obtained, and the similarity matrix is called according to the historically viewed courses to recommend a course list to the user.
[0013] Furthermore, in the above-mentioned course recommendation method based on text similarity provided by an embodiment of the present invention, the steps of determining the first optimal weight corresponding to the first vector, determining the second optimal weight corresponding to the second vector, and determining the third optimal weight corresponding to the third vector include:
[0014] Get a labeled text similarity dataset;
[0015] Calling the TF-IDF model to identify the text similarity data set to obtain a fourth vector, calling the topic model to identify the text similarity data set to obtain a fifth vector, and calling the Word2vec model to identify the text similarity data set to obtain a sixth vector;
[0016] Transversely splicing the fourth vector, the fifth vector and the sixth vector to obtain a transverse splicing vector;
[0017] Calling the ensemble tree model to train the horizontal splicing vector to obtain a first Shaply value;
[0018] Taking an absolute average of the first Shaply values according to the vertical dimension and scaling them to the [0,1] space, to obtain second Shaply values corresponding to the fourth vector, the fifth vector and the sixth vector;
[0019] The second Shaply values are summed and averaged to obtain the first optimal weight, the second optimal weight and the third optimal weight respectively.
[0020] Furthermore, in the above-mentioned course recommendation method based on text similarity provided by the embodiment of the present invention, the method further includes:
[0021] Checking whether there is a new course in the preset database at a preset time interval;
[0022] When the detection result is that there are new courses in the preset database, the number of the new courses is obtained;
[0023] Determine whether the quantity exceeds a preset new quantity threshold;
[0024] When the judgment result is that the number exceeds the preset new number threshold, calling the first updating rule to update the similarity matrix;
[0025] When the judgment result is that the number does not exceed the preset new number threshold, the second updating rule is called to update the similarity matrix.
[0026] Furthermore, in the above-mentioned course recommendation method based on text similarity provided by an embodiment of the present invention, the step of calling the first updating rule to update the similarity matrix includes:
[0027] Acquire the new content features of the new course, wherein the new content features include the title of the new course, the subject of the new course and the introduction of the new course;
[0028] Updating the course content corpus according to the newly added content features;
[0029] Call the TF-IDF model to identify the updated course content corpus to obtain an updated first vector, call the topic model to identify the updated course content corpus to obtain an updated second vector, and call the Word2vec model to identify the updated course content corpus to obtain an updated third vector;
[0030] Determine a first optimal weight corresponding to the updated first vector, determine a second optimal weight corresponding to the updated second vector, determine a third optimal weight corresponding to the updated third vector, and update the final vectors of all courses according to the first optimal weight, the second optimal weight and the third optimal weight that are newly acquired;
[0031] The similarity between any two courses is calculated according to the final vectors after all courses are updated, and an updated similarity matrix is obtained according to the similarity between the any two courses.
[0032] Furthermore, in the above-mentioned course recommendation method based on text similarity provided by an embodiment of the present invention, the step of calling the second updating rule to update the similarity matrix includes:
[0033] Acquire the new content features of the new course, wherein the new content features include the title of the new course, the subject of the new course and the introduction of the new course;
[0034] Updating the course content corpus according to the newly added content features;
[0035] Call the TF-IDF model to identify the updated course content corpus to obtain the first vector corresponding to the new course, call the topic model to identify the updated course content corpus to obtain the second vector corresponding to the new course, and call the Word2vec model to identify the updated course content corpus to obtain the third vector corresponding to the new course;
[0036] The final vector of the newly added course is obtained according to the first vector, the second vector, the third vector and the first optimal weight, the second optimal weight and the third optimal weight, and the similarity values with other courses are calculated according to the final vector of the newly added course, and the similarity matrix is updated according to the calculated similarity values.
[0037] Furthermore, in the above-mentioned course recommendation method based on text similarity provided by the embodiment of the present invention, the method further includes:
[0038] When there are multiple historically viewed courses, each of the historically viewed courses is determined as a target course;
[0039] According to the similarity matrix, the course with the first preset vector ranking of similarity to the target course is selected to form an initial list of courses to be recommended;
[0040] The courses in the initial list of courses to be recommended are arranged in descending order of similarity, and the courses with the second highest preset vector of similarity ranking are selected as the list of courses to be recommended.
[0041] Furthermore, in the above-mentioned course recommendation method based on text similarity provided by an embodiment of the present invention, the step of constructing a course content corpus according to the content features includes:
[0042] Performing data cleaning on the content features, wherein the data cleaning includes one or more of the following: removing links, removing invalid characters, and removing invalid sentences;
[0043] The content features after data cleaning are processed by Chinese word segmentation, and the results of the Chinese word segmentation processing are stored in a course content corpus, wherein the course content corpus is a blockchain database.
[0044] A second aspect of an embodiment of the present invention further provides a course recommendation device based on text similarity, the course recommendation device based on text similarity comprising:
[0045] A content feature acquisition module, used to acquire content features of courses in a preset database, wherein the content features include the title of the course, the subject of the course and the introduction of the course;
[0046] A corpus construction module, used to construct a course content corpus according to the content characteristics;
[0047] A vector acquisition module, used to call the TF-IDF model to identify the course content corpus to obtain a first vector, call the topic model to identify the course content corpus to obtain a second vector, and call the Word2vec model to identify the course content corpus to obtain a third vector;
[0048] An optimal value determination module, used to determine a first optimal weight corresponding to the first vector, determine a second optimal weight corresponding to the second vector, determine a third optimal weight corresponding to the third vector, and construct a final vector of the course according to the first optimal weight, the second optimal weight and the third optimal weight;
[0049] A matrix calculation module, used for calculating the similarity between any two courses according to the final vectors of the courses, and obtaining the similarity matrix of all courses in the preset database according to the similarity between the any two courses;
[0050] The recommended course generation module is used to obtain the historically viewed courses corresponding to the user identity information in the preset database, and call the similarity matrix according to the historically viewed courses to recommend a course list to the user.
[0051] A third aspect of an embodiment of the present invention further provides a computer device, the computer device comprising a processor, the processor being configured to implement any of the above-mentioned course recommendation methods based on text similarity when executing a computer program stored in a memory.
[0052] A fourth aspect of an embodiment of the present invention further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the course recommendation method based on text similarity as described in any one of the above items is implemented.
[0053] The embodiments of the present invention provide a course recommendation method based on text similarity, a course recommendation device based on text similarity, a computer device and a storage medium, which can recommend similar courses to users based on the text content features of the courses, and do not rely on the user's behavioral records of the courses; the similarity matrix in the present invention is the relationship between courses, and is a dense matrix. When the course similarity matrix is used for related processing, computer resources can be fully utilized; and in the present invention, when a new course is added to a preset database, it is only necessary to vectorize the new course content and calculate the similarity with the original course in the database, which can reduce the calculation complexity; and the present invention adopts a supervised learning method to determine the optimal weights corresponding to the first vector, the second vector and the third vector, which can avoid the subjectivity of manually setting weights and the tediousness of testing the optimal weights, improve the calculation efficiency of the optimal weights, and thus improve the course recommendation efficiency based on text similarity. BRIEF DESCRIPTION OF THE DRAWINGS
[0054] Figure 1 It is a flowchart of the course recommendation method based on text similarity provided by the first embodiment of the present invention.
[0055] Figure 2 It is a schematic diagram of the structure of a computer device according to one embodiment of the present invention.
[0056] Figure 3 yes Figure 2 An exemplary functional block diagram of a computer device is shown.
[0057] The following specific implementation manner will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION
[0058] In order to more clearly understand the above-mentioned purpose, features and advantages of the present invention, the present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0059] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. The embodiments described are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0060] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art of the present invention. The terms used in the specification of the present invention herein are only for the purpose of describing specific embodiments and are not intended to limit the present invention.
[0061] Figure 1 1 is a flowchart of a course recommendation method based on text similarity according to a first embodiment of the present invention. The course recommendation method based on text similarity can be applied to a computer device, which can be a smart device such as a smart phone, a laptop, a desktop / tablet computer, a smart watch, etc. Figure 1 As shown, the course recommendation method based on text similarity may include the following steps:
[0062] S11. Obtain content features of courses in a preset database, wherein the content features include the title of the course, the subject of the course, and an introduction to the course.
[0063] In at least one embodiment of the present invention, content features of courses in a preset database are obtained, wherein the content features include the title of the course, the theme of the course, and the introduction of the course. The title of the course is a short sentence indicating the content of the course; the theme of the course is a sentence indicating the central idea to be expressed by the course; the introduction of the course is a concise introduction to the content of the course. The file format of the course includes video, audio, etc. The preset database is a website, APP, etc. containing different types of courses. In one embodiment, the preset database is an APP containing internal education and training courses of the company.
[0064] S12. Construct a course content corpus according to the content features.
[0065] In at least one embodiment of the present invention, a course content corpus is constructed according to the content features, wherein the course content corpus can store the results of word segmentation of the content features of all courses in the form of an XML file.
[0066] Specifically, the step of constructing a course content corpus according to the content features includes: performing data cleaning on the content features, wherein the data cleaning includes one or more of the following: removing links, removing invalid characters, and removing invalid sentences; performing Chinese word segmentation processing on the content features after data cleaning, and storing the results of the Chinese word segmentation processing in the course content corpus. Among them, the Chinese word segmentation processing refers to the process of splitting a sequence of Chinese characters in a sentence into multiple separate words. Specifically, in practical applications, the existing Chinese word segmentation code can be used to perform Chinese word segmentation processing on the content features of the course, and punctuation marks can be removed from the word segmentation results. Further, the course content corpus in the present invention can be a blockchain database stored on a blockchain node. The information, video data, etc. obtained in this solution are all stored in a blockchain node, so that the advantages of blockchain storage itself can be utilized: the blockchain system is different from the storage method of traditional information projects in terms of storage. The storage of blockchain projects is that the nodes participating in the bookkeeping maintain the same account book content, and each bookkeeping node can view the data on the chain.
[0067] The blockchain referred to in this invention is a new application mode of computer technologies such as distributed data storage, peer-to-peer transmission, consensus mechanism, encryption algorithm, etc. Blockchain is essentially a decentralized database, a string of data blocks generated by cryptographic methods. Each data block contains a batch of network transaction information, which is used to verify the validity of its information (anti-counterfeiting) and generate the next block. Blockchain can include the underlying blockchain platform, platform product service layer, and application service layer.
[0068] S13. Calling the TF-IDF model to identify the course content corpus to obtain a first vector, calling the topic model to identify the course content corpus to obtain a second vector, and calling the Word2vec model to identify the course content corpus to obtain a third vector.
[0069] In at least one embodiment of the present invention, this solution is based on artificial intelligence technology, mainly calling the TF-IDF model, topic model and Word2vec model in the field of artificial intelligence to train the course content corpus to obtain the first vector V of the course content T , the second vector V M and the third vector V W . Wherein, the first vector V T The second vector V is the result of calling the TF-IDF model to map the course content corpus to the vector space. M The third vector V is the result of calling the topic model to map the course content corpus to the vector space. W The result of calling the Word2vec model to map the course content corpus into the vector space.
[0070] Among them, the TF-IDF model is a statistical method used to evaluate the importance of a word or phrase to a course in a corpus. TF means term frequency. The term frequency refers to the frequency of a given word in the course. This number is a normalization of the term count to prevent it from being biased towards long texts (the same word may have a higher term count in a long file than in a short file, regardless of whether the word is important or not). IDF means inverse document frequency. The IDF of a specific word can be obtained by dividing the total number of files by the number of files containing the word, and then taking the logarithm of the quotient.
[0071] The main idea of the TF-IDF model includes: when a word or phrase appears frequently in one course and less frequently in other courses, it is considered that the word or phrase has good category differentiation ability. The TF-IDF model uses the product of word frequency and inverse document frequency to represent words in a course.
[0072] Specifically, the TF-IDF model is called to identify the course content corpus to obtain the first vector V T The steps include: calling the TF-IDF model to extract the word segment corresponding to any course in the course content corpus; counting the word frequency of the word segment in the course; obtaining the inverse document frequency of the word segment; calculating the product of the word frequency and the inverse document frequency to obtain the first vector V of the course contentT Among them, the words with different numbers can be padded with 0 to ensure that the length of the first vector of all courses is consistent.
[0073] The topic model is a probability distribution with all characters in the text as the support set, which is used to represent the frequency of the characters appearing in the topic. The topic model includes a latent semantic indexing (LSI) model. The latent semantic indexing model obtains the topic of the text based on the singular value decomposition (SVD) method. The latent semantic indexing model extracts the meanings related to documents and terms by mapping high-dimensional vectors to latent semantic space, thereby analyzing the relationship between documents and terms.
[0074] Specifically, the calling topic model identifies the course content corpus to obtain a second vector V M The steps include: calling the topic model to map all courses in the course content corpus into a matrix; decomposing the matrix by a singular value decomposition method to obtain a potential topic feature vector of the course, that is, the second vector V M .
[0075] Among them, the Word2vec model is a method for distributed representation of words. The conditional probability of a word sequence is fit through a forward feedback neural network. After training, the middle layer is extracted as the vector representation of the word. The vector representation of the words in the course content is added and averaged to obtain the vector of the course content.
[0076] Specifically, the calling of the Word2vec model to identify the course content corpus obtains a third vector V W The steps include: calling the Word2vec model to calculate the word vector of any word segmentation of a course in the course content corpus; adding and averaging the word vectors of the word segmentation of any course to obtain a third vector V of the course content W .
[0077] S14. Determine a first optimal weight corresponding to the first vector, determine a second optimal weight corresponding to the second vector, determine a third optimal weight corresponding to the third vector, and construct a final vector of the course based on the first optimal weight, the second optimal weight and the third optimal weight.
[0078] In at least one embodiment of the present invention, the first optimal weight corresponding to the first vector is determined, the second optimal weight corresponding to the second vector is determined, the third optimal weight corresponding to the third vector is determined, and the final vector of the course is constructed according to the first optimal weight, the second optimal weight and the third optimal weight. The first vector, the second vector and the third vector are the results of respectively calling the TF-IDF model, the topic model and the Word2vec model to map the course content corpus to the vector space. Since the TF-IDF model is mainly based on word frequency statistics, it is more sensitive to long texts; the topic model is a statistical probability model based on word frequency to represent the text topic, which is more sensitive to long texts; the Word2vec model is based on context training of shallow neural networks to obtain vector representations of words, which is insensitive to the length of the text. Therefore, it is necessary to set corresponding weights for the first vector, the second vector and the third vector to combine the advantages and disadvantages of different models.
[0079] At present, most methods determine the weights corresponding to the three vectors by manual setting. However, this method is highly dependent on manual experience and is easily affected by human subjective factors. This method requires tedious manual debugging and consumes a lot of manpower. Moreover, the result obtained by similarity calculation of the vectors obtained by manual weighting belongs to an unsupervised method, and the accuracy of the result is difficult to evaluate. Therefore, in at least one embodiment of the present invention, a supervised learning method is used to determine the first optimal weight corresponding to the first vector, the second optimal weight corresponding to the second vector, and the third optimal weight corresponding to the third vector, which can avoid the subjectivity of manual setting of weights and the tediousness of testing optimal weights, improve the efficiency of optimal weight processing, and thus improve the efficiency of course recommendation based on text similarity.
[0080] In the actual processing process, the present invention does not know the similar courses of each course in the preset database, and the purpose of the present invention is to find similar courses for each course. Therefore, it is necessary to obtain the optimal weights through a known text similarity data set. It can be understood that since the text similarity data set and the course corpus both contain the content features of the text, and both call the TF-IDF model, the topic model and the Word2vec model to train the content features of the text, the three optimal weights obtained by calling the integrated tree model to train the text similarity data set are also the first optimal weight corresponding to the first vector in the course corpus, the second optimal weight corresponding to the second vector and the third optimal weight corresponding to the third vector.
[0081] Specifically, the steps of determining the first optimal weight corresponding to the first vector, determining the second optimal weight corresponding to the second vector, and determining the third optimal weight corresponding to the third vector include: obtaining a labeled text similarity data set; calling a TF-IDF model to identify the text similarity data set to obtain a fourth vector, calling a topic model to identify the text similarity data set to obtain a fifth vector, and calling a Word2vec model to identify the text similarity data set to obtain a sixth vector; horizontally splicing the fourth vector, the fifth vector, and the sixth vector to obtain a horizontal splicing vector; calling an integrated tree model to train the horizontal splicing vector to obtain a first shaply value; taking an absolute average of the first shaply value according to the vertical dimension and scaling it to the [0,1] interval to obtain a second shaply value corresponding to the fourth vector, the fifth vector, and the sixth vector; summing and averaging the second Shaply values to obtain the first optimal weight, the second optimal weight, and the third optimal weight, respectively. Among them, the fourth vector is the result of calling the TF-IDF model to map the text similarity dataset to the vector space, the fifth vector is the result of calling the topic model to map the text similarity dataset to the vector space, and the sixth vector is the result of calling the Word2vec model to map the text similarity dataset to the vector space.
[0082] In one embodiment, the text similarity dataset contains a large number of similar or dissimilar texts, and the labels are set to "0" and "1" respectively, wherein "0" indicates that the texts are dissimilar and "1" indicates that the texts are similar. The TF-IDF model, the topic model and the Word2vec model are respectively called to train the text similarity dataset to obtain the fourth vector, the fifth vector and the sixth vector of the text similarity dataset, wherein the fourth vector is the result of calling the TF-IDF model to map the text similarity dataset to the vector space, the fifth vector is the result of calling the topic model to map the text similarity dataset to the vector space, and the sixth vector is the result of calling the Word2vec model to map the text similarity dataset to the vector space. The specific training method will not be repeated here.
[0083] In one embodiment, the ensemble tree model includes a plurality of subtrees, and the number of subtrees included in the ensemble tree model is consistent with the number of texts in the text similarity data set. The training data of each subtree may be composed of the top 5 similar texts and the top 5 dissimilar texts of a text. The interval [0,1] may be scaled in a linear manner, and the number of second shaply values corresponding to the fourth vector, the fifth vector, and the sixth vector is n, which are shap1, shap2…shap respectively. n-1 ,shap n .
[0084] In at least one embodiment of the present invention, the step of constructing the final vector of the course according to the optimal weights comprises: transforming the first vector V T , the second vector V M , the third vector V W Substitute the optimal weights t, m, and w into the following formula (1) to obtain the final vector V of the course
[0085] V=tV T +mV M +W W Formula (1)
[0086] S15. Calculate the similarity between any two courses according to the final vectors of the courses, and obtain a similarity matrix of all courses in the preset database according to the similarity between the any two courses.
[0087] In at least one embodiment of the present invention, the similarity between any two courses is calculated based on the final vector of the course, and the similarity matrix of all courses in the preset database is obtained based on the similarity between the any two courses. a (a1,a2,..a n ), V b (b1,b2,..b n ), cosine similarity can be used to measure the similarity of courses: Finally, the similarity matrix of all courses in the preset database is obtained:
[0088]
[0089] Preferably, the courses in the preset database will be updated, and the number of courses updated in the same time period may be several, dozens, hundreds or even thousands. The steps for calculating the similarity between the newly added courses and all courses are different depending on the number of updated courses.
[0090] That is, the method also includes: detecting whether there are new courses in the preset database at preset time intervals; when the detection result is that there are new courses in the preset database, obtaining the number of new courses; judging whether the number exceeds a preset new number threshold; when the judgment result is that the number exceeds the preset new number threshold, calling the first update rule to update the similarity matrix; when the judgment result is that the number does not exceed the preset new number threshold, calling the second update rule to update the similarity matrix.
[0091] The preset time interval is preset by the computer device, for example, the preset time interval is 7 days. The preset new addition threshold is preset by the computer device, for example, the preset new addition threshold is 100. The first update rule and the second update rule are preset by the computer device, and by judging the number of new courses and the preset new addition threshold, and selecting the corresponding update rule based on the judgment to update the similarity matrix, the appropriate update rule can be selected according to the actual situation of the new courses, thereby improving the update efficiency of the similarity matrix and further improving the course recommendation efficiency.
[0092] Specifically, when the judgment result is that the number of new courses exceeds the preset new threshold, the step of calling the first update rule to update the similarity matrix includes: obtaining the new content features of the new courses, wherein the new content features include the title of the new courses, the theme of the new courses and the introduction of the new courses; updating the course content corpus according to the new content features; calling the TF-IDF model to identify the updated course content corpus to obtain an updated first vector, calling the topic model to identify the updated course content corpus to obtain an updated second vector, and calling the Word2vec model to identify the updated course content corpus to obtain an updated third vector; determining the first optimal weight corresponding to the updated first vector, determining the second optimal weight corresponding to the updated second vector, determining the third optimal weight corresponding to the updated third vector, and updating the final vectors of all courses according to the re-acquired first optimal weight, the second optimal weight and the third optimal weight; calculating the similarity between any two courses according to the updated final vectors of all courses, and obtaining an updated similarity matrix according to the similarity of the any two courses.
[0093] When the judgment result is that the number of new courses does not exceed the preset new threshold, the step of calling the second update rule to update the similarity matrix includes: obtaining the new content features of the new courses, wherein the new content features include the title of the new courses, the theme of the new courses and the introduction of the new courses; updating the course content corpus according to the new content features; calling the TF-IDF model to identify the updated course content corpus to obtain the first vector corresponding to the new courses, calling the topic model to identify the updated course content corpus to obtain the second vector corresponding to the new courses, calling the Word2vec model to identify the updated course content corpus to obtain the third vector corresponding to the new courses; obtaining the final vector of the new courses according to the first vector, the second vector, the third vector and the first optimal weight, the second optimal weight and the third optimal weight, and calculating the similarity values with other courses according to the final vector of the new courses, and updating the similarity matrix according to the calculated similarity values.
[0094] Among them, updating the similarity matrix according to the calculated similarity means adding the similarity between the new course and the original course to the last row (or rows) and the last column (or columns) of the similarity matrix to complete the updating of the similarity matrix.
[0095] It is understandable that, assuming that there are n courses in the similarity matrix constructed based on the content features of all courses in the preset database, the complexity of calculating the similarity between the newly added course and all courses is O(n); when the collaborative filtering method is used for course similarity processing, if it is item-based collaborative filtering, the similarity between items must be calculated first, and then the user's interest in the item is calculated based on the user's historical behavior. Assuming there are m users, the complexity of this calculation is O(m*n). Therefore, the course recommendation method based on text similarity provided by this proposal can reduce the complexity of the calculation.
[0096] S16: Obtain the historically viewed courses corresponding to the user identity information in the preset database, call the similarity matrix according to the historically viewed courses, and recommend a course list to the user.
[0097] In at least one embodiment of the present invention, historically viewed courses corresponding to the user's identity information are obtained from the preset database, and the similarity matrix is called based on the historically viewed courses to recommend a course list to the user. The user's identity information includes the account information used by the user when logging into the preset database. The corresponding historically viewed courses can be obtained based on the user's identity information, and the number of the historically viewed courses can be one or more.
[0098] Specifically, when the number of the historically viewed courses is multiple, the method further includes: when the number of the historically viewed courses is multiple, determining each course in the historically viewed courses as a target course; selecting the course with the first preset vector ranking in similarity to the target course according to the similarity matrix to form an initial list of recommended courses; arranging the courses in the initial list of recommended courses in descending order of similarity, and selecting the course with the second preset vector ranking in similarity as the list of recommended courses. The first preset vector and the second preset vector are preset by the computer device, for example, the first preset vector and the second preset vector are both 5.
[0099] The embodiment of the present invention provides a course recommendation method based on text similarity, which can recommend similar courses to users based on the text content features of the courses, and does not rely on the user's behavior record on the courses; the similarity matrix in the present invention is the relationship between courses, and is a dense matrix. When the course similarity matrix is used for related processing, computer resources can be fully utilized; and in the present invention, when a new course is added to a preset database, it is only necessary to vectorize the content of the new course and calculate the similarity with the original course in the database, which can reduce the calculation complexity; and the present invention adopts a supervised learning method to determine the optimal weights corresponding to the first vector, the second vector and the third vector, which can avoid the subjectivity of manually setting weights and the tediousness of testing the optimal weights, improve the calculation efficiency of the optimal weights, and thus improve the course recommendation efficiency based on text similarity.
[0100] At the same time, this solution can be applied in the field of smart education, thereby promoting the construction of smart cities.
[0101] The above is a detailed description of the method provided by the embodiment of the present invention. According to different requirements, the execution order of the blocks in the flowchart can be changed, and some blocks can be omitted. The computer device 1 provided by the embodiment of the present invention is described below.
[0102] An embodiment of the present invention further provides a computer device 1, comprising a memory 10, a processor 30, and a computer program stored in the memory 10 and executable on the processor 30, wherein the processor 30 implements the steps of the course recommendation method based on text similarity described in any of the above-mentioned embodiments when executing the program.
[0103] Figure 2 Schematic diagram of the structure of a computer device according to an embodiment of the present invention. Figure 2As shown, the computer device 1 includes a memory 10, in which the course recommendation device 100 based on text similarity is stored. The course recommendation device 100 based on text similarity can obtain content features of courses in a preset database, wherein the content features include the title of the course, the theme of the course and the introduction of the course; construct a course content corpus according to the content features; call the TF-IDF model to identify the course content corpus to obtain a first vector, call the topic model to identify the course corpus to obtain a second vector, call the Word2vec model to identify the course content corpus to obtain a third vector; determine the first optimal weight corresponding to the first vector, determine the second optimal weight corresponding to the second vector, determine the third optimal weight corresponding to the third vector, and construct the final vector of the course according to the first optimal weight, the second optimal weight and the third optimal weight; calculate the similarity of any two courses according to the final vector of the course, and obtain the similarity matrix of all courses in the preset database according to the similarity of the any two courses; obtain the historically viewed courses corresponding to the user identity information in the preset database, call the similarity matrix according to the historically viewed courses, and recommend a course list to the user. Through the embodiments of the present invention, similar courses can be recommended to users based on the text content features of the courses, and do not rely on the user's behavioral records of the courses; the similarity matrix in the present invention is the relationship between courses, and is a dense matrix. When the course similarity matrix is used for related processing, computer resources can be fully utilized; and in the present invention, when a new course is added to a preset database, it is only necessary to vectorize the new course content and calculate the similarity with the original course in the database, which can reduce the calculation complexity; and the present invention adopts a supervised learning method to determine the optimal weights corresponding to the first vector, the second vector and the third vector, which can avoid the subjectivity of manually setting weights and the tediousness of testing the optimal weights, improve the calculation efficiency of the optimal weights, and thus improve the efficiency of course recommendation based on text similarity.
[0104] In this embodiment, the computer device 1 may further include a display screen 20 and a processor 30. The memory 10 and the display screen 20 may be electrically connected to the processor 30 respectively.
[0105] The memory 10 can be different types of storage devices for storing various types of data. For example, it can be the memory or internal memory of the computer device 1, or it can be a memory card that can be externally connected to the computer device 1, such as a flash memory, a SM card (SmartMedia Card), an SD card (Secure Digital Card), etc. In addition, the memory 10 can include a non-volatile memory, such as a hard disk, an internal memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other non-volatile solid-state storage devices. The memory 10 is used to store various types of data, for example, various applications installed in the computer device 1, data set and obtained by applying the above-mentioned course recommendation method based on text similarity, and other information.
[0106] The display screen 20 is installed on the computer device 1 for displaying information.
[0107] The processor 30 is used to execute the course recommendation method based on text similarity and various software installed in the computer device 1, such as an operating system and application display software. The processor 30 includes but is not limited to a processor (Central Processing Unit, CPU), a microcontroller unit (Micro Controller Unit, MCU) and other devices used to interpret computer instructions and process data in computer software.
[0108] The course recommendation device 100 based on text similarity may include one or more modules, which are stored in the memory 10 of the computer device 1 and configured to be executed by one or more processors (a processor 30 in this embodiment) to complete the embodiment of the present invention. Figure 3 As shown, the course recommendation device 100 based on text similarity may include a content feature acquisition module 101, a corpus construction module 102, a vector acquisition module 103, an optimal value determination module 104, a matrix calculation module 105 and a recommended course generation module 106. The module referred to in the embodiment of the present invention may be a program segment that completes a specific function, which is more suitable for describing the execution process of software in the processor 30 than a program.
[0109] It can be understood that, corresponding to each embodiment of the above-mentioned course recommendation method based on text similarity, the computer device 1 may include Figure 3The functions of each module will be specifically introduced below. It should be noted that the same nouns, related nouns and their specific explanations in each implementation of the above course recommendation method based on text similarity can also be applied to the following introduction to the functions of each module. In order to save space and avoid repetition, they will not be repeated here.
[0110] The content feature acquisition module 101 can be used to acquire the content features of the courses in the preset database, wherein the content features include the title of the course, the subject of the course and the introduction of the course.
[0111] The corpus construction module 102 may be used to construct a course content corpus according to the content features.
[0112] The vector acquisition module 103 can be used to call the TF-IDF model to identify the course content corpus to obtain a first vector, call the topic model to identify the course content corpus to obtain a second vector, and call the Word2vec model to identify the course content corpus to obtain a third vector.
[0113] The optimal value determination module 104 can be used to determine the first optimal weight corresponding to the first vector, determine the second optimal weight corresponding to the second vector, determine the third optimal weight corresponding to the third vector, and construct the final vector of the course based on the first optimal weight, the second optimal weight and the third optimal weight.
[0114] The matrix calculation module 105 can be used to calculate the similarity between any two courses according to the final vectors of the courses, and obtain the similarity matrix of all courses in the preset database according to the similarity between the any two courses.
[0115] The recommended course generation module 106 may be used to obtain the historically viewed courses corresponding to the user identity information in the preset database, and call the similarity matrix according to the historically viewed courses to recommend a course list to the user.
[0116] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by the processor 30, the steps of the course recommendation method based on text similarity in any of the above-mentioned embodiments are implemented.
[0117] The course recommendation device 100 / computer device 1 / module / unit integrated by the text similarity can be stored in a computer-readable storage medium if it is implemented in the form of a software functional unit and sold or used as an independent product. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned implementation method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor 30, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form, etc. The computer-readable storage medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), etc.
[0118] The processor 30 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor 30 is the control center of the course recommendation device 100 / computer device 1 based on text similarity, and uses various interfaces and lines to connect various parts of the entire course recommendation device 100 / computer device 1 based on text similarity.
[0119] The memory 10 is used to store the computer program and / or module, and the processor 30 implements various functions of the course recommendation device 100 / computer device 1 based on text similarity by running or executing the computer program and / or module stored in the memory 10, and calling the data stored in the memory 10. The memory 10 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device 1, etc.
[0120] In the several specific embodiments provided by the present invention, it should be understood that the disclosed computer device and method can be implemented in other ways. For example, the system implementation described above is only illustrative, for example, the division of the modules is only a logical function division, and there may be other division methods in actual implementation.
[0121] It is obvious to those skilled in the art that the embodiments of the present invention are not limited to the details of the exemplary embodiments described above, and that the embodiments of the present invention can be implemented in other specific forms without departing from the spirit or essential features of the embodiments of the present invention. Therefore, from any point of view, the embodiments should be regarded as exemplary and non-restrictive, and the scope of the embodiments of the present invention is defined by the appended claims rather than the above description, and it is intended that all changes falling within the meaning and scope of the equivalent elements of the claims are included in the embodiments of the present invention. Any figure mark in a claim should not be regarded as limiting the claim involved. Multiple units, modules or devices stated in a system, device or computer device claim may also be implemented by the same unit, module or device through software or hardware.
[0122] The above implementation modes are only used to illustrate the technical solutions of the embodiments of the present invention and are not intended to limit the present invention. Although the embodiments of the present invention are described in detail with reference to the above preferred implementation modes, those skilled in the art should understand that the technical solutions of the embodiments of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. A course recommendation method based on text similarity, It is characterized in that The course recommendation method based on text similarity includes: Obtaining content features of a course in a preset database, wherein the content features include a course title, a course theme, and a course introduction; constructing a course content corpus according to the content characteristics; Calling the TF-IDF model to identify the course content corpus to obtain a first vector, calling the topic model to identify the course content corpus to obtain a second vector, and calling the Word2vec model to identify the course content corpus to obtain a third vector; Determine a first optimal weight corresponding to the first vector, determine a second optimal weight corresponding to the second vector, and determine a third optimal weight corresponding to the third vector, including: obtaining a text similarity data set with a label, wherein the label is used to identify whether the texts in the text similarity data set are similar; call a TF-IDF model to identify the text similarity data set to obtain a fourth vector, call a topic model to identify the text similarity data set to obtain a fifth vector, and call a Word2vec model to identify the text similarity data set to obtain a sixth vector; horizontally splice the fourth vector, the fifth vector, and the sixth vector to obtain a horizontal splicing vector; call an integrated tree model to train the horizontal splicing vector to obtain a first Shaply value; take an absolute average of the first Shaply value according to the vertical dimension and scale it to a [0,1] space to obtain a second Shaply value corresponding to the fourth vector, the fifth vector, and the sixth vector; sum and average the second Shaply values to obtain the first optimal weight, the second optimal weight, and the third optimal weight respectively; constructing a final vector of courses according to the first optimal weight, the second optimal weight and the third optimal weight; Calculating the similarity between any two courses according to the final vectors of the courses, and obtaining a similarity matrix of all courses in the preset database according to the similarity between the any two courses; Obtaining the historically viewed courses corresponding to the user identity information in the preset database, and calling the similarity matrix according to the historically viewed courses to recommend a course list to the user; Checking whether there is a new course in the preset database at a preset time interval; When the detection result is that there are new courses in the preset database, the number of the new courses is obtained; Determine whether the quantity exceeds a preset new quantity threshold; When the judgment result is that the number exceeds the preset new number threshold, calling the first updating rule to update the similarity matrix; When the judgment result is that the number does not exceed the preset new number threshold, the second updating rule is called to update the similarity matrix.
2. The course recommendation method based on text similarity according to claim 1, It is characterized in that The step of calling the first updating rule to update the similarity matrix comprises: Acquire the new content features of the new course, wherein the new content features include the title of the new course, the subject of the new course and the introduction of the new course; Updating the course content corpus according to the newly added content features; Call the TF-IDF model to identify the updated course content corpus to obtain an updated first vector, call the topic model to identify the updated course content corpus to obtain an updated second vector, and call the Word2vec model to identify the updated course content corpus to obtain an updated third vector; Determine a first optimal weight corresponding to the updated first vector, determine a second optimal weight corresponding to the updated second vector, determine a third optimal weight corresponding to the updated third vector, and update the final vectors of all courses according to the first optimal weight, the second optimal weight and the third optimal weight that are newly acquired; The similarity between any two courses is calculated according to the final vectors after all courses are updated, and an updated similarity matrix is obtained according to the similarity between the any two courses.
3. The course recommendation method based on text similarity according to claim 1, It is characterized in that The step of calling the second updating rule to update the similarity matrix comprises: Acquire the new content features of the new course, wherein the new content features include the title of the new course, the subject of the new course and the introduction of the new course; Updating the course content corpus according to the newly added content features; Call the TF-IDF model to identify the updated course content corpus to obtain the first vector corresponding to the new course, call the topic model to identify the updated course content corpus to obtain the second vector corresponding to the new course, and call the Word2vec model to identify the updated course content corpus to obtain the third vector corresponding to the new course; The final vector of the newly added course is obtained according to the first vector, the second vector, the third vector and the first optimal weight, the second optimal weight and the third optimal weight, and the similarity values with other courses are calculated according to the final vector of the newly added course, and the similarity matrix is updated according to the calculated similarity values.
4. The course recommendation method based on text similarity according to claim 1, It is characterized in that The method further comprises: When there are multiple historically viewed courses, each of the historically viewed courses is determined as a target course; Selecting a preset number of courses ranked before the target course in similarity according to the similarity matrix to form an initial list of courses to be recommended; The courses in the initial list of courses to be recommended are arranged in descending order of similarity, and the second preset number of courses ranked top in similarity are selected as the list of courses to be recommended.
5. The course recommendation method based on text similarity according to claim 1, It is characterized in that The step of constructing a course content corpus according to the content features comprises: Performing data cleaning on the content features, wherein the data cleaning includes one or more of the following: removing links, removing invalid characters, and removing invalid sentences; The content features after data cleaning are processed by Chinese word segmentation, and the results of the Chinese word segmentation processing are stored in a course content corpus, wherein the course content corpus is a blockchain database.
6. A course recommendation device based on text similarity, It is characterized in that The course recommendation device based on text similarity includes: A content feature acquisition module, used to acquire content features of courses in a preset database, wherein the content features include the title of the course, the subject of the course and the introduction of the course; A corpus construction module, used to construct a course content corpus according to the content characteristics; A vector acquisition module, used for calling a TF-IDF model to identify the course content corpus to obtain a first vector, calling a topic model to identify the course content corpus to obtain a second vector, and calling a Word2vec model to identify the course content corpus to obtain a third vector; The optimal value determination module is used to determine the first optimal weight corresponding to the first vector, determine the second optimal weight corresponding to the second vector, and determine the third optimal weight corresponding to the third vector, including: obtaining a text similarity data set with a label, wherein the label is used to identify whether the texts in the text similarity data set are similar; calling the TF-IDF model to identify the text similarity data set to obtain a fourth vector, calling the topic model to identify the text similarity data set to obtain a fifth vector, and calling the Word2vec model to identify the text similarity data set to obtain a sixth vector; horizontally splicing the fourth vector, the fifth vector, and the sixth vector to obtain a horizontal splicing vector; calling the integrated tree model to train the horizontal splicing vector to obtain a first Shaply value; taking an absolute average of the first Shaply value according to the vertical dimension and scaling it to the [0,1] space to obtain the second Shaply value corresponding to the fourth vector, the fifth vector, and the sixth vector; summing and averaging the second Shaply value to obtain the first optimal weight, the second optimal weight, and the third optimal weight respectively; the optimal value determination module is also used to construct a final vector of the course according to the first optimal weight, the second optimal weight, and the third optimal weight; A matrix calculation module, used for calculating the similarity between any two courses according to the final vectors of the courses, and obtaining the similarity matrix of all courses in the preset database according to the similarity between the any two courses; A recommended course generation module, used to obtain the historically viewed courses corresponding to the user identity information in the preset database, and call the similarity matrix according to the historically viewed courses to recommend a course list to the user; The matrix calculation module is also used to detect whether there are new courses in the preset database at preset time intervals; when the detection result is that there are new courses in the preset database, obtain the number of new courses; determine whether the number exceeds the preset new number threshold; when the determination result is that the number exceeds the preset new number threshold, call the first update rule to update the similarity matrix; when the determination result is that the number does not exceed the preset new number threshold, call the second update rule to update the similarity matrix.
7. A computer device, It is characterized in that The computer device comprises a processor, and the processor is used to implement the course recommendation method based on text similarity as claimed in any one of claims 1 to 5 when executing a computer program stored in a memory.
8. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the course recommendation method based on text similarity as claimed in any one of claims 1 to 5 is implemented.
Citation Information
Patent Citations
Activity participation analysis and recommendation method
CN109919793A
Intelligent semantic document recommendation method and device and computer readable storage medium
CN110222160A
Techniques for data-driven correlation of metrics
US20200104775A1
