A livestock industry news information recommendation system

Through multi-data source interest modeling and incremental crawling technology, combined with LDA theme clustering, the problems of insufficient accuracy and low timeliness of animal husbandry news recommendation in the existing technology are solved, and efficient and accurate news recommendations are achieved.

CN112732995BActive Publication Date: 2024-11-05YUNFU INTERNET OF THINGS RES INST CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110022281.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-01-08
Publication Date
2024-11-05
Estimated Expiration
2041-01-08

AI Technical Summary

Technical Problem

The existing news recommendation system has insufficient accuracy in the recommendation results in subdivided fields, has too much dependence on model experience, and is low in timeliness, making it difficult to adapt to the data fusion of diversified interest in specific industries such as animal husbandry.

Method used

A livestock industry news and information recommendation system was designed, using multi-data source interest modeling, combining user search terms, click association and browsing content, and using vector space model and named entity extraction, user interest modeling, and using incremental Crawling and deduplication processing combines LDA theme clustering and moving average prediction to achieve news recommendations.

Benefits of technology

It improves the accuracy and diversity of news and information recommendations, reduces experience dependence, ensures the timeliness and accuracy of recommendation results, and adapts to the interest modeling needs of specific industries.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN112732995B_ABST
    Figure CN112732995B_ABST
Patent Text Reader

Abstract

The present invention discloses a livestock news information recommendation system, including a user interest modeling module, a news information crawling module, a web page data preprocessing module, a news information recommendation module, a news information display feedback module and a news information retrieval module. Each module is implemented by JAVA language, PYTHON language, etc., so as to realize fast and effective server-side calculation, modeling and mining. Users can use the system of the present invention on the Internet web page. During the use of the system, the system automatically records the user's clicks, retrieval and other operations, and determines the user interest model through feature analysis, mining and collaborative training and learning. The present invention can adapt to the news information recommendation application in the livestock field, simplify user operations and timely recommend valuable livestock news information, save system resources, reduce the time of web page loading, and improve the efficiency of users in obtaining livestock news information.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of computer Internet data mining, and in particular to a livestock industry news information recommendation system. Background Art

[0002] At present, many livestock enterprises, farmers and meat processing companies often rely on search engines to obtain livestock news information. This method often requires users to actively provide search terms and cannot be actively pushed by the server. With the massive growth of information, people hope to obtain information through automatic recommendation. The current commonly used news recommendation system is based on the vector space model to make recommendations based on content or collaborative filtering. In order to adapt to all news objects, this method is difficult to model the user's interest in the industry professional field in a targeted manner. At the same time, in order to meet the news recommendation of different categories, industries or topics, the existing system often cannot build a special news object feature library, and can only rely on content recommendation based on the bag of words model or click-based collaborative filtering recommendation, resulting in the results being particularly concentrated relative to specific sub-topics, or obtaining too much irrelevant information.

[0003] Facing such problems, traditional solutions can be divided into two categories: one solution is to distribute the detailed topic news information in the candidate set according to the weight index, but this manual intervention method itself has no theoretical basis and is prone to experience dependence, that is, different data sets require different experience settings. Another solution is to treat the recommendation results differently, that is, try to select news information with different topics in the candidate set, but because of the insignificance of user interest in traditional news recommendation methods, it will lead to too much irrelevant information. Although the diversity of the recommendation list has been enhanced, the accuracy is still not guaranteed.

[0004] Based on the above background and the problems existing in traditional solutions, for specific industries, such as animal husbandry, it is necessary to propose a solution that is compatible and differentiated for the subdivided fields, and can adapt to the fusion of multi-interest data across system information, and quickly, accurately and objectively obtain news information recommendation results in the subdivided fields. Summary of the invention

[0005] The purpose of the present invention is to overcome the shortcomings and deficiencies of the prior art and propose a livestock industry news information recommendation system to solve the problems of insufficient accuracy of recommendation results in segmented fields, excessive reliance on model experience and low timeliness of existing news recommendation systems, and to facilitate integration into various news information platforms through an assembleable data interface mode.

[0006] To achieve the above purpose, the technical solution provided by the present invention is: a livestock news information recommendation system, comprising:

[0007] User interest modeling module, used to mine, build and store user interest models;

[0008] News information crawling module, used to incrementally collect news information about animal husbandry;

[0009] Web data preprocessing module, used to preprocess animal husbandry news information, including denoising, deduplication and classification;

[0010] News information recommendation module, used to provide a news recommendation list for a given user and automatically collect its click information for self-evaluation;

[0011] The news information display and feedback module is used to display recommended animal husbandry news information and hot news information and advertising information that need to be promoted to users;

[0012] The news information retrieval module is used to provide users with a retrieval interface, allowing users to accurately find relevant animal husbandry news information based on search terms and feature settings.

[0013] Furthermore, the user interest modeling module adopts interest modeling of multiple data sources, and the interest modeling data includes user search terms, click associations after user search and recommended news information, and browsed news information content; the user interest modeling module includes a user search term collection submodule, a user clicked news information association information submodule, and a user reading news information content modeling submodule;

[0014] The user search term collection submodule collects the search terms used by the user according to the user's record of using the search system, and assigns weights to the search terms according to the vector space model, and the longer the time, the lower the weight;

[0015] The submodule of information associated with user clicks on news information is used to build associations between users and news information, including two types of associations: associations clicked by users after retrieval and associations clicked by users on recommended news information;

[0016] The content modeling submodule of the user reading news information is used to perform vector space modeling on the content clicked and browsed by the user to form a vector space model; in addition, the content modeling submodule also uses the browsing residence time as a modeling element, and uses the number of words in the news information as a normalization factor. The longer the news information stays per unit word, the greater the weight it has on the user interest model.

[0017] Furthermore, the news information crawling module adopts an incremental crawling method, crawls animal husbandry-related news information from the web pages in the configured seed file every day, marks the time nodes of the crawled news, compares the publication time of the crawled news, discovers new publication information, and then crawls and stores it in the database. For target sites that do not have publication time information, the LSH minimum hash method is used to compare the stored information, so as to achieve deduplication before storage, and incremental crawling is performed from the marked nodes on the next day.

[0018] Furthermore, the webpage data preprocessing module performs not only spam and noise information filtering, clustering, but also named entity extraction and data cleaning on the news information content. The specific operations are as follows:

[0019] The livestock news information is preprocessed by deduplication, named entity extraction, and cleaning, and the preprocessed information is stored in the database; the minimum hash method is used to compare the stored news and the crawled news for deduplication. If duplicate news information is found, only its source is recorded and duplicate marks are made; the conditional random field method is used to identify named entities, and the words in the vocabulary are marked as named entities according to the vocabulary library classification, and new words outside the vocabulary are regarded as named entities to be stored in the database as new named entities; data cleaning includes three aspects: a. Cleaning of missing values. For livestock news information with missing release time, if there are duplicate news in the stored news information, the release time of the duplicate news is read, and if not, the current system time is assigned; b. Logical cleaning. The first 50 characters, the middle 50 characters and the last 50 characters of the crawled news information are intercepted as news tag values, and set as news unique tag key values. If there are duplicate news, the corresponding duplicate tags are added; c. Noise content cleaning. News with less than 15 characters are automatically judged as noise, including advertisements and overview page title noise.

[0020] Furthermore, the news information recommendation module specifically performs the following operations:

[0021] S1. By using named entity extraction method, key animal terms in animal husbandry news are extracted, and key animal husbandry company names are extracted as key entities;

[0022] S2. According to the characteristics of the entity, the entity objects are divided into animal entities and company entities, and classified according to the characteristics of animal husbandry news into breeding, sales, society, policy, and soft advertising categories;

[0023] S3. According to the statistical data of each user reading a specific category of animal husbandry news, it is mapped into a ranking score. The score is a real number in the interval [0,10] and is hashed with a normal distribution with a mean of 5, that is, the mean μ=5, and the standard deviation σ=1 is set. In this way, a user-category two-dimensional matrix is ​​formed. The elements in the matrix are the users who click to read a specific category. The higher the value of the matrix element, the more times the specific user clicks on the specific category news.

[0024] S4. For news within the recent time window of 6 months, use the LDA topic clustering method to find the topic and predict its hot value through moving average. For the news information i to be recommended, compare the Euclidean distance of the centroid of the news information i with the various clusters in the topic clustering, and classify it into the nearest topic classification, so as to obtain the hot value h(i) of the news information i;

[0025] S5. Assuming that the interest of user u in the recommended news information i is represented by the utility U(u,i), then:

[0026]

[0027] Where c(i) is the cluster ID number of the recommended news information i, which is used to distinguish different clusters, j is a news different from i, and the sim(*,*) function represents the similarity of the parameters of two vector structures. Here, sim(i,j) represents the similarity between i and j, and sim(s u ,s i ) means s u and i The similarity between two parameters; in the process of news information recommendation, since the recommended news are all new items, the corresponding news information cannot be found in the historical association data, so clusters are used to replace individual news objects to build associations. Similarly, c(u) represents the number of the news cluster that user u has browsed, t represents the time when the algorithm is running, and t i represents the publishing time of the news to be recommended, i.e., the release time, δ u,c(i) Indicates the user u’s interest in the news cluster c(i), which is obtained by the proportion of the user’s click browsing records on the news cluster. u and i They represent the named entities in the historical animal husbandry news information browsed by user u and the named entities of the news information i to be recommended; in the above formula, the remaining α i , β i , γ i and μ i are the parameters to be estimated and need to be solved in the iterative process. The joint probability density of these parameters to be estimated is defined as follows:

[0028]

[0029] Among them, r u,i The click ranking score of the topic cluster where the user clicks the current livestock news information, p(r u,i |U(u,i)) indicates that the score r is given under the condition of the utility U(u,i) u,i The probability of; when the probability value P reaches the maximum, the parameter value can be determined. In order to determine the parameters for all u and i, the log(P) function can be used to convert the multiplication probability problem into addition, and the minimum value of the partial derivative of each parameter is calculated, which is the extreme point. Then the gradient descent method is used to solve the parameter value until the parameter converges. The process of solving each parameter is as follows:

[0030]

[0031] In the above parameter solution iterative formula, α' i , β' i , γ' i and μ' i They represent the corresponding α in the iteration process. i , β i , γ i and μ i ω2~ω5 are temporary variables for controlling the iterative learning speed. The iteration stops when the residual between the iteration result of each parameter and the previous iteration result is less than 0.0001. If the number of iterations exceeds the empirical threshold of 10,000 and there is still no convergence, the algorithm tries to use other random initial values ​​and iterate again until the converged parameters are obtained.

[0032] S6. After all the parameters to be evaluated are determined, for each news information i to be recommended, its interest value can be solved by formula U(u,i); in order to prevent similar news from being recommended repeatedly, the animal husbandry news information that the user has browsed and the news information to be recommended will be compared, and the news to be recommended that has been recommended similar news will be displayed in a way of lowering its ranking score. If the reduction exceeds the set value, it will not be recommended, as shown in the following formula:

[0033]

[0034] Among them, L represents the preference score, S represents the selected candidate set, sim(q u ,q i ) represents the search terms used by the user in the system and the search terms to be recommended for news information. The calculation result of this formula is consistent with sim(s u ,s i ) and sim(q u ,q i), and negatively correlated with sim(i,j), which means that in the recommendation list, if the vector space similarity of the news information to be recommended is too close to the news that the user has clicked, the ranking will be downgraded. If the named entity of the news information to be recommended is highly correlated with the news that the user has clicked, or is highly correlated with the search term set, it will be weighted, indicating that the news information to be recommended is highly consistent with the user's previous interest in named entities and the user's precise search needs. Since the weights of each sub-item cannot be equal, the parameters of each sub-item are set to θ1~θ3, which represent the weights of each sub-item, that is, to represent the role of each sub-item, and the gradient descent method is used in the training set to obtain its parameters, as shown in the following formula:

[0035]

[0036]

[0037] Here, θ'1, θ'2, and θ'3 represent the iterative temporary variables of θ1 to θ3, respectively. Similar to the previous parameter estimation, since the sorting formula itself is a non-product formula, the log function is not used, and the partial derivatives of the L formula for each parameter are directly used; a1 to a3 are learning iteration rate control parameters. When the iterative parameter result is less than 0.0001 compared with the result of the previous iteration, the iteration terminates and the algorithm returns θ1 to θ3 as the final parameters of the model. Otherwise, when the number of iterations exceeds the empirical threshold of 10,000 times, the algorithm resets the initial value until the parameters that meet the conditions are obtained.

[0038] Further, the step S4 comprises the following steps:

[0039] S4-1. Divide the news into time period sub-datasets according to the time window Δt, and perform clustering on the sub-datasets. The specific process is as follows: perform k-means clustering on the sub-datasets, set the initial cluster k, and finally obtain its k cluster centers; then, randomly sample m iterative starting points for the k sub-clusters, and start from these starting points, calculate the Euclidean distance of the surrounding points with R as the radius. When the calculated distance is less than the threshold T, mark it as a surrounding point and include it in the cluster of the point until all points are traversed;

[0040] S4-2, taking the time window as the benchmark, the corresponding news information is associated with the time axis, and the association method is the same as the step S4-1 method; then, according to the number of clusters per day, statistics are performed to calculate the proportion of each cluster to the total number of daily livestock news information, thereby forming a daily proportion change curve, and then the exponential shift method is applied on the curve to predict hot spots; when the recommended news information i enters the system, first determine the cluster to which it belongs, and then return its hot spot value h(i) based on the hot spot prediction result.

[0041] Furthermore, the news information display feedback module is used to display the results of animal husbandry news recommendation. For the news information on the display page, if the user browses the title but does not click it, it will be recorded as a negative feedback feature in the user interest model. When the title is browsed three times but not clicked, the corresponding news information will be removed from the display list of the user; in order to ensure the timeliness of the recommended interest model, that is, during the operation of the system, the update of the interest model can timely reflect the changes in the user's interest needs, the news information display feedback module caches the user's click behavior and retrieval behavior as input features of the recommendation algorithm. When the cache library accumulates to the threshold, the user interest model algorithm is started to update the user interest model; the news information display feedback module also recommends animal husbandry product advertisements required by the user based on the user's interest, changing the mode of broadcasting advertisements to cause information interference to the user. The advertisement recommendation method is the same as the animal husbandry news information recommendation method. When the advertisement is stored in the background, the advertising manager inputs the advertisement content and classification information into the system, and builds an association with the user interest model based on the advertisement content and classification information.

[0042] Furthermore, the news information retrieval module provides a retrieval interface page and a retrieval system. The retrieval system is built based on the Lucene open source project and includes a membership system. Users need to log in to use it. When the user logs in, all behaviors will be collected, mined and modeled according to the user ID.

[0043] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0044] 1. The present invention integrates the multi-dimensional information of user historical clicks, browsing and search terms. Compared with the single-dimensional interest modeling of the traditional recommendation system, the system and its core method of the present invention are more objective and accurate in mining user interest. The animal husbandry news recommendation method not only uses the animal husbandry news that users click and browse as the source of recommendation features, but also uses the search terms as the source of recommendation features, further focusing on the user's interest features. This is due to the fact that the present invention integrates the query deceleration record of the information retrieval system into the feature collection of the recommendation system.

[0045] 2. The recommendation method of the present invention integrates hot news information topics and key named entity information, and removes duplicates from the recommended results, downgrading overly similar news information, but preventing interesting information from being mistakenly downgraded based on key named entity similarity, thereby ensuring higher accuracy and ranking. The livestock news recommendation method excludes duplicate or similar livestock news information results in the recommendation ranking, and constrains the interest focus based on the livestock named entity set, thereby ensuring the accuracy of the recommendation while optimizing the diversity of the recommendation results.

[0046] 3. User interest profiles in traditional recommendation systems often face the problem of updating timeliness, that is, it is difficult for user interest models to get fast update feedback with clicks, browsing or purchase actions. The feature feedback design of the present invention can effectively feed back changes in the system to user interest modeling in a timely manner, thereby updating user interest profiles and providing users with more accurate recommendation results based on the latest user interest model.

[0047] 4. Compared with traditional methods, the livestock industry news information recommendation method is overly dependent on parameter thresholds. The core recommendation algorithm proposed in this invention optimizes the parameters to be determined with the goal of maximizing user interest, obtains considerable model parameters, and makes the recommendation more objective and accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 This is a schematic diagram of the relationship and hierarchy of various modules in the livestock industry news information recommendation system.

[0049] Figure 2 This is a schematic diagram of the calling and running process of each module of the livestock industry news information recommendation system. DETAILED DESCRIPTION

[0050] The present invention is further described in detail below in conjunction with embodiments and drawings, but the embodiments of the present invention are not limited thereto.

[0051] like Figure 1 and Figure 2 As shown, this embodiment discloses a livestock industry news information recommendation system, including the following functional modules:

[0052] User interest modeling module, used to mine, build and store user interest models;

[0053] News information crawling module, used to incrementally collect news information about animal husbandry;

[0054] Web data preprocessing module, used to preprocess animal husbandry news information, including denoising, deduplication and classification;

[0055] News information recommendation module, used to provide a news recommendation list for a given user and automatically collect its click information for self-evaluation;

[0056] The news information display and feedback module is used to display recommended animal husbandry news information and hot news information and advertising information that need to be promoted to users;

[0057] The news information retrieval module is used to provide users with a retrieval interface, allowing users to accurately find relevant animal husbandry news information based on search terms and feature settings.

[0058] The user interest modeling module is different from the traditional single-source modeling. It adopts interest modeling of multiple data sources. The interest modeling data includes user search terms, click associations after user search and recommended news information, and browsed news information content; the user interest modeling module includes a user search term collection submodule, a user click association information submodule, and a user reading news information content modeling submodule.

[0059] The main function of the user search term collection submodule is to collect the search terms used by users according to their records of using the search system, and assign weights to them according to the vector space model. The longer the time, the lower the weight.

[0060] The main function of the submodule of information associated with user clicks on news information is to build associations between users and news information, including two types of associations: associations clicked by users after retrieval and associations clicked by users on recommended news information.

[0061] The main function of the content modeling submodule of the user reading news information is to perform vector space modeling on the content clicked and browsed by the user to form a vector space model; in addition, the content modeling submodule also uses the browsing residence time as a modeling element, and uses the number of words in the news information as a normalization factor. The longer the news information stays per unit word, the greater its weight on the user interest model.

[0062] The traditional method of crawling news information often adopts crawling first and then deduplication, or deduplication through full-text comparison. These two methods consume more system resources. The target website of animal husbandry news information updates information every day. Traditional crawlers generally crawl all in batches each time they run, and then deduplicate after crawling to the local. This method has a large amount of repetitive work and wastes a lot of computing resources and bandwidth resources. The news information crawling module of this system adopts an incremental crawling method. It crawls animal husbandry-related news information from the web pages in the configuration seed file every day, marks the time nodes of the news after crawling, compares the publication time of the crawled news, finds new release information, and then crawls and stores it in the database. For target sites that do not have release time information, the LSH minimum hash method is used to compare the stored information, so as to achieve deduplication before storage, and incremental crawling is performed from the marked nodes the next day.

[0063] Traditional methods focus on filtering and clustering news information for news information preprocessing. The web data preprocessing module of this system not only filters and clusters news information, but also extracts named entities and cleans data. The specific operations are as follows:

[0064] The livestock news information is pre-processed by deduplication, named entity extraction, and cleaning, and the pre-processed information is stored in the database; the minimum hash method is used to compare the stored news and the crawled news. If duplicate news information is found, only its source is recorded and duplicate marks are made; the conditional random field method is used to identify named entities, and the words in the vocabulary are marked as named entities according to the vocabulary library classification, and new words outside the vocabulary are regarded as named entities to be stored in the database as new named entities; data cleaning includes three aspects: a. Cleaning of missing values. For livestock news information with missing release time, if there are duplicate news in the stored news information, the release time of the duplicate news is read, and if not, the current system time is assigned; b. Logical cleaning. The first 50 characters, the middle 50 characters and the last 50 characters of the crawled news information are intercepted as news tag values, and set as news unique tag key values. If there are duplicate news, the corresponding duplicate tags are added; c. Noise content cleaning. News with less than 15 characters are automatically identified as noise, mainly advertisements and overview page title noise.

[0065] The news information recommendation module specifically performs the following operations:

[0066] S1. Through named entity extraction, key animal terms such as pig, cow, and chicken are extracted from animal husbandry news, and key animal husbandry company names such as Wens Group, Dayong Industrial and other animal husbandry companies are extracted and used as key entities;

[0067] S2. According to the characteristics of the entity, the entity objects are divided into animal entities and company entities, and classified according to the characteristics of animal husbandry news into breeding, sales, society, policy, and soft advertising categories;

[0068] S3. According to the statistical data of each user reading a specific category of animal husbandry news, it is mapped into a ranking score. The score is a real number in the interval [0,10] and is hashed with a normal distribution with a mean of 5, that is, the mean μ=5, and the standard deviation σ=1 is set. In this way, a user-category two-dimensional matrix is ​​formed. The elements in the matrix are the users who click to read a specific category. The higher the value of the matrix element, the more times the specific user clicks on the specific category news.

[0069] S4. For news within the recent time window of 6 months, the LDA topic clustering method is used to find the topic and predict its hot value through moving average. For the news information i to be recommended, the Euclidean distance of the centroid of the news information i is compared with the various clusters in the topic clustering, and the news information i is classified into the nearest topic classification, and then the hot value h(i) of the news information i is obtained. The main steps are as follows:

[0070] S4-1. Divide the news into time period sub-datasets according to the time window Δt, and perform clustering on the sub-datasets. The specific process is as follows: perform k-means clustering on the sub-datasets, set the initial cluster k, and finally obtain its k cluster centers; then, randomly sample m iterative starting points for the k sub-clusters, and start from these starting points, calculate the Euclidean distance of the surrounding points with R as the radius. When the calculated distance is less than the threshold T, mark it as a surrounding point and include it in the cluster of the point until all points are traversed;

[0071] S4-2, taking the time window as the benchmark, the corresponding news information is associated with the time axis, and the association method is the same as the step S4-1 method; then, according to the number of clusters per day, statistics are performed to calculate the proportion of each cluster to the total number of daily livestock news information, thereby forming a daily proportion change curve, and then the exponential shift method is applied on the curve to predict hot spots; when the recommended news information i enters the system, first determine the cluster to which it belongs, and then return its hot spot value h(i) based on the hot spot prediction result.

[0072] S5. Assuming that the interest of user u in the recommended news information i is represented by the utility U(u,i), then:

[0073]

[0074] Where c(i) is the cluster ID number of the recommended news information i, which is used to distinguish different clusters, j is a news different from i, and the sim(*,*) function represents the similarity of the parameters of two vector structures. Here, sim(i,j) represents the similarity between i and j, and sim(s u ,s i ) means s u and i The similarity between two parameters; in the process of news information recommendation, since the recommended news are all new items, the corresponding news information cannot be found in the historical association data, so clusters are used to replace individual news objects to build associations. Similarly, c(u) represents the number of the news cluster that user u has browsed, t represents the time when the algorithm is running, and t i represents the publishing time of the news to be recommended, i.e., the release time, δ u,c(i) Indicates the user u’s interest in the news cluster c(i), which is obtained by the proportion of the user’s click browsing records on the news cluster. u and i They represent the named entities in the historical animal husbandry news information browsed by user u and the named entities of the news information i to be recommended; in the above formula, the remaining α i , β i , γ i and μi are the parameters to be estimated and need to be solved in the iterative process. The joint probability density of these parameters to be estimated is defined as follows:

[0075]

[0076] Among them, r u,i The click ranking score of the topic cluster where the user clicks the current livestock news information, p(r u,i |U(u,i)) indicates that the score r is given under the condition of the utility U(u,i) u,i The probability of; when the probability value P reaches the maximum, the parameter value can be determined. In order to determine the parameters for all u and i, the log(P) function can be used to convert the multiplication probability problem into addition, and the minimum value of the partial derivative of each parameter is calculated, which is the extreme point. Then the gradient descent method is used to solve the parameter value until the parameter converges. The process of solving each parameter is as follows:

[0077]

[0078] In the above parameter solution iterative formula, α' i , β' i , γ' i and μ' i They represent the corresponding α in the iteration process. i , β i , γ i and μ i ω2~ω5 are temporary variables for controlling the iterative learning speed. The iteration stops when the residual between the iteration result of each parameter and the previous iteration result is less than 0.0001. If the number of iterations exceeds the empirical threshold of 10,000 and there is still no convergence, the algorithm tries to use other random initial values ​​and iterate again until the converged parameters are obtained.

[0079] S6. After all the parameters to be evaluated are determined, for each news information i to be recommended, its interest value can be solved by formula U(u,i); in order to prevent similar news from being recommended repeatedly, the animal husbandry news information that the user has browsed and the news information to be recommended will be compared, and the news to be recommended that has been recommended similar news will be displayed in a way that its ranking score is reduced. If it is reduced too much, it will not be recommended, as shown in the following formula:

[0080]

[0081] Among them, L represents the preference score, S represents the selected candidate set, sim(q u ,q i) represents the search terms used by the user in the system and the search terms to be recommended for news information. The calculation result of this formula is consistent with sim(s u ,s i ) and sim(q u ,q i ), and negatively correlated with sim(i,j), which means that in the recommendation list, if the vector space similarity of the news information to be recommended is too close to the news that the user has clicked, the ranking will be downgraded. If the named entity of the news information to be recommended is more relevant to the news that the user has clicked, or has a high correlation with the search term set, it will be weighted, indicating that the news information to be recommended has a high degree of consistency with the user's previous interest in named entities and the user's precise search needs; since the weights of each sub-item cannot be equal, the parameters of each sub-item are set to θ1~θ3, which represent the weights of each sub-item, that is, to represent the role of each sub-item, and the gradient descent method is used in the training set to obtain its parameters, as shown in the following formula:

[0082]

[0083] Here, θ'1, θ'2, and θ'3 represent the iterative temporary variables of θ1 to θ3 respectively; similar to the previous parameter estimation, but because the sorting formula itself is a non-product formula, the log function is not used, and the partial derivatives of the L formula with respect to each parameter are directly used; a1 to a3 are learning iteration rate control parameters. When the iterative parameter result is less than 0.0001 compared with the result of the previous iteration, the iteration terminates and the algorithm returns θ1 to θ3 as the final parameters of the model. Otherwise, when the number of iterations exceeds the empirical threshold of 10,000 times, the algorithm resets the initial value until the parameters that meet the conditions are obtained.

[0084] The main function of the news information display feedback module is to display the results of animal husbandry news recommendation. For the news information on the display page, if the user browses the title but does not click it, it will be recorded as a negative feedback feature in the user interest model. When the title is browsed three times but not clicked, the corresponding news information will be removed from the display list of the user. In order to ensure the timeliness of the recommended interest model, that is, during the operation of the system, the update of the interest model can timely reflect the changes in the user's interest needs, the news information display feedback module caches the user's click behavior and retrieval behavior as input features of the recommendation algorithm. When the cache library accumulates to the threshold, the user interest model algorithm is started to update the user interest model, which is more timely and effective than the traditional scheduled update. The news information display feedback module also recommends animal husbandry product advertisements required by users based on user interest, changing the mode of broadcasting advertisements to cause information interference to users. The advertisement recommendation method is the same as the animal husbandry news information recommendation method. When the advertisement is stored in the background, the advertising manager inputs the advertisement content and classification information into the system, and builds an association with the user interest model based on the advertisement content and classification information.

[0085] The news information retrieval module provides a retrieval interface page and a retrieval system. The retrieval system is built based on the Lucene open source project and includes a membership system. Users need to log in to use it. When the user logs in, all behaviors will be collected, mined and modeled according to the user ID.

[0086] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A livestock industry news information recommendation system, characterized in that: include: User interest modeling module, used to mine, build and store user interest models; News information crawling module, used to incrementally collect news information about animal husbandry; Web data preprocessing module, used to preprocess animal husbandry news information, including denoising, deduplication and classification; News information recommendation module, used to provide a news recommendation list for a given user and automatically collect its click information for self-evaluation; The news information display and feedback module is used to display recommended animal husbandry news information and hot news information and advertising information that need to be promoted to users; The news information retrieval module is used to provide users with a retrieval interface, allowing users to accurately find relevant animal husbandry news information based on search terms and feature settings; The news information recommendation module specifically performs the following operations: S1. By using named entity extraction method, key animal terms in animal husbandry news are extracted, and key animal husbandry company names are extracted as key entities; S2. According to the characteristics of the entity, the entity objects are divided into animal entities and company entities, and classified according to the characteristics of animal husbandry news into breeding, sales, society, policy, and soft advertising categories; S3. According to the statistical data of each user reading a specific category of animal husbandry news, it is mapped into a ranking score. The score is a real number in the interval [0,10] and is hashed with a normal distribution with a mean of 5, that is, the mean μ=5, and the standard deviation σ=1 is set. In this way, a user-category two-dimensional matrix is ​​formed. The elements in the matrix are the users who click to read a specific category. The higher the value of the matrix element, the more times the specific user clicks on the specific category news; S4. For news within the recent time window of 6 months, use the LDA topic clustering method to find the topic and predict its hot value through moving average. For the news information i to be recommended, compare the Euclidean distance of the centroid of the news information i with the various clusters in the topic clustering, and classify it into the nearest topic classification, so as to obtain the hot value h(i) of the news information i; S5. Assuming that the interest of user u in the recommended news information i is represented by the utility U(u,i), then: Where c(i) is the cluster ID number of the recommended news information i, which is used to distinguish different clusters, j is a news different from i, and the sim(*,*) function represents the similarity of the parameters of two vector structures. Here, sim(i,j) represents the similarity between i and j, and sim(s u ,s i ) means s u and i The similarity between two parameters; in the process of news information recommendation, since the recommended news are all new items, the corresponding news information cannot be found in the historical association data, so clusters are used to replace individual news objects to build associations. Similarly, c(u) represents the number of the news cluster that user u has browsed, t represents the time when the algorithm is running, and t i represents the publishing time of the news to be recommended, i.e., the release time, δ u,c(i) Indicates the user u’s interest in the news cluster c(i), which is obtained by the proportion of the user’s click browsing records on the news cluster. u and i They represent the named entities in the historical animal husbandry news information browsed by user u and the named entities of the news information i to be recommended; in the above formula, the remaining α i , β i , γ i and μ i The parameters to be estimated need to be solved in the iterative process; the joint probability density of these parameters to be estimated is defined as follows: Among them, r u,i The click ranking score of the topic cluster where the user clicks the current livestock news information, p(r u,i |U(u,i)) indicates that the score r is given under the condition of the utility U(u,i) u,i The probability of; when the probability value P reaches the maximum, the parameter value can be determined. In order to determine the parameters for all u and i, the log(P) function can be used to convert the multiplication probability problem into addition, and the minimum value of the partial derivative of each parameter is calculated, which is the extreme point. Then the gradient descent method is used to solve the parameter value until the parameter converges. The process of solving each parameter is as follows: In the above parameter solution iterative formula, α' i , β' i , γ' i and μ' i They represent the corresponding α in the iteration process. i , β i , γ i and μ i ω2~ω5 are temporary variables for controlling the iterative learning speed. The iteration stops when the residual between the iteration result of each parameter and the previous iteration result is less than 0.0001. If the number of iterations exceeds the empirical threshold of 10,000 and there is still no convergence, the algorithm tries to use other random initial values ​​and iterate again until the converged parameters are obtained. S6. After all the parameters to be evaluated are determined, for each news information i to be recommended, its interest value can be solved by formula U(u,i); in order to prevent similar news from being recommended repeatedly, the animal husbandry news information that the user has browsed and the news information to be recommended will be compared, and the news to be recommended that has been recommended similar news will be displayed in a way of lowering its ranking score. If the reduction exceeds the set value, it will not be recommended, as shown in the following formula: Among them, L represents the preference score, S represents the selected candidate set, sim(q u ,q i ) represents the search terms used by the user in the system and the search terms to be recommended for news information. The calculation result of this formula is consistent with sim(s u ,s i ) and sim(q u ,q i ), and negatively correlated with sim(i,j), which means that in the recommendation list, if the vector space similarity of the news information to be recommended is close to the news that the user has clicked, the ranking will be downgraded. If the named entity of the news information to be recommended is highly correlated with the news that the user has clicked, or is highly correlated with the search term set, it will be weighted, indicating that the news information to be recommended is highly consistent with the user's previous interest in named entities and the user's precise search needs. Since the weights of each sub-item cannot be equal, the parameters of each sub-item are set to θ1~θ3, which represent the weights of each sub-item, that is, to represent the role of each sub-item, and the gradient descent method is used in the training set to obtain its parameters, as shown in the following formula: Here, θ'1, θ'2, and θ'3 represent the iterative temporary variables of θ1 to θ3 respectively; similar to the previous parameter estimation, but because the sorting formula itself is a non-product formula, the log function is not used, and the partial derivatives of the L formula with respect to each parameter are directly used; a1 to a3 are learning iteration rate control parameters. When the iterative parameter result is less than 0.0001 compared with the result of the previous iteration, the iteration terminates and the algorithm returns θ1 to θ3 as the final parameters of the model. Otherwise, when the number of iterations exceeds the empirical threshold of 10,000 times, the algorithm resets the initial value until the parameters that meet the conditions are obtained.

2. The livestock industry news information recommendation system according to claim 1, characterized in that: The user interest modeling module adopts interest modeling of multiple data sources, and the interest modeling data includes user search terms, click associations after user search and recommended news information, and browsed news information content; the user interest modeling module includes a user search term collection submodule, a user clicked news information association information submodule, and a user reading news information content modeling submodule; The user search term collection submodule collects the search terms used by the user according to the user's record of using the search system, and assigns weights to the search terms according to the vector space model, and the longer the time, the lower the weight; The submodule of information associated with user clicks on news information is used to build associations between users and news information, including two types of associations: associations clicked by users after retrieval and associations clicked by users on recommended news information; The content modeling submodule of the user reading news information is used to perform vector space modeling on the content clicked and browsed by the user to form a vector space model; in addition, the content modeling submodule also uses the browsing residence time as a modeling element, and uses the number of words in the news information as a normalization factor. The longer the news information stays per unit word, the greater the weight it has on the user interest model.

3. The livestock industry news information recommendation system according to claim 1, characterized in that: The news information crawling module adopts an incremental crawling method, crawling animal husbandry-related news information from the web pages in the configured seed file every day, marking the time nodes of the crawled news, comparing the publishing time of the crawled news, discovering new publishing information, and then crawling and storing it in the database. For target sites that do not have publishing time information, the LSH minimum hash method is used to compare the stored information, so as to achieve deduplication before storage, and incremental crawling is performed from the marked nodes on the next day.

4. The livestock industry news information recommendation system according to claim 1, characterized in that: The web page data preprocessing module performs not only spam and noise information filtering, clustering, but also named entity extraction and data cleaning on news information content. The specific operations are as follows: The livestock news information is preprocessed by deduplication, named entity extraction, and cleaning, and the preprocessed information is stored in the database; the minimum hash method is used to compare the stored news and the crawled news for deduplication. If duplicate news information is found, only its source is recorded and duplicate marks are made; the conditional random field method is used to identify named entities, and the words in the vocabulary are marked as named entities according to the vocabulary library classification, and new words outside the vocabulary are regarded as named entities to be stored in the database as new named entities; data cleaning includes three aspects: a. Cleaning of missing values. For livestock news information with missing release time, if there are duplicate news in the stored news information, the release time of the duplicate news is read, and if not, the current system time is assigned; b. Logical cleaning. The first 50 characters, the middle 50 characters and the last 50 characters of the crawled news information are intercepted as news tag values, and set as news unique tag key values. If there are duplicate news, the corresponding duplicate tags are added; c. Noise content cleaning. News with less than 15 characters are automatically judged as noise, including advertisements and overview page title noise.

5. The livestock industry news information recommendation system according to claim 1, characterized in that: The step S4 comprises the following steps: S4-1. Divide the news into time period sub-datasets according to the time window Δt, and perform clustering on the sub-datasets. The specific process is as follows: perform k-means clustering on the sub-datasets, set the initial cluster k, and finally obtain its k cluster centers; then, randomly sample m iterative starting points for the k sub-clusters, and start from these starting points, calculate the Euclidean distance of the surrounding points with R as the radius. When the calculated distance is less than the threshold T, mark it as a surrounding point and include it in the cluster of the point until all points are traversed; S4-2, taking the time window as the benchmark, the corresponding news information is associated with the time axis, and the association method is the same as the step S4-1 method; then, according to the number of clusters per day, statistics are performed to calculate the proportion of each cluster to the total number of daily livestock news information, thereby forming a daily proportion change curve, and then the exponential shift method is applied on the curve to predict hot spots; when the recommended news information i enters the system, first determine the cluster to which it belongs, and then return its hot spot value h(i) based on the hot spot prediction result.

6. The livestock industry news information recommendation system according to claim 1, characterized in that: The news information display feedback module is used to display the results of animal husbandry news recommendation. For the news information on the display page, if the user browses the title but does not click it, it will be recorded as a negative feedback feature in the user interest model. When the title is browsed three times but not clicked, the corresponding news information will be removed from the display list of the user. In order to ensure the timeliness of the recommended interest model, that is, during the operation of the system, the update of the interest model can timely reflect the changes in the user's interest needs, the news information display feedback module caches the user's click behavior and retrieval behavior as input features of the recommendation algorithm. When the cache library accumulates to the threshold, the user interest model algorithm is started to update the user interest model. The news information display feedback module also recommends animal husbandry product advertisements required by the user based on the user's interest, changing the mode of broadcasting advertisements to cause information interference to the user. The advertisement recommendation method is the same as the animal husbandry news information recommendation method. When the advertisement is stored in the background, the advertisement management personnel input the advertisement content and classification information into the system, and build an association with the user interest model based on the advertisement content and classification information.

7. The livestock industry news information recommendation system according to claim 1, characterized in that: The news information retrieval module provides a retrieval interface page and a retrieval system. The retrieval system is built based on the Lucene open source project and includes a membership system. Users need to log in to use it. When the user logs in, all behaviors will be collected, mined and modeled according to the user ID.