An intelligent recommendation system and method for industrial Internet platform

Abstract

The present invention relates to the field of computer recommendation technology, and discloses an intelligent recommendation system and method for an industrial Internet platform. The system comprises a background management module, a user terminal, a recommendation management module, and a front-end display module. The method comprises the following processes: (1) performing data preprocessing on platform information in the background management module; (2) performing data preprocessing on user information; (3) extracting text-topic features of platform text information; (4) extracting user-interest features of enterprise user information; (5) converting the vector y obtained by processes (3) and (4) into CNN ,y AFM By interactively fusing low-order and high-order features and then undergoing a sigmoid nonlinear transformation, the probability output of the predicted degree of preference is obtained. The recommended information is then sorted according to the probability of the degree of preference to obtain the top k to be recommended. (6) The top k to be recommended are transmitted to the front-end display module to form a recommendation list and recommended to the corresponding enterprise users.

Description

Technical Field

[0001] The present invention relates to the field of computer recommendation technology, and in particular to an intelligent recommendation system and method for an industrial Internet platform. Background Art

[0002] In recent years, with the rapid development of the Internet era, the amount of information on the Internet has skyrocketed, leading to an era of data explosion. It is precisely because of the advent of this era that we will have the problem of information overload when browsing information on a certain platform, that is, users cannot obtain the part of information that is truly valuable to them from the excessive information, and the efficiency of using information is extremely low. In addition, with the rise of the industrial Internet in recent years, more and more industrial Internet platforms have gradually emerged. The platform aims to provide corporate users with a variety of product and service trading platforms, and can also provide them with a variety of latest related news and various policies to help corporate users keep abreast of the latest developments and supply and demand information in the industry. At present, in addition to the more well-known Haier CoaXus Industrial Internet Platform, XCMG Hanyun Industrial Internet Platform, etc., there are also many small industrial Internet platforms created by many companies.

[0003] It is precisely because of the emergence of many industrial Internet platforms that for corporate users who browse and access the platforms, it takes a lot of time and energy to find products and policy information that meet their needs, which is extremely inefficient. In recent years, some recommendations for industrial Internet platforms have emerged based on traditional recommendation algorithms based on collaborative filtering, but they cannot solve the current practical problems, such as data sparsity and cold start problems. Moreover, in the industrial Internet platform, the importance of a certain product or information to different corporate users is definitely different, so the traditional recommendation algorithm alone can no longer solve the above-mentioned pain points such as data sparsity and cold start.

[0004] Compared with other information website platforms, the information and users of the Industrial Internet platform have relatively distinct characteristics: for example, the diversity and hidden feature correlation of news information and various industry solutions in the platform, and the characteristics of users in the platform are that although the number is not as high as the order of magnitude of platform information, each user has relatively personalized characteristics and preferences (industry dynamics, solutions, etc.). In view of this characteristic of the Industrial Internet platform, this patent proposes a new recommendation model for the Industrial Internet platform involving convolutional neural networks, attention mechanisms and factor decomposition machines. The patent document with publication number CN109241424A discloses a recommendation method. Although the proposed model uses the three technologies of convolutional neural networks, attention mechanisms and factor decomposition machines, the entire data processing process and model cannot be well applied to the Industrial Internet platform. The model in the patent document models users and information in the same way, greatly ignoring the respective characteristics of users and information in the Industrial Internet platform. The present invention is innovative based on the unique characteristics of the Industrial Internet platform. Summary of the invention

[0005] In response to the above problems, the present invention provides an intelligent recommendation system and a working method for an industrial Internet platform, which first solves the problem of information overload in the industrial Internet platform, and secondly solves the cold start problem of new enterprise users in the recommendation and the problem of different importance weights of information such as a certain product and industry solution to users.

[0006] The technical solution of the present invention to solve the technical problem is:

[0007] An intelligent recommendation system for industrial Internet platform, including background management module, user terminal, recommendation management module and front-end display module.

[0008] The background management module performs feature classification on the platform's information data, and performs addition, deletion, modification, query, and storage management on the information data; the information data mainly includes industry news policies, industry solutions, various products, expert databases, etc. on the industrial Internet platform.

[0009] The user end includes a user information management module and a user behavior record management module. The user information management module is responsible for registering new users of enterprise users and storing user registration information. The user behavior record management module is responsible for recording the user's browsing record log on the platform; such as news policy browsing, industry solution browsing, platform search records, etc.

[0010] The front-end display module is mainly used to display the most suitable recommendation list for enterprise users.

[0011] The recommendation management module takes the information data of the background management module and the information data of the user end as input, performs feature modeling on the information data of the background management module through a convolutional neural network to obtain a text-topic feature model, performs feature modeling on the information data of the user end through FM (Factorization Machine) combined with an attention mechanism to obtain a user-interest feature model, performs feature interactive fusion on the feature vectors emedding of the text-topic feature model and the user-interest feature model, and then performs a sigmoid nonlinear transformation to obtain a predicted probability value of the degree of liking of the recommended information, sorts the recommended information according to the size of the probability value of the degree of liking, and obtains the recommended k results topk as output and transmits them to the front-end display module.

[0012] A working method of an intelligent recommendation system for an industrial Internet platform includes the following processes:

[0013] (1) Data preprocessing of the platform information in the background management module: Jieba word segmentation is performed on the platform text data to transform it into multiple token units, and the multiple tokens obtained by word segmentation are converted into a fixed-length vector X through the word2vector method. i ;

[0014] (2) Data preprocessing of user information: One-hot encode the user's registration information and behavior record data information and convert it into a word vector Q; this encoding form uses an n'-bit state register to encode n' states, each state has its own independent register bit, and only one bit is valid at any time. It is the most common binary vector representation method. For example, "China, the United States, and France" are respectively "100, 010, and 001" through one-hot encoding. Therefore, all data about the user is converted into a vector representation through this encoding method.

[0015] (3) Text-topic feature extraction of platform text information: The vector X obtained in process (1) i As the embedding input of the convolutional neural network part of the recommendation management module, it is then passed into the hidden layer containing the convolutional layer for convolution calculation, and finally outputs the text-topic feature vector y CNN ; The convolutional neural network can be any convolutional neural network, such as an m-layer convolutional neural network (CNN) in the prior art.

[0016] (4) User-interest feature extraction of enterprise user information: The vector Q obtained in process (2) is introduced as another part of the recommendation management module into the input part of the factorization machine (FM) of the attention mechanism network. This part calculates the user-interest feature vector y with unique information weight for each user through the factorization machine (FM) combined with the attention mechanism. AFM , the formula is as follows:

[0017]

[0018] In formula (I), y AFM is the user-interest feature vector, w 0 is a global constant bias term, n is the total number of user-interest features of enterprise user information, i and j are values ​​greater than 0 and less than n, and w i is the first-order eigenvector parameter, p T is the parameter vector, a i，j is the eigenvalue Q i and Q j The corresponding hidden vector v i and v j The attention score, v i is the K-dimensional latent vector of the user-interest feature of the i-th enterprise user information; v j is the K-dimensional latent vector of the user-interest feature of the j-th enterprise user information, Q i is the i-th eigenvector Q in the above process (2), Q j is the j-th eigenvector Q in the above process (2);

[0019] The essence of the attention mechanism network is a simple structure of a fully connected layer plus a softmax output layer. Its mathematical expression is as follows:

[0020] a' i,j =h T ReLu(W(v i ⊙v j )Q i Q j +b) (II)

[0021]

[0022] In formulas (II) and (III), h is the weight vector from the fully connected layer to the softmax output layer, h T is the transposed matrix of the weight vector h, ReLu is the activation function of the attention network, W is the weight matrix from the feature cross layer to the attention network fully connected layer, v i is the K-dimensional latent vector of the user-interest feature of the i-th enterprise user information; v jis the K-dimensional latent vector of the user-interest feature of the j-th enterprise user information, Q i is the i-th eigenvector Q in the above process (2), Q j is the jth feature vector Q in the above process (2), b is the bias vector, and the learned model parameters are the weight matrix W from the feature cross layer to the attention network fully connected layer, the bias vector b, and the weight vector h, a′ from the fully connected layer to the softmax output layer. i，j is the calculated attention score data, a i，j a′ i，j The result is mapped to an attention score between 0 and 1 through softmax.

[0023] (5) The vector y obtained by processes (3) and (4) CNN ,y AFM By integrating low-order and high-order features, and then through sigmoid nonlinear transformation, we get the probability output of predicting the degree of preference. Then, we sort the recommended information according to the probability of the degree of preference to get the topk to be recommended. The formula is as follows:

[0024] y=sigmoid(y CNN +y AFM ) (IV)

[0025] In formula (IV), y is the probability value of the degree of liking of the recommended information, ranging from 0 to 1, and topk is the k results recommended by the system;

[0026] (6) The topk to be recommended are transmitted to the front-end display module to form a recommendation list and recommended to the corresponding enterprise users.

[0027] The effects provided in the content of the invention are only the effects of the embodiments, not all the effects of the invention. One of the above technical solutions has the following advantages or beneficial effects:

[0028] The present invention solves the problem of information overload in the industrial Internet platform through convolutional neural networks and the introduction of FM's attention mechanism. The present invention separately models the information data and user information of the industrial Internet platform based on their unique characteristics. The convolutional neural network can extract the characteristics of the platform information at a deeper level, effectively solving the diversity of platform information and the correlation of hidden features. The combination of the attention mechanism and the factor decomposition machine solves the problems of sparse user-topic matrix data and different information weights (because users have personalized characteristics and preferences, they have different preferences for different information, so the information weights are different). The present invention models in different ways according to the different characteristics of users and information in the industrial Internet platform, making the recommendation results more humane.

[0029] The cold start problem of items is solved by CNN in step 3. The cold start of items refers to how to recommend new items to users who may be interested in them. The items in the present invention refer to useful information in the industrial Internet platform. The powerful local feature extraction capability of CNN in step 3 can effectively capture the semantic features of the word sequence of text information, thereby obtaining a group of items with high similarity, and finally recommending them to users (because the information of these items is highly similar), thereby solving the cold start problem of items in recommendation;

[0030] The present invention introduces an attention mechanism in step 4. The idea of ​​the attention mechanism is to let the model learn to allocate its own attention to highlight the different influences of different features on modeling. Different information in the platform is of different importance to each enterprise user. The same information has different amounts of information when modeling different users. Therefore, the introduction of the attention mechanism solves the problem of different weights of information importance.

[0031] The present invention uses a factorization machine (FM) combined with an attention mechanism in step 4. The factorization machine (FM) takes into account the second-order intersection of features, so that each vector is fully learned, thereby solving the problem of difficulty in training caused by data sparsity. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] Figure 1 It is a schematic diagram of the system working process of the present invention;

[0033] Figure 2 It is a schematic diagram of the system model of the present invention; DETAILED DESCRIPTION

[0034] In order to clearly illustrate the technical features of the present solution, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings. The disclosure below provides many different embodiments or examples for realizing different structures of the present invention. In order to simplify the disclosure of the present invention, the components and settings of specific examples are described below. In addition, the present invention may repeat reference numbers and / or letters in different examples. This repetition is for the purpose of simplification and clarity, and does not itself indicate the relationship between the various embodiments and / or settings discussed. It should be noted that the components illustrated in the accompanying drawings are not necessarily drawn to scale. The present invention omits the description of known components and processing techniques and processes to avoid unnecessary limitations on the present invention.

[0035] An intelligent recommendation system for industrial Internet platform, including background management module, user terminal, recommendation management module and front-end display module.

[0036] The background management module performs feature classification on the platform's information data, and performs addition, deletion, modification, query, and storage management on the information data; the information data mainly includes industry news policies, industry solutions, various products, expert databases, etc. on the industrial Internet platform.

[0037] The user end includes a user information management module and a user behavior record management module. The user information management module is responsible for registering new users of enterprise users and storing user registration information. The user behavior record management module is responsible for recording the user's browsing record log on the platform; such as news policy browsing, industry solution browsing, platform search records, etc.

[0038] The front-end display module is mainly used to display the most suitable recommendation list for enterprise users.

[0039] The recommendation management module takes the information data of the background management module and the information data of the user end as input, performs feature modeling on the information data of the background management module through a convolutional neural network to obtain a text-topic feature model, performs feature modeling on the information data of the user end through FM (Factorization Machine) combined with an attention mechanism to obtain a user-interest feature model, performs feature interactive fusion on the feature vectors emedding of the text-topic feature model and the user-interest feature model, and then performs a sigmoid nonlinear transformation to obtain a predicted probability value of the degree of liking of the recommended information, sorts the recommended information according to the size of the probability value of the degree of liking, and obtains the recommended k results topk as output and transmits them to the front-end display module.

[0040] A working method of an intelligent recommendation system for an industrial Internet platform is:

[0041] (1) Data preprocessing of the platform information in the background management module: Jieba word segmentation is performed on the platform text data to transform it into n token units, and the n tokens obtained by word segmentation are converted into a fixed-length vector X through the word2vector method. i ;

[0042] (2) Data preprocessing of user information: One-hot encode the user's registration information and behavior record data information and convert it into a word vector Q; this encoding form uses an n'-bit state register to encode n' states, each state has its own independent register bit, and only one bit is valid at any time. It is the most common binary vector representation method. For example, "China, the United States, and France" are respectively "100, 010, and 001" through one-hot encoding. Therefore, all data about the user is converted into a vector representation through this encoding method.

[0043] (3) Text-topic feature extraction of platform text information: The vector X obtained in process (1) i As the embedding input of the convolutional neural network part of the recommendation management module, it is then passed into the hidden layer containing the convolutional layer for convolution calculation, and finally outputs the text-topic feature vector y CNN ; The convolutional neural network can be any convolutional neural network, such as an m-layer convolutional neural network (CNN) in the prior art.

[0044] (4) User-interest feature extraction of enterprise user information: The vector Q obtained in process (2) is introduced as another part of the recommendation management module into the input part of the factorization machine (FM) of the attention mechanism network. This part calculates the user-interest feature vector y with unique information weight for each user through the factorization machine (FM) combined with the attention mechanism. AFM .

[0045] FM (Factorization Machines) is an improved version of the logistic regression model. It aims to solve the problem that model parameters are difficult to train in the sparse data scenario (forming feature vectors through one-hot encoding) in recommendation. The second-order intersection of features is considered. The principle is to add a polynomial on the basis of the linear model to describe the second-order intersection of features. The formula of FM is:

[0046]

[0047] In formula (V), y FM is the user-interest feature vector, w 0 is a global constant bias term, n is the total number of user-interest features of enterprise user information, i and j are values ​​greater than 0 and less than n, and w i is the first-order eigenvector parameter, w ij is the second-order cross-characteristic parameter, Q i is the i-th eigenvector Q in the above process (2), Q j is the j-th eigenvector Q in the above process (2);

[0048] The attention mechanism is equivalent to a weighted average, and the value of attention is the weight, which determines the importance of the interaction between different features. Therefore, the improved formula of the present invention is as follows:

[0049]

[0050] In formula (I), y AFM is the user-interest feature vector, w 0 is a global constant bias term, n is the total number of user-interest features of enterprise user information, i and j are values ​​greater than 0 and less than n, and w i is the first-order eigenvector parameter, p T is the parameter vector, a i，j is the eigenvalue Q i and Q j The corresponding hidden vector v i and v j The attention score, v i is the K-dimensional latent vector of the user-interest feature of the i-th enterprise user information; v j is the K-dimensional latent vector of the user-interest feature of the j-th enterprise user information, Q i is the i-th eigenvector Q in the above process (2), Q j is the j-th eigenvector Q in the above process (2);

[0051] The difference between the improved formula and the FM formula lies in the third term, the most important of which is the attention score a i,j , the simplest method is to use a weight parameter to represent it, but in order to prevent the weight parameter from being difficult to converge due to the sparse cross-feature data, an attention network is used between the pairwise feature cross layer and the pooling layer to generate the attention score. The essence of the attention mechanism network is a simple fully connected layer plus a softmax output layer structure, and its mathematical expression is as follows:

[0052] a' i,j =h T ReLu(W(v i ⊙v j )Q i Q j +b) (II)

[0053]

[0054] In formulas (II) and (III), h is the weight vector from the fully connected layer to the softmax output layer, h Tis the transposed matrix of the weight vector h, ReLu is the activation function of the attention network, W is the weight matrix from the feature cross layer to the attention network fully connected layer, v i is the K-dimensional latent vector of the user-interest feature of the i-th enterprise user information; v j is the K-dimensional latent vector of the user-interest feature of the j-th enterprise user information, Q i is the i-th eigenvector Q in the above process (2), Q j is the jth feature vector Q in the above process (2), b is the bias vector, and the learned model parameters are the weight matrix W from the feature cross layer to the attention network fully connected layer, the bias vector b, and the weight vector h, a′ from the fully connected layer to the softmax output layer. i，j is the calculated attention score data, a i，j a′ i，j The result is mapped to an attention score between 0 and 1 through softmax.

[0055] (5) The vector y obtained by processes (3) and (4) CNN ,y AFM By integrating low-order and high-order features, and then through sigmoid nonlinear transformation, we get the probability output of predicting the degree of preference. Then, we sort the recommended information according to the probability of the degree of preference to get the topk to be recommended. The formula is as follows:

[0056] y=sigmoid(y CNN +y AFM ) (IV)

[0057] In formula (IV), y is the probability value of the degree of liking of the recommended information, ranging from 0 to 1, and topk is the k results recommended by the system;

[0058] (6) The topk to be recommended are transmitted to the front-end display module to form a recommendation list and recommended to the corresponding enterprise users.

[0059] Although the above describes the specific implementation mode of the invention in conjunction with the drawings, it is not intended to limit the scope of protection of the invention. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. A recommendation system for an industrial Internet platform based on a convolutional neural network and an attention mechanism, characterized in that: Including backend management module, user end, recommendation management module, and front-end display module; The background management module performs feature classification on the platform's information data, and performs addition, deletion, modification, query, and storage management on the information data; The user terminal includes a user information management module and a user behavior record management module. The user information management module is responsible for registering new users of enterprise users and storing user registration information. The user behavior record management module is responsible for recording the browsing record log of users on the platform. The recommendation management module takes the information data of the background management module and the information data of the user end as input, performs feature modeling on the information data of the background management module through a convolutional neural network to obtain a text-topic feature model, and the text-topic feature model performs feature extraction on the data of the background management module; performs feature modeling on the information data of the user end through a factor decomposition machine combined with an attention mechanism to obtain a user-interest feature model, and the user-interest feature model performs feature extraction on the data of the user end; obtains a predicted probability value of the degree of liking of the recommended information by fusing the feature vectors of the text-topic feature model and the user-interest feature model through low-order and high-order feature interactions, and then undergoes a sigmoid nonlinear transformation, and sorts the recommended information according to the size of the probability value of the degree of liking, and obtains the recommended k results topk as output and transmits them to the front-end display module; The front-end display module displays the recommendation list that best matches the enterprise user.

2. A working method of a recommendation system for an industrial Internet platform based on a convolutional neural network and an attention mechanism, characterized in that: The following processes are included: (1) Data preprocessing of the platform information in the background management module: Jieba word segmentation is performed on the platform text data to transform it into multiple token units, and the multiple tokens obtained by word segmentation are converted into a fixed-length vector X through the word2vector method. i ; (2) Preprocessing user information: One-hot encode the user's registration information and behavior record data and convert them into word vectors Q; (3) Text-topic feature extraction of platform text information: The vector X obtained in process (1) i As the embedding input of the convolutional neural network part of the recommendation management module, it is then passed into the hidden layer containing the convolutional layer for convolution calculation, and finally outputs the text-topic feature vector y CNN ; (4) User-interest feature extraction of enterprise user information: The vector Q obtained in process (2) is introduced as another part of the recommendation management module into the input part of the factorization machine of the attention mechanism network. The formula is as follows: In formula (I), y AFM is the user-interest feature vector, w0 is the global constant bias term, n is the total number of user-interest features of enterprise user information, i and j are values ​​greater than 0 and less than n, w i is the first-order eigenvector parameter, p T is the parameter vector, a i，j is the eigenvalue Q i and Q j The corresponding hidden vector v i and v j The attention score, v i is the K-dimensional latent vector of the user-interest feature of the i-th enterprise user information; v j is the K-dimensional latent vector of the user-interest feature of the j-th enterprise user information, Q i is the i-th eigenvector Q in the above process (2), Q j is the j-th eigenvector Q in the above process (2); your' i,j h T ReLu(W(v i ⊙v j )Q i Q j +b) (II) In formulas (II) and (III), h is the weight vector from the fully connected layer to the softmax output layer, h T is the transposed matrix of the weight vector h, ReLu is the activation function of the attention network, W is the weight matrix from the feature cross layer to the attention network fully connected layer, v i is the K-dimensional latent vector of the user-interest feature of the i-th enterprise user information; v j is the K-dimensional latent vector of the user-interest feature of the j-th enterprise user information, Q i is the i-th eigenvector Q in the above process (2), Q j is the jth feature vector Q in the above process (2), b is the bias vector, and the learned model parameters are the weight matrix W from the feature cross layer to the attention network fully connected layer, the bias vector b, and the weight vector h, a′ from the fully connected layer to the softmax output layer. i，j is the calculated attention score data, a i，j a′ i，j Map the result to an attention score between 0 and 1 through softmax; (5) The vector y obtained by processes (3) and (4) CNN ,y AFM By integrating low-order and high-order features, and then through sigmoid nonlinear transformation, we get the probability output of predicting the degree of preference. Then, we sort the recommended information according to the probability of the degree of preference to get the topk to be recommended. The formula is as follows: y=sigmoid(y CNN +y AFM ) (IV) In formula (IV), y is the probability value of the degree of liking of the recommended information, ranging from 0 to 1, and topk is the k results recommended by the system; (6) The topk to be recommended are transmitted to the front-end display module to form a recommendation list and recommended to the corresponding enterprise users.

3. The working method of the recommendation system of the industrial Internet platform based on the convolutional neural network and the attention mechanism as claimed in claim 2, characterized in that: The encoding form in the process (2) is to use an n'-bit state register to encode n' states, each state has its own independent register bit, and only one bit is valid at any time.

Images