Symmetry-Based Deep Network and Dynamic Multi-Interaction Human Resource Position Recommendation Method

Abstract

The present invention discloses a method for recommending human resource positions based on a symmetric deep network and dynamic multi-interaction, which preprocesses the acquired data to obtain feature embedding vectors; constructs an improved symmetric deep network model and a dynamic fine-grained multi-feature interaction module for parallel training; pools the vectorized user behavior interaction information and then inputs it into the multi-interest extraction layer in the capsule network to obtain multiple interest capsule representations of the user's historical behavior; then inputs it into the symmetric deep neural network model, which is composed of a forward MLP and a reverse MLP. By adding lateral connections, the reverse MLP is allowed to contain more information; builds a dynamic fine-grained multi-feature interaction module, mines the preferences in the user's historical behavior by inputting the user behavior feature information into the Encoder module of the Transformer, and inputs it into the Attention mechanism to obtain sequence information; performs a Softmax operation on the above two modules to obtain the final prediction result to provide more accurate and timely recommendations.

Description

Technical Field

[0001] The present invention relates to the fields of natural language processing and recommendation algorithms, and particularly to a human resource recommendation method based on a symmetric deep network and a dynamic multi-interaction attention mechanism. Background Art

[0002] Recommendation systems have become a very useful tool in various fields. Researchers have been trying to improve their algorithms in order to issue better predictions to users. It has long been used to improve the user experience and provide personalized recommendations in different fields such as media, entertainment, and e-commerce.

[0003] One of the current challenges in the field of recommendation is how to make accurate predictions for a specific field and correctly evaluate the results predicted by the recommendation system algorithm; the key to recommendation is to be able to fully mine the characteristics of the data itself so that it can cope with changing needs and make accurate recommendations. Regarding fully mining the characteristics of the data itself, that is, a content-rich model, researchers have also used a context-rich model, considering context information related to interactions, such as time, location, and past interactions, to more fully extract feature representations; the main challenge for long-term inactive users is to capture the changing preferences of their historical behavior interest predictions, which is the key to achieving accurate and timely recommendations.

[0004] Capsule networks are primarily used for image training, aggregating pose information. Therefore, they can learn good results with a small amount of data, so this is also a significant improvement over CNNs, better modeling the hierarchical relationship of internal knowledge representation in neural networks. However, its training is relatively difficult, possibly due to the update coupling coefficient and the influence of convolutional layer stacking. Improving the training efficiency is a major challenge.

[0005] The Attention mechanism has achieved good results in solving the problems of long information and information loss, capturing global relationships in one step, and paying attention to the local connections of elements. When the function is calculating, it compares each element of the sequence with other elements, and the distance between each element is the same during this process. The disadvantages are also relatively obvious. Since it processes all elements of the sequence in parallel, it cannot consider the element order of the input sequence, and the results obtained in natural language processing tasks are often greatly discounted.

[0006] Methods based on deep neural networks can well capture complex relationships. However, deep neural networks applied to recommendation systems are usually a pyramid structure that maps user and item latent vectors to a low-dimensional space to learn abstract and invariant features. But this may lead to partially inseparable features. Summary of the Invention

[0007] Objective of the Invention: Aiming at the problems existing in the prior art, the present invention proposes a symmetric deep network and a dynamic multi-interaction human resource job recommendation method, which can fully extract and learn data features, learn data features from multiple angles according to the improved symmetric deep network, and secondly use fine-grained feature interaction to fully mine the features of users' historical behavior data to provide more accurate and timely recommendations.

[0008] Technical Solution: The present invention proposes a human resource job recommendation method based on a symmetric deep network and dynamic multi-interaction, including the following steps:

[0009] Step 1: Integrate the data information provided by enterprises and the job information crawled from personal networks, and preprocess the obtained user information, job information, and user behavior information;

[0010] Step 2: Perform one-hot vectorized feature representation on the preprocessed data, and perform dimensionality reduction processing on the feature vectors with high dimensions according to the characteristics of the data;

[0011] Step 3: Input the vectorized user information and user behavior interaction information into the Max Pooling layer, and then input it into the multi-interest extraction layer in the capsule network to obtain multiple interest capsules of the user's historical behavior;

[0012] Step 4: Combine the user feature vector with multiple interest capsule vectors through a fully connected operation to obtain the final multi-interest vector representation of the user;

[0013] Step 5: Construct an improved symmetric deep network model, and input the user multi-interest vector representation into the improved symmetric deep network model; the improved symmetric deep network model obtains a low-dimensional feature map through a forward MLP operation, uses the Leaky-Relu activation function between each layer, and at the same time adds a horizontal connection, and inputs the result of the forward MLP into the reverse MLP to capture high-dimensional feature relationships;

[0014] Step 6: Build a dynamic fine-grained multi-feature interaction module, and input the user behavior feature information into the improved Encoder module of the Transformer. The improved Encoder module changes the layer normalization layer in the original Transformer's encoder from post_LN to Pre-LN, performs an interaction operation on the output result with the job features, inputs the result into the Attention mechanism, and then inputs it into the MLP for linear mapping;

[0015] Step 7: Perform a SoftMax operation on the outputs of the symmetric deep network model and the dynamic fine-grained multi-feature interaction module to obtain the final predicted recommendation result.

[0016] Furthermore, the specific method of step 1 is as follows:

[0017] Step 1.1: Integrate the enterprise job data, user data, user interaction data provided by the cooperative enterprise and the recent job data D crawled from the personal network;

[0018] Step 1.2: After cleaning, de-duplicating, and removing blanks from the sorted data D, enterprise job data D1, user data D2, and user behavior data D3 are obtained, and their data all include entity and attribute information;

[0019] Step 1.3: The preprocessed feature information can be expressed as the following formula:

[0020]

[0021] where N is the dimension of the data, represents the feature of the i-th dimension.

[0022] Furthermore, the specific method of step 2 is as follows:

[0023] Step 2.1: For the preprocessed feature information X * , perform vectorized feature representation. If it is a numerical feature, it is directly used as a scalar. If it is a non-numerical feature, one-hot processing is used;

[0024] Step 2.2: For the non-numerical feature one-hot vector whose feature vector is high-dimensional, perform dimensionality reduction processing. The formula is as follows:

[0025]

[0026] For high-dimensional features, wherein is the one-hot vector, V i is the vector matrix, and non-high-dimensional features are directly scalarized, that is

[0027] Step 2.3: According to the above vectorized processing, the vectorized representations of the corresponding user, enterprise job, and user behavior features can be obtained, as follows:

[0028]

[0029]

[0030]

[0031] wherein the fine-grained embedding feature vector of user f The fine-grained embedding feature vector of enterprise position p The fine-grained embedding feature vector of user behavior 1, where F, P, and L are the total numbers of users, enterprise positions, and user behaviors respectively.

[0032] Furthermore, the specific method of step 3 is as follows:

[0033] Step 3.1: Input the vectorized user behavior interaction information e b into the Max Pooling layer, and then input it into the multi-interest extraction layer in the capsule network to calculate the corresponding number of dynamic interests K u , and the update iteration formula is as follows:

[0034] K u = max(1, min(K u , log2|e b |))

[0035] where K u is the number of dynamic interests of user u; obtain multiple interest capsules of the user's historical behavior;

[0036] Step 3.2: Calculate all user behavior routing weights. To ensure that the sum of all behavior routings is 1, use the Softmax function for operation;

[0037] Step 3.3: Calculate all interest routing weights. Perform a dot product operation on the user behavior routing weights and user behavior features. To ensure the consistency of dimensions, use the bilinear mapping matrix U, and finally perform weighted summation;

[0038] Step 3.4: To ensure the consistency of directions, use the non-linear function Squash for regulation, and then update and iterate the parameters to output the corresponding interest capsule vector u j .

[0039] Furthermore, the specific method of step 5 is as follows:

[0040] Step 5.1: Input the user multi-interest vector representation into the forward MLP network to obtain a low-dimensional feature mapping;

[0041] Step 5.2: Through the improved use of the Leaky-Relu activation function, adjust the bias. The specific hyperparameter is α, α ∈ (1, +∞), and the formula is as follows:

[0042]

[0043] Step 5.3: Add horizontal connections simultaneously. After completing a forward network training, in addition to further forward propagation training, the output is also input into the reverse neural network. Both the forward and reverse networks use three layers.

[0044] Step 5.4: Perform a Concat operation on the results after forward and reverse training, and output the corresponding feature representation information.

[0045] Furthermore, the specific method for building the dynamic fine-grained multi-feature interaction module in Step 6 is as follows:

[0046] Step 6.1: First, input the user behavior feature information into the LayerNorm layer. The formula is as follows:

[0047]

[0048] γ and β are hyperparameters for backpropagation, μ and σ are the mean and variance of the input matrix, ∈ is a positive bias factor to prevent the denominator from being zero, and l b is the output of the normalization network layer corresponding to the user behavior information;

[0049] Step 6.2: Input the network layer result into the multi-head self-attention model in the improved encoder model of Transformer, perform FFN operation, and obtain the interactive representation h l , and directly concatenate it with the representation e i of the position to obtain the representation V l of different user behavior features, and get the following formula:

[0050] V l = Concat(e i , h l )

[0051] Step 6.3: Use the output result V l as the input of the Attention mechanism, and calculate the preference weights for interactive behaviors at different times. The formula is as follows:

[0052]

[0053]

[0054] where θ is the scoring function, α l is the user behavior preference weight, is the bias factor, W l is the weight matrix, and Relu is the activation function;

[0055] Step 6.4: After obtaining the weights, perform weighted R ib, and then input it into the linear mapping of the MLP layer to obtain more fine-grained feature information and output the final feature representation:

[0056]

[0057] Among them, R ib is the feature representation of the fine-grained interaction between the job and user behavior information.

[0058] Beneficial effects:

[0059] In the data preprocessing stage of the method of the present invention, dimensionality reduction is performed on high-dimensional data, and the interest extraction model of the capsule network is used to perform multi-interest extraction on user behaviors. Then, a fully connected operation is performed in combination with job and user characteristics to fully obtain the interests of users, which are used as the input of the symmetric deep network. The horizontal connection positive and negative network models are used to better distinguish features that are difficult to separate in the low-dimensional space and improve the accuracy of recommendations. In parallel with this, the dynamic fine-grained multi-feature interaction attention module mainly performs fine-grained feature interaction on user historical behavior information and job information, mines the preferences in user historical behaviors, and the feature interaction mainly passes through the Encoder module in the Transformer model. Then, it interacts with the job representation and is input into the Attention mechanism for feature weight training, adaptively exploring the user preferences of sequential behaviors in different subspaces, further improving the sufficiency of feature extraction and the accuracy of recommendations. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 is the flow chart of the method for recommending human resource positions based on a symmetric deep network and dynamic multi-interaction of the present invention;

[0061] Figure 2 is the architecture diagram of the method for recommending human resource positions based on a symmetric deep network and dynamic multi-interaction of the present invention;

[0062] Figure 3 is the schematic diagram of the improved Transformer-Encoder module of the method for recommending human resource positions based on a symmetric deep network and dynamic multi-interaction of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0063] The present invention will be further described in detail below with reference to the accompanying drawings.

[0064] See attached Figure 1 to attached Figure 3 , the present invention discloses a method for recommending human resource positions based on a symmetric deep network and dynamic multi-interaction. For the specific flow chart, see Figure 1 , including the following steps:

[0065] Step 1: Integrate the data information provided by partner enterprises and the job information crawled through personal networks, and preprocess the obtained user information, job information, and user behavior information. The specific method is as follows:

[0066] Step 1.1: Integrate the enterprise job data, user data, user interaction data provided by partner enterprises and the recent job data D crawled through personal networks.

[0067] Step 1.2: After performing operations such as cleaning, duplicate removal, and empty value removal on the sorted data D, obtain enterprise job data D1, user data D2, and user behavior data D3, and their data all include entity and attribute information.

[0068] Step 1.3: The preprocessed feature information can be expressed as the following formula:

[0069]

[0070] where N is the dimension of the data, represents the feature of the i-th dimension.

[0071] Step 2: For the preprocessed data, perform one-hot vectorized feature representation, and perform dimensionality reduction on the feature vectors with high dimensions according to the features of the data. The specific method is as follows:

[0072] Step 2.1: For the preprocessed feature information X * , perform vectorized feature representation. If it is a numerical feature, it is directly used as a scalar. If it is a non-numerical feature, one-hot processing is used.

[0073] Step 2.2: For the non-numerical feature one-hot vector whose feature vector is high-dimensional, perform dimensionality reduction. The formula is as follows:

[0074]

[0075] For high-dimensional features, perform where is the one-hot vector, V i is the vector matrix, and non-high-dimensional features are directly scalarized, that is,

[0076] Step 2.3: According to the above vectorization processing, the vectorized representations of the corresponding user, enterprise job, and user behavior features can be obtained, which are as follows:

[0077]

[0078]

[0079]

[0080] Among them, The fine-grained embedding feature vector of user f, The fine-grained embedding feature vector of enterprise position p, The fine-grained embedding feature vector of user behavior 1, where F, P, and L are the total numbers of users, enterprise positions, and user behaviors respectively.

[0081] Step 3: Input the vectorized user behavior interaction information into the Max Pooling layer, and then input it into the multi-interest extraction layer in the capsule network to obtain multiple interest capsules of the user's historical behavior. The specific method is as follows:

[0082] Step 3.1: Input the vectorized user behavior interaction information e b into the Max Pooling layer, and then input it into the multi-interest extraction layer in the capsule network to calculate the corresponding dynamic number of interests K u , and the update iteration formula is as follows:

[0083] K u = max(1, min(K u , log2|e b |))

[0084] Among them, K u is the dynamic number of interests of user u; obtain multiple interest capsules of the user's historical behavior.

[0085] Step 3.2: Calculate the routing weights of all user behaviors. To ensure that the sum of all behavior routings is 1, use the Softmax function for operation.

[0086] Step 3.3: Calculate the routing weights of all interests. Perform a dot product operation on the user behavior routing weights and user behavior features. To ensure the consistency of dimensions, use the bilinear mapping matrix U, and finally perform weighted summation.

[0087] Step 3.4: To ensure the consistency of directions, use the nonlinear function Squash for regulation, and then update and iterate the parameters to output the corresponding interest capsule vector u j .

[0088] Step 4: Perform a Pooling operation on the position feature vector, and combine multiple interest capsule vectors through a fully connected operation to obtain the user's final multi-interest vector representation. The specific method is as follows:

[0089] Step 4.1: Input the position feature vector e i , into the pooling layer, and combine it with the interest capsule vector u jPerform a full connection to obtain a user multi-interest vector representation as the input feature of the improved symmetric deep network.

[0090] Step 5: Construct an improved symmetric deep network model and input the user multi-interest vector representation into the improved symmetric deep network model. The improved symmetric deep network model obtains a low-dimensional feature map through a forward MLP operation, uses the Leaky-Relu activation function between each layer, and simultaneously adds a lateral connection. The result of the forward MLP is input into the reverse MLP to capture high-dimensional feature relationships. The specific method is as follows:

[0091] Step 5.1: Input the user multi-interest vector representation into the forward MLP network to obtain a low-dimensional feature map.

[0092] Step 5.2: Adjust the bias by improving the use of the Leaky-Relu activation function. The specific hyperparameter is α, α ∈ (1, +∞), and the formula is as follows:

[0093]

[0094] Step 5.3: Simultaneously add a lateral connection so that after completing a forward network training, in addition to further forward propagation training, it will also be input into the reverse neural network. The number of layers of the forward and reverse networks used is 3 layers.

[0095] Step 5.4: Perform a Concat operation on the results after forward and reverse training and output the corresponding feature representation information.

[0096] Step 6: Build a dynamic fine-grained multi-feature interaction module and input the user behavior feature information into the improved Encoder module of Transformer. The improved Encoder module changes the layer normalization layer in the original Transformer's encoder from post_LN to Pre-LN, performs an interaction operation on the output result with the job characteristics, inputs the result into the Attention mechanism, and then inputs it into the MLP for linear mapping. The specific method is as follows:

[0097] Step 6.1: First, input the user behavior feature information into the LayerNorm layer, and the formula is as follows:

[0098]

[0099] γ and β are hyperparameters for backpropagation, μ and σ are the mean and variance of the input matrix, ∈ is a positive bias factor to prevent the denominator from being zero, l b The output of the normalization network layer corresponding to the user behavior information.

[0100] Step 6.2: Input the network layer result into the multi-head self-attention model in the encoder model of the improved Transformer for FFN operation to obtain the interactive representation h l , and directly concatenate it with the representation e of the position i to obtain the representation V of different user behavior features l , and get the following formula:

[0101] V l = Concat(e i , h l )

[0102] Step 6.3: Use the output result V l as the input of the Attention mechanism to calculate the preference weights for interactive behaviors at different times. The formula is as follows:

[0103]

[0104]

[0105] where θ is the scoring function, α l is the user behavior preference weight, is the bias factor, W l is the weight matrix, and Relu is the activation function

[0106] Step 6.4: After obtaining the weights, perform weighted R on the representation ib , and then input it into the MLP layer for linear mapping to obtain more fine-grained feature information and output the final feature representation:

[0107]

[0108] where R ib is the feature representation of the fine-grained interaction between the position and user behavior information

[0109] Step 7: Perform SoftMax operation on the output of the symmetric deep network and the dynamic fine-grained multi-feature interaction module to obtain the final predicted recommendation result. The specific method is as follows:

[0110] Step 7.1: Read the final output feature of the symmetric deep network and the output R of the dynamic fine-grained multi-feature interaction module ib , and perform Softmax operation to obtain the final predicted recommendation result

[0111] The above embodiments are only for illustrating the technical concept and features of the present invention, and the purpose is to enable those skilled in the art to understand the content of the present invention and implement it accordingly, and it should not be used to limit the protection scope of the present invention. Any equivalent transformation or modification made according to the spirit of the present invention should be covered within the protection scope of the present invention.

Claims

1. A method for recommending human resource positions based on a symmetric deep network and dynamic multi-interaction, characterized in that, It includes the following steps: Step 1: Integrate the data information provided by the enterprise and the job information crawled from the personal network, and preprocess the obtained user information, job information, and user behavior information; Step 2: Perform one-hot vectorized feature representation on the preprocessed data. According to the characteristics of the data, perform dimensionality reduction on the high-dimensional feature vectors. Step 3: Input the vectorized user information and user behavior interaction information into the Max Pooling layer, and then input it into the multi-interest extraction layer in the capsule network to obtain multiple interest capsules of the user's historical behavior. Step 4: Through the fully connected operation of combining the user feature vector and multiple interest capsule vectors, obtain the final multi-interest vector representation of the user. Step 5: Construct an improved symmetric deep network model and input the user multi-interest vector representation into the improved symmetric deep network model; the improved symmetric deep network model obtains a low-dimensional feature map through the forward MLP operation, uses the Leaky-Relu activation function between each layer, and at the same time adds a lateral connection, and inputs the result of the forward MLP into the reverse MLP to capture high-dimensional feature relationships. Step 6: Build a dynamic fine-grained multi-feature interaction module and input the user behavior feature information into the improved Encoder module of Transformer. In the improved Encoder module, the layernormalization layer in the original Transformer's encoder is changed from post_LN to Pre-LN, perform an interaction operation on the output result and the job feature, input the result into the Attention mechanism, and then input it into the MLP for linear mapping. Step 6.1: First, input the user behavior feature information into the LayerNorm layer, and the formula is as follows: γ and β are hyperparameters for backpropagation, μ and σ are the mean and variance of the input matrix, ∈ is a regularization factor to prevent the denominator from being zero, l b Output of the normalization network layer for applying user behavior information, e b is the vectorized user behavior interaction information; Step 6.2: Input the network layer result into the multi-head self-attention model in the improved encoder model of the Transformer, perform the FFN operation, and obtain the interactive representation h l , directly concatenate with the representation e of the position i to obtain the representation V of different user behavior features l , and obtain the following formula: V l = Concat(e i , h l ) Step 6.3: Take the output result V l as the input of the Attention mechanism, and calculate the preference weights for the interaction behaviors at different times. The formula is as follows: Among them, the θ scoring function, α l is the user behavior preference weight, is the deviation factor, W l is the weight matrix, Relu is the activation function, and L represents the total number of user behaviors; Step 6.4: After obtaining the weights, perform weighted R on the representation ib , and then input it into the MLP layer for linear mapping to obtain more fine-grained feature information and output the final feature representation: Among them, R ib is the feature representation of the fine-grained interaction between the post and user behavior information; Step 7: Perform SoftMax operation on the outputs of the symmetric deep network model and the dynamic fine-grained multi-feature interaction module to obtain the final predicted recommendation result.

2. The method for recommending human resource positions based on a symmetric deep network and dynamic multi-interaction according to claim 1, wherein The specific method of the above Step 1 is: Step 1.1: Integrate the enterprise job data, user data, user interaction data provided by the cooperative enterprise and the recent job data D crawled from the personal network. Step 1.2: After cleaning, de-duplicating, and removing blanks from the sorted data D, obtain the enterprise job data D1, user data D2, and user behavior data D3, and all of these data include entity and attribute information. Step 1.3: The preprocessed feature information is expressed as the following formula: where N is the dimension of the data, represents the feature of the i-th dimension.

3. The method for recommending human resource positions based on a symmetric deep network and dynamic multi-interaction according to claim 1, wherein The specific method of the above Step 2 is: Step 2.1: For the preprocessed feature information X * , perform vectorized feature representation. If it is a numerical feature, it is directly used as a scalar. If it is a non-numerical feature, one-hot processing is used; Step 2.2: For non-numeric features, one-hot vectors whose feature vectors are high-dimensional are subjected to dimensionality reduction processing, and the formula is as follows: For high-dimensional features Among them is a one-hot vector, V i is a vector matrix, and non-high-dimensional features are directly scalarized, that is Step 2.3: According to the above vectorization process, obtain the vectorized representations of the corresponding user, enterprise job, and user behavior features, which are as follows: Among them, The fine-grained embedding feature vector of user f, The fine-grained embedding feature vector of enterprise position p, The fine-grained embedding feature vector of user behavior l, where F, P, and L are the total numbers of users, enterprise positions, and user behaviors respectively.

4. The method for recommending human resource positions based on a symmetric deep network and dynamic multi-interaction according to claim 1, characterized in that The specific method of the above Step 3 is: Step 3.1: Input the vectorized user behavior interaction information e b into the Max Pooling layer, and then input it into the multi-interest extraction layer in the capsule network to calculate the corresponding number of dynamic interests K u , and the update iteration formula is as follows: K u = max(1, min(K u , log2|e b |)) where K u is the number of dynamic interests of user u; obtaining multiple interest capsules of the user's historical behavior; Step 3.2: Calculate all user behavior routing weights. To ensure that the sum of all behavior routings is 1, use the Softmax function for operation. Step 3.3: Calculate all interest routing weights. Perform a dot product operation on the user behavior routing weights and user behavior features. To ensure the consistency of dimensions, use the bilinear mapping matrix U, and finally perform weighted summation. Step 3.4: To ensure the consistency of direction, use the non-linear function Squash for regulation, and then update and iterate the parameters to output the corresponding interest capsule vector u j .

5. The method for recommending human resource positions based on a symmetric deep network and dynamic multi-interaction according to claim 1, characterized in that The specific method of the above Step 5 is: Step 5.1: Input the user multi-interest vector representation into the forward MLP network to obtain a low-dimensional feature mapping; Step 5.2: Adjust the bias by improving the use of the Leaky-Relu activation function. The specific hyperparameter is α, where α ∈ (1, +∞), and the formula is as follows: Step 5.3: At the same time, add a lateral connection so that after completing a forward network training, in addition to further forward propagation training, it will also be input into the reverse neural network. The number of layers of the forward and reverse networks used is 3 layers; Step 5.4: Perform a Concat operation on the results after forward and reverse training and output the corresponding feature representation information.

Images