A Person-Job Matching Method Based on Symmetric Contrastive Learning

Abstract

The present invention discloses a person-job matching method based on symmetric contrast learning, and belongs to the field of computer data processing. The present invention divides resume text into regular text and irregular text, adopts a strategy to determine the remaining resume for regular text, and classifies irregular text according to work experience, project experience, job requirements, and job requirements, and performs unsupervised text semantic representation on any part. The key loss calculation strategy adopts a symmetric contrast learning function to solve the defect that the traditional loss function cannot fully meet the contrast learning idea, and can calculate the distance between the real sample and the enhanced sample from the whole. Finally, the calculated text semantic representation is used to calculate the semantic matching degree between the irregular text between the resume information such as work experience and the recruitment information through the attention mechanism, and the score between the entire resume and the recruitment information is given in combination with the calculated score of the regular text.

Description

Technical Field

[0001] The present invention relates to the technical field of computer data processing, and more specifically, to a person-job matching method based on symmetric contrast learning. Background Art

[0002] In the era of big data, various recruitment platforms have emerged continuously. While providing rich online recruitment services for users, they also face the dilemma of "information overload". In order to extract valuable information from a large amount of resume and job information and assist in the precise matching between talents and company needs, an algorithm framework for quickly judging the matching degree between resumes and job information is required. When calculating the overlap between resume and job information, the most crucial thing is how to evaluate the semantic similarity between two texts. Currently, the supervised learning-based methods require a large number of manually labeled data sets and have a strong manual participation attribute. For unsupervised learning methods, most of them use deep learning to learn word vectors or sentence vectors, and then use different strategies to combine the word vectors to obtain sentence representations; or use statistical machine learning methods to give keywords of a paragraph or an article as the semantic expression of the entire text. However, in practice, there is still a large room for optimization in its application reliability. Summary of the Invention

[0003] 1. Technical Problems to be Solved by the Invention

[0004] Since supervised learning requires a large number of labeled texts, and with the development of unsupervised deep learning, calculating text semantic similarity through unsupervised learning has strong vitality. Therefore, the present invention intends to propose an unsupervised learning method to learn the unstructured text data in resumes and recruitment information, in order to obtain a text expression with semantics. Specifically, the contrast learning method in unsupervised learning is used as the loss calculation function of the deep learning model. The present invention improves this loss function to obtain a symmetric contrast learning loss calculation function that satisfies the distance definition and makes the text semantic expression space have better uniformity or alignment; and through the pre-training model, unsupervised semantic similarity calculation training is carried out on the unstructured texts in the resume and recruitment data sets, so as to obtain a semantic similarity measurement model in this field; then, the parameter relationship between the semantic similarity between unstructured texts and the similarity of structured text representations is calculated through a small amount of labeled data, and finally, a model that only requires a small amount of manually labeled data but can quickly help enterprises screen resumes and individuals apply accurately can be obtained.

[0005] 2. Technical Solutions

[0006] To achieve the above object, the technical solution provided by the present invention is as follows:

[0007] A person-job matching method based on symmetric contrast learning of the present invention includes the following steps:

[0008] S100. Classify resumes and recruitment information into regular text and irregular text according to rules.

[0009] S200. Use the selected irregular text to train an irregular text semantic representation model.

[0010] S300. Calculate the semantic expression vectors of the irregular text in resumes and recruitment information according to the irregular text semantic representation model.

[0011] S400. Combine the semantic expression vectors of regular text and irregular text, and train a two-layer neural network model through label data to obtain a person-job matching prediction model.

[0012] Furthermore, an initial resume screening process is also included between steps S200 and S300. Incorporate regular text into the person-job matching calculation strategy for initial screening. Specifically, the person-job matching calculation strategy includes:

[0013] Strategy 1. Use regular text as a hard condition to screen resumes; or:

[0014] Strategy 2. Use regular text as a soft condition and add it to the resume for screening.

[0015] Furthermore, the calculation of the irregular text semantic expression model described in S200 includes the following steps:

[0016] S210. Divide the irregular text in resumes and recruitment information into training data sets and validation data sets ;

[0017] S220. Use a pre-trained model as an encoder. Each input is N different sentences in the training data set ; Select the position encoding of the CLS of the pre-trained model as the output of the encoding layer , and then enter a fully connected layer FCN. The vector obtained after passing through the activation function tanh(x) is the representation vector of the sentences in the entire batch in the text semantic space;

[0018] Similarly, for each sentence in the real sample , obtain an augmented sample through augmentation , and then perform the same operations as in step S220. Finally, obtain a matrix of representation vectors of the augmented sample text in the semantic space ; The specific augmentation method can be obtained by randomly deleting some words or replacing them with synonyms to get the augmented sample;

[0019] S230, using the loss calculation function model to calculate the loss value of the irregular text semantic representation model , and back propagate;

[0020] S240, repeat steps S220 to S230 until the verification data set Uniformity on The indicator can reach the threshold .

[0021] Furthermore, the specific calculation method in step S300 includes:

[0022] S310, irregular text processing of resume;

[0023] The irregular text in the resume is divided into work experience and project experience. Each category is divided into several sentences. Assume that the work experience is expressed as , project experience is expressed as , then the semantic expression vector of each sentence can be calculated based on the semantic expression model of irregular text, that is, the semantic expression vector of work experience , the semantic expression vector of project experience ;

[0024] S320, processing of irregular text in recruitment information;

[0025] The text in the recruitment information is divided into job requirements and job requirements. Each category is divided into several sentences. Suppose the job requirements are expressed as , job requirements are expressed as , the semantic expression vector of sentences in each category is calculated based on the irregular text semantic expression model, and the semantic expression vector of job requirements is calculated. And the semantic expression vector of job requirements ;

[0026] S330, calculating the matching degree of the recruitment information matching the resume information;

[0027] The vector representation of work experience in the job requirements space, the vector representation of work experience in the job requirements space, the vector representation of project experience in the job requirements space, and the vector representation of project experience in the job requirements space are calculated according to the attention mechanism;

[0028] S340, calculating the matching degree between the recruitment information and the resume information;

[0029] Calculate the vector representation of job requirements in the work experience space based on the attention mechanism , Vector representation of job requirements in project experience space Vector Representation of Job Requirements in the Work Experience Space Vector Representation of Job Requirements in the Project Experience Space .

[0030] 3. Beneficial Effects

[0031] Adopting the technical solution provided by the present invention, compared with the prior art, it has the following beneficial effects:

[0032] The traditional resume screening method filters resumes through some regularized data, and then manually reviews other unstructured texts. In this era of information explosion, even after screening, there are still a large number of invalid resumes. This not only requires a large amount of manpower from the recruitment side to screen, but also wastes the time of the applicants, making it impossible for both sides to quickly find the most suitable target. Also, due to the lack of text semantic tagging samples, it is particularly important to use unsupervised deep learning to calculate semantic similarity. Therefore, based on the contrastive learning method in the field of unsupervised learning, by improving the problem of inconsistent loss functions and the idea of contrastive learning, the present invention gives a calculation function that satisfies non-negativity, symmetry, and transitivity in distance. This is of great significance in meeting the need for a large amount of manpower to give the similarity between unstructured texts in person-job matching, and at the same time has wide applicability in other fields that require calculating text semantic similarity. Description of the Drawings

[0033] Figure 1 Schematic diagram of the framework of the traditional contrastive learning calculation method;

[0034] Figure 2 Schematic diagram of the symmetry and uniformity of the semantic improvement method in the present invention;

[0035] Figure 3 Flow chart of the calculation of the present invention;

[0036] Figure 4 Schematic diagram of the data text classification of the present invention. Detailed Embodiments

[0037] To further understand the content of the present invention, the present invention will be described in detail with reference to the accompanying drawings.

[0038] In the description of the present invention, it should be noted that the orientation or positional relationship indicated by the terms "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. In addition, the terms "first", "second", "third" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance.

[0039] The present invention will be further described below in conjunction with embodiments.

[0040] Embodiment 1

[0041] The idea of contrastive learning is that things that are close in distance should be consistent in the expression space compared to things that are far away. Taking the representation of text data as an example, for two sentences and , which are similar in the real space, they should be close in the semantic expression space. While the sentence and the sentence are semantically opposite or dissimilar compared to , then the distance between and in the text semantic expression space should be less than the distance between and . A key issue in using contrastive learning is to construct positive and negative samples. Positive samples are samples that are close in distance to the real sample, and negative samples are samples that are far away from the real sample. As shown in Figure 1 is a schematic framework diagram of the traditional contrastive learning calculation method.

[0042] For better description and understanding, first give some symbol assumptions. Assume the dataset , and the definition of each sentence is composed of multiple tokens, that is . Randomly select N samples from the dataset as a batch , and obtain the corresponding positive samples through a certain data augmentation method , that is . Among them, data augmentation methods in the field of text processing usually include randomly deleting certain words in the sentence, randomly deleting words, synonym replacement, etc., which belong to conventional techniques and will not be elaborated here. Negative samples are non-corresponding samples in the same batch, which are negative samples. Taking the sentence as an example, its corresponding positive sample is , and the corresponding negative sample is . Each sample can obtain the corresponding representation vector through the same mapping, that is , the current traditional calculation method for contrastive learning in the industry is as follows:

[0043]

[0044] Among them, represents calculating the cosine value of the angle between two semantic vectors ; represents the temperature hyperparameter.

[0045] However, the above formula does not fully meet the idea of contrastive learning, that is, shortening the sample pairs with closer distances and pulling apart the sample pairs with farther distances. The above formula can be understood as using the enhanced sample as the target semantic space, and then the real sample learns in this semantic space. However, the relationship between the real sample and the enhanced sample is relative, that is, it can be considered that the real sample is enhanced from the enhanced sample. Therefore, the real sample can be regarded as the target semantic space, and the corresponding enhanced sample learns from it. If we only use formula (1) as the loss calculation function, then it is equivalent to determining in advance one of the enhanced sample or the real sample as the target semantic space, and the remaining one as the semantic space to be learned, which adds prior knowledge. But often we don't know which part should be most suitable as the target semantic space. On the other hand, the idea of contrastive learning itself is a symmetric concept. Whether and are close or far, this is a symmetric relationship between two sentences, that is, when saying is far from , it also means that is far from . From the perspective of the batch, formula (1) can only unidirectionally show that when using the enhanced sample as the expression space, the coordinate expression of the real sample in this space is closer to the coordinate axis of this target space, and it cannot show that the enhanced sample also conforms to this target in the real sample space.

[0046] Based on the above two parts of the description, the present invention provides an improved symmetric contrastive learning loss calculation function.

[0047]

[0048] Among them represents the sample composed of N real samples, represents the sample enhanced according to , N is the number of sentences participating in the semantic model construction each time, is the balance hyperparameter, used to adjust the sample randomness problem in the real scenario, is the temperature hyperparameter, Denote the semantic representation of the real sample, which is any one in the semantic representation of the real sample, denote the semantic representation of the augmented sample, which is any one in the semantic representation of the augmented sample.

[0049] We first give the definition of distance based on the above loss function. Assume is a random sample set with batchsize equal to N, and the data augmentation strategy set , randomly select two augmentation strategies for augmentation, then we can get , then the non - negativity, symmetry and transitivity of the above loss function can be defined respectively as:

[0050]

[0051] The following will give proofs for non - negativity, symmetry and transitivity:

[0052] Regarding non - negativity, if and only if is an orthogonal matrix , in other cases, since , then , so , similarly for the latter part of formula (2), so non - negativity is satisfied; symmetry is obviously satisfied according to the definition; for transitivity, without loss of generality, we can assume is the normalized vector. First, expand the left - hand side of formula (5) according to the definition as:

[0053]

[0054] The right - hand side can be expanded as

[0055]

[0056] where .

[0057] Then when , one term in formula (6 - 1) , and one term in formula (7 - 1)

[0058] .

[0059] When , since is obtained by augmenting , then , such that the representations of the two sentences satisfy

[0060]

[0061] Then there is:

[0062]

[0063] Among them, , represents rounding up. So each term in formula (7 - 1) is greater than or equal to that in formula (6 - 1). Similarly, a similar idea can be applied to formula (7 - 2) and formula (6 - 2), then the transitivity holds.

[0064] Research shows that the semantic expression space is more uniform and the generalization performance of the semantic space is better. To better illustrate the performance of the improved loss calculation function in these two aspects, the definitions of two metrics are given first below.

[0065]

[0066] Among them, refers to the probability distribution of similar text pairs occurring, refers to the probability distribution of each sentence in the entire sentence space, is the representation vector after mapping and normalization. Based on the idea that similar sentences should be close in distance in the semantic expression space, then the formula (9) for calculating alignment is a quantization strategy for this idea; in addition, for the formula (10) for calculating the uniformity of the entire semantic expression space, considering a limiting case where all sentence expressions are concentrated in a very concentrated neighborhood, then the value of will be very large. If all semantic representations are evenly distributed near the origin, then it will make its value in a very small neighborhood.

[0067] When the data approaches infinity, the right - hand side of formula (2) can be considered as

[0068]

[0069] When minimizing the loss function formula (2), it means that the first line of formula (11) should be minimized as much as possible, further requiring that the angle between the similar sentence representations and is smaller, which is in line with the original intention of the alignment metric (9). In addition, minimizing the last two lines of formula (11) is also consistent with the definition of uniformity. Combining the above two points, formula (2) can balance uniformity and alignment, making the semantic expression space more likely to have better generalization performance.

[0070] In the specific implementation, the last two lines of formula (11) can be written as

[0071]

[0072] Among them, each column is a representation of a sentence. The first line of formula (12) can be transformed into the second line according to Jensen's inequality. Minimizing rest means minimizing the upper bound of the third term in formula (12). The existence theory states that when the sum of the diagonal elements is fixed, minimizing the sum means minimizing the corresponding largest eigenvalue, which makes the eigenvalues of the entire tend to be flat, improving the problem of semantic expression space aggregation and increasing the uniformity of the sentence expression space.

[0073] The following will describe the irregular text semantic representation model of the present invention:

[0074] By using the Bert or RoBert model as the encoding layer, the output at the cls position or the average of the encodings of each word is used as the semantic expression vector of a sentence.

[0075] S210: Divide the irregular text in the resume and recruitment information into a training data set and a validation data set .

[0076] S220: Use a pre-trained model such as Bert or Robert as the encoder. Each input is N different sentences in the training set ; Select the position encoding of the CLS of the pre-trained model as the output of the encoding layer , and then enter a fully connected layer FCN. The vector obtained after passing through the activation function tanh(x) is the representation vector of the sentences in the entire batch in the text semantic space.

[0077]

[0078] Similarly, for each sentence in the real sample , an enhanced sample is obtained through enhancement, and then the same operation is performed. Finally, a matrix of representation vectors of the enhanced sample text semantic space is obtained ;

[0079] S230: Use the improved loss calculation function formula (2) to calculate the loss value of the entire model , and backpropagate.

[0080] S240: Repeat steps S220 to S230 until the uniformity index on the validation set can reach the threshold .

[0081] According to the irregular text semantic representation model, the semantic expression vector of irregular text in resumes and recruitment information can be calculated; the following is a detailed description:

[0082] S310: Irregular text processing of resumes;

[0083] The irregular text in the resume can be divided into work experience and project experience. Each category can be divided into several sentences. Suppose the work experience is , project experience is , then according to the above irregular text semantic calculation model, the semantic expression vector of each sentence can be obtained, that is, the semantic expression vector of work experience , the semantic expression vector of project experience .

[0084] S320: Processing of irregular text in recruitment information;

[0085] The text in the recruitment information can be divided into job requirements and job requirements. Each category can be divided into several sentences. Assume that the job requirements can be expressed as , job requirements are expressed as Then, according to the above irregular text semantic calculation model, the semantic expression of sentences in each category and the semantic expression vector of job requirements are calculated. And the semantic expression vector of job requirements .

[0086] S330: Resume → Recruitment, calculating the matching degree of the recruitment information matching the resume information;

[0087] Based on the semantic expression results obtained above, how to calculate the matching degree of the recruitment information that matches the known resume information. With the help of the attention mechanism, the vector representation of work experience in the job requirements space and the vector representation of work experience in the job requirements are calculated. The following gives the vector calculation of work experience in the job requirements space.

[0088]

[0089] in Similarly, we can get the vector representation between work experience and job requirements , the vector representation of project experience between job requirements and job demands is as well as .

[0090] S340: Recruitment → Resume, calculate the matching degree between the recruitment information and the resume information;

[0091] According to the description in step S330, the vector representation of the job requirements in the work experience space can be obtained in the same way. , Vector representation of job requirements in project experience space , vector representation of job requirements in work experience space , Vector representation of job requirements in project experience space .

[0092] The person-job matching method based on symmetric contrast learning of the present invention gives the matching degree of the corresponding resume to the known recruitment information. Figure 3 This is the overall flow chart, and the data composition is shown in Figure 4 , the specific steps include:

[0093] S100: Classify resumes and recruitment information into regular text and irregular text.

[0094] S200: Calculate a random text semantic representation model using the selected random text.

[0095] It should be noted that the present invention also integrates the calculation strategy of person-job matching into the rule text to establish a preliminary screening, and the calculation strategy of person-job matching specifically includes:

[0096] Strategy 1: The rule text is a hard condition to filter resumes, such as gender or age requirements;

[0097] Strategy 2: Add rule text to the resume as a soft condition, for example, establish a functional relationship between age and advantages. As age increases, advantages continue to decrease, but resumes will not be directly screened out.

[0098] S300: Based on the resumes that have passed the initial screening, the semantic expression vector between the resumes and the irregular text in the recruitment information is calculated according to the irregular text semantic representation model; see the above description for details.

[0099] S400: By combining the strategy of calculating regular text and the semantic expression vector between irregular text, a two-layer neural network model can be trained with a small amount of label data to obtain a person-job matching prediction model.

[0100] The following is an explanation of the effect of the irregular text semantic representation model in the present invention:

[0101] To more intuitively and effectively demonstrate the effect of the irregular text semantic representation model in the inventor-post matching model of the present invention, the following presents the results of calculating the semantic similarity of irregular texts on a public dataset by combining the text semantic representation model with different encoding layer models and different data augmentation methods, and calculates the uniformity and alignment of the semantic expression space of the trained model on the STS-B dataset. Table 1 shows the performance of the text semantic representation model under different encoding layer models and different data augmentation methods, where each number represents the Spearman coefficient, and the larger the value, the better. #-bil represents the improved model on model #, R represents the Robert model as the encoder, and B represents the Bert model as the encoder. The results in the table show that the improved method can achieve varying degrees of improvement under different models, indicating that the obtained semantic expression space has better generalization performance. At the same time Figure 2 are the two indicators of uniformity and alignment of different models. The arrow points to the improved model. The results show that the symmetric contrastive learning loss function (Formula 2) improves the generalization performance of the entire semantic expression space by improving uniformity or alignment, which is consistent with the implementation method of this function because Formula 2 adds a formula that coexists with alignment and uniformity on the original basis.

[0102] Table 1: Comparison of Different Models under Public Datasets

[0103]

[0104] The above schematically describes the present invention and its implementation manners. This description is not restrictive and is only one of the implementation manners of the present invention. In fact, it is not limited thereto. Therefore, if those of ordinary skill in the art are inspired by it and design similar structural manners and embodiments without creative efforts without departing from the spirit of the present invention, they shall fall within the protection scope of the present invention.

Claims

1. A person-job matching method based on symmetric contrastive learning, characterized in that: It includes the following steps: S100. Classify the resumes and recruitment information into regular text and irregular text according to rules; S200. Use the selected irregular text to train an irregular text semantic representation model; S300. Calculate the semantic expression vectors of the irregular text in the resumes and recruitment information according to the irregular text semantic representation model; S400. Combine the semantic expression vectors of the regular text and the irregular text, and train a two-layer neural network model through label data to obtain a person-job matching prediction model; The calculation of the irregular text semantic representation model described in S200 includes the following steps: S210. Divide the unstructured text in the resume and recruitment information into training data sets and validation data sets by sentence and validation data sets ; S220. Use the pre-trained model as the encoder, and each input is the training data set with N different sentences ; select the position encoding of the CLS of the pre-trained model as the output of the encoding layer , which will then enter a fully connected layer FCN, and the vector obtained after passing through the activation function tanh(x) is the representation vector of the sentences in the entire batch in the text semantic space; similarly, for the augmented samples of the real samples , perform the same operation to obtain the representation vector matrix of the augmented sample text semantic space ; S230. Calculate the loss value of the unstructured text semantic representation model using the loss calculation function model , and perform backpropagation; S240. Repeat steps S220 to S230 until the uniformity metric on the validation dataset reaches the threshold ; The symmetric contrastive learning loss calculation function used is: Among them represents a sample composed of N real samples represents according to the enhanced sample, where N is the number of sentences participating in semantic model construction at one time is a balance hyperparameter used to adjust the sample randomness problem in the real scenario is a temperature hyperparameter 2. The person-job matching method based on symmetric contrastive learning according to claim 1, wherein: There is also a resume preliminary screening process between S200 and S300. Incorporate the regular text into the person-job matching calculation strategy for preliminary screening. Specifically, the person-job matching calculation strategy includes: Strategy 1. Use the regular text as a hard condition to screen the resumes; or: Strategy 2. Add the regular text as a soft condition to the resumes for screening.

3. A person-job matching method based on symmetric contrastive learning according to claim 1, characterized in that: The specific calculation method in step S300 includes: S310. Process the irregular text of the resumes; The unstructured text in the resume is divided into work experiences and project experiences. Each category is divided into several sentences. Suppose the work experience is expressed as , and the project experience is expressed as . Then, according to the semantic expression model of unstructured text, the semantic expression vector of each sentence can be calculated, that is, the semantic expression vector of work experience , and the semantic expression vector of project experience ; S320. Process the irregular text in the recruitment information; The text in the recruitment information is divided into job requirements and job requirements. Each category is divided into several sentences. Suppose the job requirements are expressed as , job requirements are expressed as , the semantic expression vector of sentences in each category is calculated based on the irregular text semantic expression model, and the semantic expression vector of job requirements is calculated. And the semantic expression vector of job requirements ; S330. Calculate the matching degree of the recruitment information that matches the resume information according to the resume information; Calculate the vector representation of work experience in the space of job requirements, the vector representation of work experience in the space of job demands, the vector representation of project experience in the space of job requirements, and the vector representation of project experience in the space of job demands according to the attention mechanism; S340. Calculate the matching degree between the recruitment information and the resume information according to the recruitment information; Calculate the vector representation of job requirements in the work experience space based on the attention mechanism , Vector representation of job requirements in project experience space , vector representation of job requirements in work experience space , Vector representation of job requirements in project experience space .

Images