A method and apparatus for improving the quality of job class prediction samples

Abstract

The application discloses a method and device for improving the quality of job category prediction samples, comprising the following steps: S1, improving the data quality by using a double-linked list, and cleaning the data by using a clustering method; S2, sequentially using a pre-training labeling model, a vector model and a classification model to label, vector and classify the input processing, and integrating a job category tree knowledge graph into the model in advance; and S3, using a multi-task training method to reduce the model perplexity. In the method for improving the quality of job category prediction samples, the data quality is improved by using a double-linked list, the data is cleaned by using a clustering method, the data accuracy is enhanced, a large pre-training model is integrated with hierarchical features, the iteration efficiency is improved, a multi-task joiner training method is adopted to reduce the perplexity, a business database is used to train a vector model, a section vector model and a job category prediction model, the representation of the vector is enriched, and the quality of the job category prediction samples is improved.

Description

Technical Field

[0001] This invention relates to the field of improving the quality of job category prediction samples, and more specifically, to a method and apparatus for improving the quality of job category prediction samples. Background Technology

[0002] A job category refers to a collection of similar positions that share similarities in terms of job nature, requirements, and responsibilities. When managing positions and employees, companies typically group similar positions into the same job category for easier management and evaluation. Job categories can be adjusted and updated as needed to adapt to changes within and outside the company. As a fundamental concept in human resource management, job categories primarily serve to provide companies with a way to classify and categorize positions, facilitating the management, evaluation, and motivation of each position.

[0003] Job categories are among the most important pieces of information in the recruitment industry. On the platform, job category information appears in user workflows and is also a crucial ranking or recall strategy. However, different recruitment platforms have vast and varied job category trees, making accurate understanding and memorization extremely costly for users. Statistics show that only about 80%+ of users can understand and memorize the target job category for a job posting and correctly select from the massive tree. Effective job category classification is crucial for improving user efficiency, enhancing basic data quality, and contributing significantly to business features. Current on-platform job category information suffers from the following problems: low data quality and skewed data distribution; the number of job categories (1385) slows down model iteration due to the increased number of targets; from a training target perspective, some job categories are similar, making accurate model fitting difficult; and from a training data perspective, most positions can be divided into multiple categories, exhibiting multi-label characteristics.

[0004] Therefore, we propose a method to improve the quality of job category prediction samples to address the above problems. Summary of the Invention

[0005] In order to overcome the above-mentioned defects of the prior art, embodiments of the present invention provide a method for improving the quality of job category prediction samples to solve the problems mentioned in the background art.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a method for improving the quality of job category prediction samples, comprising the following steps: S1: using a doubly linked list to improve data quality and using clustering methods to clean the data; S2: sequentially using a pre-trained labeling model, a vector model, and a classification model to label, vectorize, and classify the input processing, and pre-integrating the job category tree knowledge graph into the model; S3: using a multi-task training method to reduce model perplexity.

[0007] In a preferred embodiment, step S2, utilizing the pre-trained vector model, includes: S21: collecting the title and description information of job positions or work experience, and feeding them into the model; S22: encoding the input using the pre-trained vector model BERT to obtain document vectors; S23: performing softmax multi-class classification on the document vectors and calculating the loss; S24: outputting the probability distribution of the job category corresponding to the current input, selecting the job category with the highest probability to enter the post-processing process, and using the key job categories as the final output after post-processing.

[0008] In a preferred embodiment, in step S2, the model uses job category tree features to first fit the model on the first-level job category and the second-level job category, and then train it on the second-level job category task, so that when the model predicts the third-level job category, it already contains the knowledge of the first and second-level job categories.

[0009] In a preferred embodiment, step S2, which involves integrating the job category tree knowledge graph into the model in advance, includes: S25: extracting job category tags based on the new three-level job category information; S26: vectorizing and reducing the dimensionality of the BERT vector.

[0010] In a preferred embodiment, in step S25, during the process of extracting job category tags based on the new three-level job category information, the job category tags are extracted based on the adjusted job category information and the division of multiple parent job categories, and the job category tags are extracted based on industry characteristics. Then, the analysis results of job descriptions and resumes are associated with different job categories to provide a data foundation for subsequent search result filtering and weight calculation.

[0011] In a preferred embodiment, step S26, Bert vectorization and dimensionality reduction includes the following: S261: Vectorizing the word chains in the job description and resume using NLP technology; S262: Considering industry characteristics, vectorizing is performed from different dimensions, including three levels: industry context, chapter, and bureau. Then, based on historical data samples, the vectorization results of different dimensions are evaluated and labeled with different weights.

[0012] In a preferred embodiment, due to the differences in specialized terms and semantics across different industries, dimensionality reduction is performed for different industries, and a unified dimensionality model is no longer used. The number of dimensions is reduced from the original 768 to 64, which improves computational efficiency and reduces data access time.

[0013] In a preferred embodiment, the training method in S3 includes: S31: loss1 is the first-level job category prediction loss, lossall = loss1; S32: loss2 is the second-level job category prediction loss, lossall = loss1 + loss2; S33: loss3 is the third-level job category prediction loss, lossall = loss1 + loss2 + loss3.

[0014] The present invention also provides the following technical solution: an apparatus for improving the quality of job category prediction samples, characterized in that it includes: a data processing module, which improves data quality using a doubly linked list and cleans the data using a clustering method; a model processing module, which sequentially uses a pre-trained labeling model, a vector model, and a classification model to label the input, perform vector and classification processing, and integrate the job category tree knowledge graph into the model in advance; and a data training module, which uses a multi-task training method to reduce model perplexity.

[0015] The technical effects and advantages of this invention are as follows:

[0016] 1. Use doubly linked lists to improve data quality, use clustering methods to clean the data and enhance data accuracy, incorporate hierarchical features into the pre-trained large model to improve iteration efficiency, and adopt a multi-task joiner training method to reduce perplexity.

[0017] 2. The current performance of the resume form shows that the new model has improved accuracy by 11.9% compared to the online model and by 4% compared to user selection. It can improve the efficiency of users filling in information, improve the quality of basic data, and can also be used for recommendation search strategies. From the inconsistency rate perspective, it can be seen that the job categories provided by the model can be regarded as similar job categories and used as new features for recall strategies.

[0018] 3. Current results show that the new model improves accuracy by 6.8% compared to the online model and by 3% compared to user selection. It can improve the efficiency of user information entry, improve the quality of basic data, and can also be used for recommendation search strategies. From the inconsistency rate, it can be seen that the job categories provided by the model can be regarded as similar job categories as new features for recall strategies.

[0019] 4. Using business data to train vector models, document vector models, and job category prediction models enriches the representation of vectors, improves the quality of job category prediction samples, and saves a lot of human resources. Attached Figure Description

[0020] Figure 1 This is a flowchart of the method for improving the quality of job category prediction samples according to the present invention;

[0021] Figure 2(A) is a schematic diagram of the labeling model structure constructed in this invention; Figure 2(B) is a schematic diagram of the vector model structure constructed in this invention; Figure 2(C) is a schematic diagram of the classification model structure constructed in this invention.

[0022] Figure 3 In the method for improving the quality of job category prediction samples in this invention, from the perspective of resumes, the current effect of the resume is shown in the comparison chart of the accuracy of the new model compared to the online model and the accuracy compared to user selection.

[0023] Figure 4 In the method for improving the quality of job category prediction samples in this invention, from the perspective of resumes, the figure shown is a comparison of the inconsistency rate between user selections and the new model predictions.

[0024] Figure 5 In the method for improving the quality of job category prediction samples in this invention, from the perspective of job description, the comparison diagram of the accuracy of the new model compared with the online model and the accuracy of user selection is shown.

[0025] Figure 6 In the method for improving the quality of job category prediction samples in this invention, a comparison chart of inconsistency rates is shown from the perspective of job descriptions;

[0026] Figure 7 This is a schematic diagram of a knowledge graph used for job classification in the existing technology;

[0027] Figure 8 This is a schematic diagram of the knowledge graph for job classification constructed in this invention, based on adjusted job category information and multiple parent job category divisions;

[0028] Figure 9 The diagram illustrates the process of extracting tags based on the adjusted new three-level job category information and multiple parent job category divisions, extracting tags based on industry characteristics, and then associating the JDCV analysis results with different job categories.

[0029] Figure 10 A schematic diagram is shown illustrating the vectorization of word chains in JD and CV using the NLP technology of this invention. Detailed Implementation

[0030] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0031] Reference Figure 1A method to improve the quality of job category prediction samples includes the following methods:

[0032] S1: Use doubly linked lists to improve data quality and use clustering methods to clean the data;

[0033] A singly linked list can be traversed in one step, but during development, it is often necessary to backtrack to the previous node. A doubly linked list can be traversed from head to tail or from tail to head. The process of linking the lists is bidirectional. A node has both forward and backward references, which greatly facilitates reverse output and provides strong data support for the subsequent vector model of this application.

[0034] Clustering methods are used to clean the data to improve its accuracy.

[0035] S2: The input is labeled, vectorized, and classified sequentially using a pre-trained labeling model, a vector model, and a classification model, and the job category tree knowledge graph is pre-integrated into the model;

[0036] Figure 2(A) is a schematic diagram of the labeling model structure constructed in this invention; Figure 2(B) is a schematic diagram of the vector model structure constructed in this invention; Figure 2(C) is a schematic diagram of the classification model structure constructed in this invention.

[0037] Referring to Figure 2, step S2, utilizing the pre-trained labeling model, includes:

[0038] S21: Extract full-text features of long texts using transformer;

[0039] S22: Use a batch layer to convert long texts into short texts and perform parallel computation to extract sequence features;

[0040] Through the above steps, the innovative new marking model of this invention improves the sample accuracy from 0.37 to 0.72.

[0041] Furthermore, in step S2 of this application, the use of a pre-trained vector model includes:

[0042] S23: Train a dual-tower structure using query-title data, and use the title tower to calculate and provide vectors;

[0043] Through the above steps, the accuracy of the innovative vector model of this invention is improved to 87.6%.

[0044] Furthermore, in step S2 of this application, the use of a pre-trained classification model includes:

[0045] S24: Collect the title and description information of the job or work experience, and feed it into the model in a flat manner;

[0046] like Figure 3 As shown, this invention provides an interface for posting job openings; it allows users to set job categories, industry requirements, and educational and experience levels.

[0047] S25: Encode the input using the pre-trained vector model BERT to obtain the document vector;

[0048] S26: Perform softmax multi-class classification on the document vectors and calculate the loss J(θ);

[0049]

[0050] Where i represents the i-th document vector sample, j represents the j-th category of the i-th document vector sample, T represents the number of attributes; m represents the number of document vector samples, k represents the number of categories; the probability predicted for each sample is P, the length of the vector is k; and y represents the true category of the document vector.

[0051] Suppose there are m sample text vectors, i.e. [x1, x2, ... xm], each sample has n attributes, i.e. xi = [a1, a2, ... an], divided into k classes, yi = [0, 0, ... 1 ... 0], where the i-th position is 1 and the rest are 0.

[0052] The probability of each class is Pk = P(y(k) = 1), and ΣP = 1.

[0053]

[0054] Represent Pk = 1{y = k}Pk in another way;

[0055] Pj represents the probability that a document vector sample x is classified into the j-th class. Here, the subscript j in the numerator θ indicates the class of the document vector, and θ is a matrix.

[0056]

[0057] Logistic regression is a vector because multi-class classification is equivalent to multiple classifiers, each with its own template. Therefore, the size of this matrix is ​​the number of categories X the number of attributes; T represents the number of attributes, k represents the number of categories, and each row represents the template of the category of a document vector, which is θj in the formula above.

[0058] S27: The output result is the probability distribution of the job category corresponding to the current input. The job category with the highest probability is selected from the distribution and enters the post-processing process. The key job categories are post-processed and used as the final output.

[0059] Furthermore, the model uses job category tree features, allowing it to first fit the first and second job categories, and then train it on the second job category tasks, so that when the model predicts the third job category, it already includes the knowledge of the first and second job categories.

[0060] From a resume perspective, through the above steps, as follows: Figure 3 As shown, the current effect of the resume form is that the new model improves the accuracy by 11.9% compared to the online model and by 4% compared to user selection. This can improve the efficiency of user information entry, enhance the quality of basic data, and can also be used for recommendation search strategies. The inconsistency rate between user selection and the new model's predictions indicates that... Figure 4 As shown, the job categories provided by the model can be regarded as similar job categories and used as new features for the recall strategy.

[0061] Through the above steps, from the job description, as follows: Figure 5 As shown, the new model achieves a 6.8% improvement in accuracy compared to the online model and a 3% improvement compared to user selection. It can improve the efficiency of user information entry, enhance the quality of basic data, and can also be used in recommendation search strategies. The inconsistency rate indicates that... Figure 6 As shown, the job categories provided by the model can be regarded as similar job categories and used as new features for the recall strategy.

[0062] The current results show that the new model improves accuracy by 6.8% compared to the online model and by 3% compared to user selection. It can improve the efficiency of user information entry, improve the quality of basic data, and can also be used for recommendation search strategies. From the inconsistency rate, it can be seen that the job categories provided by the model can be regarded as similar job categories as new features for recall strategies.

[0063] Furthermore, in step S2, the job category tree knowledge graph is pre-integrated into the model, including:

[0064] S28: Extract job category tags based on the new three-level job category information;

[0065] A knowledge graph, also known as a knowledge domain visualization or knowledge domain mapping map, is a series of different graphics that display the development process and structural relationships of knowledge. It uses visualization techniques to describe knowledge resources and their carriers, and to mine, analyze, construct, draw and display knowledge and the interrelationships between them.

[0066] In the recruitment and job-seeking industry, the most critical knowledge graph elements are the job description (JD) and the CV (CV), which mostly describe their attributes in text form and possess clear industry characteristics and user attributes. For example... Figure 7 As shown, in order to provide more accurate and faster retrieval and recommendation of JDV, it is crucial to extract keywords and mine data based on existing data to form a complete and accurate knowledge graph.

[0067] In step S28, during the process of extracting job category tags based on the new three-level job category information, the job category tags are extracted based on the adjusted job category information and the division of multiple parent job categories, and the analysis results of job descriptions and resumes are then associated with different job categories.

[0068] This provides a data foundation for subsequent search result filtering and weight calculation.

[0069] like Figure 8 As shown, based on the adjusted job category information and the division of multiple parent job categories, job category tags are extracted according to industry characteristics, and then the analysis results of JDV are associated with different job categories to provide a data foundation for subsequent search result filtering and weight calculation.

[0070] S29: Bert vectorization and dimensionality reduction.

[0071] In step S29, Bert vectorization and dimensionality reduction includes the following:

[0072] S291: Vectorize word chains in job descriptions and resumes using NLP technology;

[0073] S292: Considering the characteristics of the industry, vectorization is performed from different dimensions, including three levels: industry context, chapter, and section. Then, based on historical data samples, the vectorization results of different dimensions are evaluated and labeled with different weights.

[0074] Thus, due to the differences in specialized terms and semantics across different industries, dimensionality reduction is performed for different industries, and a unified dimensional model is no longer used. The number of dimensions is reduced from the original 768 to 64, which improves computational efficiency and reduces data access time.

[0075] like Figure 9 As shown, word chains in JD and CV are vectorized using NLP technology. Considering industry characteristics, vectorization is performed from different dimensions, including industry context, chapter, and section levels. Then, based on historical data samples, the vectorization results of different dimensions are evaluated and labeled with different weights. Due to the differences in specialized terms and semantics across different industries, dimensionality reduction is performed for different industries, abandoning the uniform dimensionality model and reducing it from the original 768 dimensions to 64 dimensions. This significantly improves computational efficiency and reduces data access time.

[0076] The method for improving the quality of job category prediction samples according to the present invention further includes the following steps:

[0077] S3: Use multi-task training to reduce model perplexity.

[0078] The training method in step S3 includes:

[0079] Step 1: loss1 is the prediction loss for the first-level job category, lossall = loss1;

[0080] Step 2: loss2 is the predicted loss for the second-level job category, lossall = loss1 + loss2;

[0081] Step 3: loss3 is the predicted loss for the third job category, lossall = loss1 + loss2 + loss3.

[0082] It should be noted that the model of this invention uses job category tree features, so that the model is first fitted on the first-level job category and the second-level job category, and then trained on the second-level job category task, so that when the model predicts the third-level job category, it already contains the knowledge of the first- and second-level job categories.

[0083] The technical effects and advantages of this invention are as follows:

[0084] 1. Use doubly linked lists to improve data quality, use clustering methods to clean the data and enhance data accuracy, incorporate hierarchical features into the pre-trained large model to improve iteration efficiency, and adopt a multi-task joiner training method to reduce perplexity.

[0085] 2. The current performance of the resume form shows that the new model has improved accuracy by 11.9% compared to the online model and by 4% compared to user selection. It can improve the efficiency of users filling in information, improve the quality of basic data, and can also be used for recommendation search strategies. From the inconsistency rate perspective, it can be seen that the job categories provided by the model can be regarded as similar job categories and used as new features for recall strategies.

[0086] 3. Work Description: The current results show that the new model improves accuracy by 6.8% compared to the online model and by 3% compared to user selection. It can improve the efficiency of user information entry, improve the quality of basic data, and can also be used for recommendation search strategies. From the inconsistency rate, it can be seen that the job categories provided by the model can be regarded as similar job categories and used as new features for recall strategies.

[0087] 4. Using business data to train vector models, document vector models, and job category prediction models enriches the representation of vectors, improves the quality of job category prediction samples, and saves a lot of human resources.

[0088] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the invention. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, it is intended that all variations falling within the meaning and scope of equivalents of the claims be included within the present invention.

[0089] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for improving the quality of job category prediction samples, characterized in that, Comprise: S1: use a double-linked list to improve data quality, and use a clustering method to clean the data; S2: sequentially use a pre-trained tagging model, a vector model, and a classification model to tag, vector, and classify the input, and integrate the job category tree knowledge graph into the model in advance; specifically comprising: S21: use a transformer to extract long text features; S22: use a batch processing layer to convert long text into short text and extract sequence features in parallel; S23: use query-title data to train a double-tower structure, and use the title tower to calculate and provide vectors; S24: collect the title and description information of the position or work experience, and send it to the model in a flat manner; S25: use a pre-trained vector model BERT to encode the input and obtain a chapter vector; S26: perform softmax multi-classification on the chapter vector and calculate the loss J(θ); ； Where i represents the i-th chapter vector sample, j represents the j-th category of the i-th chapter vector sample, T represents the number of attributes; m represents the number of chapter vector samples, k represents the number of categories; the probability predicted by each sample is P, and the length of the vector is k; y is the true category of the chapter vector; Assume that there are m chapter vector samples, i.e. [x1, x2,..., xm], each sample has n attributes, i.e. xi = [a1, a2,..., an], a total of k categories, yi = [0, 0,..., 1,..., 0], the i-th position is 1, and the rest are 0; The probability of each category is Pk = P(y(k) = 1), and ΣP = 1; Pk = 1{y = k}Pk in another way; Pj represents the probability that the chapter vector sample x is classified into the j-th category; where the subscript of θ on the numerator is j, representing the category of the chapter vector, and θ is a matrix, ； And the logistic regression is a vector, because multiple classifiers each have their own templates, so the size of the matrix is the number of categories X the number of attributes; T represents the number of attributes, k represents the number of categories, and each row represents a template for the category of a chapter vector, which is θj in the above formula; S27: output the probability distribution of the current input corresponding to the job category, select the job category with the maximum probability from it, and enter the post-processing process, and the key job category after post-processing is used as the final output; S28: extract job category labels based on new three-level job category information, and associate the analysis results of the job description and resume table with different job categories based on the adjusted job category information and multi-parent job category division, and industry characteristics; S29: Bert vectorization and dimension reduction; specifically comprising: S291: vectorize the word chains in the job description and resume table through NLP technology; S292: considering the industry characteristics, vectorize from different dimensions respectively, including industry context, chapters, and sub-chapters, then evaluate the vectorization results of different dimensions based on historical data samples, and mark different weights; S3: use a multi-task training method to reduce model confusion.

2. The method for improving the quality of promotion class prediction samples according to claim 1, characterized in that: In the step S2, the model uses the job category tree features, so that the model is first fitted on the first-level job category and the second-level job category, and then trained on the tasks of the second-level job category, so that when the model predicts the third-level job category, the knowledge of the first-level and second-level job categories is already included.

3. The method for improving the quality of promotion class prediction samples according to claim 2, characterized in that: In the step S25, in the process of extracting the job category label according to the new third-level job category information, the job category label is extracted according to the adjusted job category information and the multi-parent job category division, and the analysis results of the job description and the resume table are associated with different job categories.

4. The method of claim 1, wherein: The training method in the step S3 includes: S31: loss1 is a first-level job category prediction loss, and lossall=loss1; S32: loss2 is a second-level job category prediction loss, and lossall=loss1+loss2; S33: loss3 is a third-level job category prediction loss, and lossall=loss1+loss2+loss3.

Images