Transformer-based person-job matching method, system, device and medium

Abstract

The present invention discloses a Transformer-based person-job matching method, system, device and medium, which belongs to the technical field of cross-application of natural language processing and human resource management. The technical problem to be solved by the present invention is how to reduce subjective errors in the person-job matching process, improve the accuracy, fairness and objectivity of the matching, and at the same time improve the efficiency of manual matching. The technical scheme adopted is: data collection: collecting job information data and job seeker resume data from multiple data sources, and cleaning and collating the collected data to obtain job description text and resume text; text preprocessing: performing preprocessing operations on the collected and sorted job description text and resume text to obtain preprocessed job description text and resume text; Transformer and BERT model fusion: constructing an encoder-decoder structure based on Transformer; model optimization; multi-label text classification: constructing a multi-label classification system; matching result output and evaluation.

Description

Technical Field

[0001] The present invention relates to the technical field of cross-application of natural language processing and human resource management, and in particular to a Transformer-based person-job matching method, system, device and medium. Background Art

[0002] In today's digital age, companies' human resource management faces the challenge of efficiently screening and accurately matching talent. At the same time, job seekers often face the challenge of poor job matching during their job search. With the rapid development of artificial intelligence (AI), natural language processing (NLP) has become a key technology for solving the problem of job matching. Traditional job matching processes present the following challenges:

[0003] ① Manual resume screening is too subjective: Manual resume screening is easily affected by subjective factors such as personal experience and preferences, resulting in inconsistent evaluation criteria;

[0004] ② Inefficient matching: Faced with a massive amount of resumes and complex job requirements, manual processing speed is limited;

[0005] ③Incomplete information mining: Manual review of resumes and job descriptions often fails to fully extract the key information hidden in them;

[0006] ④ Single matching dimension: Previous matching methods were usually based on only a limited number of dimensions, such as education level and years of work experience, which was not comprehensive enough and resulted in low matching accuracy;

[0007] ⑤ Waste of talent resources: Due to the limitations of traditional matching methods, many potential job seekers may be overlooked due to certain superficial factors, and companies may miss out on suitable talents due to matching errors. At the same time, many positions that are suitable for job seekers may be difficult to accurately locate matching positions due to inaccurate and rigorous recommendation methods, resulting in a double waste and mismatch of talent and job resources.

[0008] Therefore, how to reduce subjective errors in the process of person-job matching, improve the accuracy, fairness and objectivity of matching, and at the same time improve the efficiency of manual matching is a technical problem that needs to be solved urgently. Summary of the Invention

[0009] The technical task of the present invention is to provide a Transformer-based person-job matching method, system, equipment and medium to solve the problem of how to reduce subjective errors in the person-job matching process, improve the accuracy, fairness and objectivity of the matching, and at the same time improve the efficiency of manual matching.

[0010] The technical task of the present invention is achieved in the following way: a Transformer-based person-job matching method, which is specifically as follows:

[0011] Data collection: Collecting job information and applicant resume data from multiple data sources, cleaning and organizing the collected data to obtain job descriptions and resumes. Data sources include recruitment websites, internal human resources management systems, and talent databases.

[0012] Text preprocessing: preprocess the collected and sorted job description texts and resume texts to obtain the preprocessed job description texts and resume texts;

[0013] Fusion of Transformer and BERT models: Construct an encoder-decoder structure based on Transformer, where the encoder is used to extract features and perform semantic encoding on the input text; the decoder is used to generate influential outputs in specific tasks; the encoder includes multiple multi-head attention layers, each of which focuses on different feature dimensions of the input text, and enhances the model's ability to capture text semantic relationships through parallel computing and information fusion; at the same time, the pre-trained BERT model parameters are loaded into the encoder part of the built Transformer architecture, and a specific feature fusion layer is set at the encoder output layer to fuse the pre-processed job description text and resume text encoding Features: A custom attention mechanism is added to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching, and residual connections are introduced to solve the gradient problem. At the same time, the Transformer and BERT models are properly initialized and hyperparameters are set so that the models can inherit the language knowledge and semantic understanding capabilities learned by BERT on large-scale corpora. During the loading process, some parameters are fine-tuned or frozen according to the characteristics of the person-job matching task to balance the model's generalization ability and specificity. The underlying parameters closely related to general language understanding remain frozen, while some parameters close to the output layer directly related to the multi-label text classification task are fine-tuned and trained.

[0014] Model optimization: Adjust and optimize the fused Transformer and BERT models based on the needs of the job-person matching task;

[0015] Multi-label text classification: Build a multi-label classification system, using preprocessed job description and resume text as input and the corresponding multi-label classification results as output. Train the fused Transformer and BERT models, iteratively update parameters using corresponding loss functions and optimization algorithms, and evaluate and fine-tune the Transformer and BERT models using evaluation metrics. The multi-label classification system includes job skill labels (such as data analysis capabilities, programming skills, and sales techniques), educational requirements (such as associate degree, bachelor's degree, and master's degree), work experience (such as 1-2 years and 3-5 years), industry (such as internet and finance), job nature (such as full-time, part-time, and internship), and other special requirements (such as whether to recruit people with disabilities, whether business trips are required, and whether specific certificates are required).

[0016] Matching result output and evaluation: The job text and resume text to be matched are input into the trained Transformer and BERT models, the matching scores are calculated, and the job seekers are ranked and recommended to companies or a job list is recommended to the job seekers based on the scores. At the same time, feedback information is collected for further evaluation and update of the Transformer and BERT models.

[0017] As a preference, data collection is specifically as follows:

[0018] A strategy combining scheduled crawling and real-time updating is used to collect data from external data sources. Metadata information about the data source and collection time is recorded to facilitate subsequent data traceability and management. The collected data is also checked for integrity. If any missing or erroneous data is found, it is marked and attempts are made to supplement or correct the erroneous data from other channels. For external data sources such as recruitment websites, data is crawled at set intervals (such as every hour or at specific times of the day) to obtain the latest posted positions and updated resume information.

[0019] By comprehensively integrating the submission records of various platforms and the interview invitation records issued by enterprises, and conducting meticulous cleaning, classification and labeling, we can construct pre-training data that can accurately reflect the matching characteristics and relationships between people and jobs, and form training data with a one-to-one correspondence between resumes and jobs, providing a solid and reliable data foundation for accurate modeling and efficient analysis of people-job binding.

[0020] As a preference, text preprocessing is specifically as follows:

[0021] Text cleaning: Remove non-text characters, stop words, redundant information, and sensitive information from the text; among them, non-text characters include special characters and garbled codes; stop words include words that contribute less to semantic analysis such as "de", "shi", "zai"; redundant information includes repeated paragraphs and a large number of blanks; sensitive information includes personal privacy-related information such as names, contact information, and avatars in resumes that contribute less to person-job matching.

[0022] Word segmentation: Use a Chinese word segmentation tool to split the text into a sequence of words or phrases, that is, use the Jieba word segmentation tool to accurately split the text into a sequence of words or phrases to improve the accuracy of subsequent processing.

[0023] Part-of-speech tagging: Based on the word segmentation, use the HanLP part-of-speech tagging tool to tag each word or phrase; among them, the tagged part-of-speech includes nouns, verbs, adjectives, and adverbs. Part-of-speech tagging helps to further understand the grammatical structure and semantic information of the text. For example, it can distinguish key nouns (such as welfare benefits, job titles, skill names, technical terms, etc.) and verbs (such as be responsible for, execute, manage, etc.) in the job description, so as to provide a basis for the subsequent model to extract more valuable features; through part-of-speech tagging, important information in the text can be more accurately located, which has an important auxiliary role for some feature extraction and screening strategies based on part-of-speech rules.

[0024] Stem extraction and lemmatization: For English text, perform stem extraction operations to convert words into their basic stem forms; for Chinese text, perform similar lemmatization processing according to semantic and grammatical rules to reduce the impact of lexical diversity on model training.

[0025] Text vectorization: Use a pre-trained Word2Vec word vector model to map each word or phrase to a vector of a fixed dimension, thereby converting the text data into a numerical form that the model can process; during the text vectorization process, optimize the word vectors according to actual needs; among them, the optimization methods include adjusting the vector dimension and using a specific vector initialization method.

[0026] Preferably, the model optimization is as follows:

[0027] A specific feature fusion layer is added to the encoder output layer to effectively fuse the encoded features of the job text and resume text; and a fusion method based on the attention mechanism is adopted to enable the Transformer and BERT models to better evaluate the matching relationship between the two; through the attention mechanism, the Transformer and BERT models dynamically assign weights to different feature parts of the job text and resume text, focusing on information that has a significant impact on the matching; a custom attention mechanism is added to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching; the attention mechanism guides the Transformer and BERT models to pay more attention to these key information when processing the text based on the importance weights of pre-set key factors for person-job matching (such as core skills, years of work experience, skill certificates, expected salary, etc.), thereby improving the accuracy of the matching results; for example, for the "proficient in Golang programming" mentioned in the job description, the model can focus on the job seeker's experience and skill level in Golang programming when processing the resume;

[0028] Introducing Residual Connections: Residual connections are added between the layers of the Transformer and BERT models to address the vanishing and exploding gradient problems in deep neural networks, enabling more stable model training. When information is passed through a multi-layer network, residual connections allow the original input information to skip any layer and be added to the transformed information, helping to preserve the original feature information of the text and avoiding excessive loss or distortion of information during deep network training.

[0029] Model initialization and hyperparameter setting: Perform proper initialization on the fused Transformer and BERT models. Specifically, use the Xavier initialization method to initialize the Transformer and BERT model parameters to ensure a good parameter distribution at the beginning of training, which helps accelerate training convergence and improve model performance. At the same time, set the hyperparameters for model training. These hyperparameters include the learning rate, batch size, and number of training epochs. The details are as follows:

[0030] Through multiple trials and optimizations, we found a hyperparameter combination suitable for this task. The initial learning rate was set in a small range (e.g., 0.0001-0.001), and a learning rate decay strategy was adopted as the number of training rounds increased to prevent the model from failing to converge due to an excessively high learning rate in the late stages of training, or from training too slowly due to an excessively low learning rate.

[0031] The batch size is adjusted according to hardware resources and data size, and the value is between 16-256;

[0032] The number of training rounds is determined by the performance of the model on the validation set, and is between 10 and 60 rounds. When the performance of the model on the validation set no longer improves, early stopping is used to terminate the training in advance to prevent overfitting.

[0033] Preferably, the matching result output and evaluation are as follows:

[0034] Matching calculation: The job description and resume are fed into the trained Transformer and BERT models, respectively. The Transformer and BERT models output the predicted probability of each tag and a comprehensive matching score. The matching score is calculated by weighting the predicted probability and the importance of each tag (determined through expert experience or data analysis) to quantify the degree of match between the job and the applicant.

[0035] Result sorting and recommendation: Job seekers are sorted in descending order based on their match scores, and the top-ranked candidates are recommended to corporate recruiters. At the same time, a list of highly matching positions is recommended to job seekers. During the recommendation process, a detailed match information report is provided, including a match analysis of each tag and the detailed calculation process of the match score, so that recruiters and job seekers can gain a deeper understanding of the basis for the matching results.

[0036] Evaluation and feedback: Establish a matching result evaluation mechanism to collect feedback from corporate recruiters and job seekers on the recommendation results. The feedback information on the recommendation results includes whether the job was successfully hired and the job seeker's satisfaction with the recommended position. Based on the feedback data, further evaluate and analyze the Transformer and BERT models to identify possible deviations or deficiencies in the models, such as unsatisfactory matching results for certain specific industries or job types. Use the feedback information as an important basis for optimizing the Transformer and BERT models, regularly update and adjust the models, continuously improve the accuracy and practicality of the matching results, and achieve continuous optimization and improvement of the person-job matching system.

[0037] A Transformer-based person-job matching system, comprising:

[0038] The data collection module is used to collect job information data and job seeker resume data from multiple data sources, clean and organize the collected data, and obtain the collected and organized job description text and resume text. The data sources include recruitment websites, internal human resources management systems, and talent databases.

[0039] The text preprocessing module is used to perform word segmentation, part-of-speech tagging, stemming, lemmatization, and text vectorization on the collected and organized job description and resume texts. Word segmentation uses the Chinese word segmentation tool Jieba to split the text into word or phrase sequences; part-of-speech tagging uses the part-of-speech tagging tool HanLP to tag each word or phrase; stemming and lemmatization are performed on English and Chinese texts respectively to reduce the impact of lexical diversity; and text vectorization uses a pre-trained word vector model to map words or phrases into fixed-dimensional vectors.

[0040] The Transformer and BERT model fusion module is used to build a Transformer-based encoder-decoder structure. The encoder contains multiple multi-head attention layers to extract input text features and perform semantic encoding. The pre-trained BERT model parameters are loaded into the encoder part, and a specific feature fusion layer is set at the encoder output layer to fuse the encoded features of the job text and resume text. A custom attention mechanism is added to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching, and residual connections are introduced to solve the gradient problem. At the same time, the Transformer and BERT models are properly initialized and hyperparameters are set;

[0041] The multi-label text classification module is used to build a multi-label classification system. It uses the pre-processed job text and resume text as input and the corresponding multi-label classification results as output to train the fused Transformer and BERT models, adopts appropriate loss functions and optimization algorithms to iteratively update parameters, and evaluates and tunes the Transformer and BERT models through evaluation indicators. The multi-label classification system includes job skill labels, academic qualification labels, work experience labels, industry field labels, and other special requirement labels.

[0042] The matching result output and evaluation module is used to input the job text and resume text to be matched into the trained model, calculate the matching score and sort the job seekers according to the score to recommend them to the company or recommend a job list for the job seekers, and collect feedback information for further evaluation and update of the model.

[0043] Preferably, the data acquisition module collects data from external data sources using a strategy combining scheduled capture and real-time update, records metadata information about the source of the data and the time of collection, performs integrity checks on the collected data, and marks and corrects erroneous data;

[0044] The data collection module integrates submission records and interview invitation records to form training data with a one-to-one correspondence between resumes and positions.

[0045] More optimally, the feature fusion layer in the Transformer and BERT model fusion module uses concatenation, weighted summation, and an attention-based fusion method. The custom attention mechanism guides the Transformer and BERT models to focus on key information based on the pre-set importance weights of key factors in job matching. Residual connections are set between the layers of the Transformer and BERT models, allowing the original input information to skip any layer and be added to the transformed information.

[0046] The matching result output and the matching score in the evaluation module are obtained by weighted summation based on the importance weight and predicted probability of each label. The feedback information includes whether the company recruiters and job seekers are satisfied with the recommendation results, that is, whether the company can recruit talents that meet the job requirements and whether the job seekers can quickly find their ideal jobs.

[0047] An electronic device comprising: a memory and at least one processor;

[0048] wherein the memory stores computer-executable instructions;

[0049] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the Transformer-based person-job matching method as described above.

[0050] A computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the above-mentioned Transformer-based person-job matching method.

[0051] The Transformer-based person-job matching method, system, device, and medium of the present invention have the following advantages:

[0052] (1) The present invention significantly improves matching accuracy: with the help of the powerful semantic understanding ability and multi-label text classification function of the BERT model, it can comprehensively and deeply analyze the key information in the job description and the job seeker's resume, accurately capture the implicit semantics and multi-dimensional features in the text, and thus achieve more accurate person-job matching; at the same time, it uses the BERT model's ability to process objective data to analyze and match job and resume texts based on unified algorithms and model standards, reducing subjective errors and improving the accuracy, fairness and objectivity of matching; compared with traditional matching methods, it can greatly reduce mismatches caused by insufficient or one-sided understanding of information, screen out talents that are highly compatible with job requirements for enterprises, and also recommend jobs that are more in line with their own conditions and career expectations to job seekers, effectively improving recruitment success rates and job satisfaction;

[0053] (2) The present invention greatly improves the matching efficiency: The present invention is based on the efficient Transformer architecture and the advanced BERT model, which can quickly process massive amounts of job and resume data. Through automated text analysis and matching processes, the matching calculation of large-scale data is completed in a short time, which greatly shortens the cycle of person-job matching. It not only enables companies to fill job vacancies more quickly and seize market opportunities, but also allows job seekers to obtain employment opportunities faster, reduce job waiting time, and improve the operating efficiency of the entire employment market; at the same time, the efficient text processing performance of the BERT model can quickly perform multi-label classification and semantic analysis on large amounts of text data, complete the matching of positions with the information of many job seekers in a short time, and significantly improve the overall efficiency of person-job matching;

[0054] (3) The present invention reduces human resource costs: Traditional manual job matching requires a lot of manpower, material resources and time costs, including recruiters screening resumes, arranging interviews and subsequent talent assessments. However, after the present invention realizes automated matching, it can significantly reduce manual intervention, allowing human resources departments to devote more energy to more valuable work such as talent strategy planning and employee relationship management, thereby reducing the company's human resource cost expenditure in the recruitment process. At the same time, it also reduces the energy consumption (such as job screening) and economic costs (such as resume delivery fees, transportation fees, etc.) of job seekers in the job search process.

[0055] (4) The present invention promotes the optimal allocation of talent resources: Through precise matching, the present invention enables various types of talent to flow better to corresponding positions, avoids idleness and waste of talent, and realizes the rational allocation of talent resources among different industries, enterprises and positions. This helps to improve the efficiency of human resource utilization in the entire society, promotes the balanced development of various industries, stimulates the innovative potential and work enthusiasm of talents, and provides strong talent support for economic development;

[0056] (5) The present invention has good adaptability and scalability: The Transformer-based BERT model adopted by the present invention has a broad application base and good versatility in the field of natural language processing. It can adapt to the job matching needs of different industries, different types of positions, and various complex text expressions. Moreover, with the continuous increase in data volume and the continuous development of technology, the model can easily expand its functions and performance through further training and optimization to meet the ever-changing job matching challenges in the future and maintain the technological advancement and practicality.

[0057] (6) The multi-label text classification based on the BERT model of the present invention can deeply mine multiple key elements in the text, such as the potential skills and expertise of job seekers and the in-depth ability requirements of positions, ensuring the integrity and accuracy of information utilization in the matching process and solving the problem of incomplete information mining;

[0058] (7) The present invention uses the BERT model multi-label classification to comprehensively evaluate the matching degree between positions and job seekers from multiple dimensions, including professional skills, project experience, industry knowledge, and comprehensive quality, achieving more accurate and detailed person-job matching, and solving the problem of a single matching dimension;

[0059] (8) Through precise matching, the present invention enables job seekers to better find suitable positions to fully utilize their talents, and enables enterprises to accurately acquire the talents they need, thereby reducing the waste and mismatch of talent resources, promoting the optimal allocation of human resources, and solving the problem of waste of talent resources;

[0060] (IX) The Transformer architecture of the present invention provides strong support for text feature extraction and semantic understanding with its powerful parallel computing capability and effective processing capability for long sequence data; the BERT model, as a pre-trained model based on Transformer, performs excellently in natural language processing tasks and can deeply understand text semantic information; in the job matching scenario, the model is used to perform multi-label text classification processing on job description texts and job application resume texts, and can accurately extract key information from the text, such as multi-dimensional label information such as skills required for the job, skills possessed by job seekers, job functions, and work experience of job seekers; through in-depth analysis and matching of this information, the accuracy and efficiency of job matching can be greatly improved, the time and manpower costs of the human resources department in the recruitment process can be reduced, the enterprise talent selection process can be optimized, the overall human resources management efficiency can be improved, and solid technical support can be provided for the enterprise's talent strategic layout, promoting the rapid development of human resources management towards intelligence and precision; at the same time, it can help job seekers quickly screen out suitable jobs and improve job search efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The present invention will be further described below with reference to the accompanying drawings.

[0062] Attachment Figure 1 This is a structural diagram of the Transformer-based job matching system. DETAILED DESCRIPTION

[0063] The following is a detailed description of the Transformer-based person-job matching method, system, device, and medium of the present invention with reference to the accompanying drawings and specific embodiments.

[0064] Example 1:

[0065] This embodiment provides a Transformer-based person-job matching method, which is specifically as follows:

[0066] S1. Data Collection: Collect job information data and job applicant resume data from multiple data sources, clean and organize the collected data, and obtain job description text and resume text. Data sources include recruitment websites, internal human resource management systems, and talent databases.

[0067] S2. Text preprocessing: performing preprocessing operations on the collected and sorted job description texts and resume texts to obtain preprocessed job description texts and resume texts;

[0068] S3, Transformer and BERT model fusion: Construct an encoder-decoder structure based on Transformer, where the encoder is used to extract features and perform semantic encoding on the input text; the decoder is used to generate influential output in a specific task; the encoder includes multiple multi-head attention layers, each head focuses on different feature dimensions of the input text, and enhances the model's ability to capture text semantic relationships through parallel computing and information fusion; at the same time, the pre-trained BERT model parameters are loaded into the encoder part of the Transformer architecture, and a specific feature fusion layer is set at the encoder output layer to fuse the pre-processed job description text and resume text. Encoding features: Adding a custom attention mechanism to the Transformer and BERT model heads to focus on extracting and analyzing key information for person-job matching, and introducing residual connections to address gradient issues. At the same time, properly initialize the Transformer and BERT models and set hyperparameters to enable the models to inherit the language knowledge and semantic understanding capabilities learned by BERT on large-scale corpora. During the loading process, fine-tune or freeze some parameters based on the characteristics of the person-job matching task to balance the model's generalization ability and specificity. The underlying parameters closely related to general language understanding remain frozen, while fine-tuning and training are performed on some parameters near the output layer that are directly related to the multi-label text classification task.

[0069] S4. Model optimization: Adjust and optimize the fused Transformer and BERT models based on the requirements of the job-person matching task;

[0070] S5. Multi-label text classification: Construct a multi-label classification system, using pre-processed job description text and resume text as input and the corresponding multi-label classification results as output. Train the fused Transformer and BERT models, and use the corresponding loss function and optimization algorithm to iteratively update the parameters. Evaluate and tune the Transformer and BERT models using evaluation indicators. The multi-label classification system includes job skill labels (such as data analysis ability, programming skills, sales skills, etc.), educational requirement labels (such as associate degree, bachelor's degree, master's degree, etc.), work experience labels (such as 1-2 years, 3-5 years, etc.), industry field labels (such as Internet, finance, etc.), job nature labels (such as full-time, part-time, employment internship, etc.), and other special requirement labels (such as whether to recruit disabled people, whether business travel is required, whether specific certificates are required, etc.);

[0071] S6. Matching result output and evaluation: Input the job text and resume text to be matched into the trained Transformer and BERT models, calculate the matching score, and rank the job seekers based on the score to recommend them to the company or recommend a job list to the job seekers. At the same time, collect feedback information to further evaluate and update the Transformer and BERT models.

[0072] The data collection in step S1 of this embodiment is specifically as follows:

[0073] S101. Collect data from external data sources using a strategy that combines scheduled crawling with real-time updating, and record metadata about the data source and collection time to facilitate subsequent data tracing and management. At the same time, perform integrity checks on the collected data. If any data is missing or erroneous, mark it and attempt to supplement or correct the erroneous data from other channels. For external data sources such as recruitment websites, crawl data at set intervals (e.g., every hour or at a specific time each day) to obtain the latest posted positions and updated resume information.

[0074] S102. By comprehensively integrating the submission records of various platforms and the interview invitation records issued by enterprises, and conducting detailed cleaning, classification and labeling, we can construct pre-training data that can accurately reflect the matching characteristics and relationships between people and jobs, and form training data with a one-to-one correspondence between resumes and jobs, providing a solid and reliable data foundation for accurate modeling and efficient analysis of the binding between people and jobs.

[0075] The text preprocessing in step S2 of this embodiment is specifically as follows:

[0076] S201. Text cleaning: Remove non-text characters, stop words, redundant information, and sensitive information from the text. Among them, non-text characters include special characters and garbled codes; stop words include words that contribute little to semantic analysis, such as "of", "is", "in"; redundant information includes duplicate paragraphs and a large number of blanks; sensitive information includes personal privacy information such as names, contact information, and avatars in resumes that contribute little to person-job matching.

[0077] S202. Word segmentation: Use a Chinese word segmentation tool to split the text into a sequence of words or phrases, that is, use the Jieba word segmentation tool to accurately split the text into a sequence of words or phrases to improve the accuracy of subsequent processing.

[0078] S203. Part-of-speech tagging: On the basis of word segmentation, use the HanLP part-of-speech tagging tool to tag the part of speech of each word or phrase. Among them, the tagged parts of speech include nouns, verbs, adjectives, and adverbs. Part-of-speech tagging helps to further understand the grammatical structure and semantic information of the text. For example, it can distinguish key nouns (such as welfare benefits, job titles, skill names, technical terms, etc.) and verbs (such as be responsible for, execute, manage, etc.) in job descriptions, so as to provide a basis for the subsequent model to extract more valuable features. Through part-of-speech tagging, important information in the text can be more accurately located, which has an important auxiliary role for some feature extraction and screening strategies based on part-of-speech rules.

[0079] S204. Stem extraction and lemmatization: Perform stem extraction operations on English texts to convert words into their basic stem forms; for Chinese texts, perform similar lemmatization processing according to semantic and grammatical rules to reduce the impact of lexical diversity on model training.

[0080] S205. Text vectorization: Use a pre-trained Word2Vec word vector model to map each word or phrase to a vector of a fixed dimension, thereby converting the text data into a numerical form that the model can process; during the text vectorization process, optimize the word vectors according to actual needs. Among them, the optimization methods include adjusting the vector dimension and using a specific vector initialization method.

[0081] The model optimization in step S4 of this embodiment is specifically as follows:

[0082] S401. Add a specific feature fusion layer to the encoder output layer to effectively fuse the encoded features of the job text and resume text; and adopt a fusion method based on the attention mechanism to enable the Transformer and BERT models to better evaluate the matching relationship between the two; through the attention mechanism, the Transformer and BERT models dynamically assign weights to different feature parts of the job text and resume text, focusing on information that has an important impact on the matching; add a custom attention mechanism to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching; the attention mechanism guides the Transformer and BERT models to pay more attention to these key information when processing the text based on the importance weights of pre-set key factors for person-job matching (such as core skills, years of work experience, skill certificates, expected salary, etc.), thereby improving the accuracy of the matching results; for example, for the "proficient in Golang programming" mentioned in the job description, the model can focus on the job seeker's experience and skill level in Golang programming when processing the resume;

[0083] S402. Introducing Residual Connections: Residual connections are added between the layers of the Transformer and BERT models to address the vanishing and exploding gradient problems in deep neural networks, enabling more stable model training. When information is passed through a multi-layer network, residual connections allow the original input information to skip any layer and be added to the transformed information, helping to preserve the original feature information of the text and avoid excessive loss or distortion of information during deep network training.

[0084] S403, model initialization and hyperparameter setting: Perform reasonable initialization operations on the fused Transformer and BERT models. Specifically, use the Xavier initialization method to initialize the Transformer and BERT model parameters to ensure that the model has a good parameter distribution in the early stage of training, which is conducive to accelerating training convergence and improving model performance; at the same time, set the hyperparameters of model training; among them, the hyperparameters of model training include learning rate (LearningRate), batch size (BatchSize) and number of training rounds (Epochs); the details are as follows:

[0085] ① Through multiple experiments and tuning, find the hyperparameter combination suitable for this task. The initial learning rate is set in a small range (such as 0.0001-0.001), and a learning rate decay strategy is adopted as the number of training rounds increases to avoid the model failing to converge due to excessive learning rates in the late training period or training too slowly due to too low learning rates.

[0086] ② The batch size is adjusted according to hardware resources and data size, and the value is between 16-256;

[0087] ③ The number of training rounds is determined by the performance of the model on the validation set, and is between 10 and 60 rounds. When the performance of the model on the validation set no longer improves, the early stopping method is used to terminate the training early to prevent overfitting.

[0088] The matching result output and evaluation in step S6 of this embodiment are specifically as follows:

[0089] S601. Matching Degree Calculation: The job description and resume description to be matched are fed into the trained Transformer and BERT models, respectively. The Transformer and BERT models output the predicted probability of each tag and a comprehensive matching degree score. The matching degree score is obtained by weighted summing the importance weight of each tag (determined through expert experience or data analysis) and the predicted probability, thereby quantifying the degree of match between the job and the job seeker.

[0090] S602. Result Sorting and Recommendation: Sort job seekers in descending order based on their matching scores, and recommend top-ranked candidates to company recruiters. At the same time, recommend highly matching job positions to job seekers. During the recommendation process, provide a detailed matching information report, including matching analysis of each tag and detailed matching score calculation process, so that recruiters and job seekers can gain a deeper understanding of the basis for the matching results.

[0091] S603. Evaluation and Feedback: Establish a matching result evaluation mechanism to collect feedback from corporate recruiters and job seekers on the recommendation results. The feedback on the recommendation results includes whether the job was successfully hired and the job seeker's satisfaction with the recommended position. Based on the feedback data, further evaluate and analyze the Transformer and BERT models to identify possible deviations or deficiencies in the models, such as unsatisfactory matching results for certain specific industries or job types. Use the feedback information as an important basis for optimizing the Transformer and BERT models, regularly update and adjust the models, continuously improve the accuracy and practicality of the matching results, and achieve continuous optimization and improvement of the person-job matching system.

[0092] Example 2:

[0093] As attached Figure 1 As shown, this embodiment provides a Transformer-based person-job matching system, which includes:

[0094] The data collection module is used to collect job information data and job seeker resume data from multiple data sources, clean and organize the collected data, and obtain the collected and organized job description text and resume text. The data sources include recruitment websites, internal human resources management systems, and talent databases.

[0095] The text preprocessing module is used to perform word segmentation, part-of-speech tagging, stemming, lemmatization, and text vectorization on the collected and organized job description and resume texts. Word segmentation uses the Chinese word segmentation tool Jieba to split the text into word or phrase sequences; part-of-speech tagging uses the part-of-speech tagging tool HanLP to tag each word or phrase; stemming and lemmatization are performed on English and Chinese texts respectively to reduce the impact of lexical diversity; and text vectorization uses a pre-trained word vector model to map words or phrases into fixed-dimensional vectors.

[0096] The Transformer and BERT model fusion module is used to build a Transformer-based encoder-decoder structure. The encoder contains multiple multi-head attention layers to extract input text features and perform semantic encoding. The pre-trained BERT model parameters are loaded into the encoder part, and a specific feature fusion layer is set at the encoder output layer to fuse the encoded features of the job text and resume text. A custom attention mechanism is added to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching, and residual connections are introduced to solve the gradient problem. At the same time, the Transformer and BERT models are properly initialized and hyperparameters are set;

[0097] The multi-label text classification module is used to build a multi-label classification system. It uses the pre-processed job text and resume text as input and the corresponding multi-label classification results as output to train the fused Transformer and BERT models, adopts appropriate loss functions and optimization algorithms to iteratively update parameters, and evaluates and tunes the Transformer and BERT models through evaluation indicators. The multi-label classification system includes job skill labels, academic qualification labels, work experience labels, industry field labels, and other special requirement labels.

[0098] The matching result output and evaluation module is used to input the job text and resume text to be matched into the trained model, calculate the matching score and sort the job seekers according to the score to recommend them to the company or recommend a job list for the job seekers, and collect feedback information for further evaluation and update of the model.

[0099] The data acquisition module in this embodiment uses a strategy that combines scheduled capture with real-time updating to collect data from external data sources, and records metadata information about the source of the data and the time of collection, performs integrity checks on the collected data, and marks and corrects erroneous data.

[0100] The data collection module in this embodiment integrates the submission records and interview invitation records to form training data with a one-to-one correspondence between resumes and positions.

[0101] The feature fusion layer in the Transformer and BERT model fusion module in this embodiment adopts a fusion method based on splicing, weighted summation, and attention mechanism. The custom attention mechanism guides the Transformer and BERT models to focus on key information according to the pre-set importance weights of key factors in person-job matching. The residual connection is set between the layers of the Transformer and BERT models to allow the original input information to skip any layer and be added to the transformed information.

[0102] The matching result output and the matching score in the evaluation module in this embodiment are obtained by weighted summation based on the importance weight and predicted probability of each tag, and the feedback information includes whether the company recruiters and job seekers are satisfied with the recommendation results, that is, whether the company can recruit talents that meet the job requirements and whether the job seekers can quickly find their ideal jobs.

[0103] Example 3:

[0104] This embodiment also provides an electronic device, including: a memory and at least one processor;

[0105] wherein the memory stores computer-executable instructions;

[0106] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the Transformer-based person-job matching method described in any one of the present inventions.

[0107] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0108] The memory can be used to store computer programs and / or modules. The processor implements various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area. The program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created based on the use of the terminal, etc. In addition, the memory can also include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state memory devices.

[0109] Example 4:

[0110] This embodiment further provides a computer-readable storage medium storing a plurality of instructions, which are loaded by a processor to cause the processor to execute the Transformer-based person-job matching method according to any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, wherein the storage medium stores software program code that implements the functions of any of the above-described embodiments, and a computer (or CPU or MPU) of the system or device can read and execute the program code stored in the storage medium.

[0111] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute part of the present invention.

[0112] Examples of storage media for providing program code include floppy disks, hard disks, magneto-optical disks, optical disks (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), magnetic tapes, non-volatile memory cards, and ROMs. Alternatively, the program code can be downloaded from a server computer via a communication network.

[0113] In addition, it should be clear that the functions of any of the above embodiments can be achieved not only by executing the program code read by the computer, but also by enabling the operating system operating on the computer to complete part or all of the actual operations based on the instructions of the program code.

[0114] In addition, it can be understood that the program code read from the storage medium is written into the memory provided in the expansion board inserted into the computer or into the memory provided in the expansion unit connected to the computer, and then based on the instructions of the program code, the CPU installed on the expansion board or expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the above embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Transformer-based person-job matching method, characterized in that: The method is as follows: Data collection: Collecting job information and applicant resume data from multiple data sources, cleaning and organizing the collected data to obtain job descriptions and resumes. Data sources include recruitment websites, internal human resources management systems, and talent databases. Text preprocessing: preprocess the collected and sorted job description texts and resume texts to obtain the preprocessed job description texts and resume texts; Fusion of Transformer and BERT models: Constructing a Transformer-based encoder-decoder structure, where the encoder is used to extract features and perform semantic encoding on the input text; the decoder is used to generate influential outputs in specific tasks; the encoder includes multiple multi-head attention layers, each of which focuses on different feature dimensions of the input text. Through parallel computing and information fusion, the model's ability to capture text semantic relationships is enhanced; at the same time, the pre-trained BERT model parameters are loaded into the encoder part of the built Transformer architecture, and a specific feature fusion layer is set at the encoder output layer to fuse the encoded features of the pre-processed job description text and resume text. A custom attention mechanism is added to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching, and residual connections are introduced to solve the gradient problem. At the same time, the Transformer and BERT models are reasonably initialized and hyperparameters are set; Model optimization: Adjust and optimize the fused Transformer and BERT models based on the needs of the job-person matching task; Multi-label text classification: Build a multi-label classification system, using preprocessed job description and resume text as input and the corresponding multi-label classification results as output. Train the fused Transformer and BERT models, iteratively update parameters using corresponding loss functions and optimization algorithms, and evaluate and fine-tune the Transformer and BERT models using evaluation metrics. The multi-label classification system includes labels for job skills, educational requirements, work experience, industry, job nature, and other special requirements. Matching result output and evaluation: The job text and resume text to be matched are input into the trained Transformer and BERT models, the matching scores are calculated, and the job seekers are ranked and recommended to companies or a job list is recommended to the job seekers based on the scores. At the same time, feedback information is collected for further evaluation and update of the Transformer and BERT models.

2. The Transformer-based person-job matching method according to claim 1 is characterized in that: The data collection details are as follows: Adopt a strategy that combines timed crawling and real-time updating to collect data from external data sources, and record the metadata information of the data source and the collection time. At the same time, perform integrity checks on the collected data. If data is found to be missing or incorrect, mark it and attempt to supplement or correct the incorrect data; among them, for the external data source of the recruitment website, crawl data at a set time interval to obtain the latest released job positions and updated resume information; By comprehensively integrating the delivery records of each platform and the interview invitation records sent by enterprises, and performing meticulous cleaning, classification, and annotation processing, pre-training data that can accurately reflect the characteristics and relationships of person-job matching is constructed, forming training data with a one-to-one correspondence between resumes and job positions.

3. The Transformer-based person-job matching method according to claim 1, characterized in that: The text preprocessing is as follows: Text cleaning: Remove non-text characters, stop words, redundant information, and sensitive information from the text; among them, non-text characters include special characters and garbled codes; stop words include words with little contribution to semantic analysis such as "of", "is", "in"; redundant information includes duplicate paragraphs and blanks; sensitive information includes information such as names, contact information, and avatars in resumes that involve personal privacy and have little contribution to person-job matching; Word segmentation processing: Use a Chinese word segmentation tool to split the text into a sequence of words or phrases, that is, use the Jieba word segmentation tool to accurately split the text into a sequence of words or phrases; 词性标注:在分词处理的基础上,利用HanLP词性标注工具对每个单词或短语进行词性标注;其中,标注的词性包括名词、动词、形容词及副词; Part-of-speech tagging: On the basis of word segmentation processing, use the HanLP part-of-speech tagging tool to perform part-of-speech tagging on each word or phrase; among them, the part-of-speeches to be tagged include nouns, verbs, adjectives, and adverbs; Stem extraction and lemmatization: Perform stem extraction operations on English texts to convert words into their basic stem forms; for Chinese texts, perform similar lemmatization processing according to semantic and grammatical rules; 4. The Transformer-based person-job matching method according to claim 1, characterized in that: Text vectorization: Use a pre-trained Word2Vec word vector model to map each word or phrase into a vector of a fixed dimension, thereby converting text data into a numerical form that the model can process; during the text vectorization process, optimize the word vectors according to actual needs; among them, the optimization methods include adjusting the vector dimension and using a specific vector initialization method; Model optimization is as follows: Add a specific feature fusion layer to the output layer of the encoder to effectively fuse the encoded features of the job text and the resume text; and adopt a fusion method based on the attention mechanism to enable the Transformer and BERT models to better evaluate the matching relationship between the two; through the attention mechanism, the Transformer and BERT models dynamically assign weights to different feature parts of the job text and the resume text, focusing on information that has an important impact on matching; add a custom attention mechanism to the heads of the Transformer and BERT models to focus on the extraction and analysis of key information for person-job matching; the attention mechanism guides the Transformer and BERT models to pay more attention to these key information when processing text according to the importance weights of the pre-set key elements of person-job matching, thereby improving the accuracy of the matching results; Introducing residual connections: Residual connections are added between the layers of the Transformer and BERT models. When information is passed through a multi-layer network, residual connections allow the original input information to skip any layer and be added to the transformed information. This helps preserve the original feature information of the text and avoids excessive information loss or distortion during deep network training. Model initialization and hyperparameter setting: Initialize the fused Transformer and BERT models. Specifically, use the Xavier initialization method to initialize the Transformer and BERT model parameters to ensure that the model has a good parameter distribution at the beginning of training, which is conducive to accelerating training convergence and improving model performance. At the same time, set the hyperparameters of model training. The hyperparameters of model training include learning rate, batch size, and number of training rounds. The details are as follows: Through multiple trials and optimizations, we found a hyperparameter combination suitable for this task. We set the initial learning rate to a relatively small range and adopted a learning rate decay strategy as the number of training rounds increased to prevent the model from failing to converge due to excessive learning rates in the late stages of training, or from training too slowly due to excessive learning rates. The batch size is adjusted according to hardware resources and data size, and the value is between 16-256; The number of training rounds is determined based on the performance of the model on the validation set, and ranges from 10 to 60 rounds. When the performance of the model on the validation set no longer improves, the early stopping method is used to terminate the training in advance to prevent overfitting.

5. The Transformer-based person-job matching method according to any one of claims 1 to 4, characterized in that: The matching result output and evaluation are as follows: Matching calculation: The job description and resume are fed into the trained Transformer and BERT models, respectively. The Transformer and BERT models output the predicted probability of each tag and a comprehensive matching score. The matching score is calculated by weighting the importance of each tag and the predicted probability, thereby quantifying the degree of match between the job and the applicant. Result sorting and recommendation: Sort job seekers in descending order based on their matching scores, and recommend top-ranked candidates to company recruiters. Also, recommend highly matching job positions to job seekers. During the recommendation process, provide a detailed matching information report, including matching analysis of each tag and detailed matching score calculation process. Evaluation and feedback: Establish a matching result evaluation mechanism to collect feedback from corporate recruiters and job seekers on the recommendation results. The feedback information on the recommendation results includes whether the job was successfully hired and the job seeker's satisfaction with the recommended position. Based on the feedback data, further evaluate and analyze the Transformer and BERT models. Use the feedback information as an important basis for optimizing the Transformer and BERT models. Regularly update and adjust the models to continuously improve the accuracy and practicality of the matching results, and achieve continuous optimization and improvement of the person-job matching system.

6. A Transformer-based person-job matching system, characterized by: The system includes: The data collection module is used to collect job information data and job seeker resume data from multiple data sources, clean and organize the collected data, and obtain the collected and organized job description text and resume text. The data sources include recruitment websites, internal human resources management systems, and talent databases. The text preprocessing module is used to perform word segmentation, part-of-speech tagging, stemming, lemmatization, and text vectorization on the collected and organized job description and resume texts. Word segmentation uses the Chinese word segmentation tool Jieba to split the text into word or phrase sequences; part-of-speech tagging uses the part-of-speech tagging tool HanLP to tag each word or phrase; stemming and lemmatization are performed on English and Chinese texts respectively to reduce the impact of lexical diversity; and text vectorization uses a pre-trained word vector model to map words or phrases into fixed-dimensional vectors. The Transformer and BERT model fusion module is used to build a Transformer-based encoder-decoder structure. The encoder contains multiple multi-head attention layers to extract input text features and perform semantic encoding. The pre-trained BERT model parameters are loaded into the encoder part, and a specific feature fusion layer is set at the encoder output layer to fuse the encoded features of the job text and resume text. A custom attention mechanism is added to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching, and residual connections are introduced to solve the gradient problem. At the same time, the Transformer and BERT models are properly initialized and hyperparameters are set; The multi-label text classification module is used to build a multi-label classification system. It uses the pre-processed job text and resume text as input and the corresponding multi-label classification results as output to train the fused Transformer and BERT models, adopts appropriate loss functions and optimization algorithms to iteratively update parameters, and evaluates and tunes the Transformer and BERT models through evaluation indicators. The multi-label classification system includes job skill labels, academic qualification labels, work experience labels, industry field labels, and other special requirement labels. The matching result output and evaluation module is used to input the job text and resume text to be matched into the trained model, calculate the matching score and sort the job seekers according to the score to recommend them to the company or recommend a job list for the job seekers, and collect feedback information for further evaluation and update of the model.

7. The Transformer-based person-job matching system according to claim 6, characterized in that: The data acquisition module collects data from external data sources using a strategy that combines scheduled capture with real-time updating, records metadata about the source of the data and the time of collection, performs integrity checks on the collected data, and marks and corrects erroneous data; The data collection module integrates submission records and interview invitation records to form training data with a one-to-one correspondence between resumes and positions.

8. The Transformer-based person-job matching system according to claim 6 or 7, characterized in that: The feature fusion layer in the Transformer and BERT model fusion module uses concatenation, weighted summation, and an attention-based fusion method. The custom attention mechanism guides the Transformer and BERT models to focus on key information based on the pre-set importance weights of key factors in job matching. Residual connections are set between the layers of the Transformer and BERT models to allow the original input information to skip any layer and be added to the transformed information. The matching result output and the matching score in the evaluation module are obtained by weighted summation based on the importance weight and predicted probability of each label. The feedback information includes whether the company recruiters and job seekers are satisfied with the recommendation results, that is, whether the company can recruit talents that meet the job requirements and whether the job seekers can quickly find their ideal jobs.

9. An electronic device, characterized in that: include: memory and at least one processor; wherein the memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the Transformer-based person-job matching method according to any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the Transformer-based person-job matching method as described in any one of claims 1 to 5 is implemented.

Images