Transform-based man-post matching method, system and device, and medium

Abstract

The invention discloses a person and post matching method, system and equipment based on Transform and a medium, belongs to the technical field of cross application of natural language processing and human resource management, and aims to solve the technical problem of how to reduce subjective errors in a person and post matching process, improve matching accuracy, fairness and objectivity and improve manual matching efficiency at the same time. According to the technical scheme, the method comprises the following steps: data collection: collecting post information data and job seeker resume data from a plurality of data sources, and cleaning and sorting the collected data to obtain a post description text and a resume text; text preprocessing: preprocessing the acquired and sorted post description text and resume text to obtain the preprocessed post description text and resume text; the method comprises the following steps of: fusing a Transform model and a BERT model; constructing an encoder-decoder structure based on the Transform model; optimizing the model; multi-label text classification: constructing a multi-label classification system; and outputting and evaluating a matching result.

Description

Technical Field

[0001] The present invention relates to the technical field of cross-application of natural language processing and human resource management, and in particular to a Transformer-based person-job matching method, system, device and medium. Background Art

[0002] In today's digital age, the human resource management of enterprises faces the challenge of efficient screening and accurate matching of talents. At the same time, job seekers often face the challenge of poor job matching in the process of finding a job. With the rapid development of artificial intelligence technology, natural language processing technology has become one of the key technical means to solve the problem of matching people with jobs. In the traditional matching process, there are the following problems:

[0003] ① Manual resume screening is too subjective: Manual resume screening is easily interfered by subjective factors such as personal experience and preference, resulting in inconsistent evaluation criteria;

[0004] ② Low matching efficiency: Faced with a large number of resumes and complex job requirements, manual processing speed is limited;

[0005] ③Incomplete information mining: Manual review of resumes and job descriptions often makes it difficult to fully extract the key information hidden in them;

[0006] ④ Single matching dimension: Previous matching methods usually only consider a few limited dimensions, such as education level, years of work experience, etc., which is not comprehensive enough, resulting in low matching accuracy;

[0007] ⑤ Waste of talent resources: Due to the limitations of traditional matching methods, many potential job seekers may be overlooked due to some superficial factors, and companies may miss out on suitable talents due to matching errors. At the same time, many positions that are suitable for job seekers may be difficult for job seekers to accurately locate matching positions due to inaccurate and rigorous recommendation methods, resulting in a double waste and mismatch of talent and job resources.

[0008] Therefore, how to reduce subjective errors in the process of person-job matching, improve the accuracy, fairness and objectivity of matching, and at the same time improve the efficiency of manual matching is a technical problem that needs to be solved urgently. Summary of the invention

[0009] The technical task of the present invention is to provide a Transformer-based person-job matching method, system, device and medium to solve the problem of how to reduce subjective errors in the person-job matching process, improve the accuracy, fairness and objectivity of the matching, and at the same time improve the efficiency of manual matching.

[0010] The technical task of the present invention is achieved in the following way: a person-job matching method based on Transformer, the method is as follows:

[0011] Data collection: Collect job information data and job seekers' resume data from multiple data sources, clean and organize the collected data, and obtain job description texts and resume texts; data sources include recruitment websites, internal human resource management systems and talent databases;

[0012] Text preprocessing: preprocess the collected and sorted job description texts and resume texts to obtain the preprocessed job description texts and resume texts;

[0013] Fusion of Transformer and BERT models: Construct an encoder-decoder structure based on Transformer, where the encoder is used to extract features and encode semantics of the input text; the decoder is used to generate influential outputs in specific tasks; the encoder includes multiple multi-head attention layers, each of which focuses on different feature dimensions of the input text, and enhances the model's ability to capture text semantic relationships through parallel computing and information fusion; at the same time, the pre-trained BERT model parameters are loaded into the encoder part of the Transformer architecture, and a specific feature fusion layer is set at the encoder output layer to fuse the pre-processed job description text and resume text encoding Features: add a custom attention mechanism to the head of the Transformer and BERT models to focus on the extraction and analysis of key information for person-job matching, and introduce residual connections to solve the gradient problem. At the same time, the Transformer and BERT models are initialized reasonably and hyperparameters are set so that the models can inherit the language knowledge and semantic understanding capabilities learned by BERT on large-scale corpora. During the loading process, some parameters are fine-tuned or frozen according to the characteristics of the person-job matching task to balance the generalization ability and pertinence of the model. The underlying parameters closely related to general language understanding are kept frozen, while some parameters close to the output layer that are directly related to the multi-label text classification task are fine-tuned and trained.

[0014] Model optimization: According to the requirements of the person-job matching task, the fused Transformer and BERT models are adjusted and optimized;

[0015] Multi-label text classification: Construct a multi-label classification system, use the pre-processed job description text and resume text as input, and the corresponding multi-label classification results as output, train the fused Transformer and BERT models, and use the corresponding loss function and optimization algorithm to iteratively update the parameters, and evaluate and tune the Transformer and BERT models through evaluation indicators; Among them, the multi-label classification system includes job skill labels (such as data analysis ability, programming skills, sales skills, etc.), education requirement labels (such as junior college, undergraduate, master's degree, etc.), work experience labels (such as 1-2 years, 3-5 years, etc.), industry field labels (such as Internet, finance, etc.), job nature labels (such as full-time, part-time, employment internship, etc.) and other special requirements labels (such as whether to recruit disabled people, whether business trips are required, whether specific certificates are required, etc.);

[0016] Matching result output and evaluation: Input the job text and resume text to be matched into the trained Transformer and BERT models, calculate the matching score, and sort the job seekers according to the score and recommend them to the company or recommend a job list to the job seekers. At the same time, collect feedback information to further evaluate and update the Transformer and BERT models.

[0017] As a preference, data collection is specifically as follows:

[0018] A strategy combining scheduled crawling and real-time updating is used to collect data from external data sources, and metadata information about the source of the data and the time of collection is recorded for subsequent data tracing and management. At the same time, integrity checks are performed on the collected data. If data is missing or wrong, it is marked and attempts are made to supplement or correct the wrong data from other channels. For external data sources of recruitment websites, data is crawled at set time intervals (such as every hour or at a specific time every day) to obtain the latest job posts and updated resume information.

[0019] By comprehensively integrating the submission records of various platforms and the interview invitation records issued by enterprises, and conducting detailed cleaning, classification and labeling, we can construct pre-training data that can accurately reflect the matching characteristics and relationships between people and positions, and form training data with a one-to-one correspondence between resumes and positions, providing a solid and reliable data foundation for accurate modeling and efficient analysis of person-position binding.

[0020] As a preference, the text preprocessing is specifically as follows:

[0021] Text cleaning: Remove non-text characters, stop words, redundant information, and sensitive information from the text; among them, non-text characters include special characters and garbled codes; stop words include words with little contribution to semantic analysis such as "of", "is", "in"; redundant information includes repeated paragraphs and a large number of blanks; sensitive information includes personal privacy-related information such as names, contact information, and avatars in resumes that contribute little to person-job matching.

[0022] Word segmentation: Use a Chinese word segmentation tool to split the text into a sequence of words or phrases, that is, use the Jieba word segmentation tool to accurately split the text into a sequence of words or phrases to improve the accuracy of subsequent processing.

[0023] Part-of-speech tagging: Based on the word segmentation, use the HanLP part-of-speech tagging tool to tag the part of speech of each word or phrase; among them, the tagged parts of speech include nouns, verbs, adjectives, and adverbs. Part-of-speech tagging helps to further understand the grammatical structure and semantic information of the text. For example, it can distinguish key nouns (such as welfare benefits, job titles, skill names, technical terms, etc.) and verbs (such as be responsible for, execute, manage, etc.) in job descriptions, thus providing a basis for the subsequent model to extract more valuable features; through part-of-speech tagging, important information in the text can be more accurately located, which has an important auxiliary role for some feature extraction and screening strategies based on part-of-speech rules.

[0024] Stemming and lemmatization: Perform stemming operations on English texts to convert words into their basic stem forms; for Chinese texts, perform similar lemmatization processing according to semantic and grammatical rules to reduce the impact of lexical diversity on model training.

[0025] Text vectorization: Use a pre-trained Word2Vec word vector model to map each word or phrase to a vector of a fixed dimension, thereby converting the text data into a numerical form that the model can process; during the text vectorization process, optimize the word vectors according to actual needs; among them, the optimization methods include adjusting the vector dimension and using a specific vector initialization method.

[0026] Preferably, the model optimization is as follows:

[0027] A specific feature fusion layer is added to the encoder output layer to effectively fuse the encoding features of the job text and resume text; and a fusion method based on the attention mechanism is adopted to enable the Transformer and BERT models to better evaluate the matching relationship between the two; through the attention mechanism, the Transformer and BERT models dynamically assign weights to different feature parts of the job text and resume text, focusing on information that has an important impact on the matching; a custom attention mechanism is added to the head of the Transformer and BERT models to focus on the extraction and analysis of key information for person-job matching; the attention mechanism guides the Transformer and BERT models to pay more attention to these key information when processing the text based on the importance weights of the pre-set key elements of person-job matching (such as core skills, years of work experience, skill certificates, expected salary, etc.), thereby improving the accuracy of the matching results; for example, for the "proficient in Golang programming" mentioned in the job description, the model can focus on the job seeker's experience and skill level in Golang programming when processing the resume;

[0028] Introducing Residual Connections: Adding residual connections between the layers of the Transformer and BERT models to solve the gradient vanishing and gradient exploding problems in deep neural networks, allowing the model to be trained more stably; when information is passed through a multi-layer network, residual connections allow the original input information to directly skip any layer and be added to the transformed information, which helps to preserve the original feature information of the text and avoid excessive loss or distortion of information during deep network training;

[0029] Model initialization and hyperparameter setting: Perform reasonable initialization operations on the fused Transformer and BERT models. Specifically, use the Xavier initialization method to initialize the Transformer and BERT model parameters to ensure that the model has a good parameter distribution at the beginning of training, which is conducive to accelerating the training convergence speed and improving the model performance; at the same time, set the hyperparameters of model training; among them, the hyperparameters of model training include learning rate (LearningRate), batch size (BatchSize) and number of training rounds (Epochs); the details are as follows:

[0030] Through multiple experiments and tuning, we found a hyperparameter combination suitable for this task. The initial learning rate was set in a small range (such as 0.0001-0.001), and the learning rate decay strategy was adopted as the number of training rounds increased to avoid the model failing to converge due to too large a learning rate in the later stage of training or too slow training due to too small a learning rate.

[0031] The batch size is adjusted according to the hardware resources and data size, and the value is between 16-256;

[0032] The number of training rounds is determined based on the performance of the model on the validation set, and is between 10 and 60 rounds. When the performance of the model on the validation set no longer improves, the early stopping method is used to terminate the training in advance to prevent overfitting.

[0033] Preferably, the matching result output and evaluation are as follows:

[0034] Matching degree calculation: The job text and resume text to be matched are input into the trained Transformer and BERT models respectively. The Transformer and BERT models output the predicted probability of each label and the comprehensive matching degree score. The matching degree score is obtained by weighted summation of the importance weight of each label (determined by expert experience or data analysis) and the predicted probability, thereby quantifying the matching degree between the job and the job seeker.

[0035] Result sorting and recommendation: Sort job seekers in descending order according to their matching scores, and recommend top-ranked job seekers to corporate recruiters; at the same time, recommend job seekers a list of highly matching positions; during the recommendation process, provide a detailed matching information report, including matching analysis of each tag and detailed calculation process of matching scores, so that recruiters and job seekers can have a deep understanding of the basis of matching results;

[0036] Evaluation and feedback: Establish a matching result evaluation mechanism to collect feedback from corporate recruiters and job seekers on the recommendation results. The feedback on the recommendation results includes whether the job is successfully hired and the job seeker’s satisfaction with the recommended position. Based on the feedback data, further evaluate and analyze the Transformer and BERT models to identify possible deviations or deficiencies in the models, such as unsatisfactory matching results for certain specific industries or job types. Use the feedback information as an important basis for optimizing the Transformer and BERT models, update and adjust the models regularly, continuously improve the accuracy and practicality of the matching results, and achieve continuous optimization and improvement of the job matching system.

[0037] A Transformer-based person-job matching system, the system comprising:

[0038] The data collection module is used to collect job information data and job seeker resume data from multiple data sources, and clean and organize the collected data to obtain the collected and organized job description text and resume text; the data sources include recruitment websites, internal human resource management systems of enterprises, and talent databases;

[0039] The text preprocessing module is used to perform word segmentation, part-of-speech tagging, stem extraction, lemma restoration, and text vectorization on the collected and sorted job description texts and resume texts. The Chinese word segmentation tool Jieba is used to segment the text into a sequence of words or phrases. The part-of-speech tagging tool HanLP is used to tag each word or phrase. Stem extraction and lemma restoration are performed on English texts and Chinese texts respectively to reduce the impact of vocabulary diversity. Text vectorization uses a pre-trained word vector model to map words or phrases into fixed-dimensional vectors.

[0040] The Transformer and BERT model fusion module is used to build a Transformer-based encoder-decoder structure. The encoder contains multiple multi-head attention layers to extract input text features and perform semantic encoding. The pre-trained BERT model parameters are loaded into the encoder part, and a specific feature fusion layer is set at the encoder output layer to fuse the encoding features of the job text and resume text. A custom attention mechanism is added to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching, and residual connections are introduced to solve the gradient problem. At the same time, the Transformer and BERT models are reasonably initialized and hyperparameters are set;

[0041] The multi-label text classification module is used to build a multi-label classification system, and uses the pre-processed job text and resume text as input, and the corresponding multi-label classification results as output. It trains the fused Transformer and BERT models, uses appropriate loss functions and optimization algorithms to iteratively update parameters, and evaluates and tunes the Transformer and BERT models through evaluation indicators; the multi-label classification system includes job skill labels, education requirement labels, work experience labels, industry field labels, and other special requirement labels.

[0042] The matching result output and evaluation module is used to input the job text and resume text to be matched into the trained model, calculate the matching score and sort the job seekers according to the score and recommend them to the company or recommend a job list for the job seekers, and collect feedback information for further evaluation and update of the model.

[0043] Preferably, the data acquisition module adopts a strategy combining scheduled capture and real-time update to collect data from external data sources, and records metadata information of the source of the data and the time of collection, performs integrity check on the collected data, and marks and corrects erroneous data;

[0044] The data collection module integrates the submission records and interview invitation records to form resume-position one-to-one corresponding training data.

[0045] More optimally, the feature fusion layer in the Transformer and BERT model fusion module adopts a fusion method based on splicing, weighted summation, and attention mechanism. The custom attention mechanism guides the Transformer and BERT models to focus on key information according to the pre-set importance weights of key factors in job matching. The residual connection is set between the layers of the Transformer and BERT models to allow the original input information to skip any layer and be added to the transformed information.

[0046] The matching result output and the matching score in the evaluation module are obtained by weighted summation based on the importance weight and predicted probability of each label, and the feedback information includes whether the company recruiters and job seekers are satisfied with the recommendation results, that is, whether the company can recruit talents that meet the job requirements and whether the job seekers can quickly find their ideal jobs.

[0047] An electronic device comprising: a memory and at least one processor;

[0048] Wherein, the memory stores computer-executable instructions;

[0049] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the Transformer-based person-job matching method as described above.

[0050] A computer-readable storage medium stores computer-executable instructions. When a processor executes the computer-executable instructions, the above-mentioned Transformer-based person-job matching method is implemented.

[0051] The Transformer-based person-job matching method, system, device and medium of the present invention have the following advantages:

[0052] (I) The present invention significantly improves the matching accuracy: With the powerful semantic understanding ability and multi-label text classification function of the BERT model, it can comprehensively and deeply analyze the key information in the job description and the job seeker's resume, accurately capture the implicit semantics and multi-dimensional features in the text, and thus achieve more accurate job matching; at the same time, the BERT model's objective data processing ability is used to analyze and match the job and resume texts according to a unified algorithm and model standard, reduce subjective errors, and improve the accuracy, fairness and objectivity of matching; compared with traditional matching methods, it can greatly reduce mismatches caused by insufficient or one-sided understanding of information, screen out talents that are highly compatible with job requirements for enterprises, and also recommend jobs that are more in line with their own conditions and career expectations to job seekers, effectively improving recruitment success rate and job satisfaction;

[0053] (ii) The present invention greatly improves the matching efficiency: Based on the efficient Transformer architecture and the advanced BERT model, the present invention can quickly process massive amounts of job and resume data. Through automated text analysis and matching processes, the matching calculation of large-scale data can be completed in a short period of time, greatly shortening the period of person-job matching. This not only enables companies to fill job vacancies more quickly and seize market opportunities, but also allows job seekers to obtain employment opportunities faster, reducing job search waiting time and improving the operating efficiency of the entire employment market. At the same time, the efficient text processing performance of the BERT model can quickly perform multi-label classification and semantic analysis on large amounts of text data, complete the matching of positions with the information of numerous job seekers in a short period of time, and significantly improve the overall efficiency of person-job matching.

[0054] (III) The present invention reduces the cost of human resources: Traditional manual job matching requires a lot of manpower, material resources and time costs, including recruitment personnel screening resumes, interview arrangements and subsequent talent evaluation. After the present invention realizes automated matching, it can significantly reduce manual intervention, allowing the human resources department to devote more energy to more valuable work such as talent strategy planning and employee relationship management, reducing the human cost expenditure of enterprises in the recruitment process, and also reducing the energy consumption (such as job screening) and economic costs (such as resume delivery fees, transportation fees, etc.) of job seekers in the job search process.

[0055] (IV) The present invention promotes the optimal allocation of talent resources: Through precise matching, the present invention enables various types of talents to flow to matching positions better, avoids idleness and waste of talents, and realizes the reasonable allocation of talent resources among different industries, enterprises and positions, which helps to improve the utilization efficiency of human resources in the whole society, promote the balanced development of various industries, stimulate the innovation potential and work enthusiasm of talents, and provide strong talent support for economic development;

[0056] (V) The present invention has good adaptability and expansibility: The Transformer-based BERT model adopted by the present invention has a broad application base and good versatility in the field of natural language processing, and can adapt to the job matching needs of different industries, different types of positions and various complex text expressions; and, with the continuous increase in data volume and the continuous development of technology, the model can be further trained and optimized to easily expand its functions and performance to cope with the ever-changing job matching challenges in the future and maintain the advancement and practicality of the technology;

[0057] (VI) The multi-label text classification based on the BERT model of the present invention can deeply mine multiple key elements in the text, such as the potential skills and expertise of job seekers, the deep-level ability requirements of positions, etc., to ensure the integrity and accuracy of information utilization in the matching process, and solve the problem of incomplete information mining;

[0058] (VII) The present invention can comprehensively evaluate the matching degree between positions and job seekers from multiple dimensions, including professional skills, project experience, industry knowledge, and comprehensive quality, through the multi-label classification of the BERT model, to achieve more accurate and detailed person-job matching, and solve the problem of a single matching dimension;

[0059] (VIII) Through precise matching, the present invention enables job seekers to better find suitable positions to give full play to their talents, and enterprises can accurately obtain the talents they need, thereby reducing the waste and mismatch of talent resources, promoting the optimal allocation of human resources, and solving the problem of waste of talent resources;

[0060] (IX) The Transformer architecture of the present invention provides strong support for text feature extraction and semantic understanding with its powerful parallel computing capability and effective processing capability for long sequence data; the BERT model, as a pre-trained model based on Transformer, performs excellently in natural language processing tasks and can deeply understand text semantic information; in the job matching scenario, the model is used to perform multi-label text classification processing on job description text and job application resume text, and can accurately extract key information in the text, such as multi-dimensional label information such as skills required for the job, skills possessed by job seekers, job functions, and job seekers' work experience; through in-depth analysis and matching of this information, the accuracy and efficiency of job matching can be greatly improved, the time and manpower costs of the human resources department in the recruitment process can be reduced, the enterprise talent selection process can be optimized, the overall human resources management efficiency can be improved, and solid technical support can be provided for the enterprise's talent strategic layout, promoting the rapid development of human resources management in the direction of intelligence and precision; at the same time, it helps job seekers quickly screen out suitable jobs and improve job search efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The present invention is further described below in conjunction with the accompanying drawings.

[0062] Attached Figure 1 This is a structural diagram of the Transformer-based person-job matching system. DETAILED DESCRIPTION

[0063] The Transformer-based person-job matching method, system, device and medium of the present invention are described in detail below with reference to the accompanying drawings and specific embodiments of the specification.

[0064] Embodiment 1:

[0065] This embodiment provides a Transformer-based person-job matching method, which is specifically as follows:

[0066] S1. Data collection: Collect job information data and job seeker resume data from multiple data sources, clean and organize the collected data, and obtain job description texts and resume texts; data sources include recruitment websites, internal human resource management systems and talent databases;

[0067] S2. Text preprocessing: performing preprocessing operations on the collected and sorted job description texts and resume texts to obtain the preprocessed job description texts and resume texts;

[0068] S3, Transformer and BERT model fusion: Construct an encoder-decoder structure based on Transformer. The encoder is used to extract features and encode semantics of the input text; the decoder is used to generate influential outputs in specific tasks; the encoder includes multiple multi-head attention layers, each of which focuses on different feature dimensions of the input text. Through parallel computing and information fusion, the model's ability to capture text semantic relationships is enhanced; at the same time, the pre-trained BERT model parameters are loaded into the encoder part of the Transformer architecture, and a specific feature fusion layer is set at the encoder output layer to fuse the pre-processed job description text and resume text. Encoding features, adding a custom attention mechanism to the head of the Transformer and BERT models to focus on the extraction and analysis of key information for person-job matching, and introducing residual connections to solve the gradient problem. At the same time, the Transformer and BERT models are reasonably initialized and hyperparameters are set so that the models can inherit the language knowledge and semantic understanding capabilities learned by BERT on large-scale corpora; during the loading process, some parameters are fine-tuned or frozen according to the characteristics of the person-job matching task to balance the generalization ability and pertinence of the model; the underlying parameters closely related to general language understanding are kept frozen, while some parameters close to the output layer that are directly related to the multi-label text classification task are fine-tuned and trained;

[0069] S4. Model optimization: According to the requirements of the person-job matching task, the fused Transformer and BERT models are adjusted and optimized;

[0070] S5. Multi-label text classification: Construct a multi-label classification system, use the pre-processed job description text and resume text as input, and the corresponding multi-label classification results as output, train the fused Transformer and BERT models, and use the corresponding loss function and optimization algorithm to iteratively update the parameters, and evaluate and tune the Transformer and BERT models through evaluation indicators; among them, the multi-label classification system includes job skill labels (such as data analysis ability, programming skills, sales skills, etc.), education requirement labels (such as junior college, undergraduate, master's degree, etc.), work experience labels (such as 1-2 years, 3-5 years, etc.), industry field labels (such as Internet, finance, etc.), job nature labels (such as full-time, part-time, employment internship, etc.) and other special requirements labels (such as whether to recruit disabled people, whether business trips are required, whether specific certificates are required, etc.);

[0071] S6. Matching result output and evaluation: Input the job text and resume text to be matched into the trained Transformer and BERT models, calculate the matching scores, and sort the job seekers according to the scores and recommend them to the companies or recommend a job list to the job seekers. At the same time, collect feedback information to further evaluate and update the Transformer and BERT models.

[0072] The data collection in step S1 of this embodiment is specifically as follows:

[0073] S101. Collect data from external data sources by combining scheduled crawling with real-time updating, and record metadata information of the data source and collection time for subsequent data tracing and management. At the same time, perform integrity check on the collected data. If data is missing or wrong, mark it and try to supplement or correct the wrong data from other channels. For external data sources of recruitment websites, crawl data at set time intervals (such as every hour or a specific time period every day) to obtain the latest released positions and updated resume information.

[0074] S102. By comprehensively integrating the submission records of various platforms and the interview invitation records issued by enterprises, and conducting detailed cleaning, classification and labeling processing, we can construct pre-training data that can accurately reflect the matching characteristics and relationships between people and positions, and form training data with a one-to-one correspondence between resumes and positions, providing a solid and reliable data foundation for accurate modeling and efficient analysis of person-position binding.

[0075] The text preprocessing in step S2 of this embodiment is specifically as follows:

[0076] S201. Text cleaning: Remove non-text characters, stop words, redundant information, and sensitive information from the text. Among them, non-text characters include special characters and garbled codes; stop words include words with little contribution to semantic analysis such as "de", "shi", "zai"; redundant information includes repeated paragraphs and a large number of blanks; sensitive information includes personal privacy information such as names, contact information, and avatars in resumes that contribute little to person-job matching.

[0077] S202. Word segmentation: Use a Chinese word segmentation tool to split the text into a sequence of words or phrases, that is, use the Jieba word segmentation tool to accurately split the text into a sequence of words or phrases to improve the accuracy of subsequent processing.

[0078] S203. Part-of-speech tagging: On the basis of word segmentation, use the HanLP part-of-speech tagging tool to tag the part of speech of each word or phrase. Among them, the tagged parts of speech include nouns, verbs, adjectives, and adverbs. Part-of-speech tagging helps to further understand the grammatical structure and semantic information of the text. For example, it can distinguish key nouns (such as welfare benefits, job titles, skill names, technical terms, etc.) and verbs (such as be responsible for, execute, manage, etc.) in job descriptions, thus providing a basis for the subsequent model to extract more valuable features; through part-of-speech tagging, important information in the text can be more accurately located, which has an important auxiliary role for some feature extraction and screening strategies based on part-of-speech rules.

[0079] S204. Stem extraction and lemmatization: Perform stem extraction on English text to convert words into their basic stem forms; for Chinese text, perform similar lemmatization processing according to semantic and grammatical rules to reduce the impact of lexical diversity on model training.

[0080] S205. Text vectorization: Use a pre-trained Word2Vec word vector model to map each word or phrase to a vector of a fixed dimension, thereby converting the text data into a numerical form that the model can process; during the text vectorization process, optimize the word vectors according to actual needs. Among them, the optimization methods include adjusting the vector dimension and using a specific vector initialization method.

[0081] The model optimization in step S4 of this embodiment is specifically as follows:

[0082] S401. Add a specific feature fusion layer to the encoder output layer to effectively fuse the encoding features of the job text and resume text; and adopt a fusion method based on the attention mechanism to enable the Transformer and BERT models to better evaluate the matching relationship between the two; through the attention mechanism, the Transformer and BERT models dynamically assign weights to different feature parts of the job text and resume text, focusing on information that has an important impact on the matching; add a custom attention mechanism to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching; the attention mechanism guides the Transformer and BERT models to pay more attention to these key information when processing the text based on the importance weights of the pre-set key elements of person-job matching (such as core skills, years of work experience, skill certificates, expected salary, etc.), thereby improving the accuracy of the matching results; for example, for the "proficient in Golang programming" mentioned in the job description, the model can focus on the job seeker's experience and skill level in Golang programming when processing the resume;

[0083] S402, Introducing Residual Connections: Adding residual connections between the layers of the Transformer and BERT models to solve the gradient vanishing and gradient exploding problems in deep neural networks, so that the model can be trained more stably; when information is transmitted through a multi-layer network, residual connections allow the original input information to directly skip any layer and be added to the transformed information, which helps to retain the original feature information of the text and avoid excessive loss or distortion of information during deep network training;

[0084] S403, model initialization and hyperparameter setting: Perform reasonable initialization operations on the fused Transformer and BERT models, specifically: use the Xavier initialization method to initialize the Transformer and BERT model parameters to ensure that the model has a good parameter distribution in the early stage of training, which is conducive to accelerating the training convergence speed and improving the model performance; at the same time, set the hyperparameters of model training; among which, the hyperparameters of model training include learning rate (LearningRate), batch size (BatchSize) and number of training rounds (Epochs); the details are as follows:

[0085] ① Through multiple experiments and tuning, find a hyperparameter combination suitable for this task. The initial learning rate is set in a small range (such as 0.0001-0.001), and the learning rate decay strategy is adopted as the number of training rounds increases to avoid the model failing to converge due to excessive learning rate in the later stage of training or training too slowly due to too small learning rate;

[0086] ② The batch size is adjusted according to the hardware resources and data size, and the value is between 16-256;

[0087] ③ The number of training rounds is determined based on the performance of the model on the validation set, and is between 10 and 60 rounds. When the performance of the model on the validation set no longer improves, the early stopping method is used to terminate the training in advance to prevent overfitting.

[0088] The matching result output and evaluation in step S6 of this embodiment are specifically as follows:

[0089] S601, match calculation: the job text and resume text to be matched are input into the trained Transformer and BERT models respectively, and the Transformer and BERT models output the predicted probability of each label and the comprehensive match score; wherein the match score is obtained by weighted summation of the importance weight of each label (determined by expert experience or data analysis) and the predicted probability, thereby quantifying the match degree between the job and the job seeker;

[0090] S602, result sorting and recommendation: Sort the job seekers in descending order according to the matching scores, and recommend the top-ranked job seekers to the company recruiters; at the same time, recommend a list of positions with high matching degrees to the job seekers; during the recommendation process, provide a detailed matching information report, including matching analysis of each tag and detailed calculation process of matching scores, so that recruiters and job seekers can have a deep understanding of the basis of matching results;

[0091] S603. Evaluation and feedback: Establish a matching result evaluation mechanism to collect feedback from corporate recruiters and job seekers on the recommendation results. The feedback on the recommendation results includes whether the job is successfully hired and the job seeker’s satisfaction with the recommended position. Based on the feedback data, further evaluate and analyze the Transformer and BERT models to identify possible deviations or deficiencies in the models, such as unsatisfactory matching results for certain specific industries or job types. Use the feedback information as an important basis for optimizing the Transformer and BERT models, update and adjust the models regularly, continuously improve the accuracy and practicality of the matching results, and achieve continuous optimization and improvement of the job matching system.

[0092] Embodiment 2:

[0093] As attached Figure 1 As shown, this embodiment provides a Transformer-based person-job matching system, which includes:

[0094] The data collection module is used to collect job information data and job seeker resume data from multiple data sources, and clean and organize the collected data to obtain the collected and organized job description text and resume text; the data sources include recruitment websites, internal human resource management systems of enterprises, and talent databases;

[0095] The text preprocessing module is used to perform word segmentation, part-of-speech tagging, stem extraction, lemma restoration, and text vectorization on the collected and sorted job description texts and resume texts. The Chinese word segmentation tool Jieba is used to segment the text into a sequence of words or phrases. The part-of-speech tagging tool HanLP is used to tag each word or phrase. Stem extraction and lemma restoration are performed on English texts and Chinese texts respectively to reduce the impact of vocabulary diversity. Text vectorization uses a pre-trained word vector model to map words or phrases into fixed-dimensional vectors.

[0096] The Transformer and BERT model fusion module is used to build a Transformer-based encoder-decoder structure. The encoder contains multiple multi-head attention layers to extract input text features and perform semantic encoding. The pre-trained BERT model parameters are loaded into the encoder part, and a specific feature fusion layer is set at the encoder output layer to fuse the encoding features of the job text and resume text. A custom attention mechanism is added to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching, and residual connections are introduced to solve the gradient problem. At the same time, the Transformer and BERT models are reasonably initialized and hyperparameters are set;

[0097] The multi-label text classification module is used to build a multi-label classification system, and uses the pre-processed job text and resume text as input, and the corresponding multi-label classification results as output. It trains the fused Transformer and BERT models, uses appropriate loss functions and optimization algorithms to iteratively update parameters, and evaluates and tunes the Transformer and BERT models through evaluation indicators; the multi-label classification system includes job skill labels, education requirement labels, work experience labels, industry field labels, and other special requirement labels.

[0098] The matching result output and evaluation module is used to input the job text and resume text to be matched into the trained model, calculate the matching score and sort the job seekers according to the score and recommend them to the company or recommend a job list for the job seekers, and collect feedback information for further evaluation and update of the model.

[0099] The data acquisition module in this embodiment adopts a strategy combining scheduled capture and real-time update to collect data from external data sources, and records metadata information of the source of the data and the time of collection, performs integrity checks on the collected data, and marks and corrects erroneous data.

[0100] The data collection module in this embodiment integrates the submission records and the interview invitation records to form resume-position one-to-one corresponding training data.

[0101] The feature fusion layer in the Transformer and BERT model fusion module in this embodiment adopts a fusion method based on splicing, weighted summation, and attention mechanism. The custom attention mechanism guides the Transformer and BERT models to focus on key information according to the pre-set importance weights of key factors of person-job matching. The residual connection is set between the layers of the Transformer and BERT models to allow the original input information to skip any layer and be added to the transformed information.

[0102] The matching result output in this embodiment and the matching score in the evaluation module are obtained by weighted summation based on the importance weight and predicted probability of each label, and the feedback information includes whether the company recruiters and job seekers are satisfied with the recommendation results, that is, whether the company can recruit talents that meet the job requirements and whether the job seekers can quickly find their ideal jobs.

[0103] Embodiment 3:

[0104] This embodiment also provides an electronic device, including: a memory and at least one processor;

[0105] Wherein, the memory stores computer-executable instructions;

[0106] The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the Transformer-based person-job matching method described in any one of the present inventions.

[0107] The processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor may be a microprocessor or any conventional processor, etc.

[0108] The memory can be used to store computer programs and / or modules. The processor realizes various functions of the electronic device by running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, at least one application required for a function, etc.; the data storage area can store data created according to the use of the terminal, etc. In addition, the memory can also include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash memory card, at least one disk storage period, a flash memory device, or other volatile solid-state storage devices.

[0109] Embodiment 4:

[0110] This embodiment also provides a computer-readable storage medium, which stores a plurality of instructions, which are loaded by a processor to enable the processor to execute the Transformer-based person-job matching method in any embodiment of the present invention. Specifically, a system or device equipped with a storage medium can be provided, on which a software program code that implements the functions of any of the above embodiments is stored, and a computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage medium.

[0111] In this case, the program code itself read from the storage medium can realize the function of any one of the above-mentioned embodiments, and thus the program code and the storage medium storing the program code constitute a part of the present invention.

[0112] The storage medium embodiments for providing the program code include a floppy disk, a hard disk, a magneto-optical disk, an optical disk (such as CD-ROM, CD-R, CD-RW, DVD-ROM, DVD-RAM, DVD-RW, DVD+RW), a magnetic tape, a non-volatile memory card, and a ROM. Alternatively, the program code can be downloaded from a server computer by a communication network.

[0113] In addition, it should be clear that the functions of any of the above embodiments can be implemented not only by executing the program code read by the computer, but also by enabling an operating system operating on the computer to complete part or all of the actual operations based on instructions from the program code.

[0114] In addition, it can be understood that the program code read from the storage medium is written to a memory provided in an expansion board inserted into the computer or written to a memory provided in an expansion unit connected to the computer, and then based on the instructions of the program code, a CPU installed on the expansion board or the expansion unit is enabled to perform part or all of the actual operations, thereby realizing the functions of any of the above-mentioned embodiments.

[0115] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein with equivalents. However, these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A Transformer-based person-job matching method, characterized in that: The method is as follows: Data collection: Collect job information data and job seekers' resume data from multiple data sources, clean and organize the collected data, and obtain job description texts and resume texts; data sources include recruitment websites, internal human resource management systems and talent databases; Text preprocessing: preprocess the collected and sorted job description texts and resume texts to obtain the preprocessed job description texts and resume texts; Fusion of Transformer and BERT models: Construct an encoder-decoder structure based on Transformer. The encoder is used to extract features and encode semantics of the input text; the decoder is used to generate influential outputs in specific tasks; the encoder includes multiple multi-head attention layers, each of which focuses on different feature dimensions of the input text. Through parallel computing and information fusion, the model's ability to capture text semantic relationships is enhanced; at the same time, the pre-trained BERT model parameters are loaded into the encoder part of the built Transformer architecture, and a specific feature fusion layer is set at the encoder output layer to fuse the encoding features of the pre-processed job description text and resume text. A custom attention mechanism is added to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching, and residual connections are introduced to solve the gradient problem. At the same time, the Transformer and BERT models are reasonably initialized and hyperparameters are set; Model optimization: According to the requirements of the person-job matching task, the fused Transformer and BERT models are adjusted and optimized; Multi-label text classification: Build a multi-label classification system, use the pre-processed job description text and resume text as input, and the corresponding multi-label classification results as output, train the fused Transformer and BERT models, and use the corresponding loss function and optimization algorithm to iteratively update the parameters, and evaluate and tune the Transformer and BERT models through evaluation indicators; the multi-label classification system includes job skill labels, education requirement labels, work experience labels, industry field labels, job nature labels, and other special requirement labels; Matching result output and evaluation: Input the job text and resume text to be matched into the trained Transformer and BERT models, calculate the matching score, and sort the job seekers according to the score and recommend them to the company or recommend a job list to the job seekers. At the same time, collect feedback information to further evaluate and update the Transformer and BERT models.

2. The Transformer-based person-job matching method according to claim 1, characterized in that: The data collection details are as follows: Adopt a strategy that combines timed crawling and real-time updating to collect data from external data sources, and record the metadata information of the data source and the collection time. At the same time, perform integrity checks on the collected data. If data is found to be missing or incorrect, mark it and attempt to supplement or correct the incorrect data. Among them, for the external data source of the recruitment website, data is crawled at set time intervals to obtain the latest released job positions and updated resume information. By comprehensively integrating the delivery records of each platform and the interview invitation records sent by enterprises, and performing meticulous cleaning, classification, and annotation processing, pre-training data that can accurately reflect the characteristics and relationships of person-job matching is constructed, forming training data with a one-to-one correspondence between resumes and job positions.

3. The Transformer-based person-job matching method according to claim 1, characterized in that: The text preprocessing is as follows: Text cleaning: Remove non-text characters, stop words, redundant information, and sensitive information from the text. Among them, non-text characters include special characters and garbled codes; stop words include words with little contribution to semantic analysis such as "of", "is", "in"; redundant information includes repeated paragraphs and blanks; sensitive information includes information related to personal privacy such as names, contact information, and avatars in resumes that have little contribution to person-job matching. Word segmentation: Use a Chinese word segmentation tool to split the text into a sequence of words or phrases, that is, use the Jieba word segmentation tool to accurately split the text into a sequence of words or phrases. Part-of-speech tagging: On the basis of word segmentation, use the HanLP part-of-speech tagging tool to tag the part of speech of each word or phrase. Among them, the tagged parts of speech include nouns, verbs, adjectives, and adverbs. Stem extraction and lemmatization: Perform stem extraction operations on English texts to convert words into their basic stem forms; for Chinese texts, perform similar lemmatization processing according to semantic and grammatical rules. Text vectorization: Use a pre-trained Word2Vec word vector model to map each word or phrase to a vector of a fixed dimension, thereby converting text data into a numerical form that the model can process. During the text vectorization process, optimize the word vectors according to actual needs. Among them, the optimization methods include adjusting the vector dimension and using a specific vector initialization method.

4. The Transformer-based person-job matching method according to claim 1, characterized in that: The model optimization is as follows: Add a specific feature fusion layer to the output layer of the encoder to effectively fuse the encoded features of the job text and the resume text; and adopt a fusion method based on the attention mechanism to enable the Transformer and BERT models to better evaluate the matching relationship between the two. Through the attention mechanism, the Transformer and BERT models dynamically allocate weights to different feature parts of the job text and the resume text, focusing on information that has an important impact on matching. Add a custom attention mechanism to the heads of the Transformer and BERT models to focus on the extraction and analysis of key information for person-job matching. The attention mechanism guides the Transformer and BERT models to pay more attention to these key information when processing text according to the importance weights of the key elements for person-job matching set in advance, thereby improving the accuracy of the matching results. Introducing residual connections: Adding residual connections between the layers of the Transformer and BERT models; when information is passed through a multi-layer network, residual connections allow the original input information to directly skip any layer and be added to the transformed information, which helps to preserve the original feature information of the text and avoid excessive loss or distortion of information during deep network training; Model initialization and hyperparameter setting: Initialize the fused Transformer and BERT models. Specifically, use the Xavier initialization method to initialize the Transformer and BERT model parameters to ensure that the model has a good parameter distribution at the beginning of training, which is conducive to accelerating the training convergence speed and improving the model performance; at the same time, set the hyperparameters of model training; among them, the hyperparameters of model training include learning rate, batch size and number of training rounds; the details are as follows: Through multiple experiments and tuning, we found a hyperparameter combination suitable for this task. The initial learning rate was set in a small range, and the learning rate decay strategy was adopted as the number of training rounds increased to prevent the model from failing to converge due to an excessively large learning rate in the late stage of training or training too slowly due to an excessively small learning rate. The batch size is adjusted according to the hardware resources and data size, and the value is between 16-256; The number of training rounds is determined based on the performance of the model on the validation set, and is between 10 and 60 rounds. When the performance of the model on the validation set no longer improves, the early stopping method is used to terminate the training in advance to prevent overfitting.

5. The Transformer-based person-job matching method according to any one of claims 1-4, characterized in that: The matching result output and evaluation are as follows: Matching degree calculation: The job text and resume text to be matched are input into the trained Transformer and BERT models respectively. The Transformer and BERT models output the predicted probability of each label and the comprehensive matching degree score. The matching degree score is obtained by weighted summation of the importance weight and predicted probability of each label, thereby quantifying the matching degree between the job and the job seeker. Result sorting and recommendation: Sort job seekers in descending order according to their matching scores, and recommend the top-ranked job seekers to corporate recruiters; at the same time, recommend job seekers a list of highly matching positions; during the recommendation process, provide a detailed matching information report, including matching analysis of each tag and detailed calculation process of matching scores; Evaluation and feedback: Establish a matching result evaluation mechanism to collect feedback from corporate recruiters and job seekers on the recommendation results. The feedback on the recommendation results includes whether the job is successfully hired and the job seeker’s satisfaction with the recommended position. Based on the feedback data, further evaluate and analyze the Transformer and BERT models. Use the feedback information as an important basis for optimizing the Transformer and BERT models. Regularly update and adjust the models to continuously improve the accuracy and practicality of the matching results, and achieve continuous optimization and improvement of the job matching system.

6. A Transformer-based person-job matching system, characterized in that: The system includes: The data collection module is used to collect job information data and job seeker resume data from multiple data sources, and clean and organize the collected data to obtain the collected and organized job description text and resume text; the data sources include recruitment websites, internal human resource management systems of enterprises, and talent databases; The text preprocessing module is used to perform word segmentation, part-of-speech tagging, stem extraction, lemma restoration, and text vectorization on the collected and sorted job description texts and resume texts. The Chinese word segmentation tool Jieba is used to segment the text into a sequence of words or phrases. The part-of-speech tagging tool HanLP is used to tag each word or phrase. Stem extraction and lemma restoration are performed on English texts and Chinese texts respectively to reduce the impact of vocabulary diversity. Text vectorization uses a pre-trained word vector model to map words or phrases into fixed-dimensional vectors. The Transformer and BERT model fusion module is used to build a Transformer-based encoder-decoder structure. The encoder contains multiple multi-head attention layers to extract input text features and perform semantic encoding. The pre-trained BERT model parameters are loaded into the encoder part, and a specific feature fusion layer is set at the encoder output layer to fuse the encoding features of the job text and resume text. A custom attention mechanism is added to the Transformer and BERT model heads to focus on the extraction and analysis of key information for person-job matching, and residual connections are introduced to solve the gradient problem. At the same time, the Transformer and BERT models are reasonably initialized and hyperparameters are set; The multi-label text classification module is used to build a multi-label classification system, and uses the pre-processed job text and resume text as input, and the corresponding multi-label classification results as output. It trains the fused Transformer and BERT models, uses appropriate loss functions and optimization algorithms to iteratively update parameters, and evaluates and tunes the Transformer and BERT models through evaluation indicators; the multi-label classification system includes job skill labels, education requirement labels, work experience labels, industry field labels, and other special requirement labels. The matching result output and evaluation module is used to input the job text and resume text to be matched into the trained model, calculate the matching score and sort the job seekers according to the score and recommend them to the company or recommend a job list for the job seekers, and collect feedback information for further evaluation and update of the model.

7. The Transformer-based person-job matching system according to claim 6, characterized in that: The data acquisition module adopts a strategy combining scheduled capture and real-time update to collect data from external data sources, and records metadata information about the source of the data and the time of collection, performs integrity checks on the collected data, and marks and corrects erroneous data; The data collection module integrates the submission records and interview invitation records to form resume-position one-to-one corresponding training data.

8. The Transformer-based person-job matching system according to claim 6 or 7, characterized in that: The feature fusion layer in the Transformer and BERT model fusion module adopts concatenation, weighted summation, and attention-based fusion methods. The custom attention mechanism guides the Transformer and BERT models to focus on key information based on the pre-set importance weights of key factors in job matching. The residual connection is set between the layers of the Transformer and BERT models to allow the original input information to skip any layer and be added to the transformed information. The matching result output and the matching score in the evaluation module are obtained by weighted summation based on the importance weight and predicted probability of each label, and the feedback information includes whether the company recruiters and job seekers are satisfied with the recommendation results, that is, whether the company can recruit talents that meet the job requirements and whether the job seekers can quickly find their ideal jobs.

9. An electronic device, characterized in that: include: memory and at least one processor; Wherein, the memory stores computer-executable instructions; The at least one processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the Transformer-based person-job matching method as described in any one of claims 1 to 5.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions, and when the processor executes the computer-executable instructions, the Transformer-based person-job matching method as described in any one of claims 1 to 5 is implemented.

Images