Job description automatic generation method based on deep learning

Abstract

This invention relates to the field of job descriptions, and more particularly to a deep learning-based method for automatically generating job descriptions. The method includes: preprocessing acquired job description text data to obtain multiple modules; attaching corresponding module tags to word segments in each module; constructing a module aggregation degree reflecting the semantic purity within each module based on the probability distribution of words within each module; extracting boundary local word sequences between adjacent modules and calculating information entropy, and constructing a boundary jump degree based on the module aggregation degree; constructing structural weights for each module based on the boundary jump degree and the word count ratio of each module; using the structural weights as adaptive weighting coefficients to improve the prediction loss function of the ProphetNet model, obtaining an improved ProphetNet model for generating job description text. This invention improves the writing efficiency and quality in recruitment scenarios.

Description

Technical Field

[0001] This invention relates to the field of job descriptions, and more particularly to a method for automatically generating job descriptions based on deep learning. Background Technology

[0002] With the rapid development of online recruitment platforms and the advancement of human resource digitalization, job descriptions, as the core carrier of recruitment information, not only directly influence job seekers' willingness to apply but also serve as a key basis for the underlying job-person matching algorithms to extract features. However, the writing of traditional job descriptions heavily relies on the human experience of HR or business departments, generally suffering from low writing efficiency, inconsistent formatting standards, and omissions of key information. Furthermore, manual writing struggles to cater to the differentiated expression needs of various sub-sectors, job levels, and corporate cultures. Therefore, utilizing deep learning and natural language processing technologies to achieve high-quality automated generation of job descriptions is of great significance for improving recruitment efficiency.

[0003] ProphetNet is an advanced sequence-to-sequence pre-trained language model based on a future n-tuple prediction mechanism. Compared to traditional models that only predict the next word, ProphetNet forces the model to plan the future text direction while generating the current word through multi-step prediction, thereby effectively improving the long-range coherence of the generated text.

[0004] However, directly applying the ProphetNet model in real-world job description generation scenarios has significant limitations. Job descriptions are not ordinary, straightforward text, but rather highly structured text composed of multiple functional sub-modules such as job responsibilities, job requirements, and salary / benefits. These sub-modules differ greatly in semantic boundaries and lexical style: for example, the job responsibilities module highly focuses on action verbs, the job requirements module is densely packed with skills and conditional vocabulary, while the salary / benefits module is mostly composed of formulaic phrase combinations.

[0005] Existing ProphetNet models face the following problems when dealing with this type of text: First, the lexical aggregation degree and specific paradigms of different modules differ significantly, but the model lacks the ability to perceive local semantic purity. Second, when text transitions from one functional module to another, the semantic distribution undergoes a sharp break, leading to a surge in local information entropy, making the model prone to prediction errors at the boundaries. Finally, the information density and importance of different modules vary drastically; for example, the distinguishing value of job responsibilities and requirements is far greater than that of salary and benefits, but the existing ProphetNet model's n-tuple prediction loss function assigns equal weight to all prediction positions. This undifferentiated gradient backpropagation mechanism not only causes model training resources to be tilted towards modules with simple vocabulary and low importance, but also makes the model unable to perceive the structural boundaries between modules. Ultimately, this results in serious cross-module semantic confusion, blurred boundaries, and low quality of generating core and difficult vocabulary in the generated job descriptions, severely affecting the structural integrity and practical usability of the automatically generated text. Summary of the Invention

[0006] To address the issue that existing ProphetNet models, when automatically generating job descriptions, fail to perceive the semantic boundaries and functional importance differences between modules due to the equal weighting of the loss function, resulting in poor text generation quality, this invention provides a deep learning-based method for automatically generating job descriptions.

[0007] This invention provides a method for automatically generating job descriptions based on deep learning, employing the following technical solution: A deep learning-based method for automatically generating job descriptions includes the following steps: The acquired job description text data is preprocessed to obtain multiple modules, and corresponding module labels are attached to the word segments in each module. Based on the probability distribution of words within each module, a module aggregation degree reflecting the semantic purity within the module is constructed. Boundary local word sequences between adjacent modules are extracted and information entropy is calculated. Combined with the module aggregation degree, a boundary jump degree is constructed. The boundary jump degree is positively correlated with the absolute value of the difference between the module aggregation degrees of adjacent modules and is also positively correlated with information entropy. Based on the boundary jump degree and the word count ratio of each module, the structural weights of each module are constructed. The structural weights are used as adaptive weighting coefficients to improve the prediction loss function of the ProphetNet model, resulting in an improved ProphetNet model for generating job description text. In the step of constructing structural weights, the average boundary jump degree is obtained by taking the average of the boundary jump degrees on both sides of each module. The structural weights are positively correlated with the average boundary jump degree and inversely proportional to the proportion of words in the module.

[0008] The technical advantage of this invention is that, compared to the shortcomings of the existing ProphetNet model, which uses an indiscriminate gradient backpropagation mechanism that leads to semantic confusion and blurred boundaries across modules, this invention extracts structural weights by constructing module aggregation degree and boundary jump degree, and uses them to adaptively weight and improve the model loss function. This enables the model to perceive the semantic boundaries and functional importance differences of different modules, thereby improving the structural integrity and practical usability of the automatically generated job description text.

[0009] Preferably, the steps for preprocessing the acquired job description text data to obtain multiple modules are as follows: use regular expressions to clean HTML tags, special symbols, and abnormal length samples; use jieba word segmentation combined with a custom dictionary in the recruitment field to process the job description text to obtain Chinese word segments; and split the Chinese word segments of each job description into three modules: job responsibilities, job requirements, and salary and benefits, based on the labeled boundary information.

[0010] The technical advantage of this invention is that by combining a custom dictionary for the recruitment field with boundary information splitting rules in the data preprocessing stage, it can extract distinctive functional structural modules more accurately than traditional general word segmentation methods, laying the foundation for the automated system to output text that highly conforms to the enterprise's recruitment business specifications.

[0011] Preferably, after preprocessing the acquired job description text data to obtain multiple modules, the method further includes: using the Word2Vec model to convert Chinese word segmentation into word vectors.

[0012] Preferably, the expression for the module aggregation degree is: ; in, For module Module aggregation degree; The number of high-frequency words within the module participating in the calculation; For module Chinese word frequency ranking The word; For words With modules The joint probability; For words The marginal probability; For module The percentage of words in the total number of words.

[0013] The technical effect of this invention is that it quantifies the aggregation degree of modules based on the relationship between joint probability and marginal probability. Compared with the traditional global word frequency statistics method, it can effectively eliminate the interference of low-frequency noise words and accurately capture the degree of exclusive vocabulary of specific modules in business text, thereby helping the model to distinguish the purity of different content.

[0014] Preferably, the steps for extracting the boundary local word sequence between adjacent modules and calculating the information entropy are as follows: constructing a sequence of adjacent module pairs according to the standard module order of the job description; for each pair of adjacent modules, extracting a set number of words before and after the marked boundary position to form a boundary local word sequence; counting the occurrence frequency of each word in the boundary local word sequence, recording the total number of non-repeating words in the local vocabulary, and calculating the Shannon information entropy of the boundary local word sequence based on the occurrence frequency.

[0015] The technical advantage of this invention is that by using Shannon information entropy to quantify the distribution of local vocabulary between adjacent modules, it overcomes the problem that existing models are prone to prediction loss at the boundaries, enabling the system to clearly define the context transition area when processing business descriptions with complex structures.

[0016] Preferably, the method for calculating the boundary jump degree is as follows: calculate the difference in the aggregation degree of two adjacent modules and take the absolute value of the difference; take the natural logarithm of the total number of non-repeating words and use the logarithmic result as the normalization coefficient; multiply the absolute value by the Shannon information entropy and divide the product by the normalization coefficient to obtain the boundary jump degree.

[0017] The technical effect of this invention is that it combines the aggregation degree difference with the local information entropy to calculate the boundary jump degree. Compared with the traditional method that relies on fixed position thresholds or manual labeling, it realizes dynamic perception of semantic faults, thereby greatly reducing the logical confusion of cross-module content when generating transition paragraphs.

[0018] Preferably, the method for calculating the structure weights is as follows: For each module, obtain all boundary jump degrees formed by the module and its neighboring modules, and average the obtained boundary jump degrees to obtain the average boundary jump degree of the corresponding module; count the number of words in each module in the training corpus, calculate the proportion of the number of words to the total number of words in the training set, and obtain the word count proportion of the module; calculate the sum of the average boundary jump degrees of all modules, and use it as the normalized denominator; divide the average boundary jump degree of a single module by its own word count proportion, and then divide it by the normalized denominator to obtain the structure weight of the corresponding module.

[0019] The technical effect of this invention is that by combining the average boundary jump degree of the integrated modules with the word density to calculate the structural weight, the functional importance of each module can be objectively measured in a data-driven manner, avoiding the bias of human intervention and guiding the system to focus computing power on the generation of highly discriminative information during actual writing.

[0020] Preferably, the expression for the prediction loss function of the ProphetNet model is: ; in, This represents the total prediction loss of the improved ProphetNet model; Indicates the time step in the sequence generation process; Indicates time step The structural weights corresponding to module m to which the generated words belong; Indicates at time step The original ProphetNet model calculates the standard prediction loss based on the future n-tuple prediction mechanism.

[0021] The technical effect of this invention is that it constructs an adaptive weighted target loss function, which drives the model to prioritize reducing the prediction error of core and difficult modules during training. This effectively solves the problem of unbalanced training resource allocation in the face of complex text in existing technologies and improves the generation accuracy of core business modules.

[0022] Preferably, after obtaining the improved ProphetNet model, the method further includes: training the improved ProphetNet model to obtain a text generation model; during the training process, after each training round, calculating the n-tuple prediction loss and module completeness rate of the current model on the test set; implementing an early stopping strategy based on the changing trend of the test set loss; and terminating training when the test set loss no longer decreases for several consecutive training rounds.

[0023] The technical effect of this invention is that by monitoring the changing trends of prediction loss and module completeness rate, an early stopping strategy is implemented, which effectively prevents overfitting caused by overtraining and ensures that the text generation model has excellent generalization adaptability when facing diverse and ever-changing real-world internet recruitment needs.

[0024] Preferably, the steps for generating job description text are as follows: input the job title, industry, job level, work location, and core skill keywords into the text generation model. The text generation model first generates start identifiers for each functional module, and then generates the content of each module word by word under the guidance of the improved loss function. After generation, the output sequence is formatted into a standard three-part text of job responsibilities, job requirements, and salary based on the module identifiers. After post-processing to remove duplicate sentences and formatting errors, the final result is output.

[0025] The technical advantage of this invention is that it establishes a complete end-to-end automated content generation mechanism, which can generate a well-formatted and clearly structured three-part recruitment document with just one click based on a small number of core job keywords. It also effectively eliminates repetition and formatting errors, reducing the time required for manual job description writing.

[0026] The present invention has the following technical effects: This invention proposes a deep learning-based method for automatically generating job descriptions. By quantifying the semantic purity and abrupt changes between different functional modules, objective structural weights are extracted, and then adaptively weighted improvements are made to the model's prediction loss function. This mechanism breaks through the limitations of traditional indiscriminate gradient backpropagation, driving the model to allocate resources to core modules with high prediction difficulty and high importance during generation. This effectively solves the problems of cross-module semantic confusion and boundary ambiguity, achieving automated generation of high-quality job description texts with complete structure and clear semantics, thus improving the writing efficiency and quality in recruitment scenarios. Attached Figure Description

[0027] Figure 1 This is a comparison chart of the module aggregation degree of the present invention.

[0028] Figure 2 This is a comparison chart of the structural weights of each module with the equal weight benchmark of the original ProphetNet model.

[0029] Figure 3 This is a comparison chart of the cross-module semantic confusion rates of each module.

[0030] Figure 4 This is a flowchart of the deep learning-based method for automatically generating job descriptions according to the present invention. Detailed Implementation

[0031] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0032] This invention discloses a method for automatically generating job descriptions based on deep learning, referring to... Figure 4 This includes the following steps: Step S1: Data acquisition and preprocessing.

[0033] By collecting job description text data in batches from recruitment platforms, the following preprocessing was performed on all collected job description text data in sequence: HTML tags, special symbols, and abnormal length samples were cleaned using regular expressions; the job description text was processed using jieba word segmentation combined with a custom dictionary in the recruitment field to obtain Chinese word segments; based on the labeled boundary information, the Chinese word segments of each job description were split into three modules: job responsibilities, job requirements, and salary and benefits; and corresponding module tags were attached to each word. The module tags include: job responsibilities, job requirements, and salary and benefits.

[0034] For example, if the modules corresponding to job requirements are (python, c language, java, word), then the module labels corresponding to word segmentation of python, c language, java, and word are all "job requirements"; the set of words obtained after preprocessing is used as the corpus, and the corpus is divided into training set and test set in an 8:2 ratio; finally, the Word2Vec model is used to convert Chinese word segmentation into word vectors.

[0035] Step S2: Build module aggregation degree.

[0036] During the automatic generation of job descriptions, significant lexical aggregation occurs within different functional modules due to their highly specific expression paradigms. According to distributed semantics theory, semantically similar words tend to co-occur frequently in similar contexts. This pattern is particularly pronounced in the various functional modules of the job descriptions: action verbs such as "responsible," "promoting," and "coordinating" are highly concentrated in the job responsibilities module; conditional words such as "familiar with," "master," and "possess" appear densely in the job requirements module; and numerical ranges and welfare noun phrases dominate the salary and benefits module. Stronger lexical aggregation within a module indicates higher semantic purity and clearer boundaries; conversely, weaker aggregation suggests the presence of cross-functional domain vocabulary within the module.

[0037] The original ProphetNet model's equal-weight loss cannot distinguish between highly and low-aggregated modules, causing training resources to be skewed towards less important modules, thus affecting the generation quality of core modules. Therefore, the aggregation degree of the building modules needs to be improved. Reflecting each module The degree of vocabulary aggregation lays the data foundation for the adaptive allocation of subsequent loss weights.

[0038] The training set corpora were divided into three sub-corpora based on module tags: job responsibilities, job requirements, and salary and benefits. Word counts were then performed on each sub-corpus. In the module word frequency in and total word frequency in the corpus Calculate the joint probability Marginal probability of words and module ratio ,in The total number of words in the entire corpus. For module Total word count; sorted by word frequency within each module in descending order. High-frequency words are used in subsequent calculations to eliminate interference from low-frequency noise words.

[0039] The expression for module aggregation degree is: ; in, For module The module aggregation degree; the larger the value, the higher the degree of aggregation. The number of high-frequency words within the module used in the calculation is taken as an empirical value of 50, which can be adjusted between 30 and 100 depending on the size of the corpus. For module Chinese word frequency ranking The words are ranked from highest to lowest frequency. , , Each word With modules joint probability, word Marginal probability and modules The percentage of words in each term is directly calculated from the word frequency statistics results in the preprocessing stage.

[0040] When the vocabulary aggregation features of a module are more obvious, the high-frequency words of that module... relatively The larger the ratio, the larger the numerator of the formula, while the denominator is relatively smaller due to the high frequency of the words, ultimately leading to a higher degree of module aggregation. The larger. Compared to traditional global word frequency statistics methods, By analyzing the ratio of module conditional probability to global probability, we can accurately capture the degree of exclusivity of words to specific modules and effectively distinguish between strong and weak aggregation modules.

[0041] Step S3: Construct the boundary jump degree.

[0042] In module aggregation Building upon this foundation, we further analyze the semantic transition characteristics between adjacent modules. According to the conditional entropy theory of information theory, when a sequence transitions from a region with a concentrated semantic distribution to a region with a drastically different semantic distribution, the uncertainty in predicting the next word will increase sharply, manifested as a dramatic rise in local information entropy.

[0043] In the job description, when the module aggregation degree of two adjacent modules is... When both modules have high aggregation levels but drastically different lexical domains, such as a job responsibility module dominated by verbs and a job requirement module dominated by conditional words, the degree of lexical mixing at the boundary is most severe; while the aggregation degree of the two low modules is... The transitions between modules are relatively smooth. The intensity of these semantic jumps at the boundaries directly determines the prediction difficulty of the model at the boundary locations; that is, the more drastic the jump, the easier it is for the ProphetNet model to generate cross-module obfuscation near the boundary. Existing ProphetNet models cannot perceive such positional jumps, leading to unstable generation quality in boundary regions. Therefore, based on module aggregation degree... Further construct the boundary jump degree This quantifies the abrupt change in semantic aggregation between adjacent modules, accurately locates boundary positions with high prediction difficulty, and provides structured input for module importance measurement in subsequent steps.

[0044] Construct a sequence of adjacent module pairs according to the standard module order of the job description (job responsibilities → job requirements → salary and benefits); for each pair of adjacent modules, take the marked boundary position as the center, and take the positions before and after it. A sequence of words forms a boundary local word sequence. The frequency of each word in this sequence is counted, and the size of the local vocabulary is recorded. Shannon information entropy of the boundary local word sequence based on the above frequencies. and with As a normalization coefficient, the boundary jump degree is calculated, and the expression is: ; in, For module With modules The boundary jump degree between adjacent modules; the larger the value, the more significant the semantic discontinuity between the two adjacent modules. and Let be the module aggregation degree between the i-th module and the (i+1)-th module; The Shannon information entropy of the local word sequence at the boundary is calculated by statistically analyzing the preceding and following words. The total number of words The word frequency distribution of each word is calculated, where Take an empirical value of 10, which can be adjusted between 5 and 15 based on the average module length. The higher the entropy value, the more diverse the boundary vocabulary and the more mixed the distribution. For this The number of unique words appearing in the word window; Let e ​​represent the logarithmic function with base e. The overall normalization coefficient represents the theoretical maximum value of the Shannon information entropy of the local word sequence at the boundary, ensuring that the degree of jump between different boundary positions is comparable.

[0045] When the semantic jump features between adjacent modules are more obvious, the absolute value of the difference in module aggregation degree between the two modules is... The larger the area, the more mixed the vocabulary distribution in the boundary region leads to... The larger the volume, the greater the accumulation of the two volumes. Normalization reduces the boundary jump degree The larger the value, the better. Compared to traditional methods that rely on fixed-location thresholds or manually marked boundaries, Module aggregation degree By integrating the difference information with the local information entropy of the boundary, dynamic perception and quantitative differentiation of the semantic fault intensity of different boundaries are achieved.

[0046] Step S4: Construct the structural weights.

[0047] In module aggregation Revealing the lexical aggregation features and boundary jump degree within the module Based on the strength of boundary faults between modules, this paper further analyzes the differences in functional importance of different modules within the overall structure of the job description. According to TF-IDF theory, content units with high information density and concentrated distribution have stronger distinguishing value. Combined with the analysis of the importance of job description elements in recruitment information research, the job responsibilities and requirements module is the primary basis for job seekers to judge the degree of match, and its module aggregation degree... High and boundary jump degree with adjacent modules The larger number of words indicates clear semantic boundaries and high information density; however, the vocabulary in the salary and benefits module is highly formulaic, indicating relatively low importance. More importantly, although the number of words is small, the boundary transitions are significant. Larger modules carry higher semantic uniqueness per unit word, making prediction more difficult. Existing ProphetNet models cannot distinguish this difference in prediction difficulty determined by both functional importance and semantic density, leading to an imbalance in training resource allocation. Therefore, a method based on boundary jump degree... Constructing structural weights This comprehensively reflects the semantic uniqueness and word density of each module, providing direct numerical input for the subsequent adaptive weight allocation of the loss function.

[0048] For each module The average of all boundary jump degrees formed by the module and its adjacent modules is taken to obtain the module. Average boundary jump degree The tail module has only a single-sided boundary, so the jump degree of that boundary is directly taken; at the same time, the number of words in each module is counted in the training corpus. Calculate its proportion of the total number of words in the training set. proportion The expression for the structural weights is: ; in, For module The structural weights are such that the larger the value, the higher the prediction weight that the module should be assigned in the loss function; For module The mean of the abrupt changes at both boundaries; For module The average word count percentage, calculated statistically from the training set, is as follows: for example, the job responsibilities module accounts for approximately 0.40%, the job requirements module accounts for approximately 0.35%, and the salary and benefits module accounts for approximately 0.25%. The sum of the average boundary jump degrees of all modules is used as the normalized denominator to ensure structural weights. It has consistent dimensions across different corpus sizes.

[0049] The stronger the semantic uniqueness of a module, that is, the more drastic the boundary transitions on both sides, When the numerator is larger and the percentage of words is smaller, the formula numerator... The larger the denominator The smaller the structural weight The larger. The importance of module functions is automatically measured in a data-driven manner, avoiding subjective human intervention and providing objective and reproducible quantitative basis for subsequent improvement of the loss function.

[0050] Step S5: Improve the ProphetNet model and enable automatic generation of job descriptions.

[0051] Based on the calculated structural weights An adaptive weight improvement was implemented on the n-tuple prediction loss function of the ProphetNet model. For each word in the generated sequence, the corresponding structural weight was queried based on its module label. This is used as a weighting coefficient for the predicted loss at that location, giving high-structure weights such as job responsibilities and job requirements. In this module, the prediction error of each word receives a larger gradient contribution during backpropagation, driving the model to concentrate more parameter update resources on core modules with high semantic density and high prediction difficulty, thus improving the ProphetNet model. Specifically, the expression for the n-tuple prediction loss function is: ; in, This represents the total prediction loss of the improved ProphetNet model, i.e., the adaptively weighted target loss function; This represents the time step in the sequence generation process, i.e., the nth time step in the generated sequence. The position index of each word; Indicates at time step The module tag corresponding to the word generated at the location, that is, the word belongs to the module of job responsibilities, job requirements or salary and benefits; Indicates time step The structural weights corresponding to module m to which the generated words belong; Indicates at time step The original ProphetNet model calculates the standard prediction loss based on the future n-tuple prediction mechanism, which in this embodiment is the cross-entropy loss.

[0052] The improved ProphetNet model is trained to obtain a text generation model. During the training process, after each training epoch, the n-tuple prediction loss and module completeness rate of the current model are calculated on the test set. An early stopping strategy is implemented based on the changing trend of the test set loss. Training is terminated when the test set loss no longer decreases for 5 consecutive training epochs to prevent the model from overfitting on the training set and to ensure that the improved ProphetNet model has good generalization ability.

[0053] When applying the text generation model to automatically generate job descriptions, the system uses job title, industry, job level, work location, and core skill keywords as structured input. After word segmentation and word vector embedding, the data is fed into the encoder for semantic representation. The decoding stage employs an autoregressive generation strategy: the model first generates start identifiers for each functional module, and then generates the content of each module word by word under the guidance of an improved loss function. A bundle search and temperature parameter are used to balance generation quality and diversity, with the bundle width set to 5 and the temperature parameter set to 0.8. After generation, the output sequence is formatted into a standard three-part text structure of "Job Responsibilities—Job Requirements—Salary and Benefits" based on the module identifiers. Post-processing removes duplicate sentences and formatting errors before outputting the final result, achieving automated generation of job descriptions that are structurally complete, semantically clear, and have well-defined module boundaries.

[0054] The technical effects of this invention can also be illustrated in conjunction with the accompanying drawings. Figure 1 The graph shows a comparison of the aggregation degree of the three modules. As can be seen from the graph, the aggregation degree of the job responsibility module and the job requirements module is significantly higher than that of the salary and benefits module. This indicates that the former two have stronger lexical specificity and semantic purity, which verifies the theoretical hypothesis that there are significant differences in the aggregation features among different functional modules. It proves that constructing the aggregation degree of modules can effectively quantify this difference and provide a reliable data basis for subsequent adaptive weight allocation.

[0055] Figure 2 Weights of each module A comparison chart of the equally weighted benchmark with the original ProphetNet model. The chart shows the structural weights of the "Job Responsibilities and Job Requirements" module. All were significantly higher than the equal-weighted benchmark, while the salary and benefits module was lower than the benchmark, indicating structural weighting. It can automatically identify modules with high prediction difficulty based on data-driven methods, overcoming the shortcomings of the existing ProphetNet model, which cannot distinguish the differences in module importance due to equal weights.

[0056] Figure 3This chart compares the cross-module semantic confusion rates of the original ProphetNet model and the improved ProphetNet model across different modules. As shown in the chart, the improved model has significantly lower cross-module semantic confusion rates than the original ProphetNet model in the modules of job responsibilities, job requirements, and salary / benefits, especially in the two modules with high structural weights: job responsibilities and requirements. The module showed the largest decrease, directly verifying that this method utilizes structural weights. With boundary jump degree By jointly improving the loss function, the core technical problem of cross-module semantic confusion can be effectively solved.

[0057] The above are all preferred embodiments of the present invention and are not intended to limit the scope of protection of the present invention. Therefore, all equivalent changes made in accordance with the structure, shape and principle of the present invention should be covered within the scope of protection of the present invention.

Claims

1. A method for automatically generating job descriptions based on deep learning, characterized in that, Including the following steps: The acquired job description text data is preprocessed to obtain multiple modules, and corresponding module labels are attached to the word segments in each module. Based on the probability distribution of words within each module, a module aggregation degree reflecting the semantic purity within the module is constructed. Boundary local word sequences between adjacent modules are extracted and information entropy is calculated. Combined with the module aggregation degree, a boundary jump degree is constructed. The boundary jump degree is positively correlated with the absolute value of the difference between the module aggregation degrees of adjacent modules and is also positively correlated with information entropy. Based on the boundary jump degree and the word count ratio of each module, the structural weights of each module are constructed. The structural weights are used as adaptive weighting coefficients to improve the prediction loss function of the ProphetNet model, resulting in an improved ProphetNet model for generating job description text. In the step of constructing structural weights, the average boundary jump degree is obtained by taking the average of the boundary jump degrees on both sides of each module. The structural weights are positively correlated with the average boundary jump degree and inversely proportional to the proportion of words in the module.

2. The method for automatically generating job descriptions based on deep learning according to claim 1, characterized in that, The steps for preprocessing the acquired job description text data to obtain multiple modules are as follows: use regular expressions to clean HTML tags, special symbols, and abnormal length samples; use jieba word segmentation combined with a custom dictionary in the recruitment field to process the job description text to obtain Chinese word segments; and split the Chinese word segments of each job description into three modules: job responsibilities, job requirements, and salary and benefits, based on the labeled boundary information.

3. The method for automatically generating job descriptions based on deep learning according to claim 2, characterized in that, After preprocessing the acquired job description text data to obtain multiple modules, the process also includes: using the Word2Vec model to convert Chinese word segmentation into word vectors.

4. The method for automatically generating job descriptions based on deep learning according to claim 1, characterized in that, The expression for module aggregation degree is: ； in, For module Module aggregation degree; The number of high-frequency words within the module participating in the calculation; For module Chinese word frequency ranking The word; For words With modules The joint probability; For words The marginal probability; For module The percentage of words in the total number of words.

5. The method for automatically generating job descriptions based on deep learning according to claim 1, characterized in that, The steps for extracting the boundary local word sequence between adjacent modules and calculating the information entropy are as follows: construct the adjacent module pair sequence according to the standard module order of the job description; for each pair of adjacent modules, extract a set number of words before and after the marked boundary position to form the boundary local word sequence; count the occurrence frequency of each word in the boundary local word sequence, record the total number of non-repeating words in the local vocabulary, and calculate the Shannon information entropy of the boundary local word sequence based on the occurrence frequency.

6. The method for automatically generating job descriptions based on deep learning according to claim 5, characterized in that, The method for calculating the boundary jump degree is as follows: calculate the difference in the aggregation degree of two adjacent modules and take the absolute value of the difference; take the natural logarithm of the total number of non-repeating words and use the logarithmic result as the normalization coefficient; multiply the absolute value by the Shannon information entropy and divide the product by the normalization coefficient to obtain the boundary jump degree.

7. The method for automatically generating job descriptions based on deep learning according to claim 1, characterized in that, The method for calculating structure weights is as follows: For each module, obtain all boundary jump degrees formed by the module and its neighboring modules, and average the obtained boundary jump degrees to obtain the average boundary jump degree of the corresponding module; count the number of words in each module in the training corpus, calculate the proportion of the number of words to the total number of words in the training set, and obtain the word count proportion of the module; calculate the sum of the average boundary jump degrees of all modules and use it as the normalized denominator; divide the average boundary jump degree of a single module by its own word count proportion, and then divide it by the normalized denominator to obtain the structure weight of the corresponding module.

8. The method for automatically generating job descriptions based on deep learning according to claim 1, characterized in that, The expression for the prediction loss function of the ProphetNet model is: ； in, This represents the total prediction loss of the improved ProphetNet model; Indicates the time step in the sequence generation process; Indicates time step The structural weights corresponding to module m to which the generated words belong; Indicates at time step The original ProphetNet model calculates the standard prediction loss based on the future n-tuple prediction mechanism.

9. The method for automatically generating job descriptions based on deep learning according to claim 1, characterized in that, After obtaining the improved ProphetNet model, the process also includes: training the improved ProphetNet model to obtain a text generation model. During the training process, after each training round, the n-tuple prediction loss and module completeness rate of the current model are calculated on the test set. An early stopping strategy is implemented based on the changing trend of the test set loss. Training is terminated when the test set loss no longer decreases for several consecutive training rounds.

10. The method for automatically generating job descriptions based on deep learning according to claim 9, characterized in that, The steps to generate job description text are as follows: Input the job title, industry, job level, work location and core skill keywords into the text generation model. The text generation model first generates the start identifier of each functional module, and then generates the content of each module word by word under the guidance of the improved loss function. After generation, the output sequence is formatted into a standard three-part text of job responsibilities, job requirements, and salary based on the module identifier. After post-processing to remove duplicate sentences and formatting errors, the final result is output.

Images