A talent trend analysis method based on big data

Abstract

The present invention discloses a talent trend analysis method based on big data, which relates to the field of human resource management technology. The method comprises the following steps: obtaining structured data, semi-structured data, and unstructured data of a position in an industry; processing the data into comprehensive data X; generating a context factor set C, and constructing a fusion feature using X and C; training a Stacking ensemble learning model, adjusting the weight vector R in the fusion feature based on Lasso regression, and obtaining a prediction model for predicting the number of job postings within a future time period; using recruitment strategies as individuals to generate a population, and optimizing the recruitment strategy based on a genetic algorithm to obtain the optimal recruitment strategy. The present invention fuses multi-source data and context factors to construct a fusion feature, introduces Lasso regression to dynamically adjust the context factor weights, and uses a genetic algorithm to optimize the decision structure, thereby significantly improving the prediction accuracy and adaptability of the recruitment decision support system, and providing an efficient solution for talent demand prediction and recruitment strategy formulation in complex situations.

Description

Technical Field

[0001] The present invention relates to the technical field of human resource management, and in particular to a talent trend analysis method based on big data. Background Art

[0002] In recent years, big data-driven contextual reasoning technology has demonstrated tremendous potential in areas such as recruitment decision support, talent demand forecasting, and economic trend analysis. Combining contextual reasoning with prediction algorithms, by introducing contextual factor weighting adjustments and optimizing decision structures, not only can prediction accuracy be improved, but the system's dynamic adaptability can also be enhanced, thereby supporting more efficient talent demand analysis and recruitment strategy formulation. In the field of recruitment decision support, ensuring the accuracy of prediction results and the rationality of decision structures is crucial for optimizing corporate resource allocation and improving recruitment efficiency. However, traditional prediction models have limitations in addressing the dynamic changes of complex contextual factors and optimizing decision structures, resulting in a lack of flexibility and robustness in practical applications.

[0003] XGBoost (eXtreme Gradient Boosting) is an optimized gradient boosting decision tree (GBDT) algorithm that aims to build a powerful ensemble model by integrating multiple weak learners (usually decision trees).

[0004] Random Forest is an ensemble machine learning algorithm used for classification and regression. It improves the accuracy and robustness of the model by combining the prediction results of multiple decision trees.

[0005] SVM: Support Vector Machine (SVM) is a generalized linear classifier that performs binary classification on data using supervised learning. Summary of the Invention

[0006] The purpose of the present invention is to provide a talent trend analysis method based on big data that solves the above problems, integrates multi-source data, responds to changes in the external environment in a timely manner, and makes predictions more accurate and comprehensive.

[0007] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is as follows: a talent trend analysis method based on big data, comprising the following steps;

[0008] S1, determine the industry and position to be analyzed, and obtain the structured data D of the positions in the industry s , semi-structured data D hs and unstructured data D u , including S11~S13;

[0009] S11, obtain n from the recruitment platform s Recruitment information for each position, and obtain structured features from each recruitment information to form structured data D s , , where the structural features include the number of job postings in the past ΔT time, recruitment cycle, salary level, GDP growth rate, unemployment rate, and industry growth rate, is the dimension of structured features;

[0010] S12, preset m related to the position hs -1 keyword, obtained through the Internet hs The semi-structured features are obtained from each text message to form the semi-structured data D. hs , The semi-structured features include the number of job postings in the past ΔT time, m hs -1 keyword frequency; the text information is sourced from recruitment platforms, social media, and news information;

[0011] S13, obtain n related to the position in the past ΔT time from the network t Descriptive text and n img image data, respectively constitute the text set D t and image set D img , merge D t 、D img Get D u ;

[0012] S2, based on normalization processing and feature extraction, generates comprehensive industry data X;

[0013] To D s Normalize the data to get X s , for D hs 、D t 、D img Perform feature extraction respectively to obtain the corresponding feature X hs 、X t 、X img , spliced ​​into comprehensive position data X, , treat each column of data in X as 1 sample and generate a label for each sample, which is the true value of the number of job postings in the future ΔT time period of the sample;

[0014] S3, generating a context factor set C, including steps S31 to S34;

[0015] S31, customize M context words, context factor c of the mth context word mThe initial value is 0 and the weight is w m , get the initial context factor set ,Collect industry-related policy documents and news articles to form a text dataset;

[0016] S32, identifying context words in sequence based on the rule engine, and if a context word is included in the text dataset, setting the corresponding context factor to 1;

[0017] S33, extract context words in sequence based on the BERT model. If a context word is included in the text dataset, the corresponding context factor is set to 1;

[0018] S34, update based on the results of S32 and S33 , get the context factor set C;

[0019] S4, generate the weight vector of the context factor set C , generate fusion feature X fusion , X fusion =[X,R];

[0020] S5, use X fusion Train the Stacking ensemble learning model and adjust the weight vector R in the fusion feature based on Lasso regression to obtain a prediction model that outputs the predicted value of the number of job postings in the future ΔT timeframe of the sample.

[0021] S6, the recruiting unit constructs a sample to be tested, inputs it into the prediction model, and obtains the predicted value;

[0022] S7, taking the recruitment strategy as an individual to generate a population, optimizing the recruitment strategy based on the genetic algorithm to obtain the optimal recruitment strategy;

[0023] Said recruitment strategy includes the number of job postings, salary adjustments, and channel optimization;

[0024] The fitness function of the genetic algorithm is , where Fitness(S k ) is the kth recruitment strategy S in the population k Fitness, Cost(S k )、Match(S k ,X fusion ) are S k The matching degree between recruitment cost and talent demand, α and β are Cost(S k )、Match(S k ,X fusion )’s weight;

[0025] In the genetic algorithm, individuals with small fitness are regarded as dominant individuals, and after the optimization is completed, the individuals with the smallest fitness are regarded as the optimal recruitment strategy.

[0026] As a preference: In S2, D is calculated based on the mean and standard deviation. s Perform normalization and get X s ;

[0027] Based on TF-IDF algorithm, D hs , generate the corresponding feature X hs ;

[0028] Extract D based on BERT model t Features of X t , ;

[0029] Extract D based on ResNet network img Features of X img , ;

[0030] , , .

[0031] As a preferred embodiment: step S5 specifically includes steps S51 to S53;

[0032] S51, obtain a Stacking ensemble learning model, including three base models and one meta-model. The three base models are XGBoost model, random forest model, and SVM model, which are marked as M1 to M3 respectively. The meta-model is Lasso regression model.

[0033] S52, base model is X fusion As input, output is the predicted value of the number of job postings in the future ΔT period;

[0034] The meta-model takes the fusion feature P as input and outputs the predicted value of the number of job postings in the future ΔT time, where ), 、 、 They are the outputs of M1~M3 respectively;

[0035] S53, using fusion feature X fusion Train the Stacking ensemble learning model until convergence to obtain a prediction model.

[0036] As a preference: the objective function g of the meta-model Meta (P,θ Meta )for;

[0037] ,

[0038] Where, β0 is the bias of the model, β jis the jth base model M j The weight of , 1≤j≤3, M j To X fusion The predicted value of the i-th sample in , y i is the label of the i-th sample, 1≤i≤n, and λ is the regularization parameter.

[0039] As a preference: In S7, Cost (S k )=C s +C r , Match(S k ,X fusion )=N predicted -N actual ;

[0040] Where C s 、C r S k The corresponding labor cost and recruitment channel fee, N predicted is the predicted value of the prediction model for the test sample, N actual For S k Number of job postings.

[0041] As a preferred embodiment: in S6, the recruiting unit constructs the sample to be tested as follows;

[0042] Generate a structured feature, semi-structured feature, descriptive text or image data according to your own recruitment needs. If it is a structured feature, normalize it to obtain a feature x. Otherwise, perform feature extraction to obtain feature x, and then generate the test sample x. fusion =[x,R].

[0043] Compared with the existing technology, the advantages of the present invention are: targeting the talent demand forecasting needs in complex situations, proposing a talent trend analysis method based on big data, and generating the optimal recruitment strategy. Specifically:

[0044] (1) Situational reasoning and multi-source data fusion: Obtaining multi-source data X from structured, semi-structured, and unstructured data, including but not limited to historical recruitment data, social media information, news reports, and text and image data, can comprehensively understand the changing trends of talent demand from multiple dimensions, making the model's predictions in complex recruitment environments more accurate and comprehensive.

[0045] (2) Automatic extraction and dynamic adjustment of contextual factor weights: Based on the rule engine and the BERT model, contextual words are automatically obtained, contextual factors are extracted, and a weight vector R is generated. R is then integrated with multi-source data X. The contextual factor weights are recalibrated through Lasso regression. This not only effectively handles the correlation and redundancy issues between contextual factors, improving the predictive model's adaptability to complex situations, but also enables the model to have a high degree of dynamic adaptability and to respond promptly to changes in the external environment.

[0046] (3) Optimizing the decision structure: Genetic algorithms are introduced to optimize the decision structure of the prediction model. By simulating the natural selection process, genetic algorithms iteratively search for the optimal combination of decision parameters, thereby ensuring that the system can generate the optimal recruitment strategy under multi-situational and multi-objective conditions.

[0047] In summary, this patent significantly improves the forecasting accuracy and adaptability of recruitment decision support systems by introducing Lasso regression to dynamically adjust the weights of contextual factors and employing genetic algorithms to optimize decision structures. This provides a highly efficient solution for talent demand forecasting and recruitment strategy formulation in complex scenarios. This innovation not only addresses the limitations of traditional methods in terms of dynamic adaptation of contextual factors and optimization of decision structures, but also expands the application potential of big data-driven technologies in areas such as recruitment decision support, economic trend analysis, and policy impact assessment. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 It is a flow chart of the present invention. DETAILED DESCRIPTION

[0049] The present invention will be further described below with reference to the embodiments and accompanying drawings.

[0050] Example 1: See Figure 1 ,A talent trend analysis method based on big data, comprising the following steps;

[0051] S1, determine the industry and position to be analyzed, and obtain the structured data D of the positions in the industry s , semi-structured data D hs and unstructured data D u , including S11~S13;

[0052] S11, obtain n from the recruitment platform s Recruitment information for each position, and obtain structured features from each recruitment information to form structured data D s , , where the structural features include the number of job postings in the past ΔT time, recruitment cycle, salary level, GDP growth rate, unemployment rate, and industry growth rate, is the dimension of structured features;

[0053] S12, preset m related to the position hs -1 keyword, obtained through the Internet hs The semi-structured features are obtained from each text message to form the semi-structured data D. hs , The semi-structured features include the number of job postings in the past ΔT time, m hs -1 keyword frequency; the text information is sourced from recruitment platforms, social media, and news information;

[0054] S13, obtain n related to the position in the past ΔT time from the network t Descriptive text and n img image data, respectively constitute the text set D t and image set D img , merge D t 、D img Get D u ;

[0055] S2, based on normalization processing and feature extraction, generates comprehensive industry data X;

[0056] To D s Normalize the data to get X s , for D hs 、D t 、D img Perform feature extraction respectively to obtain the corresponding feature X hs 、X t 、X img , spliced ​​into comprehensive position data X, , treat each column of data in X as 1 sample and generate a label for each sample, which is the true value of the number of job postings in the future ΔT time period of the sample;

[0057] S3, generating a context factor set C, including steps S31 to S34;

[0058] S31, customize M context words, context factor c of the mth context word m The initial value is 0 and the weight is w m , get the initial context factor set ,Collect industry-related policy documents and news articles to form a text dataset;

[0059] S32, identifying context words in sequence based on the rule engine, and if a context word is included in the text dataset, setting the corresponding context factor to 1;

[0060] S33, extract context words in sequence based on the BERT model. If a context word is included in the text dataset, the corresponding context factor is set to 1;

[0061] S34, update based on the results of S32 and S33 , get the context factor set C;

[0062] S4, generate the weight vector of the context factor set C , generate fusion feature X fusion , X fusion =[X,R];

[0063] S5, use X fusion Train the Stacking ensemble learning model and adjust the weight vector R in the fusion feature based on Lasso regression to obtain a prediction model that outputs the predicted value of the number of job postings in the future ΔT timeframe of the sample.

[0064] S6, the recruiting unit constructs a sample to be tested, inputs it into the prediction model, and obtains the predicted value;

[0065] S7, taking the recruitment strategy as an individual to generate a population, optimizing the recruitment strategy based on the genetic algorithm to obtain the optimal recruitment strategy;

[0066] Said recruitment strategy includes the number of job postings, salary adjustments, and channel optimization;

[0067] The fitness function of the genetic algorithm is , where Fitness(S k ) is the kth recruitment strategy S in the population k Fitness, Cost(S k )、Match(S k ,X fusion ) are S k The matching degree between recruitment cost and talent demand, α and β are Cost(S k )、Match(S k ,X fusion )’s weight;

[0068] In the genetic algorithm, individuals with small fitness are regarded as dominant individuals, and after the optimization is completed, the individuals with the smallest fitness are regarded as the optimal recruitment strategy.

[0069] In this implementation, in S2, based on the mean and standard deviation of D s Perform normalization and get X s ; Process D based on TF-IDF algorithm hs , generate the corresponding feature X hs ; Extract D based on BERT model t Features of X t, ; Extract D based on ResNet network img Features of X img , ; , , .

[0070] Step S5 specifically includes steps S51 to S53;

[0071] S51, obtain a Stacking ensemble learning model, including three base models and one meta-model. The three base models are XGBoost model, random forest model, and SVM model, which are marked as M1 to M3 respectively. The meta-model is Lasso regression model.

[0072] S52, base model is X fusion As input, output is the predicted value of the number of job postings in the future ΔT period;

[0073] The meta-model takes the fusion feature P as input and outputs the predicted value of the number of job postings in the future ΔT time, where ), 、 、 They are the outputs of M1~M3 respectively;

[0074] S53, using fusion feature X fusion Train the Stacking ensemble learning model until convergence to obtain a prediction model.

[0075] The objective function g of the metamodel Meta (P,θ Meta )for;

[0076] ,

[0077] Where, β0 is the bias of the model, β j is the jth base model M j The weight of , 1≤j≤3, M j To X fusion The predicted value of the i-th sample in , y i is the label of the i-th sample, 1≤i≤n, and λ is the regularization parameter.

[0078] In S6, the recruiting unit constructs the sample to be tested as follows;

[0079] Generate a structured feature, semi-structured feature, descriptive text or image data according to your own recruitment needs. If it is a structured feature, normalize it to obtain a feature x. Otherwise, perform feature extraction to obtain feature x, and then generate the test sample x. fusion =[x,R].

[0080] In S7, Cost(S k )=C s +C r , Match(S k ,X fusion )=N predicted -N actual ;

[0081] Where C s 、C r S k The corresponding labor cost and recruitment channel fee, N predicted is the predicted value of the prediction model for the test sample, N actual For S k Number of job postings.

[0082] Example 2: See Figure 1 , taking the software development industry and software engineer positions as an example.

[0083] About the structured data D in S1 s , semi-structured data D hs and unstructured data D u , specifically:

[0084] Structured data: obtained from recruitment information on recruitment platforms. Structured features include the number of job postings in the past ΔT period, recruitment cycle, salary level, GDP growth rate, unemployment rate, and industry growth rate. For example, if ΔT is 3 months, then the number of software engineer-related jobs in the past 3 months will be collected and counted. s Recruitment information, if n s =300, one of the structured features is: the number of job postings is 20, the recruitment cycle is 30 days, the salary level is 12,000 yuan, the GDP growth rate is 3.5%, the unemployment rate is 5%, and the industry growth rate is 10%; the dimension of each feature is 1, so the dimension of the structured feature is m s =6, we get , containing 300 structured feature samples.

[0085] Semi-structured data: preset m related to the position hs -1 keyword, assuming m hs =9, then 8 keywords are preset, such as deep learning, machine learning, AI talent demand, autonomous driving, etc. Recruiting units can set them according to their actual needs. hs= 150 text messages containing at least one keyword, then one text message is used to construct a semi-structured feature sample. For one text message, first obtain the number of job postings recorded in the past three months, assuming it is 10, and then use the TF-IDF method to count the frequency of each keyword. For example, the frequency of deep learning is 3‰, machine learning is 1‰, AI talent demand is 0, and autonomous driving is 0. Then 10, 3‰, 1‰, 0, 0 constitute a semi-structured feature sample with a feature dimension of 5. For 150 text messages, the generated semi-structured data .

[0086] Unstructured data u , get n related to the position in the past 3 months from the Internet t = 100 descriptive texts and n img =50 image data, each constituting a text set D t and image set D img , merge D t 、D img Get D u ;

[0087] Regarding step S2, generating comprehensive industry data X;

[0088] To D s Normalize and get X s , for D hs 、D t 、D img Feature extraction is performed to obtain X hs 、X t 、X img , and then spliced ​​into , due to the feature X extracted by the BERT model and ResNet t 、X img The dimension is 512. When concatenating X, for X s and X hs Where the dimension is less than 512, it is padded with zeros to make it 512. In this embodiment, =300+150+100+50=600, =6+5+512+512.

[0089] Regarding step S3, this embodiment takes two as an example, and customizes two context words, namely policy support and technological innovation. The recruiting unit can set it according to its own needs. The context factors c1 and c2 of these two context words are initially 0, and the weights are w1 and w2. In S32, the context words are identified in sequence based on the rule engine. If the text data set contains "policy support", then c1=1; if it does not contain technological innovation, then c2=0. In S33, context words are extracted in sequence based on the BERT model. Assuming that the BERT model recognizes "technological innovation", then c2=1. Finally, S34 is updated based on the results of S32 and S33. , and obtain the context factor set C={c1=1, c2=1}.

[0090] Regarding step S4, if w1=0.7, w2=0.6, then .

[0091] Example 3: See Figure 1 Based on Example 1, this example provides a specific operation flow of step S7, including S71 to S76;

[0092] S71, randomly initialize the population , S contains N randomly generated individuals , each individual corresponds to a recruitment strategy, for example:

[0093] ,

[0094] ,

[0095] ,

[0096] In the recruitment strategy, the values ​​for the number of job postings, salary adjustments, and channel optimization are all randomly generated and within pre-set limits;

[0097] S72, fitness assessment;

[0098] Define the fitness function of the genetic algorithm The fitness function is used to measure the quality of an individual. In the present invention, the smaller the fitness calculated by the fitness function, the better the individual. In the subsequent selection process, individuals with high fitness are more likely to be selected into the next generation.

[0099] S73, selection operation: select individuals to enter the next generation based on their fitness values. Common methods include roulette wheel selection, tournament selection, etc. In the present invention, individuals with smaller fitness values ​​are more likely to be selected;

[0100] S74, crossover, mutation, replacement and other operations generate new populations;

[0101] S75, check whether the termination conditions are met. Common termination conditions include reaching the maximum number of iterations, fitness reaching a preset target, population fitness change less than a certain threshold, etc. If not met, return to step S72, otherwise execute S76;

[0102] S76, output the individual with the smallest fitness in the population as the optimal solution, that is, the optimal recruitment strategy In this embodiment,

[0103] .

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions and improvements made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A talent trend analysis method based on big data, characterized by: The following steps are included: S1, determine the industry and position to be analyzed, and obtain the structured data D of the positions in the industry s , semi-structured data D hs and unstructured data D u , including S11~S13; S11, obtain n from the recruitment platform s Recruitment information for each position, and obtain structured features from each recruitment information to form structured data D s , , Among them, the structural features include the number of job postings in the past ΔT period, recruitment cycle, salary level, GDP growth rate, unemployment rate, industry growth rate, is the dimension of structured features; S12, preset m related to the position hs -1 keyword, obtained through the Internet hs The semi-structured features are obtained from each text message to form the semi-structured data D. hs , The semi-structured features include the number of job postings in the past ΔT time, m hs -1 keyword frequency; the text information is sourced from recruitment platforms, social media, and news information; S13, obtain n related to the position in the past ΔT time from the network t Descriptive text and n img image data, respectively constitute the text set D t and image set D img , merge D t 、D img Get D u ; S2, based on normalization processing and feature extraction, generates comprehensive industry data X; To D s Normalize the data to get X s , for D hs 、D t 、D img Perform feature extraction respectively to obtain the corresponding feature X hs 、X t 、X img , spliced ​​into comprehensive position data X, , treat each column of data in X as 1 sample and generate a label for each sample, which is the true value of the number of job postings in the future ΔT time period of the sample; S3, generating a context factor set C, including steps S31 to S34; S31, customize M context words, context factor c of the mth context word m The initial value is 0 and the weight is w m , get the initial context factor set ,Collect industry-related policy documents and news articles to form a text dataset; S32, identifying context words in sequence based on the rule engine, and if a context word is included in the text dataset, setting the corresponding context factor to 1; S33, extract context words in sequence based on the BERT model. If a context word is included in the text dataset, the corresponding context factor is set to 1; S34, update based on the results of S32 and S33 , get the context factor set C; S4, generate the weight vector of the context factor set C , generate fusion feature X fusion , X fusion =[X,R]; S5, use X fusion Train the Stacking ensemble learning model and adjust the weight vector R in the fusion feature based on Lasso regression to obtain a prediction model that outputs the predicted value of the number of job postings in the future ΔT timeframe of the sample. S6, the recruiting unit constructs a sample to be tested, inputs it into the prediction model, and obtains the predicted value; S7, taking the recruitment strategy as an individual to generate a population, optimizing the recruitment strategy based on the genetic algorithm to obtain the optimal recruitment strategy; The recruitment strategy includes the number of job postings, salary adjustments, and channel optimization; The fitness function of the genetic algorithm is , where Fitness(S k ) is the kth recruitment strategy S in the population k Fitness, Cost(S k )、Match(S k ,X fusion ) are S k The matching degree between recruitment cost and talent demand, α and β are Cost(S k )、Match(S k ,X fusion )’s weight; In the genetic algorithm, individuals with small fitness are regarded as dominant individuals, and after the optimization is completed, the individuals with the smallest fitness are regarded as the optimal recruitment strategy.

2. The talent trend analysis method based on big data according to claim 1, characterized in that: In S2, based on the mean and standard deviation of D s Perform normalization and get X s ; Based on TF-IDF algorithm to process D hs , generate the corresponding feature X hs ; Extract D based on BERT model t Features of X t , ; Extract D based on ResNet network img Features of X img , ; ， ， 。 3. The talent trend analysis method based on big data according to claim 1, characterized in that: Step S5 specifically includes steps S51 to S53; S51, obtain a Stacking ensemble learning model, including three base models and one meta-model. The three base models are XGBoost model, random forest model, and SVM model, which are marked as M1 to M3 respectively. The meta-model is Lasso regression model. S52, base model is X fusion As input, output is the predicted value of the number of job postings in the future ΔT period; The meta-model takes the fusion feature P as input and outputs the predicted value of the number of job postings in the future ΔT time, where ), 、 、 They are the outputs of M1~M3 respectively; S53, using fusion feature X fusion Train the Stacking ensemble learning model until convergence to obtain a prediction model.

4. The talent trend analysis method based on big data according to claim 3, characterized in that: The objective function g of the metamodel Meta (P,θ Meta )for; ， Where, β0 is the bias of the model, β j is the jth base model M j The weight of , 1≤j≤3, M j To X fusion The predicted value of the i-th sample in , y i is the label of the i-th sample, 1≤i≤n, and λ is the regularization parameter.

5. The talent trend analysis method based on big data according to claim 1, characterized in that: In S7, Cost(S k ) = C s + C r , Match(S k , X fusion ) = N predicted - N actual ; Where C s 、C r S k The corresponding labor cost and recruitment channel fee, N predicted is the predicted value of the prediction model for the test sample, N actual For S k Number of job postings.

6. The talent trend analysis method based on big data according to claim 1, characterized in that: In S6, the recruiting unit constructs the sample to be tested as follows; Generate a structured feature, semi-structured feature, descriptive text or image data according to your own recruitment needs. If it is a structured feature, normalize it to obtain a feature x. Otherwise, perform feature extraction to obtain feature x, and then generate the test sample x. fusion =[x,R].

Images