Talent trend analysis method based on big data

Abstract

The invention discloses a talent trend analysis method based on big data, and relates to the technical field of human resource management, and the method comprises the steps: obtaining structured data, semi-structured data and non-structured data of a position in an industry; processing the data into comprehensive data X; generating a situation factor set C, and constructing fusion features by using X and C; training a Stacking ensemble learning model, and adjusting a weight vector R in the fusion features based on Lasso regression to obtain a prediction model for predicting the number of positions published in a future time period; and taking the recruitment strategy as an individual to generate a population, and optimizing the recruitment strategy based on a genetic algorithm to obtain an optimal recruitment strategy. The multi-source data and the situation factors are fused to construct fusion features, Lasso regression is introduced to dynamically adjust the weight of the situation factors, and the decision structure is optimized by adopting the genetic algorithm, so that the prediction precision and the adaptive capacity of the recruitment decision support system are remarkably improved; and an efficient solution is provided for talent demand prediction and recruitment strategy formulation in a complex situation.

Description

Technical Field

[0001] The present invention relates to the technical field of human resource management, and in particular, to a method for analyzing talent trends based on big data. Background Art

[0002] In recent years, big data-driven context reasoning technology has shown great potential in fields such as recruitment decision support, talent demand prediction, and economic trend analysis. By combining context reasoning and prediction algorithms, and through introducing context factor weight adjustment and optimizing the decision-making structure, not only can the accuracy of prediction be improved, but also the dynamic adaptability of the system can be enhanced, thereby supporting more efficient talent demand analysis and recruitment strategy formulation. In the field of recruitment decision support, ensuring the accuracy of prediction results and the rationality of the decision-making structure is crucial for optimizing enterprise resource allocation and improving recruitment efficiency. However, traditional prediction models have limitations in dealing with the dynamic changes of complex context factors and optimizing the decision-making structure, resulting in insufficient flexibility and robustness in practical applications.

[0003] XGBoost (eXtreme Gradient Boosting) is an optimized gradient boosting decision tree (GBDT) algorithm, aiming to build a powerful integrated model by integrating multiple weak learners (usually decision trees).

[0004] Random Forest is an integrated machine learning algorithm used for classification and regression. It improves the accuracy and robustness of the model by combining the prediction results of multiple decision trees.

[0005] SVM: Support Vector Machine (SVM) is a class of generalized linear classifiers that perform binary classification on data in a supervised learning manner. Summary of the Invention

[0006] The object of the present invention is to provide a method for analyzing talent trends based on big data that solves the above problems, integrates multi-source data, responds to external environmental changes in a timely manner, and has more accurate and comprehensive predictions.

[0007] To achieve the above object, the technical solution adopted by the present invention is as follows: A method for analyzing talent trends based on big data, including the following steps; S1, determine the industry and position to be analyzed, and obtain the structured data D, s semi-structured data D, hs and unstructured data D u of the position in this industry, including S11~S13; S11. Obtain the recruitment information of n s job positions from the recruitment platform, and obtain the structured features from each recruitment information to form the structured data D s , , where the structured features include the number of job postings in the past ΔT duration, recruitment cycle, salary level, GDP growth rate, unemployment rate, and industry growth rate, which are the dimensions of the structured features; S12. Preset m hs - 1 keywords related to the job positions, and obtain n hs text information containing at least 1 keyword through the network, and obtain the semi-structured features from each text information to form the semi-structured data D hs , , the semi-structured features include the number of job postings in the past ΔT duration, m hs - 1 keyword frequencies; the text information is sourced from recruitment platforms, social media, and news; S13. Obtain n t descriptive texts and n img image data related to the job positions in the past ΔT duration from the network, and form the text set D t and the image set D img , and merge D t , D img to obtain D u ; S2. Generate the comprehensive industry data X based on normalization processing and feature extraction; Perform normalization processing on D s to obtain the data X s , perform feature extraction on D hs , D t , D img respectively to obtain the corresponding features X hs , X t , X img , and splice them into the job comprehensive data X, , take each column of data in X as 1 sample, and generate the label for each sample, where the label is the true value of the number of job postings in the future ΔT duration of this sample; S3. Generate the set of context factors C, including steps S31 to S34; S31. Customize M context words, and the initial value of the context factor c m of the m-th context word is 0 and the weight is w m , to obtain the initial set of context factors , collect the relevant policy documents and news articles in the industry to form the text data set; S32. Identify situation words sequentially based on the rule engine. If a certain situation word is included in the text dataset, the corresponding situation factor is set to 1; S33. Extract situation words sequentially based on the BERT model. If a certain situation word is included in the text dataset, the corresponding situation factor is set to 1; S34. Update according to the results of S32 and S33 to obtain the situation factor set C; S4. Generate the weight vector of the situation factor set C to generate the fused feature X fusion X fusion = [X, R]; S5. Use X fusion to train the Stacking ensemble learning model, adjust the weight vector R in the fused feature based on Lasso regression to obtain the prediction model, which is used to output the predicted value of the number of job postings in the future ΔT time period of the sample; S6. The recruitment unit constructs a sample to be tested, inputs it into the prediction model, and obtains the predicted value; S7. Take the recruitment strategy as an individual to generate a population, optimize the recruitment strategy based on the genetic algorithm to obtain the optimal recruitment strategy; The recruitment strategy includes the number of job postings, adjusted salary, and optimized channels; The fitness function of the genetic algorithm is , where Fitness(S k ) is the fitness of the k-th recruitment strategy S k in the population, Cost(S k ), Match(S k , X fusion ) are the recruitment cost and the talent demand matching degree of S k respectively, and α, β are the weights of Cost(S k ), Match(S k , X fusion ) respectively; And in the genetic algorithm, the individual with a smaller fitness is regarded as the dominant individual, and after the optimization ends, the individual with the smallest fitness is regarded as the optimal recruitment strategy.

[0008] As an optimization: In S2, normalize D s based on the mean and standard deviation to obtain X s ; Process D hs based on the TF-IDF algorithm to generate the corresponding feature X hs ; Extract the feature X t of D t based on the BERT model, ; Extract D based on the ResNet network img feature X img , ; , , .

[0009] Preferably: Step S5 specifically includes steps S51 to S53; S51, obtain a Stacking ensemble learning model, including 3 base models and 1 meta-model. The 3 base models are the XGBoost model, the random forest model, and the SVM model, which are sequentially labeled as M1 to M3, and the meta-model is the Lasso regression model; S52, the base models take X fusion as the input and output the predicted values of the number of job postings in the future ΔT duration; The meta-model takes the fused feature P as the input and outputs the predicted values of the number of job postings in the future ΔT duration, where ), , , are the outputs of M1 to M3 respectively; S53, use the fused feature X fusion to train the Stacking ensemble learning model until convergence to obtain a prediction model.

[0010] Preferably: For the meta-model, the objective function g Meta (P,θ Meta ) is; , In the formula, β0 is the bias of the meta-model, β j is the weight of the jth base model M j , 1 ≤ j ≤ 3, is the predicted value of the ith sample in X j by M fusion , y i is the label of the ith sample, 1 ≤ i ≤ n, and λ is the regularization parameter.

[0011] Preferably: In S7, Cost(S k ) = C s + C r , Match(S k , X fusion ) = N predicted - N actual ; In the formula, C s , C r are respectively S kThe corresponding labor cost and recruitment channel cost, N predicted Is the predicted value of the prediction model for the sample to be measured, N actual Is S k The number of job postings in S

[0012] Preferably: In S6, the recruitment unit constructs the sample to be measured specifically as follows; Generate a structured feature, semi-structured feature, descriptive text or image data according to its own recruitment needs. If it is a structured feature, perform normalization processing to obtain a feature x, otherwise perform feature extraction to obtain the feature x, and then generate the sample to be measured x fusion =[x, R].

[0013] Compared with the prior art, the advantages of the present invention are as follows: Aiming at the talent demand prediction requirement in complex scenarios, a talent trend analysis method based on big data is proposed to generate an optimal recruitment strategy. Specifically: (1) Situation reasoning and multi-source data fusion: Obtain multi-source data X from structured, semi-structured and unstructured data, including but not limited to historical recruitment data, social media information, news reports, and text and image data, which can comprehensively understand the changing trend of talent demand from multiple dimensions, making the prediction of the model for complex recruitment environments more accurate and comprehensive.

[0014] (2) Automatic extraction and dynamic adjustment of situation factor weights: Automatically obtain situation words and extract situation factors based on a rule engine and a BERT model to generate a weight vector R, and fuse R with multi-source data X. And recalibrate the situation factor weights through Lasso regression, which can not only effectively handle the correlation and redundancy problems between situation factors and improve the adaptability of the prediction model to complex situations, but also make the model have a high degree of dynamic adaptability and can respond to external environmental changes in a timely manner.

[0015] (3) Decision structure optimization, introduce a genetic algorithm to optimize the decision structure of the prediction model. The genetic algorithm iteratively searches for the optimal combination of decision parameters by simulating the natural selection process, so as to ensure that the system can generate an optimal recruitment strategy under multi-situation and multi-objective conditions.

[0016] In summary, this patent significantly improves the prediction accuracy and adaptability of the recruitment decision support system by introducing Lasso regression to dynamically adjust the situation factor weights and using a genetic algorithm to optimize the decision structure, providing an efficient solution for talent demand prediction and recruitment strategy formulation in complex scenarios. This innovation not only solves the limitations of traditional methods in dynamic adaptation of situation factors and decision structure optimization, but also expands the application potential of big data-driven technologies in fields such as recruitment decision support, economic trend analysis, and policy impact assessment. Description of the Drawings

[0017] Figure 1 This is the flow chart of the present invention. Detailed implementation manners

[0018] The present invention will be further described below in conjunction with embodiments and drawings.

[0019] Embodiment 1: Refer to Figure 1 , a talent trend analysis method based on big data, comprising the following steps; S1. Determine the industry and position to be analyzed, and obtain the structured data D s , semi-structured data D hs and unstructured data D u of the position in this industry, including S11~S13; S11. Obtain the recruitment information of n s positions from the recruitment platform, and obtain the structured features from each recruitment information to form the structured data D s , , wherein the structured features include the number of position releases, recruitment cycle, salary level, GDP growth rate, unemployment rate, and industry growth rate in the past ΔT period, being the dimension of the structured features; S12. Preset m hs - 1 keywords related to the position, and obtain n hs pieces of text information containing at least 1 keyword through the network, and obtain the semi-structured features from each text information to form the semi-structured data D hs , , the semi-structured features including the number of position releases in the past ΔT period and the frequency of m hs - 1 keywords; the text information is from the recruitment platform, social media, and news; S13. Obtain n t descriptive texts and n img image data related to the position in the past ΔT period from the network, and form a text set D t and an image set D img respectively, and merge D t , D img to obtain D u ; S2. Generate comprehensive industry data X based on normalization processing and feature extraction; Perform normalization processing on D s to obtain data X s , perform feature extraction on D hs , D t , D img respectively to obtain the corresponding features X hs , Xt and X img are spliced into the comprehensive job data X. Regarding each column of data in X as one sample, generate the label for each sample, where the label is the true value of the number of job postings in the future ΔT duration for this sample. S3. Generate the set of context factors C, including steps S31 to S34; S31. Define M context words, and the context factor c of the m-th context word m has an initial value of 0 and a weight of w m to obtain the initial set of context factors Collect industry-related policy documents and news articles to form a text data set. S32. Based on the rule engine, identify context words in sequence. If a certain context word is included in the text data set, the corresponding context factor is set to 1. S33. Based on the BERT model, extract context words in sequence. If a certain context word is included in the text data set, the corresponding context factor is set to 1. S34. Combine the results of S32 and S33 to update to obtain the set of context factors C; S4. Generate the weight vector of the set of context factors C to generate the fused feature X fusion X fusion = [X, R]; S5. Use X fusion to train the Stacking ensemble learning model, and adjust the weight vector R in the fused feature based on Lasso regression to obtain a prediction model for outputting the predicted value of the number of job postings in the future ΔT duration for the sample. S6. The recruitment unit constructs a sample to be tested and inputs it into the prediction model to obtain the predicted value. S7. Regard the recruitment strategy as an individual to generate a population, and optimize the recruitment strategy based on the genetic algorithm to obtain the optimal recruitment strategy. The recruitment strategy includes the number of job postings, adjusted salary, and optimized channels. The fitness function of the genetic algorithm is , where Fitness(S k ) is the fitness of the k-th recruitment strategy S in the population k , Cost(S k ), Match(S k , X fusion ) are the recruitment cost and the talent demand matching degree of S k respectively, and α, β are the weights of Cost(S k ) and Match(S k , X fusion ) respectively. In the genetic algorithm, individuals with small fitness are regarded as dominant individuals, and after the optimization is completed, the individual with the smallest fitness is regarded as the optimal recruitment strategy.

[0020] In this implementation, in S2, based on the mean and standard deviation, D s is normalized to obtain X s ; based on the TF-IDF algorithm, D hs is processed to generate the corresponding feature X hs ; based on the BERT model, the features X t of D t are extracted, ; based on the ResNet network, the features X img of D img are extracted, ; , , .

[0021] Step S5 specifically includes steps S51 to S53; S51, obtain a Stacking ensemble learning model, including 3 base models and 1 meta-model. The 3 base models are the XGBoost model, the random forest model, and the SVM model, which are sequentially labeled as M1 to M3, and the meta-model is the Lasso regression model; S52, the base models use X fusion as the input and output the predicted values of the number of job postings in the future ΔT period; The meta-model uses the fused feature P as the input and outputs the predicted values of the number of job postings in the future ΔT period, where ), , , are the outputs of M1 to M3 respectively; S53, use the fused feature X fusion to train the Stacking ensemble learning model until convergence to obtain the prediction model.

[0022] The objective function g Meta (P,θ Meta ) of the meta-model is; , In the formula, β0 is the bias of the meta-model, β j is the weight of the jth base model M j , 1 ≤ j ≤ 3, is the predicted value of the ith sample in X j by M fusion , y i is the label of the ith sample, 1 ≤ i ≤ n, and λ is the regularization parameter.

[0023] In S6, the recruitment unit constructs the sample to be measured specifically as follows: Generate a structured feature, semi-structured feature, descriptive text, or image data according to its own recruitment needs. If it is a structured feature, perform normalization processing to obtain a feature x. Otherwise, perform feature extraction to obtain feature x, and then generate the sample to be measured x fusion =[x, R].

[0024] In S7, Cost(S k ) = C s +C r , Match(S k , X fusion ) = N predicted -N actual ; In the formula, C s , C r are respectively the labor cost and recruitment channel cost corresponding to S k , N predicted is the predicted value of the prediction model for the sample to be measured, and N actual is the number of job postings in S k .

[0025] Example 2: Refer to Figure 1 , taking the software development industry and the software engineer position as an example.

[0026] Regarding the structured data D s , semi-structured data D hs and unstructured data D u in S1, specifically: Structured data: Obtained from the recruitment information on the recruitment platform. The structured features include the number of job postings in the past ΔT duration, recruitment cycle, salary level, GDP growth rate, unemployment rate, and industry growth rate. For example, if ΔT is 3 months, then collect and count the n s recruitment information related to software engineers in the past 3 months. If n s = 300, and one of them contains the following structured features: the number of job postings is 20, the recruitment cycle is 30 days, the salary level is 12,000 yuan, the GDP growth rate is 3.5%, the unemployment rate is 5%, and the industry growth rate is 10%; the dimension of each feature is 1 dimension, then the dimension m s = 6, obtaining , including 300 structured feature samples.

[0027] Semi-structured data: Preset m hs -1 keywords related to the position. Assume m hsIf it is 9, then 8 keywords are preset, such as deep learning, machine learning, AI talent demand, autonomous driving, etc., and the recruitment unit can set according to its actual needs. Assume that n hs = 150 pieces of text information containing at least one keyword are obtained through the network. Then one piece of text information is used to construct a semi-structured feature sample. For one piece of text information, first obtain the number of job postings in the past 3 months recorded in it, assume it is 10, and then use the TF-IDF method to count the frequency of each keyword. For example, the frequency of deep learning is 3‰, machine learning is 1‰, AI talent demand is 0, and autonomous driving is 0. Then 10, 3‰, 1‰, 0, 0 are used to form a semi-structured feature sample, and its feature dimension is 5. For 150 pieces of text information, the generated semi-structured data .

[0028] Unstructured data D u , obtain n t = 100 descriptive texts related to the position and n img = 50 image data from the network in the past 3 months, which respectively form the text set D t and the image set D img , and merge D t , D img to obtain D u ; Regarding step S2, generate comprehensive industry data X; Perform normalization processing on D s to obtain X s , perform feature extraction on D hs , D t , D img respectively to obtain X hs , X t , X img , and then splice them into . Since the features X t , X img extracted by the BERT model and ResNet have a dimension of 512, when splicing X, for the parts of X s and X hs that are less than 512 dimensions, pad them with zeros to 512. In this embodiment, = 300 + 150 + 100 + 50 = 600, = 6 + 5 + 512 + 512.

[0029] Regarding step S3, taking two cases in the first embodiment as an example, two custom scenario words are defined as policy support and technological innovation respectively. The recruitment unit can set according to its own needs. The initial values of the scenario factors c1 and c2 for these two scenario words are 0, and the weights are w1 and w2. In S32, based on the rule engine, the scenario words are identified in sequence. If the text dataset contains "policy support", then c1 = 1; if it does not contain technological innovation, then c2 = 0. In S33, based on the BERT model, the scenario words are extracted in sequence. Assuming the BERT model recognizes "technological innovation", then c2 = 1. Finally, in S34, the results of S32 and S33 are combined to update , and the scenario factor set C = {c1 = 1, c2 = 1} is obtained.

[0030] Regarding step S4, if w1 = 0.7 and w2 = 0.6, then .

[0031] Embodiment 3: Refer to Figure 1 , based on Embodiment 1, this embodiment gives a specific operation process for step S7, including S71 to S76; S71, randomly initialize the population , S contains N randomly generated individuals , and each individual corresponds to a recruitment strategy. For example: , , , In the recruitment strategy, the values of the number of job postings, adjusted salary, and optimized channels are all randomly generated and meet the preset limit range; S72, fitness evaluation; Define the fitness function of the genetic algorithm . This fitness function is used to measure the quality of an individual. In the present invention, the smaller the fitness calculated by the fitness function, the better the individual. In the subsequent selection process, individuals with higher fitness are more likely to be selected into the next generation; S73, selection operation; select individuals into the next generation according to the fitness value. Common methods include roulette wheel selection, tournament selection, etc. In the present invention, individuals with smaller fitness values are more likely to be selected; S74, operations such as crossover, mutation, and replacement are performed to generate a new population; S75, check whether the termination condition is met. Common termination conditions include reaching the maximum number of iterations, the fitness reaching the preset target, the change in population fitness being less than a certain threshold, etc. If not, return to step S72, otherwise execute S76; S76 outputs the individual with the minimum fitness in the population as the optimal solution, that is, the optimal recruitment strategy. , in this embodiment, .

[0032] The above are only the preferred embodiments of the present invention and are not intended to limit the present invention. Any modifications, equivalent replacements, and improvements made within the spirit and principle of the present invention shall be included within the protection scope of the present invention.

Claims

1. A talent trend analysis method based on big data, characterized in that: Including the following steps; S1. Determine the industry and position to be analyzed, and obtain the structured data D, semi-structured data D, and unstructured data D of the position in this industry, including S11 to S13; s and semi-structured data D hs and unstructured data D u , including S11 to S13; S11, Obtain the recruitment information of n s job positions from the recruitment platform, and obtain the structured features from each recruitment information to form structured data D s , , Among them, the structured features include the number of job postings in the past ΔT duration, recruitment cycle, salary level, GDP growth rate, unemployment rate, and industry growth rate, which are the dimensions of the structured features; S12. Preset m keywords related to the position hs -1 keywords, and obtain n hs pieces of text information each containing at least 1 keyword, and obtain semi-structured features from each piece of text information to form semi-structured data D hs , , where the semi-structured features include the number of job postings in the past ΔT duration, m hs -1 keyword frequencies; the text information is sourced from recruitment platforms, social media, and news S13, Obtain n descriptive texts and n image data related to the position within the past ΔT duration from the network, which respectively constitute the text set D t and the image set D img . Merge D t and D img to obtain D t ; img u ;​ S2. Generate comprehensive industry data X based on normalization processing and feature extraction; For D s perform normalization processing to obtain data X s and for D hs , D t , D img respectively perform feature extraction to obtain corresponding features X hs , X t , X img , concatenate them into the job comprehensive data X, and take each column of data in X as one sample to generate the label for each sample, where the label is the true value of the number of job postings in the future ΔT duration for this sample; S3. Generate a set of situation factors C, including steps S31 to S34; S31. Customize M scenario words, and the scenario factor c of the m-th scenario word m has an initial value of 0 and a weight of w m to obtain an initial set of scenario factors Collect industry-related policy documents and news articles to form a text data set; S32. Sequentially identify situation words based on a rule engine. If a certain situation word is included in the text dataset, the corresponding situation factor is set to 1; S33. Sequentially extract situation words based on the BERT model. If a certain situation word is included in the text dataset, the corresponding situation factor is set to 1; S34, update based on the results of S32 and S33 , to obtain the set C of context factors; S4, generate the weight vector of the situation factor set C , generate the fused feature X fusion , X fusion = [X, R]; S5, using X fusion Train a Stacking ensemble learning model, adjust the weight vector R in the fusion features based on Lasso regression to obtain a prediction model for outputting the predicted value of the number of job postings in the future ΔT duration of the sample; S6. The recruitment unit constructs a sample to be tested, inputs it into the prediction model, and obtains a predicted value; S7. Take the recruitment strategy as an individual to generate a population, and optimize the recruitment strategy based on the genetic algorithm to obtain the optimal recruitment strategy; The recruitment strategy includes the number of job postings, adjusted salary, and optimized channels; The fitness function of the genetic algorithm is , where Fitness(S k ) is the fitness of the k-th recruitment strategy S k in the population, Cost(S k ), Match(S k , X fusion ) are the recruitment cost and the talent demand matching degree of S k respectively, and α, β are the weights of Cost(S k ), Match(S k , X fusion ) respectively; And in the genetic algorithm, the individual with a smaller fitness is regarded as the dominant individual, and after the optimization is completed, the individual with the smallest fitness is regarded as the optimal recruitment strategy.

2. The method for analyzing talent trends based on big data according to claim 1, wherein: In S2, D is normalized based on the mean and standard deviation to obtain X s ; s ; Process D based on the TF-IDF algorithm hs to generate the corresponding feature X hs ; Extract feature X of D based on BERT model t t , ;​ Extract features X of D based on the ResNet network img img , ;​ ， ， 。 3. A method for analyzing talent trends based on big data according to claim 1, characterized in that: Step S5 specifically includes steps S51 to S53; S51. Obtain a Stacking ensemble learning model, including 3 base models and 1 meta-model. The 3 base models are the XGBoost model, the random forest model, and the SVM model, which are sequentially labeled as M1 to M3, and the meta-model is the Lasso regression model; S52, the base model takes X fusion as input and outputs the predicted value of the number of job postings in the future duration of ΔT; The meta-model takes the fused feature P as input and outputs the predicted value of the number of job postings in the future duration of ΔT, where ), , , are the outputs of M1 to M3 respectively; S53, using the fusion feature X fusion Train the Stacking ensemble learning model until convergence to obtain a prediction model.

4. The method for analyzing talent trends based on big data according to claim 3, wherein: The objective function g of the meta-model Meta (P, θ Meta ) is; ， where β is the bias of the meta-model, β j is the weight of the j-th base model M j , where 1 ≤ j ≤ 3, is the predicted value of M j for the i-th sample in X fusion , y i is the label of the i-th sample, where 1 ≤ i ≤ n, and λ is the regularization parameter.

5. The method for analyzing talent trends based on big data according to claim 1, wherein: In S7, Cost(S k ) = C s + C r , Match(S k , X fusion ) = N predicted - N actual ; Where, C s and C r are the corresponding labor cost and recruitment channel cost of S k respectively, N predicted is the predicted value of the prediction model for the sample to be measured, and N actual is the number of job postings in S k .

6. A method for analyzing talent trends based on big data according to claim 1, characterized in that: In S6, the specific construction of the sample to be tested by the recruitment unit is; Generate a structured feature, semi-structured feature, descriptive text, or image data according to its own recruitment needs. If it is a structured feature, perform normalization processing to obtain a feature x. Otherwise, perform feature extraction to obtain the feature x, and then generate a sample x to be tested fusion =[x, R].

Images