Human resource management method based on multi-modal data fusion and adaptive learning
By employing a human resource management approach that combines multimodal data fusion and adaptive learning, we have addressed the lack of deep learning and dynamic optimization in existing systems for talent selection and job matching. This approach enables intelligent human resource management, including person-job matching scoring, turnover risk prediction, and personalized career development path planning.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SUZHOU JIPIN NETWORK TECHNOLOGY CO LTD
- Filing Date
- 2025-11-10
- Publication Date
- 2026-04-10
Smart Images
Figure CN121836645A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of human resource management, and in particular to a human resource management method based on multimodal data fusion and adaptive learning. Background Technology
[0002] Human resource management systems are enterprise management information systems that integrate modules such as organizational planning, recruitment management, and compensation and performance management. They improve the efficiency of human resource utilization through centralized data management, aiming to increase employee satisfaction and loyalty to drive performance improvement, and help enterprises reduce management costs and accelerate value growth. However, current human resource management systems often rely on keyword matching or simple statistical analysis for functions such as talent screening, job matching, and turnover prediction, lacking deep learning and dynamic optimization capabilities.
[0003] In view of this, we propose a human resource management approach based on multimodal data fusion and adaptive learning to address the existing problems. Summary of the Invention
[0004] The purpose of this invention is to provide a human resource management method based on multimodal data fusion and adaptive learning to solve the problems mentioned in the background art.
[0005] To achieve the above objectives, the present invention provides the following technical solution: a human resource management method based on multimodal data fusion and adaptive learning, the operation steps of which include:
[0006] S1: Obtain multimodal data of the target employee or candidate. Multimodal data includes static attribute data, dynamic behavioral data, and external environment data.
[0007] S2: Input the multimodal data into the preset feature extraction model to extract features and obtain multi-dimensional feature vectors. The feature extraction model is based on a neural network.
[0008] S3: Input the multi-dimensional feature vectors into the pre-trained human resource analysis model to obtain the analysis results. The human resource analysis model is obtained by training the machine learning model using historical human resource data. The analysis results include at least one of the following: job-person matching score, turnover risk probability, and recommended career path.
[0009] S4: Based on the analysis results, execute the corresponding human resource management operations.
[0010] Furthermore, it also includes a model update step: regularly collecting newly generated human resource decision results and their actual effects feedback data; using the feedback data to incrementally learn or retrain the human resource analysis model in order to dynamically optimize the model parameters.
[0011] Furthermore, the specific operations of S1 include: asynchronously obtaining the raw data stream of the target object from heterogeneous data sources through multiple data source interfaces, where the target object is an employee or candidate; parsing and classifying the raw data stream, categorizing it into at least one of the following data modalities: static attribute data, dynamic behavioral data, and external environment data; automatically generating metadata tags for each parsed data record, whereby the metadata tags include the data source, collection timestamp, and confidence weight calculated based on the data source and collection method; and, based on predefined association rules and metadata tags, associating and integrating data from different modalities with the unique identifier of the target object to form a time-series data profile of the target object.
[0012] Furthermore, the specific operations of S2 include: inputting the acquired multimodal data into the corresponding modality-specific encoder according to modality type, with each modality-specific encoder using a neural network architecture adapted to its data structure; extracting the primary feature vector for each modality through each modality-specific encoder; inputting multiple primary feature vectors into a cross-modality feature fusion layer, and using an attention mechanism to calculate the attention weight of each primary feature vector under a specific human resource analysis task; and weighting and fusing the primary feature vectors based on the attention weights to generate a unified multidimensional feature representation vector.
[0013] Furthermore, the specific operations of S3 include: inputting the multi-dimensional feature vector of the target object into a pre-trained multi-task analysis model, which includes a shared feature encoding layer and at least two independent task-specific output layers; the shared feature encoding layer performs deep semantic encoding on the multi-dimensional feature vector to obtain a shared feature representation; the at least two independent task-specific output layers, based on the shared feature representation, compute in parallel and output at least two analysis results, the types of which include job-person matching score, turnover risk probability, and personalized development path recommendation; while outputting the analysis results, the built-in interpretability engine is invoked to generate a decision basis report corresponding to each analysis result, the report including at least the key feature factors affecting the result and their contribution.
[0014] Furthermore, the specific operations of S4 include: receiving the analysis results of the target object and the corresponding decision basis report; matching the analysis results with a predefined strategy rule base to generate one or more candidate execution operations, and calculating an expected utility value for each candidate execution operation; determining the execution mode of each candidate execution operation based on the expected utility value and the preset approval rules, including fully automatic execution, execution after notification, or execution after approval; and triggering the corresponding human resource management operation or workflow according to the determined execution mode, and recording the execution log.
[0015] Furthermore, the specific operation of S1 also includes data fusion and enhancement steps: identifying data points from different data sources that describe the same fact; resolving conflicts and fusion of data points based on confidence weights and timestamps to generate a new data record; and recording the fusion process in the data lineage map for traceability.
[0016] Furthermore, S1's specific operations also include context-aware triggering of data collection: predefined key events, including job postings, project completion, and the start of a performance evaluation cycle; when a key event is detected, a targeted data collection task for a specific target object or a specific data modality is automatically initiated.
[0017] Furthermore, the specific operation of S2 also includes a domain knowledge-guided feature learning step: predefine a set of core concept indicators in the human resources domain; during the feature extraction process, introduce an auxiliary loss function to constrain the generated multi-dimensional feature representation vector to predict the values of the core concept indicators through a linear model, thereby injecting domain knowledge into the feature learning process.
[0018] Furthermore, the specific operation of S2 also includes a hierarchical feature output step: the modality-specific encoder is configured to simultaneously output features at different levels of abstraction, including hierarchical features directly extracted from the original data and hierarchical features transformed by multiple neural networks; the cross-modality feature fusion layer fuses features at the same level of abstraction respectively, and finally outputs a feature set of feature vectors at different levels.
[0019] Compared with the prior art, the beneficial effects of the present invention are:
[0020] This invention constructs a time-series data profile of the target object using static attribute data, dynamic behavioral data, and external environment data. Multimodal data is then input into a pre-defined feature extraction model to extract multi-dimensional feature vectors. These multi-dimensional feature vectors are then input into a pre-trained human resource analysis model to obtain analysis results. Finally, based on the analysis results, corresponding human resource management operations are executed. Analysis includes person-job matching scores, turnover risk probability, and recommended development paths. This not only enables intelligent resume screening and person-job matching optimization but also predicts and provides intervention suggestions for employee turnover risk and allows for personalized employee development path planning. The algorithm framework, which integrates multimodal data fusion and dynamic adaptive learning, enhances deep learning and dynamic optimization capabilities, overcoming the limitations of keyword matching or simple statistical analysis. Attached Figure Description
[0021] Figure 1 This is a schematic diagram of the workflow of the human resource management method based on multimodal data fusion and adaptive learning of the present invention. Detailed Implementation
[0022] The technical solution of the present invention will be further described below with reference to the accompanying drawings and specific embodiments.
[0023] Example 1
[0024] like Figure 1 As shown, the human resource management method based on multimodal data fusion and adaptive learning includes the following steps:
[0025] S1: Obtain multimodal data of the target employee or candidate. Multimodal data includes static attribute data, dynamic behavioral data, and external environment data.
[0026] S2: Input the multimodal data into the preset feature extraction model to extract features and obtain multi-dimensional feature vectors. The feature extraction model is based on a neural network.
[0027] S3: Input the multi-dimensional feature vectors into the pre-trained human resource analysis model to obtain the analysis results. The human resource analysis model is obtained by training the machine learning model using historical human resource data. The analysis results include at least one of the following: job-person matching score, turnover risk probability, and recommended career path.
[0028] S4: Based on the analysis results, execute the corresponding human resource management operations.
[0029] It also includes model update steps: regularly collecting newly generated human resource decision results and their actual effects feedback data; using the feedback data to incrementally learn or retrain the human resource analysis model in order to dynamically optimize the model parameters.
[0030] When the analysis results include a job-person fit score, the S4 operation steps include: comparing the candidate's job-person fit score with a preset threshold; if the score is higher than the preset threshold, the candidate's resume information is automatically recommended to the corresponding recruitment manager or enters the next round of interviews.
[0031] When the analysis results include the probability of turnover risk, the S4 operation steps include: if the probability of turnover risk is higher than the risk threshold, generating an early warning message and sending it to the management terminal; at the same time, performing attribution analysis based on multi-dimensional feature vectors to generate a personalized intervention suggestion list for the target employee.
[0032] The human resource analysis model adopts an ensemble learning framework, which includes a first sub-model and a second sub-model. The first sub-model is used to process structured static attribute data, and the second sub-model is used to process unstructured dynamic behavioral data. The outputs of the first and second sub-models are weighted and fused to obtain the final analysis results.
[0033] Example 2
[0034] The specific operations of S1 include: asynchronously obtaining the raw data stream of the target object (employee or candidate) from heterogeneous data sources through multiple data source interfaces; parsing and classifying the raw data stream into at least one of the following data modalities: static attribute data, dynamic behavior data, and external environment data; automatically generating metadata tags for each parsed data record, including data source, collection timestamp, and confidence weight calculated based on data source and collection method; and associating and integrating data from different modalities with the unique identifier of the target object based on predefined association rules and metadata tags to form a time-series data profile of the target object.
[0035] It also includes data fusion and enhancement steps: identifying data points from different data sources that describe the same fact; resolving conflicts and fusion of data points based on confidence weights and timestamps to generate a new data record; and recording the fusion process in the data lineage map for traceability.
[0036] It also includes context-aware triggering of data collection: predefined key events, including job postings, project completion, and the start of a performance evaluation cycle; when a key event is detected, a targeted data collection task for a specific target object or a specific data modality is automatically initiated.
[0037] The specific operations for asynchronously acquiring data from heterogeneous data sources include: communicating with internal business systems through the first type of interface to acquire structured data via scheduled tasks or event triggers; internal business systems include enterprise resource planning systems, performance management systems, and project management systems; accessing internal communication platforms through the second type of interface to monitor and analyze text communication content in real time or near real time using natural language processing tools to extract unstructured dynamic behavioral data; and acquiring external environmental data from internet data sources through the third type of interface, including industry talent market salary levels and publicly available skill certification information of target individuals on professional social platforms.
[0038] In parsing and classifying the raw data stream, for unstructured text data, named entity recognition and relation extraction techniques are used to extract entities and their relationships, including skill names, project names, and collaborators, and transform them into structured dynamic behavioral data.
[0039] Example 3
[0040] The specific operations of S2 include: inputting the acquired multimodal data into the corresponding modality-specific encoder according to the modality type, with each modality-specific encoder using a neural network architecture adapted to its data structure; extracting the primary feature vector for each modality through each modality-specific encoder; inputting multiple primary feature vectors into a cross-modality feature fusion layer, and using an attention mechanism to calculate the attention weight of each primary feature vector under a specific human resource analysis task; and weighting and fusing the primary feature vectors based on the attention weights to generate a unified multidimensional feature representation vector.
[0041] It also includes a domain knowledge-guided feature learning step: predefine a set of core concept indicators in the human resources domain; during the feature extraction process, introduce an auxiliary loss function to constrain the generated multi-dimensional feature representation vector to predict the values of the core concept indicators through a linear model, thereby injecting domain knowledge into the feature learning process.
[0042] It also includes a hierarchical feature output step: the modality-specific encoder is configured to output features at different levels of abstraction simultaneously, including hierarchical features extracted directly from the original data and hierarchical features transformed by a multi-layer neural network; the cross-modality feature fusion layer fuses features at the same level of abstraction respectively, and finally outputs a feature set of feature vectors at different levels.
[0043] Modality-specific encoders include: encoders for processing static attribute modalities, which include embedding layers and fully connected layers for processing numerical features, used to map categorical and numerical features to the same vector space; encoders for processing dynamic behavioral sequence modalities, which employ recurrent neural networks or temporal convolutional networks to capture temporal dependencies in behavioral data; and encoders for processing text evaluation modalities, which employ pre-trained language models to extract semantic features from text data.
[0044] The specific steps for calculating the attention weight of each primary feature vector are as follows: concatenate multiple primary feature vectors; calculate the attention score of each modality's primary feature vector by combining the concatenated vector with a trainable task context vector; and normalize the attention scores to obtain the attention weight.
[0045] Example 4
[0046] The specific operations of S3 include: inputting the multi-dimensional feature vector of the target object into a pre-trained multi-task analysis model, which contains a shared feature encoding layer and at least two independent task-specific output layers; the shared feature encoding layer performs deep semantic encoding on the multi-dimensional feature vector to obtain a shared feature representation; the at least two independent task-specific output layers, based on the shared feature representation, compute in parallel and output at least two analysis results, including job-person matching scores, turnover risk probabilities, and personalized development path recommendations; and simultaneously outputting the analysis results, calling the built-in interpretability engine to generate a decision basis report corresponding to each analysis result, which includes at least the key feature factors affecting the results and their contribution.
[0047] It also includes establishing a feedback loop and dynamic model update steps: continuously monitoring the actual human resource decisions and their final effects corresponding to the analysis results to form feedback data; using the feedback data periodically or triggered to incrementally learn the multi-task analysis model in order to adjust the model parameters; recording model update versions and performance changes, and supporting model version rollback and management.
[0048] The multi-task analysis model is initialized using a meta-learning framework. The initialization process includes: acquiring historical human resources datasets from multiple different organizations; training the model on multiple datasets using meta-learning algorithms to enable the model to quickly adapt to the data distribution of new organizations; and when applied to a new target organization, rapidly fine-tuning the pre-trained model using a small amount of sample data from that organization to generate a target analysis model that fits the characteristics of that organization.
[0049] The output layer of the personalized development path recommendation adopts a sequence generation model. Its input is a shared feature representation and the organization's job map information, and the output is a priority-ordered sequence of development paths. Each node in the sequence includes a suggested target job or skills training program.
[0050] The interpretability engine adopts an attention-based interpretability method, specifically: the shared feature encoding layer or task-specific output layer contains an attention mechanism; the attention weights of each feature dimension in the multi-dimensional feature vector are calculated, and the key features after weight ranking are used as key feature factors in the decision-making report.
[0051] Example 5
[0052] The specific operations of S4 include: receiving the analysis results of the target object and the corresponding decision basis report; matching the analysis results with the predefined strategy rule base to generate one or more candidate execution operations, and calculating an expected utility value for each candidate execution operation; determining the execution mode of each candidate execution operation based on the expected utility value and the preset approval rules, including fully automatic execution, execution after notification, or execution after approval; triggering the corresponding human resource management operation or workflow according to the determined execution mode, and recording the execution log.
[0053] It also includes effect tracking and optimization steps: assigning a unique tracking identifier to each triggered execution operation; continuously collecting subsequent effect data related to the operation, including changes in the target object's status, behavioral feedback, and changes in business metrics; calculating the actual utility value of the execution operation based on the collected effect data, comparing it with the expected utility value, and using the difference data to optimize and adjust the rules in the strategy rule base.
[0054] Generating one or more candidate execution operations includes: retrieving corresponding intervention strategy templates from the strategy rule base according to the type and level of the analysis results; instantiating the intervention strategy templates using key feature factors from the decision basis report as variables to generate personalized candidate execution operations; and calculating the expected utility value of each candidate execution operation based on the success rate and cost of similar operations in historical data.
[0055] Triggering corresponding human resource management operations or workflows includes: when the execution mode is fully automatic, the operation is executed directly through the system API call, such as automatically sending personalized communication messages or advancing the candidate status to the next stage; when the execution mode is execution after notification or execution after approval, a task item containing a summary of analysis results and recommended operations is generated and pushed to the to-do list of relevant responsible persons (such as line managers, HRBPs), and the system automatically executes it after their confirmation or approval.
[0056] The strategy rule base supports dynamic rule matching based on organizational structure. The same analysis result will match different execution strategies for target objects in different departments and at different levels.
[0057] The above specific embodiments are merely several preferred embodiments of the present invention. Based on the technical solutions of the present invention and the relevant teachings of the above embodiments, those skilled in the art can make various alternative improvements and combinations to the above specific embodiments.
Claims
1. A human resource management method based on multimodal data fusion and adaptive learning, characterized in that, The operating steps include: S1: Obtain multimodal data of the target employee or candidate. Multimodal data includes static attribute data, dynamic behavioral data, and external environment data. S2: Input the multimodal data into the preset feature extraction model to extract features and obtain multi-dimensional feature vectors. The feature extraction model is based on a neural network. S3: Input the multi-dimensional feature vectors into the pre-trained human resource analysis model to obtain the analysis results. The human resource analysis model is obtained by training the machine learning model using historical human resource data. The analysis results include at least one of the following: job-person matching score, turnover risk probability, and recommended career path. S4: Based on the analysis results, execute the corresponding human resource management operations.
2. The human resource management method based on multimodal data fusion and adaptive learning according to claim 1, characterized in that, It also includes model update steps: regularly collecting newly generated human resource decision results and their actual effects feedback data; using the feedback data to incrementally learn or retrain the human resource analysis model in order to dynamically optimize the model parameters.
3. The human resource management method based on multimodal data fusion and adaptive learning according to claim 1, characterized in that, The specific operations of S1 include: asynchronously obtaining the raw data stream of the target object (employee or candidate) from heterogeneous data sources through multiple data source interfaces; parsing and classifying the raw data stream into at least one of the following data modalities: static attribute data, dynamic behavior data, and external environment data; automatically generating metadata tags for each parsed data record, including data source, collection timestamp, and confidence weight calculated based on data source and collection method; and associating and integrating data from different modalities with the unique identifier of the target object based on predefined association rules and metadata tags to form a time-series data profile of the target object.
4. The human resource management method based on multimodal data fusion and adaptive learning according to claim 1, characterized in that, The specific operations of S2 include: inputting the acquired multimodal data into the corresponding modality-specific encoder according to the modality type, with each modality-specific encoder using a neural network architecture adapted to its data structure; extracting the primary feature vector for each modality through each modality-specific encoder; inputting multiple primary feature vectors into a cross-modality feature fusion layer, and using an attention mechanism to calculate the attention weight of each primary feature vector under a specific human resource analysis task; and weighting and fusing the primary feature vectors based on the attention weights to generate a unified multidimensional feature representation vector.
5. The human resource management method based on multimodal data fusion and adaptive learning according to claim 1, characterized in that, The specific operations of S3 include: inputting the multi-dimensional feature vector of the target object into a pre-trained multi-task analysis model, which contains a shared feature encoding layer and at least two independent task-specific output layers; the shared feature encoding layer performs deep semantic encoding on the multi-dimensional feature vector to obtain a shared feature representation; the at least two independent task-specific output layers, based on the shared feature representation, compute in parallel and output at least two analysis results, including job-person matching scores, turnover risk probabilities, and personalized development path recommendations; and simultaneously outputting the analysis results, calling the built-in interpretability engine to generate a decision basis report corresponding to each analysis result, which includes at least the key feature factors affecting the results and their contribution.
6. The human resource management method based on multimodal data fusion and adaptive learning according to claim 1, characterized in that, The specific operations of S4 include: receiving the analysis results of the target object and the corresponding decision basis report; matching the analysis results with the predefined strategy rule base to generate one or more candidate execution operations, and calculating an expected utility value for each candidate execution operation; determining the execution mode of each candidate execution operation based on the expected utility value and the preset approval rules, including fully automatic execution, execution after notification, or execution after approval; triggering the corresponding human resource management operation or workflow according to the determined execution mode, and recording the execution log.
7. The human resource management method based on multimodal data fusion and adaptive learning according to claim 3, characterized in that, The specific operation of S1 also includes data fusion and enhancement steps: identifying data points from different data sources that describe the same fact; resolving conflicts and fusion of data points based on confidence weights and timestamps to generate a new data record; and recording the fusion process in the data lineage map for traceability.
8. The human resource management method based on multimodal data fusion and adaptive learning according to claim 3, characterized in that, S1's specific operations also include context-aware triggering of data collection: predefined key events, including job postings, project completion, and the start of a performance evaluation cycle; when a key event is detected, a targeted data collection task for a specific target object or a specific data modality is automatically initiated.
9. The human resource management method based on multimodal data fusion and adaptive learning according to claim 4, characterized in that, The specific operation of S2 also includes a domain knowledge-guided feature learning step: predefine a set of core concept indicators in the human resources domain; during the feature extraction process, introduce an auxiliary loss function to constrain the generated multi-dimensional feature representation vector to predict the values of the core concept indicators through a linear model, thereby injecting domain knowledge into the feature learning process.
10. The human resource management method based on multimodal data fusion and adaptive learning according to claim 4, characterized in that, The specific operation of S2 also includes a hierarchical feature output step: the modality-specific encoder is configured to output features at different levels of abstraction simultaneously, including hierarchical features extracted directly from the original data and hierarchical features transformed by multiple neural networks; the cross-modality feature fusion layer fuses features at the same level of abstraction respectively, and finally outputs a feature set of feature vectors at different levels.