Employee portrait generation method based on multi-modal data fusion and joint attention mechanism

Abstract

The invention discloses an employee portrait generation method based on multi-modal data fusion and a joint attention mechanism, and belongs to the technical field of employee performance evaluation and portrait generation. The method comprises the following steps: acquiring multi-source data, and preprocessing the data; designing a double-branch neural network; introducing a joint attention mechanism into the fusion layer, dynamically calculating importance weights of different modal data, and finally generating a fused feature vector; generating an employee portrait based on the fused feature vector, wherein the portrait comprises an ability dimension, a potential dimension and an adaptation dimension; training the neural network model through hyper-parameter optimization, and adjusting and optimizing the model through cross validation; and generating an optimized employee portrait according to the optimized neural network model. According to the employee portrait provided by the invention, the accuracy and the comprehensiveness are remarkably improved, and the transparency and the credibility of an HR decision are enhanced through interpretable feature weight distribution.

Description

Technical Field

[0001] The present invention belongs to the technical field of employee performance evaluation and portrait generation, and particularly relates to a method for generating an employee portrait based on multi-modal data fusion and joint attention mechanism. Background Art

[0002] Current methods for generating employee portraits usually rely on single-dimensional performance data (such as KPI indicators) or evaluation methods based on static rules. These traditional methods often suffer from insufficient data utilization and lack of mining of feature correlations. It is difficult to dynamically fuse structured indicators and unstructured behavioral data, and they lack the ability to adaptively adjust the weights of multi-source data, resulting in inflexible generated employee portraits that cannot accurately reflect the comprehensive performance and potential of employees.

[0003] In addition, existing performance evaluation methods usually only provide static evaluations of employees' work results, lacking real-time reflection of employees' dynamic behaviors. This makes it impossible to update the employee portrait over time and difficult to meet the personalized evaluation needs of different positions. Traditional methods also have difficulty effectively integrating data from different channels, unable to comprehensively consider all aspects of employees' traits, resulting in insufficient accuracy and interpretability of the portrait.

[0004] Therefore, when enterprises manage personnel and make decisions, they often rely on manual intervention. The evaluation criteria for employees are difficult to unify, and the results are not easily understood and accepted by the HR department, affecting the efficiency and accuracy of decision-making. To solve these problems, there is an urgent need for a method for generating an employee portrait that can dynamically fuse multi-source data, automatically adjust evaluation weights, and provide interpretability to help HR conduct more accurate personnel management and decision-making.

[0005] After retrieval, a patent application with the application number CN202311825962.6 and the publication date of December 27, 2023, discloses a method, device, equipment, and storage medium for generating an employee portrait. This application includes: obtaining the individual data of the target employee; analyzing the individual data of the target employee based on a preset employee feature analysis model to obtain the employee feature data of the target employee; generating an employee portrait of the target employee based on the employee feature data. The individual data of the employee can be analyzed through the preset employee feature analysis model to obtain the employee feature data, and then an employee portrait can be generated based on the employee feature data. Although this application provides a new idea for portrait generation, it fails to effectively consider the dynamic changes of the employee portrait and the correlation between multi-source data, lacks a mechanism for adaptively adjusting data weights, and there are still deficiencies in the evaluation of job fit and interpretability of the generated portrait, making it difficult to meet the needs of multi-dimensional decision-making in actual work.

[0006] For another example, an application case with a Chinese patent application number of CN202410304955.X and an application publication date of July 30, 2024 discloses a method, system, and device for generating individual portraits based on multi-source heterogeneous human data. This method first obtains structured and unstructured human resource data, and generates basic fact tags for each individual based on the structured data and preset fact tag generation rules. Then, it extracts features from the unstructured data to obtain abstract features for each individual, and quantifies and extracts key texts from the abstract features to generate quantitative and text-based competency tags for each individual. Finally, a visual individual portrait is obtained based on multiple tags. Although this method realizes the fusion of multi-source data, it lacks a mechanism for adaptively adjusting weights and fails to dynamically consider the requirements of different positions, resulting in room for improvement in the accuracy and interpretability of the generated portraits.

[0007] In summary, both of the above two methods have realized the generation of employee portraits and provided new ideas for employee evaluation. However, in practical applications, these methods lack sufficient consideration for the multi-dimensional fusion of employee portraits and the evaluation of position adaptability, and further optimization is still needed for the portrait accuracy of specific positions and the depth of multi-source data fusion. Summary of the Invention

[0008] The purpose of the present invention is to provide a method for generating an employee portrait based on multi-modal data fusion and joint attention mechanism, so as to solve problems such as insufficient data fusion and insufficient evaluation of position adaptability existing in traditional employee portrait generation methods.

[0009] To achieve the above object, the present invention proposes the following technical solutions:

[0010] The present invention provides a method for generating an employee portrait based on multi-modal data fusion and joint attention mechanism, including:

[0011] S1. Obtain multi-source data including structured data and unstructured data, and preprocess the data;

[0012] The structured data includes, but is not limited to, employees' KPI indicators, performance scores, and project participation; the unstructured data includes, but is not limited to, employees' work logs, email records, project documents, and text data corresponding to colleague feedback;

[0013] S2. Design a dual-branch neural network, where the first branch uses a fully connected neural network to extract key features from the structured data; the second branch uses a pre-trained BERT model to extract semantic features from the unstructured data;

[0014] S3. Introduce a joint attention mechanism in the fusion layer to dynamically calculate the importance weights of different modality data, and finally generate a fused feature vector;

[0015] S4. Generate an employee portrait based on the fused feature vector, and the portrait includes an ability dimension, a potential dimension, and a fit dimension;

[0016] S5. Train the neural network model through hyperparameter optimization and tune the model through cross-validation;

[0017] S6. Generate an optimized employee portrait according to the optimized neural network model.

[0018] Furthermore, the data preprocessing in S1 specifically includes: normalizing the structured data and / or using a pre-trained language model to perform text feature extraction and L2 normalization on the unstructured data to obtain a normalized text feature vector

[0019] Furthermore, the input layer of the fully connected neural network in S2 is 5-7 dimensions, the hidden layer is 2-4 layers, the activation function is ReLU, and the expression of the output feature vector is:

[0020] H structured =ReLU(W structured X structured +b structured )

[0021] where H structured is the feature representation obtained by passing the structured data through the fully connected neural network, W structured is the weight matrix of the structured data branch, X structured is the input structured data feature vector with dimension d s , and b structured is the bias vector of the structured data branch.

[0022] Furthermore, the activation function of the BERT model in S2 is ReLU, and its output feature vector expression is:

[0023]

[0024] where H unstructured is the feature representation obtained by passing the unstructured data through the fully connected layer, W unstructured is the weight matrix of the unstructured data branch, is the text feature vector obtained by preprocessing the unstructured data, and b unstructured is the bias vector of the unstructured data branch.

[0025] Further, in step S3, the importance weights of different modality data are dynamically calculated, and finally a fused feature vector is generated, which specifically includes:

[0026] (1) A single-layer MLP network is used for dynamic weight allocation, and the Softmax normalization formula is:

[0027]

[0028] where, Weight i represents the attention weight of modality i, indicating the importance of this modality in the final fusion; score i represents the importance score of modality i calculated by the neural network;

[0029] (2) The data feature vectors of different modalities are fused to obtain a fused feature vector:

[0030]

[0031] where, H fused represents the fused feature vector, and H i represents the feature vector of modality i.

[0032] Further, the sub-dimensions of the ability dimension in step S4 include but are not limited to professional skills, communication skills, and execution ability, and its calculation formula is:

[0033]

[0034] where, w i represents the importance weight of the i-th sub-dimension in the ability dimension, which can be dynamically adjusted according to the job requirements. For example, some jobs may attach more importance to "professional skills", while others may focus more on "communication skills"; f ability (H fused ) is the ability dimension score mapped through the fused feature H fused ;

[0035] The calculation of the potential dimension and the adaptation dimension is similar to that of the ability dimension.

[0036] Further, the hyperparameters in step S5 include the learning rate, hidden layer size, Dropout rate, and batch size, where the learning rate is 0.001 - 0.01, the number of neurons in the hidden layer is 32, 64, or 128, the Dropout rate is 0.2 - 0.5, and the batch size Batch Size ∈ {32, 64, 128}.

[0037] Further, it also includes generating a structured feature vector based on the optimized employee portrait, and generating a visual report of the employee portrait based on the structured feature vector;

[0038] The structured feature vector includes the performance of employees in different dimensions of ability, potential, and suitability.

[0039] A visualization report of the employee portrait generated based on the structured feature vector, including but not limited to radar charts and bar charts, to display the scores of employees in each dimension.

[0040] Furthermore, the dimension scores of the radar chart are normalized to [0, 1], and the formula is:

[0041]

[0042] where S norm is the normalized score, S is the original score, S min is the minimum score value of this dimension, and S max is the maximum score value of this dimension;

[0043] The bar chart is used to compare the scores of each dimension of different employees.

[0044] Furthermore, it also includes: calculating the comprehensive job matching degree according to the similarity between the optimized employee portrait and the job requirements, and the comprehensive matching degree formula is as follows:

[0045] D total = α·D match + β·D major

[0046] where α and β are weighting coefficients, and their sum is 1; D match is the ability matching degree, and its calculation is as follows:

[0047]

[0048] In the above formula, D final is the comprehensive feature vector of the employee's ability obtained previously, and P i represents the demand feature vector of the job, which is manually set by the enterprise HR;

[0049] D major is the professional matching degree, which is calculated by the following formula:

[0050] D major = similarity(H major , P major )

[0051] where H major is the professional feature vector of the employee, which is obtained from the professional skill sub-dimension vector D skill in the ability dimension of the employee portrait through a specific mapping:

[0052] H major = W h ·D skill + b h

[0053] Wherein, W h is the weight matrix, and b h is the bias vector, which is used to map the ability vector to the professional skill space. W h and b h are learned in a data-driven manner, and the specific process is as follows:

[0054] First, construct a historical matching dataset, which contains a large number of employees' ability feature vectors D skill and their corresponding professional skill evaluation labels (the professional skill scores of employees by business departments); then, use least squares regression to optimize W h and b h to minimize the prediction error, that is:

[0055]

[0056] Wherein, is the professional skill vector marked in the historical data, is the result predicted by the model. During the training process, optimization algorithms such as gradient descent are adopted to continuously adjust W h and b h to minimize the error. Finally, the trained W h can project the general ability features D of employees skill into the professional skill space, so that H major truly reflects the professional ability of employees.

[0057] P major represents the professional demand vector of the position, and its calculation is derived from the job description (JD). The specific steps are as follows: First, process the JD using the huggingface toolkit to extract keywords related to professional skills. For example, for "Java development engineer", extract [Java, Spring, MySQL], and then use the pre-trained professional skill word vector model (BERT) to map the keywords to the vector space to obtain the preliminary professional skill vector; subsequently, introduce the skill weights set by the HR, such as "must master Java (weight 1.0), be familiar with Spring (weight 0.8)", and calculate the final vector:

[0058]

[0059] Wherein, V i is the vector representation of skill i, and w iIt is the importance weight of this skill. P major Can accurately express the professional skill requirements of the position

[0060] By adopting the technical solution provided by the present invention, compared with the prior art, the following beneficial effects can be achieved:

[0061] By obtaining multi-source data, the present invention deeply fuses structured data, unstructured data and time-series data, thereby effectively solving the problems of insufficient data utilization and insufficient job matching degree evaluation in traditional methods;

[0062] At the same time, by introducing a joint attention mechanism to dynamically adjust the weights of various types of data, it ensures the accurate adaptation to job requirements during the generation of the employee portrait;

[0063] In addition, through the calculation of the comprehensive job matching degree index, the present invention also facilitates the accurate and rapid confirmation of the matching degree between employees and jobs;

[0064] That is to say, the employee portrait provided by the present invention not only has been significantly improved in terms of accuracy and comprehensiveness, but also enhances the transparency and credibility of HR decisions through interpretable feature weight allocation, provides more scientific and flexible support for the enterprise's human resource management, can effectively assist the enterprise in talent assessment, job matching and development planning, and has important practical application value. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 Is the method flow chart of the present invention;

[0066] Figure 2 Is the dual-branch network structure diagram proposed by the present invention;

[0067] Figure 3 Is the bar chart of the employee portrait generated by the embodiment of the present invention;

[0068] Figure 4 Is the radar chart of the employee portrait generated by the embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0069] Traditional employee portrait methods usually rely on single-dimensional performance data (such as KPI indicators) or static rule evaluation, which have problems of insufficient data utilization and insufficient mining of feature correlations, are difficult to dynamically fuse structured indicators and unstructured behavioral data, and lack the ability to adaptively adjust the weights of multi-source data, resulting in poor interpretability of the portrait results and insufficient accuracy of job matching degree analysis.

[0070] To solve the above problems, the present invention proposes an employee portrait generation method based on multi-modal data fusion and joint attention mechanism, as Figure 1 shown, specifically including:

[0071] S1. Data collection and preprocessing

[0072] The present invention conducts multi-source data collection, specifically including structured data and unstructured data of employees; among them, the structured data obtains quantitative indicators of employees from the enterprise HR system and the performance management platform, including but not limited to:

[0073] KPI indicators: such as task completion rate (calculation formula: actual completion volume / planned volume × 100%), work efficiency (output per unit time), attendance rate (actual attendance days / should-attend days × 100%);

[0074] Performance score: quarterly / annual score (scored by superiors, range 0 - 10 points);

[0075] Project participation: number of projects participated in, role weight (such as 1.0 for the person in charge, 0.5 for the collaborator).

[0076] The unstructured data collects text data from the enterprise internal collaboration platform, including but not limited to:

[0077] Work log: daily task summary, problem feedback;

[0078] Project documents: contribution descriptions in the requirements document and meeting minutes;

[0079] Colleague feedback: text comments in the 360-degree evaluation.

[0080] Existing employee portrait methods usually rely on a single data source, such as structured KPI indicators or static text analysis, and cannot comprehensively reflect the abilities and potential of employees. The present invention can more comprehensively depict employee characteristics through multi-source data collection, integrating structured data (KPI, attendance, performance score, etc.) and unstructured data (text data such as work logs, emails, colleague feedback, etc.).

[0081] Furthermore, the present invention preprocesses the collected multi-source data, including:

[0082] (1) Normalize the structured data to ensure that data with different dimensions can be compared. The normalization formula is as follows:

[0083]

[0084] where X is the original data, X max 、X min are the minimum and maximum values of the data respectively, and X norm is the normalized data.

[0085] (2) Use a pre-trained language model to extract text features and perform L2 normalization on unstructured data to obtain a normalized text feature vector.

[0086] As a preferred method, the present invention uses the pre-trained language model BERT to extract features from text, and the input text data T is converted into a text feature vector. And perform L2 normalization. The normalization formula is as follows:

[0087]

[0088] Where, is the L2 norm of the vector. represents the normalized text feature vector, represents the text feature vector before normalization.

[0089] S2. Construct a dual-branch neural network, where the first branch uses a fully connected neural network for extracting key features from structured data; the second branch uses a pre-trained BERT model for extracting semantic features from unstructured data.

[0090] Traditional methods often use a unified model to process all data, which easily leads to information loss or insufficient feature extraction. The present invention uses a dual-branch neural network: the structured data branch uses a fully connected neural network (FCN) to extract quantitative features such as KPIs; the unstructured data branch uses a BERT model to extract semantic information, thereby solving the above problems existing in traditional methods. The structure diagram of the dual-branch neural network of the present invention is as Figure 2 shown.

[0091] Specifically, the structured data branch uses a fully connected neural network (FCN). Its input layer is structured data, the activation function is ReLU, the hidden layer size is set according to the data dimension, and the output is the feature representation of the structured data. The formula is:

[0092] H structured = ReLU(W structured X structured + b structured )

[0093] Where, H structured is the feature representation obtained by the structured data after passing through the fully connected neural network, W structured is the weight matrix of the structured data branch, X structured is the input structured data feature vector, including KPI indicators, performance scores, etc., with a dimension of d s , b structuredThe bias vector for the structured data branch. Further preferably, the input layer of the fully connected neural network is 5-7 dimensions, the hidden layer is 2-4 layers, more preferably the input layer is 6 dimensions, the hidden layer is 2 layers, and each layer has 64 neurons, and finally a 64-dimensional feature vector is output.

[0094] The unstructured data branch uses BERT to process text data. The input is text T, and a feature vector is generated through BERT, and then mapped to the final feature space through a fully connected layer. The formula is:

[0095]

[0096] Among them, H unstructured is the feature representation of the unstructured data after passing through the fully connected layer, W unstructured is the weight matrix of the unstructured data branch, is the text vector (768 dimensions) output by BERT, and b unstructured is the bias vector of the unstructured data branch.

[0097] S3. Introduce a joint attention mechanism to dynamically adjust the importance of different modality data.

[0098] Existing methods usually assign fixed weights to multi-modal data and are difficult to adapt to different positions and evaluation criteria. By introducing a joint attention mechanism, the present invention can dynamically adjust the weights of structured data and unstructured data according to different position requirements, thereby facilitating ensuring the rationality of data fusion.

[0099] Specifically, for each modality data H structured and H unstructured , the attention mechanism assigns a weight to each modality. More preferably, the dynamic weight assignment uses a single-layer MLP network, the input is a 64-dimensional feature, and the output is a 1-dimensional score. The calculation formula of the attention weight Weight i is as follows:

[0100]

[0101] Among them, Weight i represents the attention weight of modality i, indicating the importance of this modality in the final fusion; score i is the importance score of modality i calculated through the neural network. Specifically, it is calculated using a single-layer fully connected network (MLP):

[0102] score i = W s ·H i + b s

[0103] Among them, W s and b s are the weight matrix and the bias vector respectively, and H i is the eigenvector of modality i, that is, H structured or H unstructured ;

[0104] Fuse the eigenvectors of data features of different modalities to obtain the fused eigenvector H fused :

[0105]

[0106] S4. Employee Portrait Generation

[0107] Generate an employee portrait based on the fused feature H fused and calculate the score of each dimension. Preferably, the employee portrait includes the following dimensions:

[0108] Capability Dimension D ability : Professional skills, communication skills, execution ability;

[0109] Potential Dimension D potential : Growth potential, innovation ability, leadership potential;

[0110] Fit Dimension D adopt : Compatibility with the position, suitable development direction.

[0111] Among them, the calculation formula for the capability dimension is:

[0112]

[0113] In the above formula, w i represents the importance weight of the i-th sub-dimension in the capability dimension, which can be dynamically adjusted according to the position requirements. For example, the weights of professional skills, communication skills, and execution ability are 0.4, 0.3, and 0.3 respectively; f ability (H fused ) is the score of the capability dimension mapped through the fused feature H fused ; The calculations of the potential dimension and the fit dimension are similar, and the specific sub-dimensions and their weight distributions can be adjusted according to the needs of the enterprise.

[0114] Most traditional portrait methods focus on a single dimension, such as performance scoring, but lack the assessment of future potential. The present invention designs three core dimensions: the capability dimension (skills, communication, execution ability), the potential dimension (growth potential, innovation ability, leadership), and the fit dimension (position matching degree, development direction), so as to effectively ensure the comprehensiveness of the employee portrait, and in particular, can effectively evaluate the future potential of employees.

[0115] S6. Optimization and Adaptive Adjustment

[0116] The present invention further trains the neural network model through hyperparameter optimization and trains the model through methods such as cross-validation. The validation set is used to adjust the hyperparameters, so as to further optimize the model, improve the accuracy of the model and the suitability between employees and positions.

[0117] Furthermore, the hyperparameter optimization includes learning rate (0.001 - 0.01), the number of neurons in the hidden layer (32 / 64 / 128), and Dropout rate (0.2 - 0.5); 5-fold cross-validation is adopted, and the ratio of the training set to the test set is 7:3.

[0118] As a further optimization of the technical solution of the present invention, it further includes: visualizing the generated employee portrait and outputting a visualization report, specifically including:

[0119] (1) Model output and structured feature vector generation

[0120] The structured feature vector includes the performance of employees in dimensions such as ability, potential, and job suitability, which is the concatenation of scores for each dimension and serves as the basis for subsequent job matching, talent assessment, and development planning. The calculation of the structured feature vector is as follows:

[0121] D final =[D ability ,D potential ,D adopt

[0122] (2) Generating a visualization report of the employee portrait based on the structured feature vector D final Generating a visualization report of the employee portrait

[0123] Taking the scores of each dimension in the structured feature vector D final =[D ability ,D potential ,D adopt as input, various types of charts are generated, and the types of charts include but are not limited to radar charts and bar charts. The score of each dimension will be used as the input data for the chart to display the comprehensive performance of the employee in terms of ability, potential, and suitability.

[0124] Specifically:

[0125] Radar chart generation: The radar chart is used to display the scores of employees in each dimension (ability, potential, suitability). The X-axis represents different dimensions (ability, potential, suitability), and the Y-axis represents the scores. Each point on the graph presents the score of the employee in the corresponding dimension. In this way, the HR can intuitively see the advantages and areas for improvement of the employee. Among them, the dimension scores of the radar chart need to be normalized to [0,1], and the formula is: ​

[0126]

[0127] where S norm is the score after standardization, and S is the original score. For example, the original rating of an employee in the "ability dimension" is S min is the minimum score value of this dimension (i.e., the lowest score of all employees in this dimension), and S max is the maximum score value of this dimension (i.e., the highest score of all employees in this dimension).

[0128] Bar chart generation: Based on D ability , D potential , D adopt scores, bar charts for each employee in each dimension are generated. The bar charts help to show the relative scores of employees in different dimensions, facilitating HR to conduct comparative analysis and evaluate in which areas employees perform outstandingly and in which areas they may need improvement.

[0129] As a further optimization of the technical solution of the present invention, it further includes: calculating the job fitness and generating the comprehensive matching degree. Specifically, the present invention calculates the job fitness according to the similarity between the employee portrait and the job requirements.

[0130] The similarity adopts the cosine similarity calculation formula:

[0131]

[0132] where D final represents the comprehensive ability feature vector (structured feature vector) of the employee, that is, generated by the fused feature vector H fused and includes the performances in the ability dimension, potential dimension and fitness dimension; P i represents the demand feature vector of the job, which is composed of features such as the ability requirements and experience requirements required by the job, and is manually set by the enterprise HR. This similarity represents the matching degree between the employee portrait and the job requirements. The higher the similarity, the more suitable the employee is for this job.

[0133] Finally, by weighted calculation of the ability matching degree D match and the professional matching degree D major , the final comprehensive matching degree is obtained:

[0134] D total = α·D match + β·D major

[0135] D total is the comprehensive matching degree, α and β are weighted coefficients, and their sum is 1, which can be dynamically adjusted according to the job requirements. For example:

[0136] Technical positions: α = 0.7, β = 0.3 (more focused on ability matching);

[0137] Management positions: α = 0.5, β = 0.5 (ability and professional matching are equally important).

[0138] The above ability matching degree D match is calculated as follows:

[0139]

[0140] The professional matching degree D major is calculated by the following formula:

[0141] D major = similarity(H major , P major )

[0142] where H major is the professional feature vector of the employee, which is derived from the professional skill sub-dimension vector D in the employee portrait skill obtained through a specific mapping:

[0143] H major = W h ·D skill + b h

[0144] where W h is the weight matrix, and b h is the bias vector, which is used to map the ability vector to the professional skill space. W h and b h are learned in a data-driven manner, and the specific process is as follows:

[0145] First, construct a historical matching dataset, which contains the ability feature vectors D of a large number of employees skill and their corresponding professional skill evaluation labels (the professional skill scores of employees by business departments); then, use least squares regression to optimize W h and b h to minimize the prediction error, that is:

[0146]

[0147] where, is the professional skill vector marked in the historical data, is the result predicted by the model. During the training process, optimization algorithms such as gradient descent are used to continuously adjust W h and b h to minimize the error. Finally, the trained W h can map the general ability characteristics D of employeesskill Projected onto the professional skill space, so that H major truly reflects the professional capabilities of employees.

[0148] P major represents the professional requirement vector of the position, and its calculation is derived from the job description (JD). The specific steps are as follows: First, process the JD using the huggingface toolkit to extract keywords related to professional skills. For example, for "Java Development Engineer", extract [Java, Spring, MySQL]. Then, use the pre-trained professional skill word vector model (BERT) to map the keywords to the vector space to obtain the preliminary professional skill vector. Subsequently, introduce the skill weights set by the HR, such as "must master Java (weight 1.0), be familiar with Spring (weight 0.8)", and calculate the final vector:

[0149]

[0150] where, V i is the vector representation of skill i, and w i is the importance weight of this skill. P major can accurately express the professional skill requirements of the position.

[0151] To further understand the content of the present invention, the present invention will be described in detail below in conjunction with specific embodiments.

[0152] The embodiments of the present invention specifically include the following steps:

[0153] I. Multi-source data collection and preprocessing

[0154] The structured data in this embodiment comes from the enterprise HR system and the performance management platform, including the following data:

[0155] KPI indicators: task completion rate (actual completion volume / planned volume × 100%), work efficiency (output per unit time, such as the number of tasks processed per day), attendance rate (actual attendance days / should attend days × 100%);

[0156] Performance score: quarterly or annual score (range 0 - 10 points, scored by the superior);

[0157] Project participation: number of projects participated in, role weight (the person in charge is counted as 1.0, the collaborator is counted as 0.5), as shown in Table 1 below:

[0158] Table 1 The structured data situation of this embodiment

[0159]

[0160]

[0161] The unstructured data in this embodiment comes from the internal collaboration platform, work logs, project documents, and 360-degree evaluations, specifically including the following data:

[0162] The task summary and problem feedback text in the work log (e.g., "Completed the requirements analysis of Project A and assisted in solving 3 technical difficulties");

[0163] The contribution description text in the project document (e.g., "Responsible for the code development and testing of Module B");

[0164] The 360-degree evaluation text of colleagues (e.g., "Efficient communication and can effectively coordinate cross-departmental resources").

[0165] The time-series data comes from historical performance records, including the monthly performance score time series of the past 12 months (e.g., [7.0, 7.5, 8.0, 7.8, 8.2]) and the performance scores at the key nodes of the project (requirement confirmation, delivery and acceptance) (example: the score at the requirement confirmation stage is 8.5, and the score at the delivery and acceptance stage is 9.0).

[0166] Furthermore, the following preprocessing is performed on the multi-source data in this embodiment:

[0167] (1) Structured data normalization

[0168] The normalization method is used to eliminate the dimensional difference, and the formula is:

[0169]

[0170] For example, the original value of an employee's task completion rate is 85% (X min = 60%, X max = 95%), after normalization, X norm = 0.714.

[0171] (2) Unstructured text feature extraction

[0172] Use the pre-trained BERT model (bert-base-chinese) to encode the text, output a 768-dimensional vector, and perform L2 normalization:

[0173]

[0174] Among them, is the L2 norm of the vector. represents the normalized text feature vector, represents the text feature vector before normalization.

[0175] For example, the vector of the text "Completed the requirements analysis of Project A" after being processed by BERT is The norm of the normalized vector is 1.

[0176] II. Construction of Dual-Branch Neural Network

[0177] (1) Structured Data Branch: It includes an input layer, a hidden layer, and an output layer;

[0178] Input Layer: 6-dimensional normalized structured data (corresponding to task completion rate, work efficiency, attendance rate, performance score, number of projects, role weight).

[0179] Hidden Layer: A 2-layer fully connected network with 64 neurons in each layer, and the activation function is ReLU.

[0180] Output Layer: A 64-dimensional feature vector H structured , and the calculation formula is:

[0181] H structured = ReLU(W structured X structured + b structured

[0182] (2) Unstructured Data Branch: It includes an input layer, a fully connected layer, and an output layer;

[0183] Input Layer: A 768-dimensional normalized vector output by BERT

[0184] Fully Connected Layer: 64 neurons, and the activation function is ReLU;

[0185] Output Layer: A 64-dimensional feature vector, and the calculation formula is:

[0186]

[0187] III. Introduction of Joint Attention Mechanism

[0188] First, perform dynamic weight allocation, and calculate the importance scores of structured and unstructured features through a single-layer MLP network:

[0189] score i = Linear(H i )

[0190] Then use the Softmax function to normalize the weights:

[0191]

[0192] Finally, perform cross-modal feature fusion: weighted summation to generate a 64-dimensional fused feature vector. For example, if the weight of the structured branch is 0.6 and the weight of the unstructured branch is 0.4, then the fused feature vector is the weighted sum of the two.

[0193] IV. Employee Portrait Generation

[0194] (1) Calculation of Ability Dimension:

[0195] Sub-dimensions: Professional Skills (weight 0.4), Communication Skills (weight 0.3), Execution Ability (weight 0.3). The calculation formula is as follows:

[0196]

[0197] (2) Calculation of Potential Dimension and Fit Dimension:

[0198] Potential Dimension: Growth Potential, Innovation Ability, Leadership Potential (weights are dynamically adjusted);

[0199] Fit Dimension: Job Fit Degree, Development Direction Fit Degree (weights are dynamically adjusted);

[0200] The calculation methods are the same as those of the ability dimension, and the formulas are as follows respectively:

[0201]

[0202] (3) Generation of Structured Feature Vector: Concatenate the scores of each dimension into a 9-dimensional vector, which is specifically represented as follows:

[0203] D final =[D ability ,D potential ,D adopt

[0204] As shown in Table 2 below:

[0205] Table 2 Distribution of Structured Feature Vectors in This Embodiment

[0206] Employee Professional skills Communication skills Execution ability Growth potential Innovation ability Leadership potential Job fit Development direction fit 001 0.87 0.75 0.82 0.68 0.55 0.72 0.91 0.78

[0207] V. Model Output and Generation of Structured Feature Vector

[0208] The generated structured feature vector D final is used as the basic data for subsequent job matching, talent evaluation, and development planning. It is stored in JSON or database form as follows:

[0209]

[0210] V. Generation of Visual Report

[0211] (1) Generation of Radar Chart: Standardize the scores of each dimension to the [0, 1] interval. The formula is:

[0212]

[0213] ​The three-dimensional radar chart shows the standardized scores, with the axes being ability, potential, and fit (as Figure 4 shown).

[0214] (2) Bar chart generation: The horizontal bar chart shows the dimensional score differences of different employees (as Figure 3 shown).

[0215] VI. Optimization and Adaptive Adjustment

[0216] (1) Hyperparameter optimization:

[0217] Optimization method: Grid search method, with the parameter range including:

[0218] Learning rate: 0.001, 0.005, 0.01;

[0219] Number of neurons in the hidden layer: 32, 64, 128;

[0220] Dropout rate: 0.2, 0.3, 0.5;

[0221] Cross-validation: The dataset is divided into a training set and a test set in a 7:3 ratio, and 5-fold cross-validation is used to evaluate the generalization ability of the model.

[0222] The mean squared error (MSE) of the final model on the test set is 0.023, and the accuracy rate is 92.5%.

[0223] VII. Calculation of Job Fit and Generation of Comprehensive Matching Degree

[0224] (1) Cosine similarity calculation:

[0225]

[0226] For example, if there is an employee 001 with a feature vector D final = [0.85, 0.75,..., 0.78], and the job requirement vector is [0.9, 0.8,..., 0.85], the similarity is 0.93.

[0227] (2) Generation of comprehensive matching degree: Weighted fusion of job fit and professional matching degree, with the formula:

[0228] D total = α·D ability + β·D major

[0229] For example, the comprehensive matching degree of employee 001 is 8.736, and that of employee B is 6.936.

[0230] In summary, the present invention can effectively improve the comprehensiveness and accuracy of portraits through multimodal fusion; at the same time, the attention mechanism can also realize the dynamic weight allocation of different job evaluation criteria and achieve the accuracy of job matching.

Claims

1. A method for generating an employee portrait based on multi-modal data fusion and joint attention mechanism, characterized in that Including: S1. Obtain multi-source data including structured data and unstructured data, and preprocess the data; The structured data includes but is not limited to employees' KPI indicators, performance scores, and project participation; the unstructured data includes but is not limited to employees' work logs, email records, project documents, and text data corresponding to colleague feedback; S2. Design a dual-branch neural network, where the first branch uses a fully connected neural network for key feature extraction of structured data; the second branch uses a pre-trained BERT model for semantic feature extraction of unstructured data; S3. Introduce a joint attention mechanism in the fusion layer to dynamically calculate the importance weights of different modal data, and finally generate a fused feature vector; S4. Generate an employee portrait based on the fused feature vector, and the portrait includes an ability dimension, a potential dimension, and a suitability dimension; S5. Train the neural network model through hyperparameter optimization, and tune the model through cross-validation; S6. Generate an optimized employee portrait according to the optimized neural network model.

2. The method for generating an employee portrait according to claim 1, wherein In S1, preprocessing the data specifically includes: normalizing the structured data, and / or performing text feature extraction and L2 normalization on the unstructured data using a pre-trained language model to obtain a normalized text feature vector 3. The method for generating an employee portrait according to claim 1, wherein In S2, the input layer of the fully connected neural network is 5-7 dimensions, the hidden layer is 2-4 layers, the activation function is ReLU, and the expression of the output feature vector is: H structured = ReLU(W structured X structred + b structured ) Among them, H structured is the feature representation obtained after the structured data passes through the fully connected neural network, and W structured is the weight matrix of the structured data branch, X structured is the input structured data feature vector with a dimension of d s , and b structured is the bias vector of the structured data branch.

4. The employee portrait generation method according to claim 1, characterized in that, In S2, the activation function of the BERT model is ReLU, and the expression of its output feature vector is: Among them, H unstructured is the feature representation after the fully connected layer for the unstructured data, W unstructured is the weight matrix of the unstructured data branch, is the text feature vector obtained after preprocessing the unstructured data, b unstructured is the bias vector of the unstructured data branch.

5. The method for generating an employee portrait according to any one of claims 1-4, characterized in that In S3, dynamically calculate the importance weights of different modal data, and finally generate a fused feature vector, specifically including: (1) Use a single-layer MLP network for dynamic weight allocation, and the Softmax normalization formula is: Among them, Weight i represents the attention weight of modality i, indicating the importance of this modality in the final fusion; score i represents the importance score of modality i calculated by the neural network; (2) Fuse the data feature vectors of different modalities to obtain a fused feature vector: Among them, H fused represents the fused feature vector, and H i represents the feature vector of modality i.

6. The method for generating an employee portrait according to claim 5, wherein In S4, the sub-dimensions of the ability dimension include but are not limited to professional skills, communication skills, and execution ability, and its calculation formula is: Among them, w i represents the importance weight of the i-th sub-dimension in the ability dimension, which can be dynamically adjusted according to the job requirements; f ability (H fused ) is the ability dimension score obtained by mapping through the fused feature H fused . The calculation of the potential dimension and the suitability dimension is similar to that of the ability dimension.

7. The method for generating an employee portrait according to claim 5, characterized in that, The hyperparameters in S5 include the learning rate, hidden layer size, Dropout rate, and batch size, where the learning rate is 0.001-0.01, the number of hidden layer neurons is 32, 64, or 128, the Dropout rate is 0.2-0.5, and the batch size Batch Size ∈ {32, 64, 128}.

8. The method for generating an employee portrait according to any one of claims 1-4, characterized in that It also includes generating a structured feature vector based on the optimized employee portrait, and generating a visualization report of the employee portrait based on the structured feature vector; The structured feature vector contains the performance of employees in different dimensions of ability, potential, and suitability; The visualization report of the employee portrait generated based on the structured feature vector includes but is not limited to radar charts and bar charts to display the scores of employees in each dimension.

9. The method for generating an employee portrait according to claim 8, wherein, The dimension scores of the radar chart are normalized to [0, 1], and the formula is: Among them, S norm is the standardized score, S is the original score, S min is the minimum score value of this dimension, S max is the maximum score value of this dimension; The bar chart is used to compare the scores of different employees in each dimension.

10. The method for generating an employee portrait according to any one of claims 1-4, characterized in that, It also includes: Calculate the comprehensive matching degree of the position according to the similarity between the optimized employee portrait and the position requirements, and the comprehensive matching degree formula is as follows: D total = α·D match + β·D major where α and β are weighting coefficients and their sum is 1; D match is the ability matching degree, and its calculation is as follows: In the above formula, D final is the comprehensive ability feature vector of employees obtained previously, and P i represents the demand feature vector of the position, which is manually set by the enterprise HR; D major which is calculated by the following formula for professional matching degree: D major = similarity(H major , P major ) In the above formula, H major is the professional feature vector of employees, which is the professional information extracted from structured data; P major is the professional requirement vector of the position.

Images