Employee work ability assessment method based on multi-modal health data fusion

Abstract

The invention discloses an employee work ability assessment method based on multi-modal health data fusion. The method comprises the steps of obtaining a plurality of health index values of physical examination data of an employee to be assessed, a plurality of self-assessment scale scores of psychological assessment data and a plurality of physiological monitoring index values of wearable device data; performing standardization processing on the acquired data, and converting the standardized data into feature vectors respectively; respectively inputting the feature vectors corresponding to the physical examination data and the psychological evaluation data into a trained multi-layer perceptron for feature extraction to obtain a physical examination feature vector and an evaluation feature vector; inputting the feature vector of the wearable device data into a trained long-short-term memory network for feature extraction to obtain a wearable feature vector; and inputting the physical examination feature vector, the evaluation feature vector and the wearing feature vector into a fusion module of a trained multi-modal learning model to perform feature fusion, and inputting the fused features into a DNN model of the multi-modal learning model to generate a work ability score.

Description

Technical Field

[0001] The present invention relates to enterprise employee work ability assessment technology, and in particular to an employee work ability assessment method based on multimodal health data fusion. Background Art

[0002] As companies' demand for employee health management continues to increase, healthy employees can not only improve production efficiency, but also reduce companies' medical expenses and human resource management costs.

[0003] In a complex working environment, the health status of employees is crucial to their work ability. For example, shale gas exploration covers multiple links from equipment operation, drilling operations to field exploration. Employees in different positions not only have different natures of work, but also face their own unique occupational health risks. For example, outdoor exploration personnel: they spend most of their time in the field conducting geological exploration, facing the challenges of extreme weather, rugged terrain and high physical energy consumption, and their physiological health risks are significantly increased, such as physical overdraft, joint injuries, skin problems, etc. At the same time, long-term work in an environment far away from the city and family may lead to social isolation and increased psychological pressure, which in turn affects their mental health and work performance. Therefore, companies need to accurately evaluate employees' work ability through comprehensive integration and analysis of health information, so as to adjust their positions in time when their work ability declines, ensure the physical and mental health of employees, reduce the company's medical expenses, and ensure the efficient operation of the company to reduce human resource management costs.

[0004] At present, traditional health management models usually rely on regular physical examinations and updates of health records to discover potential health problems. However, these methods often make it difficult to evaluate employees' work ability in real time and dynamically, and there is a lag. In addition, existing health management systems usually focus on employees' physical health, ignoring the impact of mental health and work environment on employees' work ability, and are difficult to cope with the diverse needs of the work environment. Summary of the invention

[0005] In view of the above-mentioned deficiencies in the prior art, the employee work ability assessment method based on multimodal health data fusion provided by the present invention solves the problem that the prior art cannot accurately assess the work ability of employees in different working environments.

[0006] In order to achieve the above-mentioned object of the invention, the technical solution adopted by the present invention is:

[0007] A method for evaluating employee work ability based on multimodal health data fusion is provided, which comprises the following steps:

[0008] S1. Obtain multiple health index values ​​of the physical examination data of the employee to be evaluated, multiple self-assessment scale scores of the psychological evaluation data, and multiple physiological monitoring index values ​​of the wearable device data;

[0009] S2, standardize the health index values, self-assessment scale scores and physiological monitoring index values, and convert the standardized physical examination data, wearable device data and work performance data into feature vectors respectively;

[0010] S3, inputting the feature vectors corresponding to the physical examination data and the psychological assessment data into the trained multi-layer perceptron for feature extraction, and obtaining a physical examination feature vector and an assessment feature vector;

[0011] S4, inputting the feature vector of the wearable device data into the trained long short-term memory network for feature extraction to obtain a wearable feature vector;

[0012] S5. Input the physical examination feature vector, the assessment feature vector and the wearing feature vector into the fusion module of the trained multimodal learning model to perform feature fusion, and input the fused features into the DNN model of the multimodal learning model to generate a work ability score.

[0013] Furthermore, the method for the fusion module to perform feature fusion includes:

[0014] S51, perform feature splicing on the physical examination feature vector and the wearing feature vector to obtain the first splicing feature h physical :

[0015] h physical =Concat(h health ,h wearabel )

[0016] Among them, concat() is a concatenation function used to combine two or more feature vectors into one during machine learning or data processing; h health is the physical examination feature vector; h wearabel is the wear feature vector;

[0017] S52, use the fully connected layer to the first concatenated feature h physical Perform dimensionality reduction operation to obtain the first dimensionality reduction data of 128 dimensions:

[0018] h′ physical =f(W physical h physical +b physical )

[0019] Among them, h′ physical is the first dimension reduction data, f() is the activation function Relu; W physical is the weight matrix of the physical examination data features; bphysical is the bias vector;

[0020] S53, perform feature concatenation on the evaluation feature vector and the first dimension reduction data to obtain the global feature h fusion :

[0021] h fusion =Concat(h psuch , h′ physical )

[0022] Among them, h psych is the evaluation feature vector;

[0023] S54, use the fully connected layer to analyze the global feature h fusion Perform dimensionality reduction operation to obtain the 64-dimensional fused feature h′ fusion :

[0024] h′ fusion =f(W fusion h fusion +b fusiotn )

[0025] Among them, W fusion is the feature weight matrix after modality fusion; b fusion is the bias vector.

[0026] The beneficial effect of the above technical solution is: this solution does not splice all modal data at once during the data fusion process, but adopts a phased fusion strategy based on the correlation between the data. First, the modalities with high correlation (physical examination feature vectors and wearing feature vectors) are locally fused, and then these fused features are gradually fused with other modal data (evaluation feature vectors). This method avoids the noise or information loss problems that may be caused by direct data splicing, can better maintain the independence of each modality, and gradually build a more effective global feature expression in the subsequent global fusion process.

[0027] Furthermore, the total loss function of the multimodal learning model is expressed as:

[0028]

[0029] Among them, L total is the total loss function; α i is the weight of the i-th feature vector in step S5; L i is the loss value of the i-th feature vector in step S5; λ1 and λ2 are both regularization coefficients; h i and h jare the i-th and j-th feature vectors in step S5 respectively; n is the total number of feature vectors input into the multimodal learning model for each employee in step S5; T is the transpose.

[0030] The beneficial effects of the above technical solution are as follows: the loss function of this solution can dynamically adjust the weight according to the contribution of each modality to the prediction task, thereby improving the prediction performance of the model. In addition, the regularization terms of the second and third terms on the right side of the loss function can effectively prevent a certain modality from monopolizing too much weight, ensuring that the features of each modality are reasonably utilized in the model. At the same time, the modal diversity constraint (the first term on the right side of the loss function) further encourages the differences between different modal features, reduces feature redundancy, and improves the generalization ability of the model. That is, the loss function constructed by this solution can flexibly respond to complex multimodal data scenarios while ensuring feature balance, significantly improving the model's prediction effect on employee work ability.

[0031] Furthermore, the DNN model includes multiple layers of fully connected layers and output layers, and each layer uses a nonlinear activation function ReLU;

[0032] The first hidden layer contains 256 neurons, and its nonlinear activation function ReLU is:

[0033] h1=ReLU(W1·h′ fusion +b1)

[0034] Among them, W1 is the weight matrix of the first hidden layer, its dimension is 256×64; b1 is the bias vector of the first hidden layer, its dimension is 256; h1 is the output of the first hidden layer, its dimension is 256;

[0035] The second hidden layer contains 128 neurons, and its nonlinear activation function ReLU is:

[0036] h2=ReLU(W2·h1+b2)

[0037] Among them, W2 is the weight matrix of the second hidden layer, and its dimension is 128×256; b2 is the bias vector of the second hidden layer, and its dimension is 128; h2 is the output of the second hidden layer, and its dimension is 128;

[0038] The third hidden layer contains 64 neurons, and its nonlinear activation function ReLU is:

[0039] h3=ReLU(W3·h2+b3)

[0040] Among them, W3 is the weight matrix of the third hidden layer, and its dimension is 64×128; b3 is the bias vector of the third hidden layer, and its dimension is 64; h3 is the output of the third hidden layer, and its dimension is 64;

[0041] The expression of the output layer is:

[0042]

[0043] Among them, W output is the weight matrix of the output layer, whose dimension is 1×64; b output is the bias vector of the output layer; Rate work ability.

[0044] The beneficial effects of the above technical solution are: by adopting the above method to carry out multimodal data fusion, a comprehensive employee health feature vector can be constructed, which can not only reflect the current health status of the employee, but also capture the changing trend of the employee's health status through dynamic updates, so as to more accurately evaluate their work ability. Enterprise managers can adjust the work arrangements of employees in real time according to the work ability score, prevent production risks caused by changes in health status, and optimize human resource management.

[0045] Furthermore, the method for extracting features from the feature vector of the physical examination data by the multi-layer perceptron includes:

[0046] S31. Use the first connection layer of the multi-layer perceptron to map the feature vector of the physical examination data to a high-dimensional space:

[0047] h 1，health =f(W 1，health x henlth +b 1,health )

[0048] Among them, h 1,health is the high-dimensional feature of the physical examination data, and its dimension is 128; W 1，health is the weight matrix of the first connection layer; b 1,health is the bias term of the first connection layer; f(.) = max(0, x health ) is the activation function, max(.) is the maximum value function, x health is the feature vector of physical examination data;

[0049] S32, use the second connection layer of the multi-layer perceptron to extract the high-dimensional features of the physical examination data to obtain a 64-dimensional physical examination feature vector h 2,health :

[0050] h 2，health =f(W 2，health h 1，health +b 2，health )

[0051] Among them, W 2,health is the weight matrix of the second connection layer; b 2,health is the bias term of the second connection layer;

[0052] S33, use the output layer of the multi-layer perceptron to output the physical examination feature vector h 2,health ;

[0053] The method of extracting the physical examination feature vector and the evaluation feature vector by the multi-layer perceptron is the same. The output dimension of the first connection layer of the evaluation feature vector is 256, and the output dimension of the second connection layer is 128;

[0054] The method of extracting features of the feature vector of wearable device data using the long short-term memory network includes:

[0055] S41. Calculate the output qt of the input gate of the long short-term memory network at the tth time step:

[0056] q t =σ(W q [h t-1 , x wearable (t)]+b q )

[0057] Among them, W q , b q are the weight and bias of the input gate respectively; h t-1 is the hidden state at the t-1th time step; is the M-dimensional time series feature vector of the wearable device data at the tth time step; M is the length of the time window; σ(.) is the sigmoid function; is the feature vector of M dimension;

[0058] S42. Calculate the output f of the forget gate of the long short-term memory network at the tth time step t :

[0059] f t =σ(W f [h t-1 , x wearable (t)]+b f )

[0060] Among them, W f and b f are the weight and bias of the forget gate respectively;

[0061] S43. Update the cell state at the tth time step according to the output of the input gate and the output of the forget gate:

[0062]

[0063] Among them, C t and C t-1 are the cell states at the tth and t-1th time steps respectively; is the tth time step is the parameter calculated from the input of the tth time step and the hidden state of the previous time step; W C and b C They are The weights and biases of ; tanh(.) is the hyperbolic tangent function;

[0064] S44. Calculate the output O of the output gate of the long short-term memory network at the tth time step t :

[0065] o t =σ(W o [h t-1 , x wearable (t)]+b o )

[0066] Among them, W o and W o are the weight and bias of the output gate respectively;

[0067] S45. Calculate the feature h extracted by the long short-term memory network at the tth time step according to the output of the output gate and the unit state. t :

[0068] h t =o t tanh(C t )

[0069] S46, repeat steps S41 to S45 until t = M, and output the feature vector h extracted at the Mth time step M , as the wear feature vector.

[0070] The beneficial effects of the above technical solution are: by extracting features from the time series data of wearable devices through the long short-term memory network (LSTM), it is possible to better capture the changes in the health status of employees at different time points, and combine physical examination and psychological assessment data to ensure that the generated work ability score is more comprehensive and accurate.

[0071] Furthermore, the training method of the multimodal learning model includes:

[0072] Using the method of steps S1 to S4, obtaining physical examination feature vectors, assessment feature vectors, and wearing feature vectors of a number of employees;

[0073] The method of steps S51 to S54 is used to obtain the fused features of each employee, and obtain the work ability score of each employee;

[0074] The fused features of each employee are used as the model input features in the training set, and the work ability score of each employee is used as the model output feature in the training set;

[0075] The training set is input into the DNN model in batches, and the DNN model is trained using the adaptive modal weight optimization algorithm AMWO until the model converges or the number of training times reaches the preset number, thereby obtaining a trained DNN model;

[0076] The fusion module and the trained DNN model are used to form a trained multimodal learning model.

[0077] Furthermore, when training the DNN model, the expression for updating the model parameters of the i-th feature vector is:

[0078]

[0079] in, and are the model parameters of the i-th feature vector in step S5 during the m+1-th and m-th trainings respectively; for The gradient of i is the learning rate of the i-th feature vector in step S5; β is the hyperparameter controlling the adjustment of the learning rate; η0 is the initial learning rate; α i is the weight of the i-th feature vector in step S5; L i is the loss value of the i-th feature vector in step S5.

[0080] The beneficial effect of the above technical solution is: due to the fact that different data sources may have different effects on employees' work ability during the fusion process of multimodal data (this solution includes 3 modes, namely physical examination data, wearable device data and psychological assessment data), this solution dynamically adjusts the weight of each modality data to ensure that the evaluation results can accurately reflect the importance of each data source, that is, this solution will optimize the contribution of each modality according to the actual performance of the data and health status.

[0081] The update of model parameters can intelligently handle the complex relationship between multimodal data, avoid excessive impact of a single data source on the evaluation results, and improve the accuracy and robustness of work ability evaluation. At the same time, it can also adaptively learn the importance of each modal feature, optimize the evaluation process, and avoid data bias in the evaluation.

[0082] Furthermore, the method for obtaining the work ability score includes:

[0083] Obtain multiple health index values ​​of the employee's physical examination data, scores of multiple self-assessment scales of psychological assessment data, and scores of multiple scoring standard indicators of work performance data;

[0084] According to the scoring table corresponding to the physical examination data, the score of each health index value is obtained;

[0085] According to the scores of health index values, self-assessment scales, and scoring criteria, the health data score, mental health score, and work performance score are obtained by weighted average.

[0086] The work ability score is obtained by taking a weighted average based on the health data score, mental health score and work performance score.

[0087] Furthermore, multiple health indicators include BMI, blood pressure and blood sugar; multiple self-assessment scales include anxiety self-assessment scale and depression self-assessment scale; multiple scoring standard indicators include performance indicators, attendance rate, work efficiency, teamwork and communication skills and subjective scores of managers; multiple physiological monitoring indicators include heart rate, sleep and activity level.

[0088] Compared with the prior art, the present invention has the following beneficial effects:

[0089] This solution can fully reflect the health status of employees in complex jobs and working environments by integrating multimodal data (physical examination data in physical examination reports, wearable device data collected by wearable devices, and psychological assessment data). In this way, the health status of employees can be evaluated from multiple dimensions. Combined with the multimodal learning model, it can accurately capture the impact of employee health on work ability, and thus obtain a relatively accurate work ability score for employees based on multimodal data. Based on the score, corporate managers can accurately know whether employees are currently competent for their jobs, so as to make timely adjustments to their jobs, ensure the physical and mental health of employees, reduce the company's medical expenses, and ensure efficient operation of the company to reduce human resource management costs.

[0090] Employees in different occupations face different occupational health risks. For example, the health assessment of equipment operators focuses on the burden on the musculoskeletal system, while the assessment of prospectors needs to pay more attention to the physiological and psychological stress in extreme working environments. Through multimodal data fusion technology and artificial intelligence algorithms, this solution can provide personalized work ability assessments for employees in different occupations based on the actual work scenarios of different positions (the data collected by wearable devices can reflect the impact of different work scenarios on the physiological monitoring data of employees), so as to accurately reflect the specific health status of each employee, help companies identify high-risk employees, optimize health management strategies, and ensure the physical and mental health of employees.

[0091] When evaluating this solution, the wearable device data can be the latest collected data, and the physical examination report data and psychological assessment data can be the most recently acquired data, so that changes in employee health status can be captured in a timely manner to avoid health risks caused by delayed evaluation. Enterprises can adjust employees' work arrangements in a timely manner based on the latest work ability assessment results to avoid a decline in work ability due to changes in health status, thereby effectively reducing the risk of production accidents.

[0092] Through accurate work capacity assessment, this solution can effectively reduce the loss of working hours caused by health problems. At the same time, through personalized health intervention and early warning, enterprises can reduce the incidence of major diseases or health problems, thereby reducing high medical expenses. It can improve the company's ability to optimize human resources management, reduce economic losses caused by employee health problems, and enhance employees' work enthusiasm and long-term work ability, ultimately improving the company's overall production efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] Figure 1 Flowchart of the employee work ability assessment method based on multimodal health data fusion. DETAILED DESCRIPTION

[0094] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0095] refer to Figure 1 , Figure 1 A flowchart of an employee work ability assessment method based on multimodal health data fusion is shown; Figure 1 As shown, the method S includes steps S1 to S5.

[0096] In step S1, multiple health index values ​​of the physical examination data of the employee to be evaluated, multiple self-assessment scale scores of the psychological assessment data, and multiple physiological monitoring index values ​​of the wearable device data are obtained; the multimodality in the subject of this scheme refers to three modal data: physical examination data, psychological assessment data, and wearable device data. Multiple health indicators include BMI, blood pressure, and blood sugar; multiple self-assessment scales include anxiety self-assessment scales and depression self-assessment scales; multiple physiological monitoring indicators include heart rate, sleep, and activity.

[0097] In step S2, the health index values, self-assessment scale scores and physiological monitoring index values ​​are standardized, and the standardized physical examination data, wearable device data and work performance data are converted into feature vectors respectively; the health index values ​​and self-assessment scale scores are Z-score standardized, and the physiological parameters measured in the wearable device data (such as heart rate, number of steps, etc.) are Min-Max standardized.

[0098] The standardized data of the three modes are vectorized respectively:

[0099] (1) Physical examination report data is usually structured tabular data, and the value of each health indicator (such as BMI, blood pressure, blood sugar, etc.) is already in numerical form. Assume that the physical examination report contains d h health indicators (such as BMI, blood pressure, blood sugar, etc.), and the physical examination data is represented as a vector in the following form:

[0100]

[0101] in, is the feature vector of the physical examination data, is d h dimensional feature vector; The 1st, 2nd, ...d h health index value.

[0102] (2) Psychological assessment data are usually scale scores, which indicate the mental health status of employees. Each assessment scale corresponds to multiple scoring items. Assume that the psychological assessment includes d p There are several scoring items, such as the Self-Rating Anxiety Scale (SAS) and the Self-Rating Depression Scale (SDS), each of which has several sub-item scores. The vectorized form of psychological assessment data is:

[0103]

[0104] in, is the feature vector of psychological assessment data, is d p dimensional feature vector; The 1st, 2nd, and 3rd items of the anxiety self-rating scale or depression self-rating scale are p Item rating.

[0105] (3) Wearable device data is usually time series data, recording changes in employees’ physiological characteristics (such as heart rate, sleep, number of steps, etc.). The sliding window method is used to convert time series data into feature vectors:

[0106] x wearable (t) = [w t , w t+1 , ...w t+M-1 ]

[0107] in, is the M-dimensional time series feature vector of the wearable device data at the tth time step; w t , w t+1 , ..., w t+M-1 They are the wearable device data collected at the t, t+1...t+M-1 time steps respectively.

[0108] In step S3, the feature vectors corresponding to the physical examination data and the psychological assessment data are respectively input into the trained multi-layer perceptron for feature extraction to obtain a physical examination feature vector and an assessment feature vector;

[0109] During implementation, the method for extracting features from the feature vector of the physical examination data by using a multi-layer perceptron preferably includes:

[0110] S31. Use the first connection layer of the multi-layer perceptron to map the feature vector of the physical examination data to a high-dimensional space: h 1，health =f(W 1，health x health +b 1,health )

[0111] Among them, h i,health is the high-dimensional feature of the physical examination data, and its dimension is 128; W 1,health is the weight matrix of the first connection layer; b 1,health is the bias term of the first connection layer; f(.) = max(0, x health ) is the activation function, max(.) is the maximum value function, x health is the feature vector of physical examination data;

[0112] S32, use the second connection layer of the multi-layer perceptron to extract the high-dimensional features of the physical examination data to obtain a 64-dimensional physical examination feature vector h 2,health :

[0113] h 2，health =f(W 2，health h 1，health +b 2，health )

[0114] Among them, W 2,health is the weight matrix of the second connection layer; b 2,health is the bias term of the second connection layer;

[0115] S33, use the output layer of the multi-layer perceptron to output the physical examination feature vector h 2,health ;

[0116] The method of extracting the physical examination feature vector and the evaluation feature vector by the multilayer perceptron is the same. The output dimension of the first connection layer of the evaluation feature vector is 256, and the output dimension of the second connection layer is 128.

[0117] In step S4, the feature vector of the wearable device data is input into the trained long short-term memory network for feature extraction to obtain a wear feature vector; the specific implementation method of this step includes:

[0118] S41. Calculate the output qt of the input gate of the long short-term memory network at the tth time step:

[0119] q t =σ(W q [h t-1 , x wearable (t)]+b q )

[0120] Among them, W q , b q are the weight and bias of the input gate respectively; h t-1 is the hidden state at the t-1th time step; is the M-dimensional time series feature vector of the wearable device data at the tth time step; M is the length of the time window; σ(.) is the sigmoid function; its function is to limit the output to the range of [0, 1]; is the feature vector of M dimension;

[0121] S42. Calculate the output f of the forget gate of the long short-term memory network at the tth time step t :

[0122] f t =σ(W f [h t-1 , x wearable (t)]+b f )

[0123] Among them, W f and b f are the weight and bias of the forget gate respectively;

[0124] S43. Update the cell state at the tth time step according to the output of the input gate and the output of the forget gate:

[0125]

[0126] Among them, C t and C t-1 are the cell states at the tth and t-1th time steps respectively; is the parameter calculated from the input of the tth time step and the hidden state of the previous time step; W C and b C They are weights and biases; tanh(.) is the hyperbolic tangent function, which is an activation function that compresses the input value into the interval [-1, 1].

[0127] S44. Calculate the output o of the output gate of the long short-term memory network at the tth time step t :

[0128] o t =σ(W o [h t-1, x wearable (t)]+b o )

[0129] Among them, W o and W o are the weight and bias of the output gate respectively;

[0130] S45. Calculate the feature h extracted by the long short-term memory network at the tth time step according to the output of the output gate and the unit state. t :

[0131] h t =o t tanh(C t )

[0132] S46, repeat steps S41 to S45 until t = M, and output the feature vector h extracted at the Mth time step M , as the wear feature vector.

[0133] In step S5, the physical examination feature vector, the assessment feature vector and the wearing feature vector are input into the fusion module of the trained multimodal learning model for feature fusion, and the fused features are input into the DNN model of the multimodal learning model to generate a work ability score.

[0134] In one embodiment of the present invention, the method for the fusion module to perform feature fusion includes:

[0135] S51, perform feature splicing on the physical examination feature vector and the wearing feature vector to obtain the first splicing feature h physical :

[0136] h physical =Concat(h health ,h wearabel )

[0137] Among them, concat() is a concatenation function used to combine two or more feature vectors into one during machine learning or data processing; h health is the physical examination feature vector; h wearabel is the wear feature vector;

[0138] S52, use the fully connected layer to the first concatenated feature h physical Perform dimensionality reduction operation to obtain the first dimensionality reduction data of 128 dimensions:

[0139] h′ physical =f(W physical h physical +b physical )

[0140] Among them, h′ physicalis the first dimension reduction data; f() is the activation function Relu, which is used to perform nonlinear processing on the linearly transformed input in order to introduce the complexity and expression ability of the model; Wphysical is the weight matrix of the physical examination data features, which is used for h physical Perform linear changes; b physical is the bias vector;

[0141] S53, perform feature concatenation on the evaluation feature vector and the first dimension reduction data to obtain the global feature h fusion :

[0142] h fusion =Concat(h psuch , h′ physical )

[0143] Among them, h psych is the evaluation feature vector;

[0144] S54, use the fully connected layer to analyze the global feature h fusion Perform dimensionality reduction operation to obtain the 64-dimensional fused feature h′ fusion :

[0145] h′ fusion =f(W fusion h fusion +b fusion )

[0146] Among them, W fusion is the feature weight matrix after modality fusion; b fusion is the bias vector.

[0147] During implementation, the expression of the total loss function of the multimodal learning model preferred by this scheme is:

[0148]

[0149] Among them, L total is the total loss function; α i is the weight of the i-th feature vector in step S5; L i is the loss value of the i-th feature vector in step S5; λ1 and λ2 are both regularization coefficients; h i and h j are the i-th and j-th feature vectors in step S5 respectively; n is the total number of feature vectors input into the multimodal learning model for each employee in step S5; T is the transpose.

[0150] The loss function of this solution can dynamically adjust the weight according to the contribution of each modality to the prediction task, thereby improving the prediction performance of the model. In addition, the regularization terms of the second and third terms on the right side of the loss function can effectively prevent a certain modality from monopolizing too much weight, ensuring that the features of each modality are reasonably used in the model. At the same time, the modal diversity constraint (the first term on the right side of the loss function) further encourages the differences between different modal features, reduces feature redundancy, and improves the generalization ability of the model. That is, the loss function constructed by this solution can flexibly respond to complex multimodal data scenarios while ensuring feature balance, significantly improving the model's prediction effect on employee work ability.

[0151] In one embodiment of the present invention, the DNN model includes multiple layers of fully connected layers and output layers, each layer uses a nonlinear activation function ReLU;

[0152] The first hidden layer contains 256 neurons, and its nonlinear activation function ReLU is:

[0153] h1=ReLU(W1·h′ fusion +b1)

[0154] Among them, W1 is the weight matrix of the first hidden layer, its dimension is 256×64; b1 is the bias vector of the first hidden layer, its dimension is 256; h1 is the output of the first hidden layer, its dimension is 256;

[0155] The second hidden layer contains 128 neurons, and its nonlinear activation function ReLU is:

[0156] h2=ReLU(W2·h1+b2)

[0157] Among them, W2 is the weight matrix of the second hidden layer, and its dimension is 128×256; b2 is the bias vector of the second hidden layer, and its dimension is 128; h2 is the output of the second hidden layer, and its dimension is 128;

[0158] The third hidden layer contains 64 neurons, and its nonlinear activation function ReLU is:

[0159] h3=ReLU(W3·h2+b3)

[0160] Among them, W3 is the weight matrix of the third hidden layer, and its dimension is 64×128; b3 is the bias vector of the third hidden layer, and its dimension is 64; h3 is the output of the third hidden layer, and its dimension is 64;

[0161] The expression of the output layer is:

[0162]

[0163] Among them, W outputis the weight matrix of the output layer, whose dimension is 1×64; b output is the bias vector of the output layer; Rate work ability.

[0164] In one embodiment of the present invention, the training method of the multimodal learning model includes:

[0165] Using the method of steps S1 to S4, obtaining physical examination feature vectors, assessment feature vectors, and wearing feature vectors of a number of employees;

[0166] The method of steps S51 to S54 is used to obtain the fused features of each employee, and obtain the work ability score of each employee;

[0167] The fused features of each employee are used as the model input features in the training set, and the work ability score of each employee is used as the model output feature in the training set;

[0168] The training set is input into the DNN model in batches, and the DNN model is trained using the adaptive modal weight optimization algorithm AMWO until the model converges or the number of training times reaches the preset number, thereby obtaining a trained DNN model;

[0169] The fusion module and the trained DNN model are used to form a trained multimodal learning model.

[0170] When training the DNN model, the expression for updating the model parameters of the i-th feature vector is:

[0171]

[0172] in, and are the model parameters of the i-th feature vector in step S5 during the m+1-th and m-th trainings respectively; for The gradient of i is the learning rate of the i-th feature vector in step S5; β is the hyperparameter controlling the adjustment of the learning rate; η0 is the initial learning rate; α i is the weight of the i-th feature vector in step S5; L i is the loss value of the i-th feature vector in step S5.

[0173] During implementation, the method for obtaining the work ability score preferably includes:

[0174] Obtain multiple health index values ​​of employees' physical examination data, scores of multiple self-assessment scales of psychological assessment data, and scores of multiple scoring standard indicators of work performance data; multiple scoring standard indicators include performance indicators, attendance rate, work efficiency, teamwork and communication skills, and subjective scores of managers;

[0175] According to the scoring table corresponding to the physical examination data, the score of each health index value is obtained;

[0176] According to the scores of health index values, self-assessment scales, and scoring criteria, the health data score, mental health score, and work performance score are obtained by weighted average:

[0177]

[0178] Among them, w BMI 、w 血压 and w 血糖 are the weights of BMI, blood pressure and blood sugar respectively; SAS and w SDS are the indicator weights of anxiety self-assessment scores and depression self-assessment scores respectively; w 绩效 、w 绩效 、w 出勤 、w 效率 、w 团队合作 and w 主观评估 They are the weights of performance score, attendance score, efficiency score, teamwork score and subjective score of managers.

[0179] Based on the health data score, mental health score and work performance score, the work ability score is obtained by weighted average:

[0180]

[0181] Among them, w 健康 、w 健康 and w 工作表现 These are the weights of health data score, mental health score and work performance score respectively.

[0182] To sum up, the evaluation method of this scheme generates relatively comprehensive health fusion data by integrating multi-source data (physical examination data, wearable device data, psychological assessment data, etc.). This data fusion goes beyond the limitations of traditional single-modal health management systems and can provide more accurate work ability assessments based on employees' health status, so that managers can accurately know whether employees are currently competent for their jobs, so as to make timely adjustments to their jobs, ensure the physical and mental health of employees, reduce economic losses caused by employee health problems, and enhance employees' work enthusiasm and long-term work ability, ultimately improving the overall production efficiency of the enterprise.

Claims

1. An employee work ability assessment method based on multimodal health data fusion, characterized in that: Includes steps: S1. Obtain multiple health index values ​​of the physical examination data of the employee to be evaluated, multiple self-assessment scale scores of the psychological evaluation data, and multiple physiological monitoring index values ​​of the wearable device data; S2, standardize the health index values, self-assessment scale scores and physiological monitoring index values, and convert the standardized physical examination data, wearable device data and work performance data into feature vectors respectively; S3, inputting the feature vectors corresponding to the physical examination data and the psychological assessment data into the trained multi-layer perceptron for feature extraction, and obtaining a physical examination feature vector and an assessment feature vector; S4, inputting the feature vector of the wearable device data into the trained long short-term memory network for feature extraction to obtain a wearable feature vector; S5. Input the physical examination feature vector, the assessment feature vector and the wearing feature vector into the fusion module of the trained multimodal learning model to perform feature fusion, and input the fused features into the DNN model of the multimodal learning model to generate a work ability score.

2. The employee work ability assessment method based on multimodal health data fusion according to claim 1 is characterized in that: The method of feature fusion performed by the fusion module includes: S51, perform feature splicing on the physical examination feature vector and the wearing feature vector to obtain the first splicing feature h physical : h physical =Concat(h health ,h wearabel ) Among them, concat() is the concatenation function; h health is the physical examination feature vector; h wearabel is the wear feature vector; S52, use the fully connected layer to the first concatenated feature h physical Perform dimensionality reduction operation to obtain the first dimensionality reduction data of 128 dimensions: h′ physical =f(W physical h physical +b physical ) Among them, h′ physical is the first dimension reduction data; f( ) is the activation function Re l u;W physical is the weight matrix of the physical examination data features; b physical is the bias vector; S53, perform feature concatenation on the evaluation feature vector and the first dimension reduction data to obtain the global feature h fusion : h fusion =Concat(h hpsuch ,h′ physical ) Among them, h psych is the evaluation feature vector; S54, use the fully connected layer to analyze the global feature h fusion Perform dimensionality reduction operation to obtain the 64-dimensional fused feature h′ fusion : h′ fusion =f(W fusion h fusion +b fusion ) Among them, W fusion is the feature weight matrix after modality fusion; b fusion is the bias vector.

3. The employee work ability assessment method based on multimodal health data fusion according to claim 1 or 2 is characterized in that: The expression of the total loss function of the multimodal learning model is: Among them, L total is the total loss function; α i is the weight of the i-th eigenvector in step S5; Li is the loss value of the i-th eigenvector in step S5; λ1 and λ2 are both regularization coefficients; h i and h j are the i-th and j-th feature vectors in step S5 respectively; n is the total number of feature vectors input into the multimodal learning model for each employee in step S5; T is the transpose.

4. The employee work ability assessment method based on multimodal health data fusion according to claim 2 is characterized in that: The DNN model includes multiple layers of fully connected layers and output layers, each layer uses a nonlinear activation function ReLU; The first hidden layer contains 256 neurons, and its nonlinear activation function ReLU is: h1=ReLU(W1 h′ fusion +b1) Among them, W1 is the weight matrix of the first hidden layer, its dimension is 256×64; b1 is the bias vector of the first hidden layer, its dimension is 256; h1 is the output of the first hidden layer, its dimension is 256; The second hidden layer contains 128 neurons, and its nonlinear activation function ReLU is: h2=ReLU(W2·h1+b2) Among them, W2 is the weight matrix of the second hidden layer, and its dimension is 128×256; b2 is the bias vector of the second hidden layer, and its dimension is 128; h2 is the output of the second hidden layer, and its dimension is 128; The third hidden layer contains 64 neurons, and its nonlinear activation function ReLU is: h3=ReLU(W3·h2+b3) Among them, W3 is the weight matrix of the third hidden layer, and its dimension is 64×128; b3 is the bias vector of the third hidden layer, and its dimension is 64; h3 is the output of the third hidden layer, and its dimension is 64; The expression of the output layer is: Among them, W output is the weight matrix of the output layer, whose dimension is 1×64; b output is the bias vector of the output layer; Rate work ability.

5. The employee work ability assessment method based on multimodal health data fusion according to claim 2 is characterized in that: The method of extracting features from the feature vector of physical examination data by a multi-layer perceptron includes: S31. Use the first connection layer of the multi-layer perceptron to map the feature vector of the physical examination data to a high-dimensional space: h 1，heaith =f(W 1，health x health +b 1，health ) Among them, h 1，health is the high-dimensional feature of the physical examination data, and its dimension is 128; W 1,health is the weight matrix of the first connection layer; b 1,health is the bias term of the first connection layer; f(.) = max(0, x health ) is the activation function, max(.) is the maximum value function, x health is the feature vector of physical examination data; S32, use the second connection layer of the multi-layer perceptron to extract the high-dimensional features of the physical examination data to obtain a 64-dimensional physical examination feature vector h 2,health : h 2，health =f(W 2，health h 1，health +b 2，health ) Among them, W 2,health is the weight matrix of the second connection layer; b 2,health is the bias term of the second connection layer; S33, use the output layer of the multi-layer perceptron to output the physical examination feature vector h 2,health ; The method of extracting the physical examination feature vector and the evaluation feature vector by the multi-layer perceptron is the same. The output dimension of the first connection layer of the evaluation feature vector is 256, and the output dimension of the second connection layer is 128; The method of extracting features of the feature vector of wearable device data using the long short-term memory network includes: S41. Calculate the output q of the input gate of the long short-term memory network at the tth time step t : q t =σ(W q [h t-1 ,x wearable (t)]+b q ) Among them, W q , b q are the weight and bias of the input gate respectively; h t-1 is the hidden state at the t-1th time step; is the M-dimensional time series feature vector of the wearable device data at the tth time step; M is the length of the time window; σ(.) is the sigmoid function; is the feature vector of M dimension; S42. Calculate the output f of the forget gate of the long short-term memory network at the tth time step t : f t =σ(W f [h t-1 ,x wearable (t)]+b f ) Among them, W f and b f are the weight and bias of the forget gate respectively; S43. Update the cell state at the tth time step according to the output of the input gate and the output of the forget gate: Among them, C t and C t―1 are the cell states at the tth and t-1th time steps respectively; is the parameter calculated from the input of the tth time step and the hidden state of the previous time step; W C and b C They are The weights and biases of ; tanh(.) is the hyperbolic tangent function; S44. Calculate the output o of the output gate of the long short-term memory network at the tth time step t : o t =σ(W o [h t-1 ,x wearable (t)]+b o ) Among them, W o and W o are the weight and bias of the output gate respectively; S45. Calculate the feature h extracted by the long short-term memory network at the tth time step according to the output of the output gate and the unit state. t : h t =o t ·tanh(C t ) S46, repeat steps S41 to S45 until t = M, and output the feature vector h extracted at the Mth time step M , as the wear feature vector.

6. The employee work ability assessment method based on multimodal health data fusion according to claim 5 is characterized in that: The training methods of multimodal learning models include: Using the method of steps S1 to S4, obtaining physical examination feature vectors, assessment feature vectors, and wearing feature vectors of a number of employees; The method of steps S51 to S54 is used to obtain the fused features of each employee, and obtain the work ability score of each employee; The fused features of each employee are used as the model input features in the training set, and the work ability score of each employee is used as the model output feature in the training set; The training set is input into the DNN model in batches, and the DNN model is trained using the adaptive modal weight optimization algorithm AMWO until the model converges or the number of training times reaches the preset number, thereby obtaining a trained DNN model; The fusion module and the trained DNN model are used to form a trained multimodal learning model.

7. The employee work ability assessment method based on multimodal health data fusion according to claim 6 is characterized in that: When training the DNN model, the expression for updating the model parameters of the i-th feature vector is: in, and are the model parameters of the i-th feature vector in step S5 during the m+1-th and m-th trainings respectively; for The gradient of i is the learning rate of the i-th feature vector in step S5; β is the hyperparameter controlling the adjustment of the learning rate; η0 is the initial learning rate; α i is the weight of the i-th feature vector in step S5; L i is the loss value of the i-th feature vector in step S5.

8. The employee work ability assessment method based on multimodal health data fusion according to claim 6 is characterized in that: The method for obtaining the work ability score includes: Obtain multiple health index values ​​of the employee's physical examination data, scores of multiple self-assessment scales of psychological assessment data, and scores of multiple scoring standard indicators of work performance data; According to the scoring table corresponding to the physical examination data, the score of each health index value is obtained; According to the scores of health index values, self-assessment scales, and scoring criteria, the health data score, mental health score, and work performance score are obtained by weighted average. The work ability score is obtained by taking a weighted average based on the health data score, mental health score and work performance score.

9. The employee work ability assessment method based on multimodal health data fusion according to claim 8 is characterized in that: Multiple health indicators include BMI, blood pressure and blood sugar; multiple self-assessment scales include anxiety self-assessment scale and depression self-assessment scale; multiple scoring standard indicators include performance indicators, attendance rate, work efficiency, teamwork and communication skills and subjective scores of managers; multiple physiological monitoring indicators include heart rate, sleep and activity level.

Images