A method and device for evaluating work behavior based on bone motion phase

Abstract

The application discloses a kind of based on skeleton movement phase's work behavior evaluation method and device, wherein the method includes: based on human dynamic capture or camera sensor obtains action picture and carries out the way of deep learning to obtain human motion skeleton data;To human motion skeleton data carry out joint node movement state automatic label strategy processing, obtain joint node movement state automatic label result;Based on joint node movement state automatic label result carries out the calculation processing of skeleton movement phase, obtains the calculation result of skeleton movement phase;Based on the calculation result of skeleton movement phase carries out the work behavior fast evaluation processing of staff, obtains the overall evaluation result to the behavior of work personnel.In the embodiment of the application, more automated, intelligent whole body evaluation analysis of staff can be provided, and then real-time monitoring and timely early warning prediction of high-risk situations of staff during work process.

Description

Technical Field

[0001] This invention relates to the field of computer vision technology, and in particular to a method and apparatus for evaluating work behavior based on skeletal motion phase. Background Technology

[0002] The complexity and inherent characteristics of the construction environment lead construction workers to perform many hazardous tasks. While occupational diseases have been gradually brought under control with advancements in science and technology, work-related musculoskeletal disorders are chronic, cumulative illnesses that account for over 30% of work-related injuries and illnesses leading to absenteeism across all industries. Repetitive tasks, improper work movements, and prolonged periods of high-intensity labor significantly impact people's daily lives. Therefore, scientifically assessing workers' postures during work is crucial for early detection and prevention of potential occupational health risks.

[0003] Traditional methods for assessing worker posture include self-assessment and observational assessment. Self-assessment involves analyzing work diaries, interviews, and questionnaires to collect data on unfavorable factors affecting worker performance in the workplace. Observational assessment, on the other hand, relies heavily on on-site managers' subjective judgment and professional knowledge when evaluating worker behavior and potential risks. Both methods have significant limitations in practical application. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art. This invention provides a method and device for evaluating work behavior based on skeletal motion phase, which can provide more automated and intelligent full-body assessment and analysis of workers, and then monitor in real time during the work process and provide timely early warning and prediction of high-risk situations caused by improper posture or long-term high-intensity work.

[0005] To address the aforementioned technical problems, embodiments of the present invention provide a method for evaluating work behavior based on skeletal motion phases, the method comprising:

[0006] Human motion skeleton data is obtained by acquiring motion images using human motion capture devices or camera sensors and then performing deep learning.

[0007] The human skeletal data is processed using an automated labeling strategy for joint motion states to obtain automated labeling results for joint motion states.

[0008] Based on the automated labeling results of the joint motion state, the bone motion phase is calculated to obtain the bone motion phase calculation result.

[0009] The calculation results of skeletal motion phase based on the rapid whole-body assessment strategy are used to evaluate work behavior and obtain a holistic evaluation result of work behavior.

[0010] Optionally, the method of obtaining human motion skeleton data based on acquiring motion images using a human motion capture device or camera sensor and performing deep learning includes:

[0011] When obtaining human motion skeleton data based on human motion capture, the bone point data in the human motion skeleton data is three-dimensional data;

[0012] When obtaining human motion skeleton data by acquiring motion images based on camera sensors and performing deep learning, the bone point data in the human motion skeleton data is two-dimensional data.

[0013] Optionally, the mathematical formula for the automated labeling strategy for joint motion states is expressed as follows:

[0014]

[0015] in, This indicates the position of the human motion skeleton data k in the i-th frame; The position of the human motion skeleton data k in frame i-1 is represented; ΔT represents the time interval between the detections of the two pose estimators; v min This represents the minimum value of the threshold parameter for the primary condition adjusted according to image resolution; v max This represents the maximum value of the threshold parameter of the primary condition adjusted according to the image resolution; This represents the automated labeling result of the joint motion state of human motion skeleton data k in the i-th frame; N represents the number of frames; d min This represents the secondary condition threshold adjusted based on an N-frame window.

[0016] Optionally, the step of calculating the skeletal motion phase based on the automated labeling results of the joint motion states to obtain the skeletal motion phase calculation results includes:

[0017] Based on the automated labeling results of the joint motion state, the original motion state function is normalized to obtain normalized motion state data.

[0018] The normalized motion state data is filtered using a Butterworth low-pass filter to obtain the filtering result.

[0019] The filtering result is subjected to curve fitting processing, and the bone motion phase is calculated based on the curve fitting result to obtain the bone motion phase calculation result.

[0020] Optionally, the formula for filtering the normalized motion state data based on the Butterworth low-pass filter is as follows:

[0021]

[0022] in, This represents a Butterworth low-pass filter, where Z represents the order of the Butterworth low-pass filter, and Z takes the value of 3; w n The parameters related to the cutoff frequency are represented, which are calculated based on the Shannon-Nyquist sampling theorem; G(i) represents the low-pass filtering result of the i-frame data; and Y represents the result of normalizing the original motion state label data.

[0023] Optionally, the curve fitting process for the filtered result includes:

[0024] The filtering results are then subjected to curve fitting based on a sine function.

[0025] The formula for curve fitting is as follows:

[0026] Ω(F i ) = a i ·sin(f i ·i+s i )+b i ;

[0027] Among them, F i Let F represent the objective function. i Parameterization yields F i =(a i ,f i ,s i ,b i ), where i is the index frame. This curve fitting process minimizes the following root mean square error loss within an N-frame window for each frame i:

[0028]

[0029]

[0030] Where, φ i f represents the motion phase at the i-th frame for any skeletal joint, used to reflect the stage of periodic motion; i Indicates the frequency of skeletal movement; a i This represents the optimized fitted amplitude parameter; s i b is the initial phase parameter; i is the offset parameter; N is the window width; t is the time index within the window with a center width of N in the i-th frame; G(t) is the result value after Butterworth low-pass filtering at time t.

[0031] Optionally, the calculation of the skeletal motion phase based on the curve fitting results includes:

[0032] When the skeleton is stationary, the motion phase is an uncertain situation, which is the optimized fitting amplitude parameter 'a'. i Combined, an updated motion phase is generated as follows:

[0033] P i =S1(a i )·φ i

[0034]

[0035] Where S1(.) represents the left side a min , right side a max The smoothing function; when the skeleton is not moving, the motion phase modulated by the amplitude parameter becomes zero; φ i P represents the motion phase at the i-th frame for any skeletal joint, used to reflect the stage of periodic motion; i This indicates an update of the motion phase.

[0036] Optionally, the calculation results of skeletal motion phase based on the rapid whole-body assessment strategy are processed for job behavior assessment to obtain a holistic assessment result of job behavior, including:

[0037] Based on the calculation results of the bone motion phase, the angle of joint bending is calculated to obtain the joint bending angle.

[0038] Fuzzy logic processing is performed based on the bending angle of the joints, and the whole-body assessment results are obtained based on the fuzzy logic processing results.

[0039] Optionally, the step of performing fuzzy logic processing based on the joint bending angle and obtaining the whole-body assessment result based on the fuzzy logic processing result includes:

[0040] The fuzzy logic processing based on the joint bending angle is as follows:

[0041]

[0042] Where, μ A (θ) represents the fuzzy logic processing result of the joint bending angle; θ represents the joint bending angle, θ∈[α,β]; I is the introduced extended parameter;

[0043] The scoring and evaluation results for individual joints are as follows:

[0044] μ B (θ i )=f i ×(S1(ai )·φ i +μ A (θ i ));

[0045] The scoring and assessment of all joint points in the body are as follows:

[0046] Score = ∑ i∈C μ B (θ i );

[0047] Where, μ B (θ i S1(.) represents the scoring evaluation result of the i-th joint; S1(.) represents the left a min , right side a max The smoothing function; φ i C represents the motion phase of any skeletal joint in the i-th frame, used to reflect the stage of periodic motion; C represents the set of joints to be observed as required by the monitoring.

[0048] In addition, embodiments of the present invention also provide a work behavior assessment device based on skeletal motion phases, the device comprising:

[0049] Acquisition module: used to obtain human motion skeleton data by acquiring motion images based on human motion capture devices or camera sensors and performing deep learning.

[0050] The tagging strategy processing module is used to perform automated tagging strategy processing on the human motion skeleton data to obtain automated tagging results of joint motion states.

[0051] Calculation module: used to calculate the skeletal motion phase based on the automated labeling results of the joint motion state, and obtain the calculation results of the skeletal motion phase;

[0052] Assessment module: Used to process the calculation results of skeletal motion phase based on a rapid whole-body assessment strategy to assess job behavior and obtain a holistic assessment result of job behavior.

[0053] In this embodiment of the invention, a more automated and intelligent full-body assessment and analysis of workers can be provided, thereby enabling real-time monitoring during the work process and timely early warning and prediction of high-risk situations caused by improper posture and prolonged high-intensity work. This significantly reduces occupational diseases caused by improper work posture and prolonged high-intensity work. Specifically, by acquiring information on skeletal joint points through modern sensors, a new work posture risk assessment tool is constructed to evaluate and score workers' work posture over time, reminding workers of the potential hazards caused by improper work posture and prolonged repetitive work. This can improve workers' work habits and help avoid the occurrence of work-related musculoskeletal diseases. Attached Figure Description

[0054] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0055] Figure 1 This is a flowchart illustrating the work behavior assessment method based on skeletal motion phase in an embodiment of the present invention.

[0056] Figure 2 This is a schematic diagram of the structural composition of the work behavior assessment device based on skeletal motion phase in an embodiment of the present invention. Detailed Implementation

[0057] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0058] Example 1

[0059] Please see Figure 1 , Figure 1 This is a flowchart illustrating the work behavior assessment method based on skeletal motion phase in an embodiment of the present invention.

[0060] like Figure 1 As shown, a method for assessing job behavior based on skeletal motion phases is described, the method comprising:

[0061] S11: Human motion skeleton data is obtained by acquiring motion images based on human motion capture devices or camera sensors and performing deep learning.

[0062] In the specific implementation of this invention, the method of obtaining human motion skeleton data by acquiring motion images based on a human motion capture device or a camera sensor and performing deep learning includes: when obtaining human motion skeleton data based on a human motion capture device, the bone point data in the human motion skeleton data is three-dimensional data; when obtaining human motion skeleton data based on acquiring motion images based on a camera sensor and performing deep learning, the bone point data in the human motion skeleton data is two-dimensional data.

[0063] Specifically, with the in-depth development of sensor technology, deep learning, and other technologies, methods for acquiring human skeletal point data are becoming increasingly mature. In this invention, the methods for acquiring human skeletal joint point data can be: 1) acquiring data through a human motion capture device; 2) acquiring motion images through a camera sensor and estimating them using deep learning methods. Through these two methods, the skeletal point data information p of the human body can be obtained. i Where i∈[0,16], mainly including left and right eyes, left and right ears, neck, waist, left and right upper arms, left and right lower arms, left and right wrist joints, left and right knee joints, and left and right leg joints; when using sensor devices such as motion capture devices, the human body's skeletal point data information p i = (x, y, z); when using a camera plus deep learning method, the obtained data is two-dimensional skeletal point data p. i = (x, y).

[0064] S12: Process the human motion skeleton data using an automated labeling strategy for joint motion states to obtain automated labeling results for joint motion states.

[0065] In a specific implementation of this invention, the mathematical formula for the automated labeling strategy for joint motion states is expressed as follows:

[0066]

[0067] in, This indicates the position of the human motion skeleton data k in the i-th frame; The position of the human motion skeleton data k in frame i-1 is represented; ΔT represents the time interval between the detections of the two pose estimators; v min This represents the minimum value of the threshold parameter for the primary condition adjusted according to image resolution; v max This represents the maximum value of the threshold parameter of the primary condition adjusted according to the image resolution; This represents the automated labeling result of the joint motion state of human motion skeleton data k in the i-th frame; N represents the number of frames; d min This represents the secondary condition threshold adjusted based on an N-frame window.

[0068] Specifically, to calculate motion phase features, the motion state of bones under rules is first extracted by detecting whether bones are moving. To reduce the manual cost of annotation and avoid inconsistencies caused by manual annotation errors, an automatic motion state data annotation module is introduced; motion state labels s can be calculated through a two-level conditional decision. A primary condition is checking whether the bone velocity is within a reasonable threshold. If this condition is true, a secondary condition of distance movement within a time series window is used for further checking. Therefore, another minimum bone movement distance threshold is proposed to filter inaccurate motion state data. This can be mathematically expressed as:

[0069]

[0070] in, This indicates the position of the human motion skeleton data k in the i-th frame; The position of the human motion skeleton data k in frame i-1 is represented; ΔT represents the time interval between the detections of the two pose estimators; v min This represents the minimum value of the threshold parameter for the primary condition adjusted according to image resolution; v max This represents the maximum value of the threshold parameter of the primary condition adjusted according to the image resolution; This represents the automated labeling result of the joint motion state of human motion skeleton data k in the i-th frame; N represents the number of frames; d min This represents the secondary condition threshold adjusted based on an N-frame window.

[0071] S13: Based on the automated labeling results of the joint motion state, perform bone motion phase calculation processing to obtain the bone motion phase calculation results;

[0072] In a specific implementation of this invention, the step of calculating the skeletal motion phase based on the automated labeling results of the joint motion state to obtain the skeletal motion phase calculation result includes: normalizing the original motion state function based on the automated labeling results of the joint motion state to obtain normalized motion state data; filtering the normalized motion state data based on a Butterworth low-pass filter to obtain a filtering result; performing curve fitting on the filtering result, and calculating the skeletal motion phase based on the curve fitting result to obtain the skeletal motion phase calculation result.

[0073] Furthermore, the formula for filtering the normalized motion state data based on the Butterworth low-pass filter is as follows:

[0074]

[0075] in, This represents a Butterworth low-pass filter, where Z represents the order of the Butterworth low-pass filter, and Z takes the value of 3; w n The parameters related to the cutoff frequency are represented, which are calculated based on the Shannon-Nyquist sampling theorem; G(i) represents the low-pass filtering result of the i-frame data; and Y represents the result of normalizing the original motion state label data.

[0076] Furthermore, the curve fitting process for the filtered result includes:

[0077] The filtering results are then subjected to curve fitting based on a sine function.

[0078] The formula for curve fitting is as follows:

[0079] Ω(F i ) = a i ·sin(f i ·i+s i )+b i ;

[0080] Among them, F i Let F represent the objective function. i Parameterization yields F i =(a i ,f i ,s i ,b i ), where i is the index frame. This curve fitting process minimizes the following root mean square error loss within an N-frame window for each frame i:

[0081]

[0082]

[0083] Where, φ i f represents the motion phase at the i-th frame for any skeletal joint, used to reflect the stage of periodic motion; i Indicates the frequency of skeletal movement; a i This represents the optimized fitted amplitude parameter; s i b is the initial phase parameter; i is the offset parameter; N is the window width; t is the time index within the window with a center width of N in the i-th frame; G(t) is the result value after Butterworth low-pass filtering at time t.

[0084] Furthermore, the calculation of the skeletal motion phase based on the curve fitting results includes:

[0085] When the skeleton is stationary, the motion phase is an uncertain situation, which is the optimized fitting amplitude parameter 'a'. iCombined, an updated motion phase is generated as follows:

[0086] P i =S1(a i )·φ i

[0087]

[0088] Where S1(.) represents the left side a min , right side a max The smoothing function; when the skeleton is not moving, the motion phase modulated by the amplitude parameter becomes zero; φ i P represents the motion phase at the i-th frame for any skeletal joint, used to reflect the stage of periodic motion; i This indicates an update of the motion phase.

[0089] Specifically, after obtaining the motion state labels of human skeletal joints through the above strategy, the motion phase characteristics of each skeletal joint are automatically calculated using the algorithm with uniformity proposed below.

[0090] First, the original motion state function is normalized. According to the mathematical formula for the automated labeling strategy of joint motion states, the value is set to 1 if the bone is moving, and 0 if it is not moving. Then, z-score data normalization is applied within a window W (N frames) centered on the i-th frame.

[0091]

[0092] in, These represent the mean and standard deviation within the time window, respectively. After applying this standardization, faster movement results in larger positive values ​​and smaller negative values, and vice versa. This shows that the motion state can remain consistent regardless of different speeds and frequencies.

[0093] Then, a Butterworth low-pass filter is applied to filter the normalized motion state data:

[0094]

[0095] in, This represents a Butterworth low-pass filter, where Z represents the order of the Butterworth low-pass filter, and Z takes the value of 3; w n The parameters related to the cutoff frequency are represented, which are calculated based on the Shannon-Nyquist sampling theorem; G(i) represents the low-pass filtering result of the i-th frame of data; and Y represents the result of normalizing the original motion state label data.

[0096] After this filtering, a smoother motion state data curve is obtained without loss of features; next, curve fitting is performed on the Butterworth-filtered data, the motion phase is calculated, and the parameters are fitted using a sine function (Eq.4):

[0097] Ω(F i ) = a i ·sin(f i ·i+s i )+b i ;

[0098] Among them, F i Let F represent the objective function. i Parameterization yields F i =(a i ,f i ,s i ,b i ), where i is the index frame. This curve fitting process minimizes the following root mean square error loss within an N-frame window for each frame i:

[0099]

[0100]

[0101] Where, φ i f represents the motion phase at the i-th frame for any skeletal joint, used to reflect the stage of periodic motion; i Indicates the frequency of skeletal movement; a i This represents the optimized fitted amplitude parameter; s i b is the initial phase parameter; i is the offset parameter; N is the window width; t is the time index within the window with a center width of N in the i-th frame; G(t) is the result value after Butterworth low-pass filtering at time t.

[0102] However, the problem arises when the skeleton is stationary, in which case the motion phase is quite uncertain. To address this, the optimized fitting amplitude parameter 'a' needs to be adjusted. i Combining these elements, we can generate a more general motion phase, denoted as:

[0103] P i =S1(a i )·φ i

[0104]

[0105] Where S1(.) represents the left side a min , right side a maxThe smoothing function; when the skeleton is not moving, the motion phase modulated by the amplitude parameter becomes zero; φ i P represents the motion phase at the i-th frame for any skeletal joint, used to reflect the stage of periodic motion; i This indicates an update of the motion phase.

[0106] S14: Based on the rapid whole-body assessment strategy, the calculation results of skeletal motion phase are processed for job behavior assessment to obtain the overall assessment results of job behavior.

[0107] In the specific implementation of this invention, the calculation results of the skeletal motion phase based on the rapid whole-body assessment strategy are used to perform work behavior assessment processing to obtain a holistic assessment result of the work behavior, including: calculating the angle of joint bending based on the calculation results of the skeletal motion phase to obtain the joint bending angle; performing fuzzy logic processing based on the joint bending angle, and obtaining a whole-body assessment result based on the fuzzy logic processing result.

[0108] Furthermore, the step of performing fuzzy logic processing based on the joint bending angle and obtaining the whole-body assessment result based on the fuzzy logic processing result includes:

[0109] The fuzzy logic processing based on the joint bending angle is as follows:

[0110]

[0111] Where, μ A (θ) represents the fuzzy logic processing result of the joint bending angle; θ represents the joint bending angle, θ∈[α,β]; I is the introduced extended parameter;

[0112] The scoring and evaluation results for individual joints are as follows:

[0113] μ B (θ i )=f i ×(S1(a i )·φ i +μ A (θ i ));

[0114] The scoring and assessment of all joint points in the body are as follows:

[0115] Score = ∑ i∈C μ B (θ i );

[0116] Where, μ B (θ i S1(.) represents the scoring evaluation result of the i-th joint; S1(.) represents the left a min , right side amax The smoothing function; φ i C represents the motion phase of any skeletal joint in the i-th frame, used to reflect the stage of periodic motion; C represents the set of joints to be observed as required by the monitoring.

[0117] Specifically, calculate the angle of joint bending: through the two bones connected at joint i. The bending angle of the joint point is calculated using the following formula:

[0118]

[0119] Let's take calculating the angle of the elbow joint as an example to illustrate, right lower arm right upper arm It indicates that, among them, S8 and S 10 S6 and S8 are the two ends of the right lower arm joints, and S8 are the two ends of the right upper arm joints. The corresponding angles can be calculated based on the relationship of spatial vectors.

[0120] Fuzzy logic is applied to the calculated angles:

[0121]

[0122] Where, μ A (θ) represents the fuzzy logic processing result of the joint bending angle; θ represents the joint bending angle, θ∈[α,β]; I is the introduced extended parameter;

[0123] The scoring and evaluation results for individual joints are as follows:

[0124] μ B (θ i )=f i ×(S1(a i )·φ i +μ A (θ i ));

[0125] The scoring and assessment of all joint points in the body are as follows:

[0126] Score = ∑ i∈C μ B (θ i );

[0127] Where, μ B (θ i S1(.) represents the scoring evaluation result of the i-th joint; S1(.) represents the left a min , right side a max The smoothing function; φ iC represents the motion phase of any skeletal joint in the i-th frame, used to reflect the stage of periodic motion; C represents the set of joints to be observed as required by the monitoring.

[0128] Where C is the set of joints to be observed according to the monitoring requirements. Generally, when only the upper body needs to be considered:

[0129] C upper = {neck, waist, upper arms, lower arms, wrists};

[0130] When considering the whole body:

[0131] C body = {neck, waist, left and right upper arms, left and right lower arms, left and right wrist joints, left and right knee joints, left and right leg joints}; at the same time, the monitoring tasks for some special engineering scenarios can be adjusted according to the situation, such as only focusing on the hands and feet in climbing operations.

[0132] When the score is greater than or equal to ThresHold1, the operational risk at a certain moment exceeds the limit, and an early warning is issued; at the same time, the operational risk over a period of time can be accumulated. This indicates that the workers have been in a high-risk state after working for a period of time and need to be reminded and given a break.

[0133] In this embodiment of the invention, a more automated and intelligent full-body assessment and analysis of workers can be provided, thereby enabling real-time monitoring during the work process and timely early warning and prediction of high-risk situations caused by improper posture and prolonged high-intensity work. This significantly reduces occupational diseases caused by improper work posture and prolonged high-intensity work. Specifically, by acquiring information on skeletal joint points through modern sensors, a new work posture risk assessment tool is constructed to evaluate and score workers' work posture over time, reminding workers of the potential hazards caused by improper work posture and prolonged repetitive work. This can improve workers' work habits and help avoid the occurrence of work-related musculoskeletal diseases.

[0134] Example 2

[0135] Please see Figure 2 , Figure 2 This is a schematic diagram of the structural composition of the work behavior assessment device based on skeletal motion phase in an embodiment of the present invention.

[0136] like Figure 2 As shown, a work behavior assessment device based on skeletal motion phases is provided, the device comprising:

[0137] Module 21: Used to obtain human motion skeleton data by acquiring motion images based on human motion capture devices or camera sensors and performing deep learning.

[0138] In the specific implementation of this invention, the method of obtaining human motion skeleton data by acquiring motion images based on a human motion capture device or a camera sensor and performing deep learning includes: when obtaining human motion skeleton data based on a human motion capture device, the bone point data in the human motion skeleton data is three-dimensional data; when obtaining human motion skeleton data based on acquiring motion images based on a camera sensor and performing deep learning, the bone point data in the human motion skeleton data is two-dimensional data.

[0139] Specifically, with the in-depth development of sensor technology, deep learning, and other technologies, methods for acquiring human skeletal point data are becoming increasingly mature. In this invention, the methods for acquiring human skeletal joint point data can be: 1) acquiring data through a human motion capture device; 2) acquiring motion images through a camera sensor and estimating them using deep learning methods. Through these two methods, the skeletal point data information p of the human body can be obtained. i Where i∈[0,16], mainly including left and right eyes, left and right ears, neck, waist, left and right upper arms, left and right lower arms, left and right wrist joints, left and right knee joints, and left and right leg joints; when using sensor devices such as motion capture devices, the human body's skeletal point data information p i = (x, y, z); when using a camera plus deep learning method, the obtained data is two-dimensional skeletal point data p. i = (x, y).

[0140] Tag strategy processing module 22: used to perform automated tagging strategy processing on the human motion skeleton data to obtain automated tagging results of joint motion states;

[0141] In a specific implementation of this invention, the mathematical formula for the automated labeling strategy for joint motion states is expressed as follows:

[0142]

[0143] in, This indicates the position of the human motion skeleton data k in the i-th frame; The position of the human motion skeleton data k in frame i-1 is represented; ΔT represents the time interval between the detections of the two pose estimators; v min This represents the minimum value of the threshold parameter for the primary condition adjusted according to image resolution; v max This represents the maximum value of the threshold parameter of the primary condition adjusted according to the image resolution; This represents the automated labeling result of the joint motion state of human motion skeleton data k in the i-th frame; N represents the number of frames; d min This represents the secondary condition threshold adjusted based on an N-frame window.

[0144] Specifically, to calculate motion phase features, the motion state of bones under rules is first extracted by detecting whether bones are moving. To reduce the manual cost of annotation and avoid inconsistencies caused by manual annotation errors, an automatic motion state data annotation module is introduced; motion state labels s can be calculated through a two-level conditional decision. A primary condition is checking whether the bone velocity is within a reasonable threshold. If this condition is true, a secondary condition of distance movement within a time series window is used for further checking. Therefore, another minimum bone movement distance threshold is proposed to filter inaccurate motion state data. This can be mathematically expressed as:

[0145]

[0146] in, This indicates the position of the human motion skeleton data k in the i-th frame; The position of the human motion skeleton data k in frame i-1 is represented; ΔT represents the time interval between the detections of the two pose estimators; v min This represents the minimum value of the threshold parameter for the primary condition adjusted according to image resolution; v max This represents the maximum value of the threshold parameter of the primary condition adjusted according to the image resolution; This represents the automated labeling result of the joint motion state of human motion skeleton data k in the i-th frame; N represents the number of frames; d min This represents the secondary condition threshold adjusted based on an N-frame window.

[0147] Calculation module 23: used to calculate the skeletal motion phase based on the automated labeling results of the joint motion state, and obtain the calculation results of the skeletal motion phase;

[0148] In a specific implementation of this invention, the step of calculating the skeletal motion phase based on the automated labeling results of the joint motion state to obtain the skeletal motion phase calculation result includes: normalizing the original motion state function based on the automated labeling results of the joint motion state to obtain normalized motion state data; filtering the normalized motion state data based on a Butterworth low-pass filter to obtain a filtering result; performing curve fitting on the filtering result, and calculating the skeletal motion phase based on the curve fitting result to obtain the skeletal motion phase calculation result.

[0149] Furthermore, the formula for filtering the normalized motion state data based on the Butterworth low-pass filter is as follows:

[0150]

[0151] in, This represents a Butterworth low-pass filter, where Z represents the order of the Butterworth low-pass filter, and Z takes the value of 3; w n The parameters related to the cutoff frequency are represented, which are calculated based on the Shannon-Nyquist sampling theorem; G(i) represents the low-pass filtering result of the i-frame data; and Y represents the result of normalizing the original motion state label data.

[0152] Furthermore, the curve fitting process for the filtered result includes:

[0153] The filtering results are then subjected to curve fitting based on a sine function.

[0154] The formula for curve fitting is as follows:

[0155] Ω(F i ) = a i ·sin(f i ·i+s i )+b i ;

[0156] Among them, F i Let F represent the objective function. i Parameterization yields F i =(a i ,f i ,s i ,b i ), where i is the index frame. This curve fitting process minimizes the following root mean square error loss within an N-frame window for each frame i:

[0157]

[0158]

[0159] Where, φ i f represents the motion phase at the i-th frame for any skeletal joint, used to reflect the stage of periodic motion; i Indicates the frequency of skeletal movement; a i This represents the optimized fitted amplitude parameter; s i b is the initial phase parameter; i is the offset parameter; N is the window width; t is the time index within the window with a center width of N in the i-th frame; G(t) is the result value after Butterworth low-pass filtering at time t.

[0160] Furthermore, the calculation of the skeletal motion phase based on the curve fitting results includes:

[0161] When the skeleton is stationary, the motion phase is an uncertain situation, which is the optimized fitting amplitude parameter 'a'. i Combined, an updated motion phase is generated as follows:

[0162] P i =S1(a i )·φ i

[0163]

[0164] Where S1(.) represents the left α min , right side a max The smoothing function; when the skeleton is not moving, the motion phase modulated by the amplitude parameter becomes zero; φ i P represents the motion phase at the i-th frame for any skeletal joint, used to reflect the stage of periodic motion; i This indicates an update of the motion phase.

[0165] Specifically, after obtaining the motion state labels of human skeletal joints through the above strategy, the motion phase characteristics of each skeletal joint are automatically calculated using the algorithm with uniformity proposed below.

[0166] First, the original motion state function is normalized. According to the mathematical formula for the automated labeling strategy of joint motion states, the value is set to 1 if the bone is moving, and 0 if it is not moving. Then, z-score data normalization is applied within a window W (N frames) centered on the i-th frame.

[0167]

[0168] in, These represent the mean and standard deviation within the time window, respectively. After applying this standardization, faster movement results in larger positive values ​​and smaller negative values, and vice versa. This shows that the motion state can remain consistent regardless of different speeds and frequencies.

[0169] Then, a Butterworth low-pass filter is applied to filter the normalized motion state data:

[0170]

[0171] in, This represents a Butterworth low-pass filter, where Z represents the order of the Butterworth low-pass filter, and Z takes the value of 3; w nThe parameters related to the cutoff frequency are represented, which are calculated based on the Shannon-Nyquist sampling theorem; G(i) represents the low-pass filtering result of the i-th frame of data; and Y represents the result of normalizing the original motion state label data.

[0172] After this filtering, a smoother motion state data curve is obtained without loss of features; next, curve fitting is performed on the Butterworth-filtered data, the motion phase is calculated, and the parameters are fitted using a sine function (Eq.4):

[0173] Ω(F i ) = a i ·sin(f i ·i+s i )+b i ;

[0174] Among them, F i Let F represent the objective function. i Parameterization yields F i =(a i ,f i ,s i ,b i ), where i is the index frame. This curve fitting process minimizes the following root mean square error loss within an N-frame window for each frame i:

[0175]

[0176]

[0177] Where, φ i f represents the motion phase at the i-th frame for any skeletal joint, used to reflect the stage of periodic motion; i Indicates the frequency of skeletal movement; a i This represents the optimized fitted amplitude parameter; s i b is the initial phase parameter; i is the offset parameter; N is the window width; t is the time index within the window with a center width of N in the i-th frame; G(t) is the result value after Butterworth low-pass filtering at time t.

[0178] However, the problem arises when the skeleton is stationary, in which case the motion phase is quite uncertain. To address this, the optimized fitted amplitude parameters ai need to be combined to generate a more general motion phase, denoted as:

[0179] P i =S1(a i )·φ i

[0180]

[0181] Where S1(.) represents the left side a min , right side a max The smoothing function; when the skeleton is not moving, the motion phase modulated by the amplitude parameter becomes zero; φ i P represents the motion phase at the i-th frame for any skeletal joint, used to reflect the stage of periodic motion; i This indicates an update of the motion phase.

[0182] Assessment Module 24: Used to process the calculation results of skeletal motion phase based on the rapid whole-body assessment strategy to assess job behavior and obtain overall assessment results of job behavior.

[0183] In the specific implementation of this invention, the calculation results of the skeletal motion phase based on the rapid whole-body assessment strategy are used to perform work behavior assessment processing to obtain a holistic assessment result of the work behavior, including: calculating the angle of joint bending based on the calculation results of the skeletal motion phase to obtain the joint bending angle; performing fuzzy logic processing based on the joint bending angle, and obtaining a whole-body assessment result based on the fuzzy logic processing result.

[0184] Furthermore, the step of performing fuzzy logic processing based on the joint bending angle and obtaining the whole-body assessment result based on the fuzzy logic processing result includes:

[0185] The fuzzy logic processing based on the joint bending angle is as follows:

[0186]

[0187] Where, μ A (θ) represents the fuzzy logic processing result of the joint bending angle; θ represents the joint bending angle, θ∈[α,β]; I is the introduced extended parameter;

[0188] The scoring and evaluation results for individual joints are as follows:

[0189] μ B (θ i )=f i ×(S1(a i )·φ i +μ A (θ i ));

[0190] The scoring and assessment of all joint points in the body are as follows:

[0191] Score = ∑ i∈C μ B (θ i );

[0192] Where, μ B (θ i S1(.) represents the scoring evaluation result of the i-th joint; S1(.) represents the left a min , right side a max The smoothing function; φ i C represents the motion phase of any skeletal joint in the i-th frame, used to reflect the stage of periodic motion; C represents the set of joints to be observed as required by the monitoring.

[0193] Specifically, calculate the angle of joint bending: through the two bones connected at joint i. The bending angle of the joint point is calculated using the following formula:

[0194]

[0195] Let's take calculating the angle of the elbow joint as an example to illustrate, right lower arm right upper arm It indicates that, among them, S8 and S 10 S6 and S8 are the two ends of the right lower arm joints, and S8 are the two ends of the right upper arm joints. The corresponding angles can be calculated based on the relationship of spatial vectors.

[0196] Fuzzy logic is applied to the calculated angles:

[0197]

[0198] Where, μ A (θ) represents the fuzzy logic processing result of the joint bending angle; θ represents the joint bending angle, θ∈[α,β]; I is the introduced extended parameter;

[0199] The scoring and evaluation results for individual joints are as follows:

[0200] μ B (θ i )=f i ×(S1(a i )·φ i +μ A (θ i ));

[0201] The scoring and assessment of all joint points in the body are as follows:

[0202] Score = ∑ i∈C μ B (θ i );

[0203] Where, μ B (θ iS1(.) represents the scoring evaluation result of the i-th joint; S1(.) represents the left a min , right side a max The smoothing function; φ i C represents the motion phase of any skeletal joint in the i-th frame, used to reflect the stage of periodic motion; C represents the set of joints to be observed as required by the monitoring.

[0204] Where C is the set of joints to be observed according to the monitoring requirements. Generally, when only the upper body needs to be considered:

[0205] C upper = {neck, waist, upper arms, lower arms, wrists};

[0206] When considering the whole body:

[0207] C body = {neck, waist, left and right upper arms, left and right lower arms, left and right wrist joints, left and right knee joints, left and right leg joints}; at the same time, the monitoring tasks for some special engineering scenarios can be adjusted according to the situation, such as only focusing on the hands and feet in climbing operations.

[0208] When the score is greater than or equal to ThresHold1, the operational risk at a certain moment exceeds the limit, and an early warning is issued; at the same time, the operational risk over a period of time can be accumulated. This indicates that the workers have been in a high-risk state after working for a period of time and need to be reminded and given a break.

[0209] In this embodiment of the invention, a more automated and intelligent full-body assessment and analysis of workers can be provided, thereby enabling real-time monitoring during the work process and timely early warning and prediction of high-risk situations caused by improper posture and prolonged high-intensity work. This significantly reduces occupational diseases caused by improper work posture and prolonged high-intensity work. Specifically, by acquiring information on skeletal joint points through modern sensors, a new work posture risk assessment tool is constructed to evaluate and score workers' work posture over time, reminding workers of the potential hazards caused by improper work posture and prolonged repetitive work. This can improve workers' work habits and help avoid the occurrence of work-related musculoskeletal diseases.

[0210] Those skilled in the art will understand that all or part of the steps in the various methods of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, which may include: read-only memory (ROM), random access memory (RAM), disk or optical disk, etc.

[0211] Furthermore, the above provides a detailed description of the work behavior assessment method and apparatus based on skeletal motion phase provided by the embodiments of the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A method for assessing job performance based on skeletal motion phase, characterized in that, The method includes: Human skeletal data is obtained by acquiring motion images using human motion capture devices or camera sensors and then performing deep learning. The human skeletal data is processed using an automated labeling strategy for joint motion states to obtain automated labeling results for joint motion states. Based on the automated labeling results of the joint motion state, the bone motion phase is calculated to obtain the bone motion phase calculation result. The calculation results of skeletal motion phase based on the rapid whole-body assessment strategy are used to assess the work behavior of workers and obtain the overall assessment results of work behavior. The mathematical formula for the automated labeling strategy for joint motion states is expressed as follows: ； in, This indicates the position of the human motion skeleton data k in the i-th frame; This indicates the position of the human motion skeleton data k in the (i-1)th frame; This represents the time interval between the detections of two pose estimators; This represents the minimum value of the threshold parameter of the primary condition adjusted according to the image resolution; This represents the maximum value of the threshold parameter of the primary condition adjusted according to the image resolution; This represents the automated labeling result of the joint motion state of human skeletal data k in the i-th frame; Indicates the number of frames; This represents the secondary condition threshold adjusted based on an N-frame window; The process of calculating the skeletal motion phase based on the automated labeling results of the joint motion states to obtain the skeletal motion phase calculation results includes: Based on the automated labeling results of the joint motion state, the original motion state function is normalized to obtain normalized motion state data. The normalized motion state data is filtered using a Butterworth low-pass filter to obtain the filtering result. The filtering result is subjected to curve fitting processing, and the bone motion phase is calculated based on the curve fitting result to obtain the bone motion phase calculation result. The calculation and processing of skeletal motion phase based on curve fitting results includes: When the skeleton is stationary, the motion phase is an uncertain situation, which is the optimized fitting amplitude parameters. Combined, an updated motion phase is generated as follows: ； in, Indicates the left side ,right The smoothing function; when the skeleton is not moving, the motion phase modulated by the amplitude parameter becomes zero; Represents the first joint of any bone. The framed motion phase is used to reflect the stages of periodic motion; This indicates an update of the motion phase.

2. The job performance evaluation method according to claim 1, characterized in that, The method of obtaining human motion skeleton data based on acquiring motion images using human motion capture devices or camera sensors and performing deep learning includes: When obtaining human motion skeleton data based on human motion capture, the bone point data in the human motion skeleton data is three-dimensional data; When obtaining human motion skeleton data by acquiring motion images based on camera sensors and performing deep learning, the bone point data in the human motion skeleton data is two-dimensional data.

3. The job performance evaluation method according to claim 1, characterized in that, The formula for filtering normalized motion state data based on a Butterworth low-pass filter is as follows: ； in, This represents a Butterworth low-pass filter, where Z represents the order of the Butterworth low-pass filter, and Z takes the value of 3. This represents a parameter related to the cutoff frequency, which is calculated based on the Shannon-Nyquist sampling theorem. This represents the low-pass filtering result of the i-frame data; This represents the result after normalizing the original motion state label data.

4. The job performance evaluation method according to claim 1, characterized in that, The curve fitting process for the filtered result includes: The filtering results are then subjected to curve fitting based on a sine function. The formula for curve fitting is as follows: ； in, Describe the objective function, and Parameterization will yield the following results. , For index frames, this curve fitting process is performed on each frame. Minimize the following root mean square error loss within an N-frame window: ； ； in, Represents the first joint of any bone. The framed motion phase is used to reflect the stages of periodic motion; Indicates the frequency of skeletal movement; This represents the optimized fitted amplitude parameters; These are the initial phase parameters; is the offset parameter; N is the window width; t is the time index within the window with a center width of N in the i-th frame; This represents the result of Butterworth low-pass filtering at time t.

5. The job performance evaluation method according to claim 1, characterized in that, The calculation results of skeletal motion phase based on the rapid whole-body assessment strategy are used to evaluate the work behavior and obtain a holistic assessment result, including: Based on the calculation results of the bone motion phase, the angle of joint bending is calculated to obtain the joint bending angle. Fuzzy logic processing is performed based on the bending angle of the joints, and the whole-body assessment results are obtained based on the fuzzy logic processing results.

6. The job performance evaluation method according to claim 5, characterized in that, The process of performing fuzzy logic processing based on the joint bending angle and obtaining the whole-body assessment result based on the fuzzy logic processing result includes: The fuzzy logic processing based on the joint bending angle is as follows: ； in, The result of fuzzy logic processing represents the bending angle of the joint. Indicates the bending angle of the joint. ; For the introduced extended parameters; The scoring and evaluation results for individual joints are as follows: ； The scoring and assessment of all joint points in the body are as follows: ； in, This represents the scoring evaluation result of the i-th joint. Indicates the left side ,right The smoothing function; Represents the first joint of any bone. The framed motion phase is used to reflect the stages of periodic motion; This represents the set of key points to be observed as required by the monitoring requirements.

7. A work behavior assessment device based on skeletal motion phase, characterized in that, The device includes: Acquisition module: used to obtain human motion skeleton data by acquiring motion images based on human motion capture devices or camera sensors and performing deep learning. The tagging strategy processing module is used to perform automated tagging strategy processing on the human motion skeleton data to obtain automated tagging results of joint motion states. Calculation module: used to calculate the skeletal motion phase based on the automated labeling results of the joint motion state, and obtain the calculation results of the skeletal motion phase; Assessment module: Used to process the calculation results of skeletal motion phase based on a rapid whole-body assessment strategy to assess work behavior and obtain a holistic assessment result of work behavior; The mathematical formula for the automated labeling strategy for joint motion states is expressed as follows: ； in, This indicates the position of the human motion skeleton data k in the i-th frame; This indicates the position of the human motion skeleton data k in the (i-1)th frame; This represents the time interval between the detections of two pose estimators; This represents the minimum value of the threshold parameter of the primary condition adjusted according to the image resolution; This represents the maximum value of the threshold parameter of the primary condition adjusted according to the image resolution; This represents the automated labeling result of the joint motion state of human skeletal data k in the i-th frame; Indicates the number of frames; This represents the secondary condition threshold adjusted based on an N-frame window; The process of calculating the skeletal motion phase based on the automated labeling results of the joint motion states to obtain the skeletal motion phase calculation results includes: Based on the automated labeling results of the joint motion state, the original motion state function is normalized to obtain normalized motion state data. The normalized motion state data is filtered using a Butterworth low-pass filter to obtain the filtering result. The filtering result is subjected to curve fitting processing, and the bone motion phase is calculated based on the curve fitting result to obtain the bone motion phase calculation result. The calculation and processing of skeletal motion phase based on curve fitting results includes: When the skeleton is stationary, the motion phase is an uncertain situation, which is the optimized fitting amplitude parameters. Combined, an updated motion phase is generated as follows: ； in, Indicates the left side ,right The smoothing function; when the skeleton is not moving, the motion phase modulated by the amplitude parameter becomes zero; Represents the first joint of any bone. The framed motion phase is used to reflect the stages of periodic motion; This indicates an update of the motion phase.

Images