Target person fatigue identification method based on machine learning

Abstract

The invention discloses a fatigue recognition method for a target person based on machine learning, and the method comprises the steps: taking data obtained from a worker database in a preset industrial scene as to-be-analyzed data; performing difference comparison on the to-be-analyzed data and the reference database, taking the to-be-analyzed data with difference as first data, and otherwise, taking the to-be-analyzed data as uncertain data; performing consistency comparison on the uncertain data by adopting the undetermined database, taking data without differences as problem data, and adding the problem data into the first data; performing fatigue feature extraction on the first data to obtain key features, performing personalized time sequence analysis on the key features to obtain second data, and performing multi-modal dynamic fusion on the second data by adopting an attention mechanism to obtain fused data; and performing fatigue judgment according to the fusion data based on a dynamic threshold value to obtain identification data, and constructing a target person fatigue identification model according to the identification data.

Description

Technical Field

[0001] The present invention relates to the field of fatigue recognition, and in particular to a method for fatigue recognition of a target person through machine learning. Background Art

[0002] With the advent of Industry 4.0, the widespread adoption of intelligent manufacturing and automated production lines has significantly improved production efficiency. However, this process has also led to increasing concerns about worker health and safety. In particular, worker fatigue, particularly in long, intensive work environments, has become a significant factor impacting both productivity and safety. To effectively monitor and manage worker fatigue, implement timely intervention measures, and ensure production safety and efficiency, it is crucial to develop an efficient and accurate machine learning method for identifying fatigue in target personnel.

[0003] Currently, monitoring worker fatigue in industrial settings primarily relies on traditional physiological indicators, such as heart rate and blood pressure, and observation of worker behavior. However, these methods are often subject to high subjectivity and lack accuracy. To overcome these limitations, a growing number of companies are introducing intelligent monitoring systems that collect and analyze worker data, including physical monitoring data, image data, and work data, to comprehensively assess worker fatigue.

[0004] However, faced with a vast amount of pending data, efficiently screening for key information related to fatigue status has become a pressing challenge. Existing methods often rely on simple difference comparisons, comparing the pending data with a reference database to identify discrepancies as potential fatigue signatures. However, this approach ignores the individual characteristics of the data in the pending database, which can easily lead to misjudgments and missed detections.

[0005] With the accelerating advancement of industrial intelligence, developing a machine learning-based method for identifying fatigue in target personnel has become an inevitable trend in industrial safety and health management. A new method for identifying fatigue in target personnel is urgently needed. By comprehensively utilizing various data from a personnel database and combining it with advanced machine learning techniques, it can accurately identify and effectively manage the fatigue status of target personnel, providing a strong guarantee for the safety and efficiency of industrial production. Summary of the Invention

[0006] The purpose of the present invention is to provide a method for fatigue recognition of a target person based on machine learning.

[0007] To achieve the above object, the present invention is implemented according to the following technical solutions:

[0008] The present invention comprises the following steps:

[0009] Data obtained from a database of workers in a preset industrial scenario within a specified time period is used as data to be analyzed; the worker database includes a reference database and a pending database; the pending data includes body monitoring data, image data, and work data; the body monitoring data includes physiological data and behavioral data; the pending database has a higher degree of personalization than the reference database; the reference data is worker data under health standards;

[0010] Comparing the data to be analyzed with the reference database for differences, and treating the data to be analyzed with differences as first data, and treating the data with differences as uncertain data;

[0011] Using the pending database to perform consistency comparison on the uncertain data, treating data without differences as problem data, and adding the problem data to the first data;

[0012] Fatigue feature extraction is performed on the first data to obtain key features, personalized time series analysis is performed on the key features to obtain second data, and multimodal dynamic fusion is performed on the second data using an attention mechanism to obtain fused data; the key features are stored in person units;

[0013] Fatigue determination is performed on the fusion data based on a dynamic threshold to obtain recognition data, a target person fatigue recognition model is constructed based on the recognition data, and fatigue recognition is performed based on the target person fatigue recognition model.

[0014] Furthermore, the method of extracting fatigue features from the first data to obtain key features includes:

[0015] Use neural network algorithms to extract behavioral features from image data, biomedical signal processing algorithms to extract biosignal features from body monitoring data, and autoencoders to extract state features from working data;

[0016] The behavioral features, biological signal features, and state features are input into the support vector machine algorithm as sample feature vectors. Given the loss coefficient and loss function, the expression is:

[0017]

[0018] QUR x (z T u+v)≥1,β>0

[0019] The normal vector perpendicular to the hyperplane is z, and the loss function of the x-th sample feature vector is The loss coefficient is β, the displacement term is v, the sample feature vector is u, and the label of the x-th sample feature vector is Q x , transposed to T, the number of sample feature vectors is Nx , the decision hyperplane function of the normal vector z is

[0020] According to the kernel function, the decision hyperplane function is given and the expression is:

[0021]

[0022] The Lagrange multiplier of the xth sample eigenvector is γ x , the i-th sample feature vector is u i , the jth sample feature vector is u j , sample feature vector u i and the sample feature vector u j The kernel function is The sign function is sgn(·), the weight coefficients are λ and α, and the kernel function parameter is τ;

[0023] If the result of the decision hyperplane function is 1, the sample feature vector is in a non-fatigue state; if the result of the decision hyperplane function is -1, the sample feature vector is in a fatigue state; the sample vector in the fatigue state is output as the key feature.

[0024] Furthermore, the method of performing personalized time series analysis on the key features to obtain second data includes:

[0025] Sort the key features in descending order by individual, and use a recurrent neural network to capture the long-term relationship of the sorted key feature set to obtain personalized change data;

[0026] The personalized change data caused by pathology is eliminated according to the medical data of the individual, and the personalized change data after elimination is output as the second data.

[0027] Furthermore, a method for dynamically fusing the second data using an attention mechanism to obtain fused data includes:

[0028] The behavioral features and state features in the second data are integrated into behavioral state features. The energy of specific frequency bands is extracted from the biological signal features through short-time Fourier transform. The self-attention mechanism is introduced in posture estimation to upgrade the behavioral state features.

[0029] The biological signal features are used as keys K or values ​​V for interaction, and a gated recurrent unit is added after the exponential shift.

[0030] The behavioral state features and biological signal features are gated and adaptively fused, and the expression is:

[0031]

[0032] ζ=σ(Z a[h1;h2])

[0033] The learnable dynamic weight is ζ, the element-by-element multiplication is ⊙, the state feature at the s-th moment is h1(s), the biological signal feature at the s-1-th moment is h2(s), and the fusion feature at the s-th moment is The state feature is h1, the biological signal feature is h2, the concatenation of the state feature h1 and the biological signal feature h2 is [h1; h2], and the learnable weight matrix is ​​Z a , the activation function is σ(·);

[0034] The fused features are input into the multimodal fusion attention module MFA, and the MFA encoded output is nonlinearly transformed by the self-gated nonlinear adaptive activation function. The expression is as follows:

[0035]

[0036] The fusion feature after MFA encoding is The learnable bias parameter is P a , the nonlinear adaptive activation function of self-gating is SiLU(·), the nonlinear transformation result is D, and the gating parameter is The number of multimodal models is N a ;

[0037] Calculate weights and biases:

[0038] C=Z C ⊙D+ω C

[0039] K=Z K ⊙D+ω K

[0040]

[0041] The learnable weight matrices are Z C , Z K , Z V , the query is C, the key is K, the value is V, and the learnable bias parameter is P V , the learnable bias vectors are ω C 、ω K ;

[0042] Given the attention function, the expression is:

[0043]

[0044] The attention function is Input comprehensive characteristic length is The relative position is encoded as P ref ;

[0045] Adjust the dynamic weight, the expression is:

[0046]

[0047] The adjusted dynamic weight is The penalty parameter is κ, the predicted probability is pt, and the false positive sample is κ FN , the false negative sample is κ FP ; Output the adjusted fusion features as fusion data.

[0048] Furthermore, the method for obtaining recognition data by performing fatigue determination based on the fusion data based on a dynamic threshold includes:

[0049] According to the given judgment criteria of the fusion data, the expression is:

[0050]

[0051] The weight of closing eyes is ρ1, the weight of opening mouth is ρ2, the weight of stopping hands is ρ3, the weight of stopping feet is ρ3, and the time of closing eyes in unit working time is n eye , the required working time of the eyes in unit working time is N eye , the time spent opening the mouth in a unit of working time is n mouth , the required working time of the mouth in unit working time is N mouth The time occupied by manual stop in unit working time is n hand , the required working time per unit working time is N hand The time spent on foot rest in a unit of working time is n foot , the required working time of the foot in unit working time is N foot , fatigue value is

[0052] By drawing ROC curves for different types of fusion data, the maximum difference between the true positive rate and the false positive rate is used as the initial threshold;

[0053] Dynamically adjust the threshold, the expression is:

[0054]

[0055] The number of modes is N m , the classification confidence of each mode m is d m , the classification result of each mode of m is G m , the base confidence is d r , the initial threshold is The adjusted dynamic threshold is The penalty coefficient is φ, and the distribution probability of the biosignal modality is pbio , the distribution probability of the behavior mode is p beh , distribution probability p bio and distribution probability p beh The divergence is U KL (p bio ||p beh );

[0056] A dynamic threshold is used to perform fatigue discrimination on the fused data, and the fused data with fatigue values ​​greater than the dynamic threshold is output as recognition data.

[0057] Furthermore, a method for constructing a target person fatigue recognition model based on the recognition data includes:

[0058] Construct the objective function based on the recognition data:

[0059]

[0060] The objective function is Fatigue value is The number of modes is N m , the monitoring value of the mth mode is f m , the normal value of the mth mode is f o , the actual fatigue recognition result is y, and the predicted fatigue recognition result is Actual fatigue recognition result y and predicted fatigue recognition result The loss function is The weight coefficient of the mth mode is l m ;

[0061] The target personnel fatigue recognition model includes the random forest algorithm, the spherical k-means clustering algorithm, and the deep machine learning algorithm;

[0062] The random forest algorithm randomly divides the input data into training data and test data in a ratio of 7:2;

[0063] The spherical k-means clustering algorithm divides the training data distributed on the sphere into multiple clusters to minimize the cosine dissimilarity within the cluster and obtain classified data;

[0064] The deep machine learning algorithm simulates the neural network structure of the human brain and automatically learns features and fatigue recognition patterns from classified data based on the objective function, and then performs fatigue recognition based on the features and fatigue recognition patterns.

[0065] The test data is input into the target person fatigue recognition model to obtain the predicted fatigue recognition result. According to the variance between the predicted fatigue recognition and the actual fatigue recognition, the learning rate of the target person fatigue recognition model is adjusted until the variance is less than 0.0011.

[0066] In a second aspect, an embodiment of the present application further provides an electronic device, including:

[0067] A processor; and a memory arranged to store computer executable instructions, which when executed cause the processor to perform the method steps described in the first aspect.

[0068] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores one or more programs. When the one or more programs are executed by an electronic device including multiple applications, the electronic device executes the method steps described in the first aspect.

[0069] The beneficial effects of the present invention are:

[0070] The present invention is a method for identifying fatigue of a target person based on machine learning. Compared with the prior art, the present invention has the following technical effects:

[0071] The present invention can improve the accuracy of user intention recognition in multi-dimensional human-computer interaction scenarios through difference comparison, consistency comparison, fatigue feature extraction, personalized time series analysis, multimodal dynamic fusion, fatigue judgment and model construction steps, thereby improving the accuracy of fatigue recognition of target personnel. Optimizing fatigue recognition of target personnel can greatly save resources and improve work efficiency. It can realize intelligent recognition of fatigue of target personnel, perform personalized time series analysis and multimodal dynamic fusion on fatigue recognition of target personnel in real time, which is of great significance to fatigue recognition of target personnel, can adapt to fatigue recognition of target personnel with different standards and fatigue recognition needs of different target personnel, and has a certain universality. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A flowchart of the steps of a method for identifying fatigue of a target person using machine learning according to the present invention;

[0073] Figure 2 This is a schematic diagram of the structure of an electronic device in an embodiment of this specification. DETAILED DESCRIPTION

[0074] The present invention will be further described below through specific examples. The illustrative examples and descriptions of the present invention are used to explain the present invention but are not intended to limit the present invention.

[0075] A method for fatigue recognition of a target person based on machine learning of the present invention comprises the following steps:

[0076] like Figure 1 As shown, in this embodiment, the following steps are included:

[0077] Data obtained from a database of workers in a preset industrial scenario within a specified time period is used as data to be analyzed; the worker database includes a reference database and a pending database; the pending data includes body monitoring data, image data, and work data; the body monitoring data includes physiological data and behavioral data; the pending database has a higher degree of personalization than the reference database; the reference data is worker data under health standards;

[0078] In the actual evaluation, worker A, an assembly line worker in Workshop 3 of a mobile phone parts factory, was selected as the research subject. Worker A had been working continuously for 13 hours. His physiological data included a heart rate of 120 bpm, heart rate variability of 37 ms, body temperature of 36.9°C, electromyogram of 148 μV, acceleration of 0.95 g, respiratory rate of 26 breaths / minute, and blood oxygen saturation of 95%. Reference data included a heart rate of 70-85 bpm, heart rate variability of 50-70 ms, body temperature of 36.2-36.8°C, electromyogram of 50-100 μV, acceleration of 1.5-2.5 g, respiratory rate of 12-18 breaths / minute, and blood oxygen saturation of 97-99%. Behavioral data included prolonged eye closing and mouth opening, and slow body movements.

[0079] Comparing the data to be analyzed with the reference database for differences, and treating the data to be analyzed with differences as first data, and treating the data with differences as uncertain data;

[0080] Using the pending database to perform consistency comparison on the uncertain data, treating data without differences as problem data, and adding the problem data to the first data;

[0081] Fatigue feature extraction is performed on the first data to obtain key features, personalized time series analysis is performed on the key features to obtain second data, and multimodal dynamic fusion is performed on the second data using an attention mechanism to obtain fused data; the key features are stored in person units;

[0082] In the actual assessment, the key features are heart rate, heart rate variability, body temperature, electromyography, acceleration, respiratory rate, blood oxygen saturation, eye closure, mouth opening, and slow limb movements;

[0083] Performing fatigue determination based on the fusion data based on a dynamic threshold to obtain recognition data, constructing a target person fatigue recognition model based on the recognition data, and performing fatigue recognition based on the target person fatigue recognition model;

[0084] In the actual evaluation, the identification result is: Worker A is tired.

[0085] In this embodiment, the method of extracting fatigue features from the first data to obtain key features includes:

[0086] Use neural network algorithms to extract behavioral features from image data, biomedical signal processing algorithms to extract biosignal features from body monitoring data, and autoencoders to extract state features from working data;

[0087] The behavioral features, biological signal features, and state features are input into the support vector machine algorithm as sample feature vectors. Given the loss coefficient and loss function, the expression is:

[0088]

[0089] QUR x (z T u+v)≥1,β>0

[0090] The normal vector perpendicular to the hyperplane is z, and the loss function of the x-th sample feature vector is The loss coefficient is β, the displacement term is v, the sample feature vector is u, and the label of the x-th sample feature vector is Q x , transposed to T, the number of sample feature vectors is N x , the decision hyperplane function of the normal vector z is

[0091] According to the kernel function, the decision hyperplane function is given and the expression is:

[0092]

[0093] The Lagrange multiplier of the xth sample eigenvector is γ x , the i-th sample feature vector is u i , the jth sample feature vector is u j , sample feature vector u i and the sample feature vector u j The kernel function is The sign function is sgn(·), the weight coefficients are λ and α, and the kernel function parameter is τ;

[0094] If the result of the decision hyperplane function is 1, the sample feature vector is in a non-fatigue state; if the result of the decision hyperplane function is -1, the sample feature vector is in a fatigue state; the sample vector in the fatigue state is output as the key feature.

[0095] In this embodiment, the method of performing personalized time series analysis on the key features to obtain the second data includes:

[0096] Sort the key features in descending order by individual, and use a recurrent neural network to capture the long-term relationship of the sorted key feature set to obtain personalized change data;

[0097] The personalized change data caused by pathology is eliminated according to the medical data of the individual, and the personalized change data after elimination is output as the second data.

[0098] In this embodiment, the method of using the attention mechanism to perform multimodal dynamic fusion on the second data to obtain fused data includes:

[0099] The behavioral features and state features in the second data are integrated into behavioral state features. The energy of specific frequency bands is extracted from the biological signal features through short-time Fourier transform. The self-attention mechanism is introduced in posture estimation to upgrade the behavioral state features.

[0100] The biological signal features are used as keys K or values ​​V for interaction, and a gated recurrent unit is added after the exponential shift.

[0101] The behavioral state features and biological signal features are gated and adaptively fused, and the expression is:

[0102]

[0103] ζ=σ(Z a [h1;h2])

[0104] The learnable dynamic weight is ζ, the element-by-element multiplication is ⊙, the state feature at the s-th moment is h1(s), the biological signal feature at the s-1-th moment is h2(s), and the fusion feature at the s-th moment is The state feature is h1, the biological signal feature is h2, the concatenation of the state feature h1 and the biological signal feature h2 is [h1; h2], and the learnable weight matrix is ​​Z a , the activation function is σ(·);

[0105] The fused features are input into the multimodal fusion attention module MFA, and the MFA encoded output is nonlinearly transformed by the self-gated nonlinear adaptive activation function. The expression is as follows:

[0106]

[0107] The fusion feature after MFA encoding is The learnable bias parameter is P a , the nonlinear adaptive activation function of self-gating is SiLU(·), the nonlinear transformation result is D, and the gating parameter is The number of multimodal models is N a ;

[0108] Calculate weights and biases:

[0109] C=Z C ⊙D+ω C

[0110] K=Z K ⊙D+ω K

[0111]

[0112] The learnable weight matrices are Z C , Z K , Z V , the query is C, the key is K, the value is V, and the learnable bias parameter is P V , the learnable bias vectors are ω C 、ω K ;

[0113] Given the attention function, the expression is:

[0114]

[0115] The attention function is Input comprehensive characteristic length is The relative position is encoded as P ref ;

[0116] Adjust the dynamic weight, the expression is:

[0117]

[0118] The adjusted dynamic weight is The penalty parameter is κ, the predicted probability is pt, and the false positive sample is κ FN , the false negative sample is κ FP ; Output the adjusted fusion features as fusion data.

[0119] In this embodiment, the method for obtaining recognition data by performing fatigue determination based on the fusion data based on a dynamic threshold includes:

[0120] According to the given judgment criteria of the fusion data, the expression is:

[0121]

[0122] The weight of closing eyes is ρ1, the weight of opening mouth is ρ2, the weight of stopping hands is ρ3, the weight of stopping feet is ρ3, and the time of closing eyes in unit working time is n eye , the required working time of the eyes in unit working time is N eye , the time spent opening the mouth in a unit of working time is n mouth , the required working time of the mouth in unit working time is N mouth The time occupied by manual stop in unit working time is n hand , the required working time per unit working time is Nhand The time spent on foot rest in a unit of working time is n foot , the required working time of the foot in unit working time is N foot , fatigue value is

[0123] By drawing ROC curves for different types of fusion data, the maximum difference between the true positive rate and the false positive rate is used as the initial threshold;

[0124] Dynamically adjust the threshold, the expression is:

[0125]

[0126] The number of modes is N m , the classification confidence of each mode m is d m , the classification result of each mode of m is G m , the base confidence is d r , the initial threshold is The adjusted dynamic threshold is The penalty coefficient is φ, and the distribution probability of the biosignal modality is p bio , the distribution probability of the behavior mode is p beh , distribution probability p bio and distribution probability p beh The divergence is U KL (p bio ||p beh );

[0127] A dynamic threshold is used to perform fatigue discrimination on the fused data, and the fused data with fatigue values ​​greater than the dynamic threshold is output as the recognition data;

[0128] In the actual evaluation, the fatigue value before adjustment was 0.312, the threshold of the biological signal modality was 0.37, and that of the behavioral modality was 0.32; the fatigue value was 0.297.

[0129] In this embodiment, the method for constructing a target person fatigue recognition model based on the recognition data includes:

[0130] Construct the objective function based on the recognition data:

[0131]

[0132] The objective function is Fatigue value is The number of modes is N m , the monitoring value of the mth mode is f m , the normal value of the mth mode is f o , the actual fatigue recognition result is y, and the predicted fatigue recognition result is Actual fatigue recognition result y and predicted fatigue recognition result The loss function is The weight coefficient of the mth mode is l m ;

[0133] The target personnel fatigue recognition model includes the random forest algorithm, the spherical k-means clustering algorithm, and the deep machine learning algorithm;

[0134] The random forest algorithm randomly divides the input data into training data and test data in a ratio of 7:2;

[0135] The spherical k-means clustering algorithm divides the training data distributed on the sphere into multiple clusters to minimize the cosine dissimilarity within the cluster and obtain classified data;

[0136] The deep machine learning algorithm simulates the neural network structure of the human brain and automatically learns features and fatigue recognition patterns from classified data based on the objective function, and then performs fatigue recognition based on the features and fatigue recognition patterns.

[0137] The test data is input into the target person fatigue recognition model to obtain the predicted fatigue recognition result. According to the variance between the predicted fatigue recognition and the actual fatigue recognition, the learning rate of the target person fatigue recognition model is adjusted until the variance is less than 0.0011.

[0138] Figure 2 This is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. Figure 2 At the hardware level, the electronic device includes a processor and, optionally, an internal bus, a network interface, and memory. The memory may include internal memory, such as high-speed random-access memory (RAM), or non-volatile memory, such as at least one disk drive. Of course, the electronic device may also include other hardware required for its services.

[0139] The processor, network interface, and memory can be interconnected via an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus, or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 2 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0140] The memory is used to store programs. Specifically, the program may include program code, which includes computer operating instructions. The memory may include internal memory and non-volatile memory, and provides instructions and data to the processor.

[0141] The processor reads the corresponding computer program from the non-volatile memory into the internal memory and then runs it, logically forming a device for machine learning to identify fatigue in a target person. The processor executes the program stored in the memory and is specifically configured to perform any of the aforementioned machine learning methods for identifying fatigue in a target person.

[0142] The above application Figure 1 The illustrated embodiment discloses a method for fatigue identification of a target person using machine learning that can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be performed by hardware integrated logic circuits in the processor or by software instructions. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. The various methods, steps, and logic block diagrams disclosed in the embodiments of this application can be implemented or executed. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium well-known in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory, and the processor reads the information in the memory and, in conjunction with its hardware, completes the steps of the above method.

[0143] The electronic device may also perform Figure 1 A machine learning method for fatigue recognition of target personnel and its implementation Figure 1 The functions of the illustrated embodiment will not be described in detail in the embodiments of the present application.

[0144] An embodiment of the present application also proposes a computer-readable storage medium, which stores one or more programs, and the one or more programs include instructions. When the instructions are executed by an electronic device including multiple applications, they perform any of the aforementioned machine learning methods for fatigue identification of target personnel.

[0145] Those skilled in the art will appreciate that the embodiments of the present application can be provided as methods, systems, or computer program products. Therefore, the present application can adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application can adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0146] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the steps in the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0147] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0148] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 The steps for the function specified in one or more boxes.

[0149] In a typical configuration, a computing device includes one or more processors (CPUs), input / output interfaces, network interfaces, and memory.

[0150] Memory may include non-permanent storage in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. Memory is an example of a computer-readable medium.

[0151] Computer-readable media includes permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transitory computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0152] It should also be noted that the terms "comprises," "includes," or any other variations thereof are intended to encompass non-exclusive inclusion, such that a process, method, commodity, or apparatus that includes a series of elements includes not only those elements but also other elements not explicitly listed, or includes elements inherent to such process, method, commodity, or apparatus. In the absence of further limitations, an element defined by the phrase "comprises a ..." does not exclude the presence of other identical elements in the process, method, commodity, or apparatus that includes the element.

[0153] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present application may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0154] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for identifying fatigue of a target person based on machine learning, characterized in that: The following steps are involved: Data obtained from a database of workers in a preset industrial scenario within a specified time period is used as data to be analyzed; the worker database includes a reference database and a pending database; the pending data includes body monitoring data, image data, and work data; the body monitoring data includes physiological data and behavioral data; the pending database has a higher degree of personalization than the reference database; the reference data is worker data under health standards; Comparing the data to be analyzed with the reference database for differences, and treating the data to be analyzed with differences as first data, and treating the data with differences as uncertain data; Using the pending database to perform consistency comparison on the uncertain data, treating data without differences as problem data, and adding the problem data to the first data; Fatigue feature extraction is performed on the first data to obtain key features, personalized time series analysis is performed on the key features to obtain second data, and multimodal dynamic fusion is performed on the second data using an attention mechanism to obtain fused data; the key features are stored in person units; Fatigue determination is performed on the fusion data based on a dynamic threshold to obtain recognition data, a target person fatigue recognition model is constructed based on the recognition data, and fatigue recognition is performed based on the target person fatigue recognition model.

2. The method for identifying fatigue of a target person based on machine learning according to claim 1, characterized in that: The method of extracting fatigue features from the first data to obtain key features includes: Use neural network algorithms to extract behavioral features from image data, biomedical signal processing algorithms to extract biosignal features from body monitoring data, and autoencoders to extract state features from working data; The behavioral features, biological signal features, and state features are input into the support vector machine algorithm as sample feature vectors. Given the loss coefficient and loss function, the expression is: s.t.Q x (z T u+v)≥1,β>0 The normal vector perpendicular to the hyperplane is z, and the loss function of the x-th sample feature vector is The loss coefficient is β, the displacement term is v, the sample feature vector is u, and the label of the x-th sample feature vector is Q x , transposed to T, the number of sample feature vectors is N x , the decision hyperplane function of the normal vector z is According to the kernel function, the decision hyperplane function is given and the expression is: The Lagrange multiplier of the xth sample eigenvector is γ x , the i-th sample feature vector is u i , the jth sample feature vector is u j , sample feature vector u i and the sample feature vector u j The kernel function is The sign function is sgn(·), the weight coefficients are λ and α, and the kernel function parameter is τ; If the result of the decision hyperplane function is 1, the sample feature vector is in a non-fatigue state; if the result of the decision hyperplane function is -1, the sample feature vector is in a fatigue state; the sample vector in the fatigue state is output as the key feature.

3. The method for identifying fatigue of a target person based on machine learning according to claim 1, characterized in that: The method for performing personalized time series analysis on the key features to obtain second data includes: Sort the key features in descending order by individual, and use a recurrent neural network to capture the long-term relationship of the sorted key feature set to obtain personalized change data; The personalized change data caused by pathology is eliminated according to the medical data of the individual, and the personalized change data after elimination is output as the second data.

4. The method for identifying fatigue of a target person based on machine learning according to claim 1, characterized in that: The method of using an attention mechanism to perform multimodal dynamic fusion on the second data to obtain fused data includes: The behavioral features and state features in the second data are integrated into behavioral state features. The energy of specific frequency bands is extracted from the biological signal features through short-time Fourier transform. The self-attention mechanism is introduced in posture estimation to upgrade the behavioral state features. The biological signal features are used as keys K or values ​​V for interaction, and a gated recurrent unit is added after the exponential shift. The behavioral state features and biological signal features are gated and adaptively fused, and the expression is: The dynamic weights that can be learned are The element-by-element multiplication is ⊙, the state feature at the s-th moment is h1(s), the biological signal feature at the s-1-th moment is h2(s), and the fusion feature at the s-th moment is The state feature is h1, the biological signal feature is h2, the concatenation of the state feature h1 and the biological signal feature h2 is [h1; h2], and the learnable weight matrix is ​​Z a , the activation function is σ(·); The fused features are input into the multimodal fusion attention module MFA, and the MFA encoded output is nonlinearly transformed by the self-gated nonlinear adaptive activation function. The expression is as follows: The fusion feature after MFA encoding is The learnable bias parameter is P a , the nonlinear adaptive activation function of self-gating is SiLU(·), the nonlinear transformation result is D, and the gating parameter is The number of multimodal models is N a ; Calculate weights and biases: C=Z C ⊙D+ω C K=Z K ⊙D+ω K The learnable weight matrices are Z C , Z K , Z V , the query is C, the key is K, the value is V, and the learnable bias parameter is P V , the learnable bias vectors are ω C 、ω K ; Given the attention function, the expression is: The attention function is Input comprehensive characteristic length is The relative position is encoded as P ref ; Adjust the dynamic weight, the expression is: The adjusted dynamic weight is The penalty parameter is κ, the predicted probability is pt, and the false positive sample is κ FN , the false negative sample is κ FP ; Output the adjusted fusion features as fusion data.

5. The method for identifying fatigue of a target person based on machine learning according to claim 1, characterized in that: The method for obtaining recognition data by performing fatigue determination based on the fusion data based on a dynamic threshold comprises: According to the given judgment criteria of the fusion data, the expression is: The weight of closing eyes is ρ1, the weight of opening mouth is ρ2, the weight of stopping hands is ρ3, the weight of stopping feet is ρ3, and the time of closing eyes in unit working time is n eye , the required working time of the eyes in unit working time is N eye , the time spent opening the mouth in a unit of working time is n mouth , the required working time of the mouth in unit working time is N mouth The time occupied by manual stop in unit working time is n hand , the required working time per unit working time is N hand The time spent on foot rest in a unit of working time is n foot , the required working time of the foot in unit working time is N foot , fatigue value is By drawing ROC curves for different types of fusion data, the maximum difference between the true positive rate and the false positive rate is used as the initial threshold; Dynamically adjust the threshold, the expression is: The number of modes is N m , the classification confidence of each mode m is d m , the classification result of each mode of m is G m , the base confidence is d r , the initial threshold is The adjusted dynamic threshold is The penalty coefficient is φ, and the distribution probability of the biosignal modality is p bio , the distribution probability of the behavior mode is p beh , distribution probability p bio and distribution probability p beh The divergence is U KL (p bio ||p beh ); A dynamic threshold is used to perform fatigue discrimination on the fused data, and the fused data with fatigue values ​​greater than the dynamic threshold is output as recognition data.

6. The method for identifying fatigue of a target person based on machine learning according to claim 1, characterized in that: The method for constructing a target person fatigue recognition model based on the recognition data includes: Construct the objective function based on the recognition data: The objective function is Fatigue value is The number of modes is N m , the monitoring value of the mth mode is f m , the normal value of the mth mode is f o , the actual fatigue recognition result is y, and the predicted fatigue recognition result is Actual fatigue recognition result y and predicted fatigue recognition result The loss function is The weight coefficient of the mth mode is l m ; The target personnel fatigue recognition model includes the random forest algorithm, the spherical k-means clustering algorithm, and the deep machine learning algorithm; The random forest algorithm randomly divides the input data into training data and test data in a ratio of 7:2; The spherical k-means clustering algorithm divides the training data distributed on the sphere into multiple clusters to minimize the cosine dissimilarity within the cluster and obtain classified data; The deep machine learning algorithm simulates the neural network structure of the human brain and automatically learns features and fatigue recognition patterns from classified data based on the objective function, and then performs fatigue recognition based on the features and fatigue recognition patterns. The test data is input into the target person fatigue recognition model to obtain the predicted fatigue recognition result. According to the variance between the predicted fatigue recognition and the actual fatigue recognition, the learning rate of the target person fatigue recognition model is adjusted until the variance is less than 0.0011.

7. An electronic device comprising: processor; as well as A memory arranged to store computer executable instructions, which, when executed, cause the processor to perform the method according to any one of claims 1 to 6.

8. A computer-readable storage medium storing one or more programs, wherein when the one or more programs are executed by an electronic device including a plurality of application programs, the electronic device executes the method according to any one of claims 1 to 6.

Images