Working load evaluation method, device, equipment and medium

Abstract

The invention provides a work load assessment method, device, equipment and medium, and the method comprises the steps: collecting the multi-modal physiological data of a work user in the task execution process of the work user, the multi-modal physiological data including electrocardiogram data, human body temperature and the oxyhemoglobin concentration of cerebral cortex; extracting physiological signal features in each mode from the multi-mode physiological data; and inputting the physiological signal features in each mode into a pre-trained work load classification model to obtain a classification result output by the work load classification model, the classification result being used for indicating the work load level of the work user. In this way, by collecting the multi-modal physiological data of the operation user, the comprehensiveness and accuracy of operation load evaluation are improved, the operation load level of the operation user is obtained through the operation load classification model, targeted analysis of the operation user is achieved, the accuracy of operation load evaluation is improved, and the user experience is improved. Therefore, operation distribution can be guided more scientifically and effectively.

Description

Technical Field

[0001] This application relates to the field of thermal reaction technology, and more specifically, to a method, apparatus, equipment, and medium for assessing work load. Background Technology

[0002] Workload is an important indicator for measuring the degree of burden an individual bears when performing a task, and it can be divided into cognitive, physiological, and psychological load. Cognitive load mainly involves the neural resource demands an individual has on attention, working memory, information processing, and decision-making abilities during task performance, and is influenced by factors such as task complexity, decision-making pressure, and time constraints. Physiological load reflects the cardiovascular response, fatigue accumulation, and changes in muscle activity triggered by the task; for example, high-intensity work can lead to a decline in physiological function and affect work efficiency. Psychological load is related to emotional state and stress perception; excessive psychological load may weaken cognitive performance and increase work-related risks.

[0003] When conducting workload assessments, a single indicator is often used, resulting in insufficient assessment basis and making it difficult to achieve a comprehensive and accurate assessment of workload. Consequently, it fails to provide scientific and effective guidance for workload allocation. Summary of the Invention

[0004] In view of this, this application provides a method, apparatus, medium and equipment for assessing workload, which can perform comprehensive and accurate workload assessment by using multimodal physiological data of the work user, and provide scientific and effective guidance for work allocation.

[0005] Specifically, this application is implemented through the following technical solution: According to a first aspect of this application, a workload assessment method is provided, the method comprising: During the process of the user performing the task, multimodal physiological data of the user is collected, including electrocardiogram data, body temperature and oxyhemoglobin concentration in the cerebral cortex; Physiological signal features under each modality were extracted from the multimodal physiological data; The physiological signal features under each modality are input into a pre-trained workload classification model to obtain the classification result output by the workload classification model. The classification result is used to indicate the workload level of the user.

[0006] In one optional implementation, the step of extracting physiological signal features from the multimodal physiological data for each modality includes: According to the preset window length and preset sliding step size, the multimodal physiological data is dynamically sampled in each modality to obtain the sampled physiological data in each modality; For each modality, physiological signal features of that modality are extracted from the physiological data sampled under that modality.

[0007] In one optional implementation, the step of extracting physiological signal features of a specific modality from the physiological data sampled under that modality includes: For each modality, the physiological data sampled under that modality are processed by signal rearrangement to obtain the feature matrix under that modality; The feature matrix under this mode is normalized, and the normalized feature matrix is ​​color mapped to obtain a color image under this mode. The color image is used as the physiological signal feature under this mode.

[0008] In one optional implementation, the workload classification model is trained through the following steps: During the human thermal response experiment performed by the test user in the target environment, the test user's multimodal physiological data, environmental parameter information under various test environmental parameters, and response result data under various test tasks were collected. Based on the test multimodal physiological data, the environmental parameter information, and the response result data, the true value of the workload level is determined; The physiological signal features of each modality were extracted from the multimodal physiological data of the test. The test physiological signal features are input into a neural network to obtain the predicted classification result output by the neural network; Based on the error loss between the true value of the workload level and the predicted classification result, the parameters of the neural network are adjusted to obtain a trained workload model.

[0009] In one optional implementation, determining the true value of the workload level based on the test multimodal physiological data, the environmental parameter information, and the response result data includes: Cognitive load was assessed based on the test multimodal physiological data, the environmental parameter information, and the response result data to obtain a scoring matrix; The scoring weights corresponding to the scoring matrix are determined based on a genetic algorithm. The elements in the scoring matrix are weighted and summed according to the scoring weights to obtain the weighted scoring matrix. The weighted scoring matrix is ​​subjected to K-means clustering, and the true value of the workload level is determined based on the clustering results.

[0010] In one optional implementation, the workload classification model includes at least a feature enhancement module, a feature processing module, a multimodal competitive query module, a bidirectional long short-term memory module, and a classification module; The step of inputting physiological signal features from various modalities into a pre-trained workload classification model to obtain the classification results output by the workload classification model includes: The feature enhancement module is used to locally enhance the physiological signal features at the oxyhemoglobin concentration to obtain enhanced physiological signal features at the oxyhemoglobin concentration. The feature processing module performs structural normalization on the physiological signal features under the electrocardiogram data, the physiological signal features under the human body temperature, and the physiological signal features under the enhanced oxyhemoglobin concentration to obtain the physiological features under each modality after structural normalization. The multimodal competitive query module filters out the target physiological features that contribute the most to the classification of workload from the physiological features of each modality, and then fuses the target physiological features to obtain a fused feature matrix. The bidirectional long short-term memory module is used to perform bidirectional time analysis on the fused feature matrix to obtain bidirectional feature vectors. The classification module determines the classification result based on the bidirectional feature vector.

[0011] In one optional implementation, the step of locally enhancing the physiological signal features at the oxyhemoglobin concentration using the feature enhancement module to obtain enhanced physiological signal features at the oxyhemoglobin concentration includes: The feature enhancement module performs threshold segmentation on the physiological signal features at the oxyhemoglobin concentration to extract the target region from the physiological signal features at the oxyhemoglobin concentration. The physiological signal characteristics at the oxyhemoglobin concentration in the target region are filtered according to the target region to generate target region features; Based on the physiological signal characteristics at the oxyhemoglobin concentration and the characteristics of the target region, the enhanced physiological signal characteristics at the oxyhemoglobin concentration are obtained.

[0012] In one optional implementation, the step of using the multimodal competitive query module to filter out target physiological features from physiological features under various modalities that contribute the most to the workload classification in a preset number of modalities, and fusing the target physiological features to obtain a fused feature matrix, includes: The multimodal competitive query module maps physiological features from different modalities to the same query space. Physiological features under different modalities within the same query space are scored to obtain the importance score corresponding to each physiological feature under each modality. The importance score is used to indicate the contribution of the physiological feature under that modality to the workload classification. Select the target physiological features with the highest importance score from the physiological features of each modality; The target physiological features are weighted and summed according to normalized weights to obtain the fusion feature matrix.

[0013] In one optional implementation, the step of performing bidirectional temporal analysis on the fused feature matrix through the bidirectional long short-term memory module to obtain a bidirectional feature vector includes: The bidirectional long short-term memory module performs forward time analysis and reverse time analysis on the fused feature matrix to obtain forward eigenvectors and reverse eigenvectors; the forward time analysis is used to perform analysis according to the default flow direction of time. The positive feature vector and the negative feature vector are fused to obtain the bidirectional feature vector.

[0014] According to a second aspect of this application, a workload assessment apparatus is provided, the apparatus comprising: The data acquisition module is used to collect the multimodal physiological data of the user during the execution of the task. The multimodal physiological data includes electrocardiogram data, body temperature and oxyhemoglobin concentration in the cerebral cortex. The feature extraction module is used to extract physiological signal features of each modality from the multimodal physiological data; The result determination module is used to input the physiological signal features under each modality into a pre-trained workload classification model to obtain the classification result output by the workload classification model. The classification result is used to indicate the workload level of the user.

[0015] According to a third aspect of this application, a computer device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the steps of the workload assessment method described in the first aspect above.

[0016] According to a fourth aspect of this application, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps of the workload assessment method described in the first aspect above.

[0017] The workload assessment method, apparatus, equipment, and medium provided in this application improve the comprehensiveness and accuracy of workload assessment by collecting multimodal physiological data of workers, including electrocardiogram data, body temperature, and oxyhemoglobin concentration in the cerebral cortex. Through a workload classification model, the workload level of workers is obtained, enabling targeted analysis and enhancing the applicability and individualization of workload assessment. This allows for more scientific and effective guidance of workload allocation, better protection of workers' safety and health during task execution, especially in high-temperature environments, and improved work efficiency, ultimately ensuring the effectiveness of workers.

[0018] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and are not intended to limit the technical solutions of this disclosure.

[0019] To make the above-mentioned objects, features and advantages of this disclosure more apparent and understandable, preferred embodiments are described below in detail with reference to the accompanying drawings. Attached Figure Description

[0020] Figure 1 This is a flowchart illustrating a workload assessment method according to an exemplary embodiment of this application; Figure 2 This is a schematic diagram illustrating a data acquisition method according to an exemplary embodiment of this application; Figure 3 This is a schematic diagram of a thermal perception assessment questionnaire shown in an exemplary embodiment of this application; Figure 4 This is a schematic diagram of a workload level assessment questionnaire illustrated in an exemplary embodiment of this application; Figure 5 This is a schematic diagram illustrating an exemplary embodiment of the letter memory test task of this application; Figure 6 This is a schematic diagram illustrating a path tracing test task according to an exemplary embodiment of this application; Figure 7 This is a schematic diagram illustrating an addition test task according to an exemplary embodiment of this application; Figure 8 This is a schematic diagram illustrating a Stroop test task according to an exemplary embodiment of this application; Figure 9 This is a schematic diagram illustrating the structure of a workload classification model according to an exemplary embodiment of this application; Figure 10 This is a schematic diagram illustrating the working principle of a bidirectional long short-term memory module according to an exemplary embodiment of this application; Figure 11This is a schematic diagram of a task scheduling device shown in an exemplary embodiment of this application; Figure 12 This is a schematic diagram of the structure of a computer device shown in an exemplary embodiment of this application. Detailed Implementation

[0021] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0022] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0023] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0024] Studies have found that environmental factors such as temperature, noise, and lighting often affect workload, thereby reducing job performance and impacting job safety. Particularly in high-temperature environments, individual workload increases significantly, primarily manifested as a simultaneous increase in physiological and cognitive load. High temperatures not only lead to elevated core body temperature, increased heart rate, and blood flow redistribution, but may also inhibit neural activity in the prefrontal cortex, weakening attention, cognitive control, and information processing abilities, thus affecting working memory and decision-making, ultimately resulting in decreased job performance. However, workload assessments often rely on single indicators, resulting in insufficient evaluation criteria and difficulty in comprehensively quantifying workload. Furthermore, workload grading largely depends on a uniform classification based on ambient temperature and metabolic levels, neglecting the influence of individual differences.

[0025] Based on the above research, this application provides a workload assessment method. By collecting multimodal physiological data of workers, including electrocardiogram data, body temperature, and oxyhemoglobin concentration, it helps to improve the comprehensiveness and accuracy of workload assessment. By obtaining the workload level of workers through a workload classification model, it enables targeted analysis of workers, which helps to improve the accuracy of workload assessment and thus guides work allocation more scientifically and effectively.

[0026] To facilitate understanding of this embodiment, a detailed description of the workload assessment method disclosed in this application is provided first. The execution entity of the workload assessment method provided in this application is generally an electronic device with a certain computing power. This electronic device can be a server, which can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud storage, big data, and artificial intelligence platforms. In some possible implementations, this workload assessment method can be implemented by a processor calling computer-readable instructions stored in memory.

[0027] The following description, in conjunction with the accompanying drawings, illustrates a method for assessing workload provided in an embodiment of this application.

[0028] See Figure 1 The diagram shown is a flowchart illustrating a workload assessment method according to an exemplary embodiment of this application. Figure 1 As shown in the figure, the workload assessment method provided in this embodiment includes steps S101 to S103, wherein: S101: During the process of the user performing the task, collect the user's multimodal physiological data, including electrocardiogram data, body temperature and oxyhemoglobin concentration in the cerebral cortex.

[0029] The electrocardiogram (ECG) data may include, for example, heart rate (HR) and heart rate variability (HRV). Optionally, the ECG data of the user can be collected by an ECG device (such as a physiological recorder) and a heart rate monitor, for example, by recording the user's cardiovascular activity data in real time at a preset frequency (such as 2000Hz), thereby ensuring that the waveform characteristics of the ECG data can be captured.

[0030] The human body temperature may include, for example, core temperature and skin temperature. Optionally, the core temperature of the worker can be measured using a body temperature capsule, and the skin temperature can be measured using a skin temperature device (such as a skin patch). In the case of multiple skin temperature collection sites, the final collected data can be the average skin temperature. For example, the skin temperature device can be attached to multiple sites of the worker, such as the forehead, chest, upper arm, forearm, thigh, and calf, and the skin temperature of each site can be weighted and averaged to obtain the final average skin temperature. The specific value of the sampling time interval between the core temperature and the skin temperature can be determined according to the needs of the workload assessment and is not limited here.

[0031] The concentration of oxyhemoglobin in the cerebral cortex can be measured by a brain oxygenation device (such as a functional near-infrared spectroscopy device) to measure the response of the prefrontal cortex (PFC) of the user. The near-infrared spectroscopy device can capture the dynamic changes in the concentration of oxyhemoglobin through the absorption and scattering effects of near-infrared light.

[0032] S102: Extract physiological signal features of each modality from the multimodal physiological data.

[0033] In this step, physiological signal features of the following modalities can be extracted from the physiological data of electrocardiogram (ECG), body temperature, and oxyhemoglobin concentration, respectively.

[0034] In some possible implementations, extracting physiological signal features from the multimodal physiological data for each modality includes: According to the preset window length and preset sliding step size, the multimodal physiological data is dynamically sampled in each modality to obtain the sampled physiological data in each modality; For each modality, physiological signal features of that modality are extracted from the physiological data sampled under that modality.

[0035] In the above steps, for the physiological data of each modality, a dynamic window sampling method is used to segment the physiological data using a sliding window of a preset window length, preserving temporal continuity with each slide. For example, the preset window length can be set to 66 seconds and the preset sliding step size to 1 second to ensure data consistency within the same window. After obtaining the sampled physiological data for each modality, physiological signal features for that modality can be extracted from the sampled physiological data for that modality.

[0036] In this way, dynamic window sampling can optimize the temporal distribution of data, improve the representativeness of the time dimension, better capture the dynamic changes of high-frequency sampled data, reduce the dispersion of data points, and ensure comprehensive coverage and temporal dependence. It is suitable for physiological signals with high requirements for temporal correlation, so as to ensure that more representative input data is provided for subsequent feature extraction and model processing.

[0037] In other embodiments, the multimodal physiological data can be intersampling in each modality according to preset intra-data step sizes and preset inter-data step sizes to obtain sampled physiological data for each modality. In this way, for the physiological data in each modality, intersampling is used with preset intra-data step sizes and preset inter-data step sizes. This method is suitable for scenarios where feature changes are stable and the requirements for temporal continuity are relatively low.

[0038] In some possible implementations, the step of extracting physiological signal features from the physiological data sampled under each modality includes: For each modality, the physiological data sampled under that modality are processed by signal rearrangement to obtain the feature matrix under that modality; The feature matrix under this mode is normalized, and the normalized feature matrix is ​​color mapped to obtain a color image under this mode. The color image is used as the physiological signal feature under this mode.

[0039] In the above steps, for each modality, the sampled physiological data under that modality can be rearranged in both row and column dimensions to obtain the feature matrix for that modality. This allows the subsequent workload classification model to capture local temporal features in the row direction and multi-channel spatial distribution features in the column direction. Since the numerical ranges of different features vary significantly, peak normalization can be used to normalize the feature matrix for that modality, ensuring that each feature remains within the same numerical range. This helps improve the efficiency of the workload classification model in processing multi-dimensional features. Color mapping is then applied to the normalized feature matrix to obtain a color image for that modality. This color image is used as the physiological signal feature for that modality, enabling the subsequent workload classification model to comprehensively capture the spatiotemporal distribution features of that modality as workload changes.

[0040] Optionally, for oxyhemoglobin concentration, the physiological data sampled under this modality can be directly processed by signal rearrangement to obtain the feature matrix under this modality; for electrocardiogram data and human body temperature, the physiological data sampled under this modality can be spatially encoded and then processed by signal rearrangement to obtain the spatial encoding matrix under this modality.

[0041] In this way, the physiological data sampled under each modality are rearranged, normalized, and color mapped to ensure the stability and balanced distribution of features, effectively improving the ability of the workload classification model to perceive workload.

[0042] S103: Input the physiological signal features under each modality into the pre-trained workload classification model to obtain the classification result output by the workload classification model. The classification result is used to indicate the workload level of the user.

[0043] In this step, physiological signal features from each modality are input into a pre-trained workload classification model. Multimodal fusion can effectively integrate the advantages of different features and achieve feature complementarity between different modalities. Oxyhemoglobin concentration can reflect the brain's oxygen supply status, body temperature can characterize the thermoregulation ability, and electrocardiogram data can reflect cardiovascular load. The combination of these three features enables a dual-dimensional characterization of workload status from both cognitive and physiological perspectives, which helps to improve the robustness and reliability of the classification results.

[0044] In some possible implementations, the workload classification model is trained through the following steps: During the human thermal response experiment performed by the test user in the target environment, the test user's multimodal physiological data, environmental parameter information under various test environmental parameters, and response result data under various test tasks were collected. In the above steps, an environmental chamber can be set up to create a target environment in which test users can perform human thermal response experiments. Optionally, to ensure the accuracy of the collected data, the collection of the test user's multimodal physiological data, environmental parameter information under various test environmental parameters, and response result data under various test tasks can begin at least after the test user has entered the environmental chamber for a preset time (e.g., 20 minutes).

[0045] For example, see [link / reference] Figure 2 This is a schematic diagram illustrating a data acquisition method as shown in an exemplary embodiment of this application. Figure 2 As shown, the system can be divided into a stimulus presentation system, a task performance acquisition system, a realistic environment simulation system, and a physiological monitoring system. In the realistic environment simulation system, the target environment is constructed. In the stimulus presentation system, a display can be provided to present the test task to the test user. In the task performance acquisition system, a keyboard and mouse compatible with the display can be provided to allow the test user to perform the test task. In the physiological monitoring system, a terminal device (such as a computer) can be provided for data collection, to collect the test user's multimodal physiological data, environmental parameter information under various test environment parameters, and response result data under various test tasks.

[0046] Optionally, a unified time-series labeling method can be used to achieve simultaneous acquisition of multiple data types, thereby improving the accuracy of subsequent model training. For example, labels can be set, and after the labels are transmitted, the test user begins performing a human thermal response experiment. The test user's multimodal physiological data can be captured from the moment the label is received by the terminal device used for data acquisition.

[0047] Here, the process of collecting the test user's multimodal physiological data is similar to the process of collecting the work user's multimodal physiological data in the previous embodiment. For details, please refer to the previous embodiment, which will not be repeated here.

[0048] The environmental parameters include, but are not limited to, temperature, humidity, and wind speed. The testing tasks may include psychological testing tasks and cognitive testing tasks; the psychological testing tasks may include thermal perception assessment tests and workload level assessment tests.

[0049] For example, see [link / reference] Figure 3 This is a schematic diagram illustrating a thermal perception assessment questionnaire, as shown in an exemplary embodiment of this application. Figure 3 As shown, the thermal perception assessment questionnaire includes Thermal Sensation Vote (TSV), Thermal Comfort Vote (TCV), Thermal Acceptance (TA), Sweating Sensation Vote (SSV), Warning Sensation Vote (WSV), and Mood Vote (MV). This questionnaire allows for the quantification of users' subjective ratings of various heat-related experiences in experimental environments related to human thermal response.

[0050] For example, see [link / reference] Figure 4 This is a schematic diagram illustrating a workload level assessment questionnaire, as shown in an exemplary embodiment of this application. Figure 4 As shown, the workload level assessment questionnaire uses the Task Load Index (TLX), which quantifies the level of work satisfaction through six dimensions: Physical Demand (PD), Mental Demand (MD), Temporal Demand (TD), Performance (P), Effort (EF), and Frustration (FR). Each dimension has a score range of 0 to 100, with higher scores indicating a greater workload.

[0051] The cognitive testing tasks may include an n-back test, a trail-making test, an addition test, and a Stroop test, which respectively measure the user's cognitive functions in various aspects such as working memory, visual search, information processing, and cognitive interference.

[0052] For example, see [link / reference] Figure 5 This is a schematic diagram illustrating a letter memory test task as an exemplary embodiment of this application. Figure 5 As shown, in the n-Back task, 'n' represents the stimulus that needs to be compared with the stimulus presented n times earlier, and whether it matches. This example uses 2-back, where the task consists of three blocks, each with 12 trials. Each trial first presents a "+" transition symbol for the target duration (e.g., 1.7 seconds), followed by a random letter for the target duration (e.g., 1.7 seconds). The user needs to determine if the current letter is the same as the letter presented in the previous two attempts (i.e., the 4th letter is compared with the 2nd letter). If they are the same, press the "←" key; otherwise, press the "→" key. A rest period of a certain duration (e.g., 20 seconds) is set after each block.

[0053] For example, see [link / reference] Figure 6 This is a schematic diagram illustrating a path tracing test task as an exemplary embodiment of this application. Figure 6 As shown, in the path tracing test task, the numbers 1-9 and the letters AI are randomly distributed on the screen. The test user needs to connect the numbers and letters alternately in ascending order within a certain time (e.g., 40 seconds), such as "1-A-2-B-3-C".

[0054] For example, see [link / reference] Figure 7 This is a schematic diagram illustrating an addition test task as an exemplary embodiment of this application. Figure 7 As shown, in the addition test task, this example task contains 2 blocks, each block containing 5 trials. Each trial first presents a "+" transition symbol for a certain duration (e.g., 2 seconds), followed by an addition problem of 3 two-digit numbers for a certain duration (e.g., 8 seconds). Participants must complete the addition calculation within the time limit (e.g., 8 seconds) and input and submit the corresponding answer. A rest period of a certain duration (e.g., 20 seconds) is set after each block.

[0055] For example, see [link / reference] Figure 8 This is a schematic diagram illustrating a Stroop test task as an exemplary embodiment of this application. Figure 8As shown, in the Stroop test task, this example task consists of three blocks, each comprising 20 trials. Each trial includes a stimulus presentation phase of a certain duration (e.g., 2 seconds), followed by a "+" transition phase of a certain duration (e.g., 0.7 seconds). During the stimulus presentation phase, a word representing a randomly assigned color is displayed on the computer screen, with the word itself displayed in the same or a different color (e.g., a blue word displayed in yellow). Participants are required to ignore the semantics of the word and identify the color of the displayed word as quickly as possible. A rest period of a certain duration (e.g., 10 seconds) is provided after each block.

[0056] After collecting the test user's multimodal physiological data, environmental parameter information under various test environment parameters, and response result data under various test tasks, the true value of the workload level can be determined based on the test multimodal physiological data, the environmental parameter information, and the response result data. Test physiological signal features under each modality are extracted from the test multimodal physiological data, and the test physiological signal features are input into a neural network to obtain the predicted classification result output by the neural network. Then, based on the error loss between the true value of the workload level and the predicted classification result, the parameters of the neural network are adjusted until the training cutoff condition is met (e.g., the number of training iterations reaches the target number, or the error loss is less than the target loss threshold, etc.), and a trained workload model is obtained.

[0057] Here, the process of extracting the test physiological signal features of each modality from the multimodal physiological data is similar to the process of extracting the physiological signal features of each modality from the multimodal physiological data in the previous embodiment. For details, please refer to the previous embodiment, and it will not be repeated here.

[0058] Traditional workload assessment methods use relatively insufficient evaluation indicators, making it difficult to achieve comprehensive quantification of workload. In this embodiment, the true value of workload level is determined based on the test user's multimodal physiological data, environmental parameter information under various test environment parameters, and response result data under various test tasks. This provides standardized training data for model training, integrates physiological and cognitive factors, and improves the accuracy and comprehensiveness of workload assessment.

[0059] In some possible implementations, determining the true value of the workload level based on the test multimodal physiological data, the environmental parameter information, and the response result data includes: Cognitive load was assessed based on the test multimodal physiological data, the environmental parameter information, and the response result data to obtain a scoring matrix; The scoring weights corresponding to the scoring matrix are determined based on a genetic algorithm. The elements in the scoring matrix are weighted and summed according to the scoring weights to obtain the weighted scoring matrix. The weighted scoring matrix is ​​subjected to K-means clustering, and the true value of the workload level is determined based on the clustering results.

[0060] In the above steps, cognitive load assessment can be performed by integrating the multimodal physiological data from the test, the environmental parameter information, and the response result data to generate a scoring matrix. Then, a genetic algorithm is used, through population initialization, fitness calculation, genetic operations (including selection, crossover, and mutation), and iterative optimization, to determine the scoring weights corresponding to the scoring matrix. Next, the elements in the scoring matrix are weighted and summed according to the scoring weights to obtain a weighted scoring matrix. K-means clustering is then performed on the weighted scoring matrix, and the true value of the workload level is determined based on the clustering results.

[0061] Here, workload can be divided into three levels: low, medium, and high, corresponding to categories 1, 2, and 3 respectively, to reflect the physiological and cognitive state of individuals under different working conditions. Low workload corresponds to a relatively low level of physiological and cognitive stress, allowing individuals to maintain high efficiency for extended periods. Medium workload reflects a certain degree of fatigue and stress accumulation, requiring appropriate adjustments to the work rhythm to prevent a decline in work efficiency. High workload indicates a significant increase in the individual's physiological and cognitive load, leading to a decrease in work capacity. Continuing to work in this state may affect personnel safety and work efficiency, necessitating appropriate rest or optimized work arrangements.

[0062] In this way, by using cluster analysis to integrate multimodal physiological data, environmental parameter information, and response result data, a scoring matrix is ​​constructed. Then, a genetic algorithm is used to dynamically optimize the scoring weights. Finally, through weighted summation and K-means clustering, the true value of the workload level is scientifically determined, which effectively improves the objectivity and accuracy of workload level classification, enhances adaptability to different working environments and individual states, and provides a highly reliable benchmark label for subsequent model training and real-time evaluation, thereby improving the adaptive performance of workload classification.

[0063] For example, see [link / reference] Figure 9 This is a schematic diagram illustrating the structure of a workload classification model, as shown in an exemplary embodiment of this application. Figure 9 As shown, the workload classification model includes at least a feature enhancement module, a feature processing module, a multimodal competitive query module, a bidirectional long short-term memory module, and a classification module.

[0064] In some possible implementations, the step of inputting physiological signal features under each modality into a pre-trained workload classification model to obtain the classification result output by the workload classification model includes: The feature enhancement module is used to locally enhance the physiological signal features at the oxyhemoglobin concentration to obtain enhanced physiological signal features at the oxyhemoglobin concentration. The feature processing module performs structural normalization on the physiological signal features under the electrocardiogram data, the physiological signal features under the human body temperature, and the physiological signal features under the enhanced oxyhemoglobin concentration to obtain the physiological features under each modality after structural normalization. The multimodal competitive query module filters out the target physiological features that contribute the most to the classification of workload from the physiological features of each modality, and then fuses the target physiological features to obtain a fused feature matrix. The bidirectional long short-term memory module is used to perform bidirectional time analysis on the fused feature matrix to obtain bidirectional feature vectors. The classification module determines the classification result based on the bidirectional feature vector.

[0065] In the above steps, the feature enhancement module can be used to screen and enhance the physiological signal features under the oxyhemoglobin concentration, thereby highlighting key regional information and optimizing the spatial distribution of data representation. The feature processing module performs structural normalization on the physiological signal features under the ECG data, the physiological signal features at body temperature, and the enhanced physiological signal features under the oxyhemoglobin concentration, ensuring consistent structure and dimension of physiological features across different modalities after structural normalization, effectively eliminating biases between different modalities. The multimodal competitive query module filters physiological features across different modalities, highlighting target physiological features in the preset number of modalities that contribute the most to workload classification, and fuses these target physiological features to obtain a fused feature matrix, thereby suppressing redundant information interference. The Bidirectional Long Short-Term Memory (Bi-LSTM) module performs bidirectional temporal analysis on the fused feature matrix to obtain bidirectional feature vectors, effectively capturing the temporal correlation between features. Through a bidirectional propagation mechanism, it extracts dependencies across time steps, strengthens the dynamic interaction between different physiological signals, and improves classification accuracy and model generalization ability. The classification module determines the classification result based on the bidirectional feature vector, thus completing the final classification decision.

[0066] In some possible implementations, the step of locally enhancing the physiological signal features at the oxyhemoglobin concentration using the feature enhancement module to obtain enhanced physiological signal features at the oxyhemoglobin concentration includes: The feature enhancement module performs threshold segmentation on the physiological signal features at the oxyhemoglobin concentration to extract the target region from the physiological signal features at the oxyhemoglobin concentration. The physiological signal characteristics at the oxyhemoglobin concentration in the target region are filtered according to the target region to generate target region features; Based on the physiological signal characteristics at the oxyhemoglobin concentration and the characteristics of the target region, the enhanced physiological signal characteristics at the oxyhemoglobin concentration are obtained.

[0067] In the above steps, the feature enhancement module can optimize the physiological signal features at the oxyhemoglobin concentration using Region-Guided Attention (RGA), guiding attention to the target region within the physiological signal features at the oxyhemoglobin concentration. Threshold segmentation distinguishes between foreground regions (i.e., the target region) containing important information and background regions containing irrelevant information. Then, feature filtering is performed on the physiological signal features at the oxyhemoglobin concentration according to the target region to generate target region features. This helps reduce the interference of background noise on the classification task while preserving the regional dynamics of the signal. Finally, the physiological signal features at the oxyhemoglobin concentration and the target region features are combined to obtain the enhanced physiological signal features at the oxyhemoglobin concentration. By incorporating the original physiological signal features at the oxyhemoglobin concentration, information loss due to uncertainty in target region selection can be avoided, while simultaneously increasing the attention mechanism's ability to model global features.

[0068] Optionally, the above RGA process can be embedded into the corresponding layers of the network, and through layer-by-layer region-guided feature enhancement, it can be ensured that the model processing can accurately focus on the target regions in the physiological signal features under oxyhemoglobin concentration.

[0069] In this way, obtaining the enhanced physiological signal characteristics at the oxyhemoglobin concentration through the above method helps to improve the spatiotemporal characterization ability at the oxyhemoglobin concentration, thereby improving the ability to distinguish the characteristics at the oxyhemoglobin concentration.

[0070] In some possible implementations, the step of using the multimodal competitive query module to filter out target physiological features from the physiological features of each modality that contribute the most to the workload classification in a preset number of modalities, and fusing the target physiological features to obtain a fused feature matrix, includes: The multimodal competitive query module maps physiological features from different modalities to the same query space. Physiological features under different modalities within the same query space are scored to obtain the importance score corresponding to each physiological feature under each modality. The importance score is used to indicate the contribution of the physiological feature under that modality to the workload classification. Select the target physiological features with the highest importance score from the physiological features of each modality; The target physiological features are weighted and summed according to normalized weights to obtain the fusion feature matrix.

[0071] In the above steps, the multimodal competitive query module can map physiological features under different modalities to the same query space, thereby ensuring that physiological features under different modalities are compared in the same space and improving the consistency of modal features. Multimodal-Guided Competitive Query Selection (MCQS) is performed on the physiological features under different modalities. Feature scoring is performed on the physiological features under different modalities within the same query space to obtain the importance score corresponding to each physiological feature under each modality. The importance score indicates the contribution of the physiological feature under that modality to the workload classification. The importance scores corresponding to the physiological features under different modalities can be sorted, for example, from high to low or from low to high. The target physiological features under a preset number of modalities with the highest importance scores are selected, and the target physiological features are weighted and summed according to normalized weights to obtain the fused feature matrix.

[0072] In this way, by using feature importance scoring and Top-K selection strategies, the optimal feature combination can be dynamically determined, thereby achieving more efficient information fusion.

[0073] In some possible implementations, the step of performing bidirectional time analysis on the fused feature matrix through the bidirectional long short-term memory module to obtain a bidirectional feature vector includes: The bidirectional long short-term memory module performs forward time analysis and reverse time analysis on the fused feature matrix to obtain forward eigenvectors and reverse eigenvectors; the forward time analysis is used to perform analysis according to the default flow direction of time. The positive feature vector and the negative feature vector are fused to obtain the bidirectional feature vector.

[0074] For example, see [link / reference] Figure 10 This is a schematic diagram illustrating the working principle of a bidirectional long short-term memory module, as shown in an exemplary embodiment of this application. Figure 10As shown, the fused feature matrix is ​​simultaneously passed to both a forward LSTM and a backward LSTM to perform forward time analysis and backward time analysis on the fused feature matrix, respectively. The forward time analysis is used to analyze according to the default flow direction of time, and the backward time analysis generates forward feature vectors. and inverse eigenvectors By concatenating the positive and negative feature vectors into a bidirectional feature vector, the model can simultaneously focus on both historical and future information of the features. Compared to traditional unidirectional LSTM, it can learn patterns in time series more comprehensively and is more suitable for modeling long-term dependent physiological signals.

[0075] In this way, by using a bidirectional long short-term memory module and a bidirectional propagation mechanism, the forward and backward dependency information of sequence data is fully captured, effectively integrating the global context and enhancing the temporal expressiveness of features. This structure is suitable for handling long-term dependency problems and exhibits higher classification accuracy and robustness in tasks such as physiological signal processing and temporal data modeling.

[0076] The classification module includes a fully connected layer and a Softmax classifier. After obtaining the bidirectional feature vector, the bidirectional feature vector is passed to the fully connected layer for mapping, and the classification result is obtained through the Softmax classifier.

[0077] Here, the classification results can include three levels: low workload, medium workload, and high workload, corresponding to category 1, category 2, and category 3, respectively, to reflect the physiological and cognitive states of individuals under different working conditions. Under low workload, the user experiences less psychological and physiological stress, has high concentration, and is suitable for long-term, high-efficiency, and complex task processing. Under medium workload, the user often experiences some fatigue and stress, requiring appropriate adjustments to their pace to maintain performance. Under high workload, the user's stress increases significantly, attention and work capacity decline, requiring timely rest or recovery measures to avoid safety risks.

[0078] The workload assessment method provided in this application improves the comprehensiveness and accuracy of workload assessment by collecting multimodal physiological data of workers, including electrocardiogram data, body temperature, and oxyhemoglobin concentration in the cerebral cortex. It obtains the workload level of workers through a workload classification model, enabling targeted analysis and enhancing the applicability and individualization of workload assessment. This allows for more scientific and effective guidance of workload allocation, better protection of workers' safety and health during task execution, especially in high-temperature environments, and improved work efficiency, thus ensuring the effectiveness of workers.

[0079] Corresponding to the embodiments of the aforementioned workload assessment method, this application also provides embodiments of a workload assessment device.

[0080] Please see Figure 11 This is a schematic diagram illustrating a workload assessment device according to an exemplary embodiment of this application. Figure 11 As shown in the figure, the workload assessment device 1100 provided in this application embodiment includes: The data acquisition module 1101 is used to collect the multimodal physiological data of the user during the execution of the task. The multimodal physiological data includes electrocardiogram data, body temperature and oxyhemoglobin concentration in the cerebral cortex. Feature extraction module 1102 is used to extract physiological signal features of each modality from the multimodal physiological data; The result determination module 1103 is used to input the physiological signal features under each modality into a pre-trained workload classification model to obtain the classification result output by the workload classification model. The classification result is used to indicate the workload level of the user.

[0081] In some possible implementations, the feature extraction module 1102 is specifically used for: According to the preset window length and preset sliding step size, the multimodal physiological data is dynamically sampled in each modality to obtain the sampled physiological data in each modality; For each modality, physiological signal features of that modality are extracted from the physiological data sampled under that modality.

[0082] In some possible implementations, the feature extraction module 1102, when extracting physiological signal features from the physiological data sampled for each modality, is specifically used for: For each modality, the physiological data sampled under that modality are processed by signal rearrangement to obtain the feature matrix under that modality; The feature matrix under this mode is normalized, and the normalized feature matrix is ​​color mapped to obtain a color image under this mode. The color image is used as the physiological signal feature under this mode.

[0083] In some possible implementations, the workload assessment device 1100 further includes a model training module 1104, which is used to train the workload classification model through the following steps: During the human thermal response experiment performed by the test user in the target environment, the test user's multimodal physiological data, environmental parameter information under various test environmental parameters, and response result data under various test tasks were collected. Based on the test multimodal physiological data, the environmental parameter information, and the response result data, the true value of the workload level is determined; The physiological signal features of each modality were extracted from the multimodal physiological data of the test. The test physiological signal features are input into a neural network to obtain the predicted classification result output by the neural network; Based on the error loss between the true value of the workload level and the predicted classification result, the parameters of the neural network are adjusted to obtain a trained workload model.

[0084] In some possible implementations, the model training module 1104, when determining the true value of the workload level based on the test multimodal physiological data, the environmental parameter information, and the response result data, is specifically used for: Cognitive load was assessed based on the test multimodal physiological data, the environmental parameter information, and the response result data to obtain a scoring matrix; The scoring weights corresponding to the scoring matrix are determined based on a genetic algorithm. The elements in the scoring matrix are weighted and summed according to the scoring weights to obtain the weighted scoring matrix. The weighted scoring matrix is ​​subjected to K-means clustering, and the true value of the workload level is determined based on the clustering results.

[0085] In some possible implementations, the workload classification model includes at least a feature enhancement module, a feature processing module, a multimodal competitive query module, a bidirectional long short-term memory module, and a classification module; The result determination module 1103 is specifically used for: The feature enhancement module is used to locally enhance the physiological signal features at the oxyhemoglobin concentration to obtain enhanced physiological signal features at the oxyhemoglobin concentration. The feature processing module performs structural normalization on the physiological signal features under the electrocardiogram data, the physiological signal features under the human body temperature, and the physiological signal features under the enhanced oxyhemoglobin concentration to obtain the physiological features under each modality after structural normalization. The multimodal competitive query module filters out the target physiological features that contribute the most to the classification of workload from the physiological features of each modality, and then fuses the target physiological features to obtain a fused feature matrix. The bidirectional long short-term memory module is used to perform bidirectional time analysis on the fused feature matrix to obtain bidirectional feature vectors. The classification module determines the classification result based on the bidirectional feature vector.

[0086] In some possible implementations, when the result determination module 1103 is used to perform local enhancement processing on the physiological signal features at the oxyhemoglobin concentration through the feature enhancement module to obtain enhanced physiological signal features at the oxyhemoglobin concentration, it is specifically used for: The feature enhancement module performs threshold segmentation on the physiological signal features at the oxyhemoglobin concentration to extract the target region from the physiological signal features at the oxyhemoglobin concentration. The physiological signal characteristics at the oxyhemoglobin concentration in the target region are filtered according to the target region to generate target region features; Based on the physiological signal characteristics at the oxyhemoglobin concentration and the characteristics of the target region, the enhanced physiological signal characteristics at the oxyhemoglobin concentration are obtained.

[0087] In some possible implementations, when the result determination module 1103 is used to filter out the target physiological features with the highest contribution to the workload classification from the physiological features under each modality through the multimodal competitive query module, and to fuse the target physiological features to obtain a fused feature matrix, it is specifically used for: The multimodal competitive query module maps physiological features from different modalities to the same query space. Physiological features under different modalities within the same query space are scored to obtain the importance score corresponding to each physiological feature under each modality. The importance score is used to indicate the contribution of the physiological feature under that modality to the workload classification. Select the target physiological features with the highest importance score from the physiological features of each modality; The target physiological features are weighted and summed according to normalized weights to obtain the fusion feature matrix.

[0088] In some possible implementations, when the result determination module 1103 performs bidirectional time analysis on the fused feature matrix through the bidirectional long short-term memory module to obtain bidirectional feature vectors, it is specifically used for: The bidirectional long short-term memory module performs forward time analysis and reverse time analysis on the fused feature matrix to obtain forward eigenvectors and reverse eigenvectors; the forward time analysis is used to perform analysis according to the default flow direction of time. The positive feature vector and the negative feature vector are fused to obtain the bidirectional feature vector.

[0089] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0090] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this application according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0091] Based on the same technical concept, this application also provides a computer device 1200, referring to... Figure 12 The diagram shown is a schematic representation of the structure of a computer device according to an exemplary embodiment of this application, comprising: The processor 1210, memory 1220, and bus 1230 are included. The memory 1220 is used to store execution instructions and includes main memory 1221 and external memory 1222. The main memory 1221, also known as internal memory, is used to temporarily store the operation data in the processor 1210 and the data exchanged with external memory 1222 such as hard disk. The processor 1210 exchanges data with external memory 1222 through main memory 1221.

[0092] In this embodiment, the memory 1220 is specifically used to store application code that executes the solution of this application, and its execution is controlled by the processor 1210. That is, when the electronic device 1200 is running, the processor 1210 communicates with the memory 1220 through the bus 1230, or the processor 1210 communicates with the memory 1220 through other means, so that the processor 1210 executes the application code stored in the memory 1220, and then executes the steps of the workload assessment method described in any of the foregoing embodiments.

[0093] The memory 1220 may be, but is not limited to, random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), etc.

[0094] Processor 1210 may be an integrated circuit chip with signal processing capabilities. The aforementioned processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application Specific Integrated Circuit (ASIC), a Field Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this invention. The general-purpose processor can be a microprocessor or any conventional processor.

[0095] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 1200. In other embodiments of this application, the electronic device 1200 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0096] This disclosure also provides a computer-readable storage medium storing a computer program that, when executed by a processor, performs the steps of the workload assessment method described in the above-described method embodiments. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0097] This disclosure also provides a computer program product, which stores a computer program. When the computer program is run by a processor, it executes the steps of the workload assessment method provided in any of the above embodiments of this disclosure. For details, please refer to the above method embodiments, which will not be repeated here.

[0098] The aforementioned computer program product can be implemented through hardware, software, or a combination thereof. In one optional embodiment, the computer program product is specifically embodied in a computer storage medium, which can be a volatile or non-volatile computer-readable storage medium. In another optional embodiment, the computer program product is specifically embodied in a software product, such as a software development kit (SDK), etc.

[0099] Furthermore, embodiments of the subject matter and functional operation described in this specification can be implemented in the following ways: digital electronic circuits, tangibly embodied computer software or firmware, computer hardware including the structures disclosed in this specification and their structural equivalents, or combinations thereof. Embodiments of the subject matter described in this specification can be implemented as one or more computer programs, i.e., one or more modules of computer program instructions encoded on a tangible, non-transitory program carrier for execution by a data processing apparatus or for controlling the operation of a data processing apparatus. Alternatively or additionally, program instructions may be encoded on artificially generated propagation signals, such as machine-generated electrical, optical, or electromagnetic signals, which are generated to encode information and transmit it to a suitable receiving device for execution by the data processing apparatus. The computer storage medium may be a machine-readable storage device, a machine-readable storage substrate, a random or serial access memory device, or combinations thereof.

[0100] The processing and logic flow described in this specification can be executed by one or more programmable computers that execute one or more computer programs to perform corresponding functions by operating on input data and generating output. The processing and logic flow can also be executed by dedicated logic circuitry—such as FPGAs (Field-Programmable Gate Arrays) or ASICs (Application-Specific Integrated Circuits), and the device can also be implemented as dedicated logic circuitry.

[0101] Suitable computers for executing computer programs include, for example, general-purpose and / or special-purpose microprocessors, or any other type of central processing unit. Typically, the central processing unit receives instructions and data from read-only memory and / or random access memory. The basic components of a computer include a central processing unit for implementing or executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will also include one or more mass storage devices for storing data, such as disks, magneto-optical disks, or optical disks, or the computer will be operatively coupled to such mass storage devices to receive data from or transfer data to them, or both. However, a computer is not required to have such devices. Furthermore, a computer can be embedded in another device, such as a mobile phone, a personal digital assistant (PDA), a mobile audio or video player, a game console, a global positioning system (GPS) receiver, or a portable storage device such as a universal serial bus (USB) flash drive, to name a few.

[0102] Computer-readable media suitable for storing computer program instructions and data include all forms of non-volatile memory, media, and memory devices, such as semiconductor memory devices (e.g., EPROM, EEPROM, and flash memory devices), magnetic disks (e.g., internal hard disks or removable disks), magneto-optical disks, and CD-ROM and DVD-ROM disks. Processors and memory may be supplemented by or incorporated into dedicated logic circuitry.

[0103] While this specification contains numerous specific implementation details, these should not be construed as limiting the scope of any invention or the scope of the claims, but rather are primarily intended to describe features of specific embodiments of a particular invention. Certain features described in the various embodiments herein may also be implemented in combination in a single embodiment. Conversely, various features described in a single embodiment may also be implemented separately in various embodiments or in any suitable sub-combination. Furthermore, while features may function in certain combinations as described above and even initially claimed in this way, one or more features from a claimed combination may be removed from that combination in some cases, and a claimed combination may refer to a sub-combination or a variation thereof.

[0104] Similarly, although the operations are depicted in a specific order in the accompanying drawings, this should not be construed as requiring these operations to be performed in the specific order shown or sequentially, or requiring all illustrated operations to be performed to achieve the desired result. In some cases, multitasking and parallel processing may be advantageous. Furthermore, the separation of various system modules and components in the above embodiments should not be construed as requiring such separation in all embodiments, and it should be understood that the described program components and systems can generally be integrated together in a single software product or packaged into multiple software products.

[0105] Thus, specific embodiments of the subject matter have been described. Other embodiments are within the scope of the appended claims. In some cases, the actions recited in the claims may be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings are not necessarily shown in a specific order or sequence to achieve the desired result. In some implementations, multitasking and parallel processing may be advantageous.

[0106] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A method for assessing workload, characterized in that, The method includes: During the process of the user performing the task, multimodal physiological data of the user is collected, including electrocardiogram data, body temperature and oxyhemoglobin concentration in the cerebral cortex; Physiological signal features under each modality were extracted from the multimodal physiological data; The physiological signal features under each modality are input into a pre-trained workload classification model to obtain the classification result output by the workload classification model. The classification result is used to indicate the workload level of the user.

2. The method according to claim 1, characterized in that, The extraction of physiological signal features from the multimodal physiological data for each modality includes: According to the preset window length and preset sliding step size, the multimodal physiological data is dynamically sampled in each modality to obtain the sampled physiological data in each modality; For each modality, physiological signal features of that modality are extracted from the physiological data sampled under that modality.

3. The method according to claim 2, characterized in that, For each modality, extracting physiological signal features from the sampled physiological data of that modality includes: For each modality, the physiological data sampled under that modality are processed by signal rearrangement to obtain the feature matrix under that modality; The feature matrix under this mode is normalized, and the normalized feature matrix is ​​color mapped to obtain a color image under this mode. The color image is used as the physiological signal feature under this mode.

4. The method according to claim 1, characterized in that, The workload classification model is trained through the following steps: During the human thermal response experiment performed by the test user in the target environment, the test user's multimodal physiological data, environmental parameter information under various test environmental parameters, and response result data under various test tasks were collected. Based on the test multimodal physiological data, the environmental parameter information, and the response result data, the true value of the workload level is determined; The physiological signal features of each modality were extracted from the multimodal physiological data of the test. The test physiological signal features are input into a neural network to obtain the predicted classification result output by the neural network; Based on the error loss between the true value of the workload level and the predicted classification result, the parameters of the neural network are adjusted to obtain a trained workload model.

5. The method according to claim 4, characterized in that, The process of determining the true value of the workload level based on the tested multimodal physiological data, the environmental parameter information, and the response result data includes: Cognitive load was assessed based on the test multimodal physiological data, the environmental parameter information, and the response result data to obtain a scoring matrix; The scoring weights corresponding to the scoring matrix are determined based on a genetic algorithm. The elements in the scoring matrix are weighted and summed according to the scoring weights to obtain the weighted scoring matrix. The weighted scoring matrix is ​​subjected to K-means clustering, and the true value of the workload level is determined based on the clustering results.

6. The method according to claim 1, characterized in that, The workload classification model includes at least a feature enhancement module, a feature processing module, a multimodal competitive query module, a bidirectional long short-term memory module, and a classification module; The step of inputting physiological signal features from various modalities into a pre-trained workload classification model to obtain the classification results output by the workload classification model includes: The feature enhancement module is used to locally enhance the physiological signal features at the oxyhemoglobin concentration to obtain enhanced physiological signal features at the oxyhemoglobin concentration. The feature processing module performs structural normalization on the physiological signal features under the electrocardiogram data, the physiological signal features under the human body temperature, and the physiological signal features under the enhanced oxyhemoglobin concentration to obtain the physiological features under each modality after structural normalization. The multimodal competitive query module filters out the target physiological features that contribute the most to the classification of workload from the physiological features of each modality, and then fuses the target physiological features to obtain a fused feature matrix. The bidirectional long short-term memory module is used to perform bidirectional time analysis on the fused feature matrix to obtain bidirectional feature vectors. The classification module determines the classification result based on the bidirectional feature vector.

7. The method according to claim 6, characterized in that, The feature enhancement module performs local enhancement processing on the physiological signal features at the oxyhemoglobin concentration to obtain enhanced physiological signal features at the oxyhemoglobin concentration, including: The feature enhancement module performs threshold segmentation on the physiological signal features at the oxyhemoglobin concentration to extract the target region from the physiological signal features at the oxyhemoglobin concentration. The physiological signal characteristics at the oxyhemoglobin concentration in the target region are filtered according to the target region to generate target region features; Based on the physiological signal characteristics at the oxyhemoglobin concentration and the characteristics of the target region, the enhanced physiological signal characteristics at the oxyhemoglobin concentration are obtained.

8. The method according to claim 6, characterized in that, The multimodal competitive query module filters out target physiological features from physiological features under various modalities that contribute the most to the workload classification of a preset number of modalities, and fuses the target physiological features to obtain a fused feature matrix, including: The multimodal competitive query module maps physiological features from different modalities to the same query space. Physiological features under different modalities within the same query space are scored to obtain the importance score corresponding to each physiological feature under each modality. The importance score is used to indicate the contribution of the physiological feature under that modality to the workload classification. Select the target physiological features with the highest importance score from the physiological features of each modality; The target physiological features are weighted and summed according to normalized weights to obtain the fusion feature matrix.

9. The method according to claim 6, characterized in that, The step of performing bidirectional time analysis on the fused feature matrix through the bidirectional long short-term memory module to obtain bidirectional feature vectors includes: The bidirectional long short-term memory module performs forward time analysis and reverse time analysis on the fused feature matrix to obtain forward eigenvectors and reverse eigenvectors; the forward time analysis is used to perform analysis according to the default flow direction of time. The positive feature vector and the negative feature vector are fused to obtain the bidirectional feature vector.

10. A workload assessment device, characterized in that, The device includes: The data acquisition module is used to collect the multimodal physiological data of the user during the execution of the task. The multimodal physiological data includes electrocardiogram data, body temperature and oxyhemoglobin concentration in the cerebral cortex. The feature extraction module is used to extract physiological signal features of each modality from the multimodal physiological data; The result determination module is used to input the physiological signal features under each modality into a pre-trained workload classification model to obtain the classification result output by the workload classification model. The classification result is used to indicate the workload level of the user.

11. A computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the steps of the workload assessment method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the program is executed by the processor, it implements the steps of the workload assessment method according to any one of claims 1 to 9.

Images