Human-computer interface adjustment method and system based on dynamic cognitive load management and risk early warning

Abstract

The invention relates to the technical field of cognitive engineering, and discloses a human-computer interface adjustment method and system based on dynamic cognitive load management and risk early warning. The human-computer interface comprises an information module. The method comprises the following steps: firstly, acquiring comprehensive cognitive load data of an operator in the process of operating a human-computer interface; then, acquiring an attention distribution state score corresponding to the information module and attention distraction risk data of the operator; and finally, under the condition that the comprehensive cognitive load data and / or the attention distraction risk data meet a preset condition, adjusting an information output mode of the information module based on an attention distribution state score corresponding to the information module and an information feature parameter corresponding to the information module. According to the method and the system, an operator can quickly obtain required information in a high-load or complex task, and interference of low-priority information is reduced, so that the possibility of misoperation is reduced, the overall task execution quality is improved, and the operator can complete the complex task more efficiently.

Description

Technical Field

[0001] The present invention relates to the field of cognitive engineering technology, and in particular to a human-machine interface adjustment method and system based on dynamic cognitive load management and risk early warning. Background Art

[0002] In the fields of transportation, aviation, aerospace, etc., personnel's cognitive load management and attention allocation are important factors in ensuring safety.

[0003] In the related art, simple early warning prompts are given according to the current importance of the task. However, these prompts are often passive and fail to evaluate and adapt to the cognitive load and attention state of the personnel in real time. Summary of the invention

[0004] The embodiments of this specification are intended to solve at least one of the technical problems in the related art to a certain extent. To this end, the embodiments of this specification propose a human-computer interface adjustment method and system based on dynamic cognitive load management and risk warning.

[0005] The present specification provides a method for adjusting a human-machine interface based on dynamic cognitive load management and risk warning, wherein the human-machine interface includes an information module, and the method includes:

[0006] Acquiring comprehensive cognitive load data of an operator during operation of the human-machine interface;

[0007] Acquire the attention allocation state score corresponding to the information module and the attention distraction risk data of the operator;

[0008] When the comprehensive cognitive load data and / or the attention distraction risk data meet preset conditions, the information output mode of the information module is adjusted based on the attention allocation state score corresponding to the information module and the information characteristic parameters corresponding to the information module, wherein the information characteristic parameters include at least one of the value parameter of the information module, the occurrence probability parameter of the information module, and the difficulty parameter of obtaining the information module.

[0009] In one embodiment, adjusting the information output mode of the information module based on the attention allocation state score corresponding to the information module and the information feature parameter corresponding to the information module includes:

[0010] Determining an adjustment parameter corresponding to the information module based on the attention allocation state score corresponding to the information module and the information feature parameter corresponding to the information module;

[0011] Based on the adjustment parameter, the information output mode of the information module is adjusted.

[0012] In one embodiment, adjusting the information output mode of the information module based on the adjustment parameter includes at least one of the following:

[0013] When the adjustment parameter satisfies a preset visual prompt condition, adjusting the information output mode of the information module to a saliency enhancement mode;

[0014] When the adjustment parameter satisfies the preset voice prompt condition, adjusting the information output mode of the information module to the voice prompt mode;

[0015] When the adjustment parameter satisfies a preset hiding condition, the information output mode of the information module is adjusted to an information hiding mode.

[0016] In one embodiment, the obtaining of comprehensive cognitive load data of an operator during operation of the human-machine interface includes:

[0017] Based on at least two of the operator's operation cognitive load data on the information module, the human cognitive load data extracted from the EEG signal, the physiological load data extracted from the ECG signal, the environmental load data and the task load data, multi-factor weighted fusion is performed to obtain the comprehensive cognitive load data.

[0018] In one embodiment, obtaining operation cognitive load data includes:

[0019] Based on the current task, determining information characteristic parameters corresponding to the information module from the preset task information relationship, wherein the information characteristic parameters include at least one of a value parameter of the information module, an occurrence probability parameter of the information module, a significance parameter of the information module, and a difficulty parameter for obtaining the information module;

[0020] The operation cognitive load data is determined based on the information feature parameters corresponding to the information module.

[0021] In one embodiment, obtaining the attention allocation state score corresponding to the information module includes:

[0022] Based on the gaze duration corresponding to the information module and the information characteristic parameters corresponding to the information module, the attention allocation state score corresponding to the information module is determined, wherein the information characteristic parameters include at least one of the value parameter of the information module, the occurrence probability parameter of the information module, the significance parameter of the information module, and the difficulty parameter of obtaining the information module.

[0023] In one embodiment, the information module includes a low-value module, and obtaining the operator's attention distraction risk data includes:

[0024] Based on the gaze duration corresponding to the low-value module, attention distraction risk data of the operator is determined.

[0025] The embodiment of this specification provides a human-machine interface adjustment system based on dynamic cognitive load management and risk warning, wherein the human-machine interface includes an information module, and the system includes:

[0026] A cognitive load determination module, used to obtain comprehensive cognitive load data of an operator during operation of the human-machine interface;

[0027] An attention allocation and risk determination module, used to obtain the attention allocation state score corresponding to the information module and the attention distraction risk data of the operator;

[0028] An interface adjustment module is used to adjust the information output mode of the information module based on the attention allocation state score corresponding to the information module and the information characteristic parameters corresponding to the information module when the comprehensive cognitive load data and / or the attention distraction risk data meet the preset conditions, wherein the information characteristic parameters include at least one of the value parameter of the information module, the occurrence probability parameter of the information module, and the difficulty parameter of obtaining the information module.

[0029] An embodiment of the present specification provides an electronic device, which includes: a memory, and one or more processors communicatively connected to the memory; the memory stores instructions executable by the one or more processors, and the instructions are executed by the one or more processors to enable the one or more processors to implement the steps of the method described in any of the above embodiments.

[0030] The embodiments of this specification provide a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method described in any of the above embodiments are implemented.

[0031] An embodiment of the present specification provides a computer program product, wherein the computer program product includes instructions, and when the instructions are executed by a processor of a computer device, the computer device is enabled to perform the steps of the method described in any one of the above embodiments.

[0032] In the above-mentioned specification implementation mode, the human-machine interface includes an information module. First, the comprehensive cognitive load data of the operator in the process of operating the human-machine interface is obtained, and the cognitive state of the operator can be monitored in real time, thereby improving the accuracy and comprehensiveness of the operator status monitoring. Then, the attention allocation state score corresponding to the information module and the attention distraction risk data of the operator are obtained, which provides a basis for adaptive adjustment. Finally, when the comprehensive cognitive load data and / or the attention distraction risk data meet the preset conditions, it is considered that the cognitive load of the operator reaches a high risk or the attention is seriously distracted, and intervention measures need to be triggered in time. By adjusting the information output mode of the information module based on the attention allocation state score corresponding to the information module and the information characteristic parameters corresponding to the information module, it is helpful for the operator to quickly obtain the required information in high-load or complex tasks, reduce the interference of low-priority information, thereby reducing the possibility of operating errors, improving the quality of overall task execution, and the operator can complete complex tasks more efficiently. Among them, the information characteristic parameters include at least one of the value parameters of the information module, the probability parameters of the occurrence of the information module, and the difficulty parameters of obtaining the information module. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 A flowchart of a human-computer interface adjustment method based on dynamic cognitive load management and risk warning provided in an embodiment of this specification;

[0034] Figure 2 A schematic diagram of a flow chart of adjusting the information output mode of the information module provided in an embodiment of this specification;

[0035] Figure 3 A schematic diagram of a process for obtaining operation cognitive load data provided in an embodiment of this specification;

[0036] Figure 4 A schematic diagram of a human-machine interface adjustment system based on dynamic cognitive load management and risk warning provided in an embodiment of this specification;

[0037] Figure 5 This is a diagram of the internal structure of an electronic device provided in accordance with an embodiment of this specification. DETAILED DESCRIPTION

[0038] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0039] In the fields of transportation, aviation, aerospace, etc., personnel's cognitive load management and attention allocation are directly related to the safety and efficiency of operations.

[0040] In the related technologies, the operation interface adopts a fixed layout, and the information is statically distributed on the dashboard or display screen. The operator needs to independently judge the importance and priority of each piece of information. This rigid interface design is more effective under low load or routine operation, but under high load or complex tasks (such as emergency disposal, equipment failure, or multi-tasking operation), the operator's cognitive load increases rapidly, which can easily lead to distraction, missing key task information, and even operational errors. And there is a lack of comprehensive cognitive load management tools in the related technologies. The operator's cognitive load may increase sharply during high-intensity tasks, but the interface design fails to respond in time, and the load cannot be reduced through interface optimization.

[0041] In related technologies, cockpit systems have begun to introduce physiological monitoring technologies, such as eye tracking, electroencephalogram (EEG), electrocardiogram (ECG), etc., to monitor the physiological and cognitive status of operators. Although these systems can collect data in real time and identify problems such as fatigue, stress, and distraction, most systems currently lack a strong feedback mechanism. They are limited to status monitoring and do not make intelligent adjustments to the interface or task information based on the collected data.

[0042] In the related art, early warning prompts (such as warnings when fuel is insufficient or an emergency failure) are triggered based on the importance of the task. Most of these prompts are passive responses, such as prompts through voice or visual signals.

[0043] Based on this, the embodiment of this specification provides a human-machine interface adjustment method based on dynamic cognitive load management and risk warning. The human-machine interface includes an information module. First, the comprehensive cognitive load data of the operator during the operation of the human-machine interface is obtained, and the cognitive state of the operator can be monitored in real time, thereby improving the accuracy and comprehensiveness of the operator status monitoring. Then, the attention allocation state score corresponding to the information module and the attention distraction risk data of the operator are obtained, which provides a basis for adaptive adjustment. Finally, when the comprehensive cognitive load data and / or the attention distraction risk data meet the preset conditions, it is considered that the cognitive load of the operator reaches a high risk or the attention is seriously distracted, and intervention measures need to be triggered in time. By adjusting the information output mode of the information module based on the attention allocation state score corresponding to the information module and the information feature parameters corresponding to the information module, it is helpful for the operator to quickly obtain the required information in high-load or complex tasks, reduce the interference of low-priority information, thereby reducing the possibility of operating errors, improving the quality of overall task execution, and the operator can complete complex tasks more efficiently. Among them, the information feature parameters include at least one of the value parameters of the information module, the probability parameters of the occurrence of the information module, and the difficulty parameters of obtaining the information module.

[0044] This specification provides a method for adjusting the human-computer interface based on dynamic cognitive load management and risk warning. Figure 1 The human-machine interface includes an information module. The human-machine interface adjustment method based on dynamic cognitive load management and risk warning may include the following steps:

[0045] S110. Obtaining comprehensive cognitive load data of the operator during the operation of the human-machine interface.

[0046] S120, obtaining the attention allocation state score corresponding to the information module and the operator's attention distraction risk data.

[0047] The human-machine interface includes multiple information modules, which can be data displays involving multiple dimensions, such as fuel quantity, radar signals, speed, temperature, warning information, etc. The display mode (such as numbers, charts, color changes, etc.) and information update frequency of each information module may affect the operator's cognitive load and attention allocation.

[0048] Specifically, the comprehensive cognitive load data can be the psychological pressure and information processing complexity that the operator bears during the operation of the human-machine interface. The comprehensive cognitive load data can be obtained by real-time monitoring in a variety of ways and quantified according to the equipment parameters obtained by monitoring. For example, it can be quantified by real-time monitoring of physiological data (such as brain waves, heart rate, eye movements, etc.) and behavioral data (such as operation response time, error rate, etc.). The attention allocation state score reflects the operator's attention to different information modules in a multi-task environment, such as the allocation of visual and cognitive resources when monitoring key indicators such as fuel volume and radar information. The attention allocation state score corresponding to the information module is determined based on the operator's eye movement data obtained. The operator's attention distraction risk data can be used to determine whether the operator is at risk of distraction at a certain moment, such as whether it is easy to ignore certain key data or react slowly under high-load task conditions. The operator's attention distraction risk data is determined based on the operator's eye movement data obtained.

[0049] S130. When the comprehensive cognitive load data and / or the attention distraction risk data meet preset conditions, the information output mode of the information module is adjusted based on the attention allocation state score corresponding to the information module and the information characteristic parameter corresponding to the information module.

[0050] The information characteristic parameters include at least one of a value parameter of the information module, an occurrence probability parameter of the information module, and a difficulty parameter of obtaining the information module.

[0051] Specifically, since the comprehensive cognitive load data is the amount of thinking resources required by the operator when processing the current task, the risk of distraction is the risk that the operator cannot concentrate due to environmental interference or information overload. Therefore, when the comprehensive cognitive load data and / or the risk of distraction data meet the preset conditions, it is necessary to help the operator refocus on important information to ensure operational safety. Based on the attention allocation state score corresponding to the information module and the information feature parameter corresponding to the information module, the information module that needs to be adjusted is determined, and the information output mode of the information module is adaptively adjusted to improve the information processing efficiency of the operator. The information feature parameter includes at least one of the value parameter of the information module, the probability parameter of the occurrence of the information module, and the difficulty parameter of obtaining the information module. Among them, the preset condition can be a pre-set threshold, which can be determined according to the task stage and operation requirements.

[0052] Exemplarily, when the comprehensive cognitive load data CL is greater than 7, or the attention distraction risk data DR is greater than 0.3, it can be considered that the comprehensive cognitive load data and / or the attention distraction risk data meet the preset conditions.

[0053] In some embodiments, a closed-loop control system is formed by dynamically adjusting the interface layout and intervention strategy based on the operator's state changes (such as comprehensive cognitive load data) through a real-time feedback mechanism. This design ensures the continued effectiveness of the system in the current task and can optimize information display based on the operator's real-time state.

[0054] The real-time feedback mechanism can continuously monitor and optimize the operator's comprehensive cognitive load data and / or distraction risk data, and adjust the interface in time to reduce the risk of overload or distraction. The closed-loop control system adjusts based on this real-time data to ensure that the system always operates effectively and safely.

[0055] In the above-mentioned implementation, the human-machine interface includes an information module. First, the comprehensive cognitive load data of the operator during the operation of the human-machine interface is obtained, so that the cognitive state of the operator can be monitored in real time, thereby improving the accuracy and comprehensiveness of the operator status monitoring. Then, the attention allocation state score corresponding to the information module and the attention distraction risk data of the operator are obtained, which provides a basis for adaptive adjustment. Finally, when the comprehensive cognitive load data and / or the attention distraction risk data meet the preset conditions, it is considered that the cognitive load of the operator reaches a high risk or the attention is seriously distracted, and intervention measures need to be triggered in time. By adjusting the information output mode of the information module based on the attention allocation state score corresponding to the information module and the information characteristic parameters corresponding to the information module, it is helpful for the operator to quickly obtain the required information in high-load or complex tasks, reduce the interference of low-priority information, thereby reducing the possibility of operating errors, improving the quality of overall task execution, and the operator can complete complex tasks more efficiently. Among them, the information characteristic parameters include at least one of the value parameters of the information module, the probability parameters of the occurrence of the information module, and the difficulty parameters of obtaining the information module.

[0056] In some embodiments, see Figure 2 , based on the attention allocation state score corresponding to the information module and the information feature parameter corresponding to the information module, adjusting the information output mode of the information module may include the following steps:

[0057] S210. Determine an adjustment parameter corresponding to the information module based on the attention allocation state score corresponding to the information module and the information feature parameter corresponding to the information module.

[0058] S220: Adjust the information output mode of the information module based on the adjustment parameters.

[0059] Specifically, in order to improve the efficiency of information processing and output, first, the current state of the information module is analyzed based on the attention allocation state score of the information module and the characteristic parameters related thereto. The attention allocation state score reflects the degree of attention of the information module in the current task or situation. The characteristic parameters corresponding to the information module are the attributes related to the information processing of the module, such as the information characteristic parameters including at least one of the value parameter of the information module, the probability parameter of the occurrence of the information module, and the difficulty parameter of obtaining the information module. Next, the adjustment parameters corresponding to each information module are determined by comprehensively analyzing the attention allocation state score corresponding to the information module and the information characteristic parameters corresponding to the information module. According to the adjustment parameters, the information module that needs to be adjusted and optimized and the information output mode corresponding to the information module are adjusted.

[0060] Exemplarily, the adjustment parameter corresponding to the information module is determined by the following formula:

[0061]

[0062] Among them, maxFi represents the adjustment parameter corresponding to the i-th information module, AR represents the attention allocation state score corresponding to the i-th information module, vi represents the value parameter of the i-th information module, pi represents the occurrence probability parameter of the i-th information module, and Ei represents the difficulty parameter of obtaining the i-th information module.

[0063] In the above implementation, based on the attention allocation state score corresponding to the information module and the information feature parameters corresponding to the information module, the adjustment parameters corresponding to the information module are determined, and based on the adjustment parameters, the information output mode of the information module is adjusted, and the adjustment is adaptively performed to improve flexibility and efficiency.

[0064] In some embodiments, adjusting the information output mode of the information module based on the adjustment parameters may include: when the adjustment parameters meet a preset visual prompt condition, adjusting the information output mode of the information module to a saliency enhancement mode.

[0065] Specifically, when operators face information overload or extremely high cognitive load, non-critical information will interfere with task execution and reduce operational efficiency. In order to avoid this situation, it is necessary to dynamically adjust the information modules in the interface to ensure that operators can focus on the most important task information, thereby optimizing the operating experience. Each information module has an adjustment parameter that indicates its priority and importance. The higher the value of the parameter, the higher the importance and urgency of the information module. By analyzing the adjustment parameters of each information module, when the preset visual prompt conditions are met, it is necessary to adjust the information output mode of the information module corresponding to the adjustment parameters that meet the preset visual prompt conditions to a saliency enhancement mode to highlight high-priority task information, attract the attention of operators, help them quickly identify key information, avoid missing key data due to information overload, and reduce the cognitive burden of information acquisition. Among them, the preset visual prompt condition can be that the adjustment parameter of the information module is the maximum value of the adjustment parameters of all information modules.

[0066] In some embodiments, for information modules that meet preset visual prompt conditions, they are placed within the operator's optimal field of view and magnified, thereby reducing the cognitive burden of information acquisition and improving information processing efficiency in high-load tasks.

[0067] In some embodiments, for information modules that meet the preset visual prompt conditions, an adaptive interface adjustment algorithm is used to dynamically optimize the display mode of the information modules. For example, by highlighting the edge of the information module, it is more prominent among many information modules; by changing the color of the information module (such as changing the high-priority information module from the normal color to red or orange) to attract attention; for urgent information modules (such as flight altitude or fuel warnings), a flashing effect is used to ensure the operator's attention.

[0068] It should be noted that the highlight of the information module will be dynamically adjusted based on the operator's line of sight tracking data to ensure that prompts are still effective when the operator does not notice it in time, and will gradually fade out when the operator's line of sight returns to the information module to avoid interference caused by continuous prompts.

[0069] In the above implementation, when the adjustment parameters meet the preset visual prompt conditions, the information output mode of the information module is adjusted to the saliency enhancement mode, the interface layout is optimized in real time according to the attention distribution of the personnel, and the attention is reasonably guided to ensure that the key and high-value information modules appear first in the field of vision of the operator, thereby reducing the cognitive burden of information acquisition and improving the information processing efficiency of the operator under high-load tasks.

[0070] In some implementations, adjusting the information output mode of the information module based on the adjustment parameters may include: adjusting the information output mode of the information module to a voice prompt mode when the adjustment parameters meet a preset voice prompt condition.

[0071] Specifically, when the operator faces information overload or extremely high cognitive load, voice prompts are used to strengthen the operator's task awareness, help them concentrate, and avoid a decrease in processing power due to excessive visual information. Each information module has an adjustment parameter that indicates its priority and importance. The higher the value of the parameter, the higher the importance and urgency of the information module. By analyzing the adjustment parameters of each information module, when the preset voice prompt conditions are met, it is necessary to adjust the information output mode of the information module corresponding to the adjustment parameters that meet the preset voice prompt conditions to the voice prompt mode to enhance the operator's attention and convey the priority and urgency of the task. Among them, the preset voice prompt condition can be that the adjustment parameter of the information module is the maximum value of the adjustment parameters of all information modules.

[0072] In some embodiments, the frequency and volume of voice prompts are optimized to ensure that operators can pay attention to important task information in a timely manner. When the operator's concentration decreases (such as ignoring high-priority task prompts for more than a set time threshold, such as not paying attention to the fuel level or flight altitude for 5 seconds), the intensity and frequency of the voice prompts will be automatically increased to ensure that it is sufficient to attract the operator's attention. After the operator makes an effective response, the voice prompt will be stopped and the response will be recorded to further optimize the intervention strategy in future tasks. Exemplarily, when it is detected that high-priority information related to the current task is ignored, a voice prompt will be triggered, and the prompt content is directly related to the current task, such as: "Please check the fuel level", "Flight altitude decreases, pay attention to the flight track", "The radar ahead shows an obstacle, please pay attention", etc.

[0073] In other embodiments, voice prompts may be used in combination with visual prompts to complement each other, thereby ensuring that operators can be effectively reminded in different situations.

[0074] In the above implementation, when the adjustment parameters meet the preset voice prompt conditions, the information output mode of the information module is adjusted to the voice prompt mode to reasonably guide attention, help operators adjust their attention in time, reduce operating errors, and improve operation accuracy.

[0075] In some implementations, adjusting the information output mode of the information module based on the adjustment parameter may include: adjusting the information output mode of the information module to the information hiding mode when the adjustment parameter satisfies a preset hiding condition.

[0076] Specifically, when operators face information overload or extremely high cognitive load, non-critical information will interfere with task execution and reduce operational efficiency. In order to avoid this situation, it is necessary to dynamically adjust the information modules in the interface to ensure that operators can focus on the most important task information, thereby optimizing the operational experience. Each information module has an adjustment parameter that indicates its priority and importance. The lower the value of the parameter, the lower the importance and urgency of the information module. By analyzing the adjustment parameters of each information module, when the preset hiding conditions are met, it is necessary to adjust the information output mode of the information module corresponding to the adjustment parameters that meet the preset hiding conditions to the information hiding mode, so as to reduce information overload and optimize the user's operational experience, help operators concentrate, ensure that operators can focus on critical task information, and avoid low-priority information interfering with operational efficiency when operators face excessive cognitive load or information overload. Among them, the preset hiding condition can be that the adjustment parameter of the information module is less than the preset threshold, and the preset threshold can be determined according to the task stage and operational requirements.

[0077] In some embodiments, information modules that meet the preset hiding conditions may be completely removed from the interface, and only when the comprehensive cognitive load data and / or the attention distraction risk data drop to an acceptable range will these information modules be redisplayed.

[0078] In other embodiments, information modules that meet the preset hiding conditions can be presented in a more concise form, such as converted into small icons or using a minimalist display mode. In this way, the information module still exists, but does not occupy the main display area of ​​the interface, avoiding distraction of the operator's attention.

[0079] In some other embodiments, information modules that meet the preset hiding conditions can be placed in a secondary area of ​​the interface, such as the bottom or sidebar, for operators to view in non-emergency situations. Operators can manually restore access to these information modules when needed.

[0080] In the above implementation, when the preset hiding conditions are met, the information output mode of the information module is adjusted to the information hiding mode, which helps the operator to adjust his attention in time, reduce operating errors, and improve the accuracy of operation.

[0081] In some embodiments, obtaining comprehensive cognitive load data of an operator during operation of a human-machine interface may include: performing multi-factor weighted fusion based on at least two of the operator's operation cognitive load data on an information module, human cognitive load data extracted from EEG signals, physiological load data extracted from ECG signals, environmental load data, and task load data to obtain comprehensive cognitive load data.

[0082] Specifically, in order to evaluate the comprehensive cognitive load of operators when performing tasks, data can be collected from multiple dimensions and analyzed comprehensively. The operational cognitive load data comes from the operator's task operation cognition of the information module, which can be evaluated through cognitive load scales and other methods, reflecting the workload perceived during the operation.

[0083] Within the time window, real-time analysis of EEG signals can identify the operator's cognitive state, fatigue level, and alertness during task execution. Focus on analyzing the changes in alpha waves, beta waves, and theta waves to understand the operator's psychological state. Specifically, alpha waves are associated with a state of relaxation, beta waves reflect alertness and task-related cognitive activities, and an increase in theta waves usually means fatigue or mild relaxation. By combining spectral analysis, waveform analysis and other algorithms, spectral analysis and waveform analysis of these brain waves can quantify the changes in these brain waves, provide objective cognitive load evaluation indicators, and obtain human cognitive load data. It should be noted that multiple sets of human cognitive load data can be included in the time window.

[0084] In the time window, heart rate variability (HRV) is an important indicator for measuring physiological load and can reflect the balance state of the operator's autonomic nervous system. Changes in heart rate variability reflect the operator's stress level and stress perception during the task. Real-time analysis of ECG signals is performed to extract heart rate variability (HRV) and quantify it to obtain physiological load data. Among them, lower HRV values ​​usually indicate higher physiological load and stress response, while higher HRV values ​​mean better physiological adaptability and lower stress levels. It should be noted that multiple sets of physiological load data can be included in the time window.

[0085] Task load data is quantified by evaluating the complexity of the current task, the accuracy requirements of the operation, and the dynamic adjustment of the current task. Based on the complexity and urgency, the task load can be divided into multiple levels (such as low load, medium load, and high load). When the task load is high, the operator may feel greater psychological pressure, resulting in an increase in cognitive load and physiological load. Therefore, by evaluating the nature and requirements of the task, the impact of the task on the operator's cognitive load can be further quantified to obtain the task load data.

[0086] Environmental parameters are also important factors that affect cognitive load. For example, bad weather or complex instrument conditions may increase the complexity of the task, thereby increasing cognitive load. Environmental load data is obtained by monitoring environmental parameters and evaluating the impact of environmental changes on tasks and quantifying them.

[0087] In order to comprehensively consider the impact of various factors on the operator's cognitive load, a comprehensive cognitive load calculation model can be constructed by combining at least two of the operational cognitive load data, human cognitive load data, physiological load data, environmental load data, and task load data through multi-source data fusion technology. Different weights are assigned to the load data corresponding to different data sources according to the importance of different data sources and the specific requirements of the task. Multi-factor weighted fusion is performed based on at least two of the operational cognitive load data, human cognitive load data, physiological load data, environmental load data, and task load data and the corresponding weights to obtain comprehensive cognitive load data.

[0088] In some implementations, the comprehensive cognitive load data is determined by the following formula:

[0089] CL=K+β·EEG c +γ·ECG c +δ·Environmental load data+ζ·Task load data

[0090] Among them, CL represents comprehensive cognitive load data, K represents operational cognitive load data, and EEG c Indicates human cognitive load data, ECG crepresents physiological load data. β, γ, δ, ζ represent the weights of their corresponding load data.

[0091] In some embodiments, when the comprehensive cognitive load data CL is less than or equal to 3, it is considered that the operator is in a low-load state during task execution, and the difficulty and complexity of the current task are relatively small. The operator can easily handle the task, has abundant cognitive resources, and has less work pressure. When the comprehensive cognitive load data CL is greater than 3 and less than or equal to 6, it is considered that the operator is in a medium-load state during task execution, the difficulty and cognitive requirements of the current task are moderate, and the operator needs a certain amount of attention and cognitive resources to handle the task, but is usually able to cope with it. When the comprehensive cognitive load data CL is greater than 6, it is considered that the operator is in a high-load state during task execution, and the complexity, time pressure or multi-task requirements of the current task exceed the operator's ability to bear. In this state, it may be difficult for the operator to handle the task efficiently, and the decision-making quality and attention allocation may be significantly affected, requiring interface intervention and early warning.

[0092] In the above implementation, based on the operator's operation cognitive load data on the information module, the human cognitive load data extracted from the EEG signal, the physiological load data extracted from the ECG signal, the environmental load data and at least two of the task load data, multi-factor weighted fusion is performed to obtain comprehensive cognitive load data, which can track the operator's cognitive state in real time, dynamically respond to changes in the comprehensive cognitive load data, and identify high-load states. When the operator enters a high-load state, an accurate warning is issued, thereby improving the accuracy and comprehensiveness of state monitoring. This provides a data basis for subsequently reducing cognitive load and thus improving the efficiency of current task execution.

[0093] In some embodiments, see Figure 3 , obtaining the operation cognitive load data may include the following steps:

[0094] S310: Based on the current task, determine information feature parameters corresponding to the information module from the preset task information relationship.

[0095] S320: Determine the operation cognitive load data based on the information feature parameters corresponding to the information module.

[0096] The information characteristic parameters include at least one of a value parameter of the information module, an occurrence probability parameter of the information module, a significance parameter of the information module, and a difficulty parameter of obtaining the information module.

[0097] Specifically, based on the requirements of the current task, the information characteristic parameters corresponding to the information module are retrieved and matched from the preset task information relationship. The information characteristic parameters include at least one of the value parameter of the information module, the probability parameter of the occurrence of the information module, the significance parameter of the information module, and the difficulty parameter of obtaining the information module. Based on the preset cognitive load model, the information characteristic parameters corresponding to the information module are integrated to calculate the operation cognitive load data. The operation cognitive load data reflects the cognitive pressure or complexity that may be generated in the operation process when completing the task.

[0098] Exemplarily, the operational cognitive load data is determined by the following formula:

[0099]

[0100] Among them, K represents the operation cognitive load data, α represents the weight, vi represents the value parameter of the i-th information module, pi represents the occurrence probability parameter of the i-th information module, Si represents the significance parameter of the i-th information module, Ei represents the difficulty parameter of obtaining the i-th information module, and n represents the total number of information modules.

[0101] Exemplarily, the value parameters vi of the information modules: the value parameters of key information modules such as altimeter, fuel gauge, and flight speed are set to high, the value parameters of the navigation information module are set to medium, and the value parameters of the temperature and humidity information modules are set to low.

[0102] The appearance probability parameter pi of the information module: According to the flight mission scenario, the appearance probability parameters of the speed and altimeter are relatively high, the appearance probability parameter of the fuel gauge is medium, and the appearance probability parameters of information modules such as temperature are relatively low.

[0103] The significance parameters Si of the information module: the significance parameters of speed and altimeter are high, and the significance parameters of fuel gauge and navigation are medium.

[0104] The difficulty parameter Ei of the information module: According to eye tracking data, pilots stare at the speedometer and altimeter for a long time, while the fuel gauge and navigation information are stared at for a shorter time.

[0105] In the above implementation, based on the current task, the information characteristic parameters corresponding to the information module are determined from the preset task information relationship, and based on the information characteristic parameters corresponding to the information module, the operational cognitive load data is determined to provide a data basis for determining the comprehensive cognitive load data.

[0106] In some embodiments, obtaining the attention allocation state score corresponding to the information module may include: determining the attention allocation state score corresponding to the information module based on the gaze duration corresponding to the information module and the information feature parameters corresponding to the information module.

[0107] The information characteristic parameters include at least one of a value parameter of the information module, an occurrence probability parameter of the information module, a significance parameter of the information module, and a difficulty parameter of obtaining the information module.

[0108] Specifically, based on the needs of the current task, the information module related to the task is retrieved and matched from the module relationship comparison table. The current task is retrieved and matched with the preset task information relationship to determine the information feature parameters corresponding to the information module. The information feature parameters include at least one of the value parameter of the information module, the probability parameter of the occurrence of the information module, the significance parameter of the information module, and the difficulty parameter of obtaining the information module. Through eye tracking technology or other gaze tracking means, the operator's gaze time on the information module is monitored in real time. The gaze time reflects the operator's attention to the information module and is an important indicator for measuring attention allocation. Based on the attention allocation model, the attention allocation state score corresponding to the information module can be calculated through the gaze time corresponding to the information module and the information feature parameter corresponding to the information module. The attention allocation state score reflects the operator's attention to the information module during the task execution process. The higher the attention allocation state score, the more likely the operator is to give priority to the information module.

[0109] Exemplarily, the attention allocation state score corresponding to the information module is determined by the following formula:

[0110]

[0111] Among them, AR represents the attention allocation state score corresponding to the i-th information module, and the fixation duration i represents the operator's gaze duration on the i-th information module, vi represents the value parameter of the i-th information module, pi represents the occurrence probability parameter of the i-th information module, Si represents the significance parameter of the i-th information module, and Ei represents the difficulty parameter of obtaining the i-th information module.

[0112] In the above implementation, the attention allocation state score corresponding to the information module is determined based on the gaze duration corresponding to the information module and the information feature parameters corresponding to the information module, providing a basis for the subsequent adaptive adjustment of the information output mode of the information module.

[0113] In some embodiments, the information module includes a low-value module, and obtaining the operator's attention distraction risk data may include: determining the operator's attention distraction risk data based on the gaze duration corresponding to the low-value module.

[0114] Specifically, based on the requirements of the current task, the information modules related to the task and which of the included information modules are low-value modules are retrieved and matched from the module relationship comparison table. Low-value modules can be parts that have little impact on the success of the task. Through eye tracking technology or other gaze tracking means, the operator's gaze time on low-value modules is monitored in real time. If the operator gazes at these low-value modules for too long during the task, it may mean that their attention is distracted and they fail to focus on more critical and high-value modules. This behavior may increase the risk of errors in the operation and affect the overall efficiency and safety of the task. According to the gaze time corresponding to the low-value module, a distraction risk data can be calculated, which reflects the operator's concentration during the work process. If the distraction risk data exceeds a certain threshold, it means that the operator has a high risk of distraction, so appropriate intervention measures need to be taken, such as reminders, adjustment of task processes, etc., to reduce the incidence of errors and improve work efficiency and safety. The distraction risk data will be used as core data for the adjustment of the information output mode of subsequent information modules.

[0115] Exemplarily, the distraction risk data is determined by the following formula:

[0116]

[0117] Among them, DR represents the attention distraction risk data, n represents the total number of low-value modules, and the fixation duration i represents the operator's gaze duration on the i-th low-value module.

[0118] In some embodiments, when the attention distraction risk data DR is less than or equal to 0.2, it is considered that the attention distraction of the operator during task execution is a low risk, indicating that the operator is focused and can effectively maintain focus on the core task module, and usually no intervention is required. When the attention distraction risk data DR is greater than 0.2 and less than or equal to 0.4, it is considered that the attention distraction of the operator during task execution is a medium risk, indicating that the operator has a slight attention distraction, and it is necessary to monitor the attention allocation of the personnel so as to take appropriate measures in time. When the attention distraction risk data DR is greater than 0.4, it is considered that the attention distraction of the operator during task execution is a high risk, indicating that the operator's attention is highly distracted, which has a serious impact on the completion quality and efficiency of the task, and it is necessary to activate the early warning mechanism and intervene immediately.

[0119] In the above implementation, based on the gaze duration corresponding to the low-value module, the operator's attention distraction risk data is determined, providing a basis for the subsequent adaptive adjustment of the information output mode of the information module.

[0120] The implementation method of this specification also provides a use scenario of a human-machine interface adjustment method based on dynamic cognitive load management and risk warning. In a complex flight mission, the pilot needs to monitor multiple flight parameters at the same time, including altitude, fuel level, flight speed and external weather conditions. During the mission, the pilot also needs to adjust the flight route according to the instructions of the control tower, and the task load is relatively high. In order to ensure the pilot's operational safety and mission execution efficiency, the system comprehensively monitors and analyzes the pilot's physiological signals, mission status and environmental parameters at the beginning of the flight mission.

[0121] Based on the value parameter of the information module, the occurrence probability parameter of the information module, the significance parameter of the information module, and the difficulty parameter of obtaining the information module, the operator's operation cognitive load data K on the information module is obtained.

[0122] The pilot's alpha and beta wave frequencies are monitored and extracted by an electroencephalogram to obtain the human cognitive load data EEG. c Among them, during the cruise phase, the alpha wave gradually increased, indicating that the pilot was in a state of mild relaxation, but as the task load increased, the beta wave gradually increased, indicating that the pilot's cognitive load increased.

[0123] ECG signals are collected through wearable heart rate monitoring equipment, and the pilot's heart rate variability (HRV) is calculated in real time to assess stress levels and obtain physiological load data ECG. c Among them, in the middle of the flight mission, the heart rate increased and the HRV decreased, indicating that the pilots' psychological stress was increasing.

[0124] Currently, pilots need to perform flight monitoring, navigation adjustment and weather monitoring at the same time, which is a task with high complexity. The system sets the task load level to "high" and quantifies it to obtain task load data. The external weather conditions are complex, including strong winds and thunderstorm areas, which bring additional operational loads to the flight. Therefore, the external weather conditions are quantified to obtain environmental load data.

[0125] The system uses the cognitive load model CL=K+β·EEG based on the acquired operational cognitive load data, human cognitive load data, physiological load data, environmental load data and task load data. c +γ·ECG c +δ·environmental load data+ζ·task load data are calculated to obtain the comprehensive cognitive load data CL=7.5. 7.5 exceeds the high load threshold of 6, indicating that the pilot is currently in a high load state.

[0126] The infrared eye tracker monitors the pilot's gaze point and gaze time in real time. Based on the eye tracking data, the gaze duration corresponding to the information module is determined. Based on the gaze duration corresponding to the information module, the value parameter of the information module, the probability parameter of the information module, the significance parameter of the information module, and the difficulty parameter of obtaining the information module, the system determines the attention allocation state score corresponding to the information module, indicating that the pilot spends too much time focusing on the navigation display and temperature display, while the value of these modules is relatively low. At the same time, the gaze time of the fuel gauge and radar display is short.

[0127] Based on the fixation duration corresponding to the low-value module, the formula After calculation, the operator's attention distraction risk data DR was obtained as 0.42. 0.42 is higher than the attention distraction risk threshold of 0.3 set by the system, indicating that the pilot stayed on low-value information for too long, and it is believed that there is a high risk of pilot distraction.

[0128] Therefore, when the pilot is currently in a high-load state and there is a high risk of distraction, according to the formula Calculation is performed to determine the adjustment parameters corresponding to the information module. The higher the value of the adjustment parameter, the higher the importance and urgency of the information module. The lower the value of the adjustment parameter, the lower the importance and urgency of the information module. It is thus determined that high-value modules include fuel gauges and radar displays, and low-value modules include temperature and humidity.

[0129] The adaptive interface adjustment mechanism is then activated, automatically enlarging high-value modules (such as the fuel gauge and radar display) and moving them to the pilot's optimal field of view, highlighting and slightly flashing them to attract attention, while issuing a voice reminder: "Please check the fuel gauge and radar information and adjust the flight altitude." Low-value modules (such as temperature and humidity) are reduced or hidden to reduce visual interference.

[0130] After the system adaptively adjusted the interface and issued voice prompts, the pilot's operating behavior changed significantly. According to the monitored eye tracking data, the pilot refocused his attention on the fuel gauge and radar display, and the gaze time increased significantly. The comprehensive cognitive load data CL dropped from 7.5 to 5.3, entering a medium load state. The attention distraction risk data DR dropped from 0.42 to 0.2, indicating that the pilot's concentration has improved significantly. Through the system's adaptive interface optimization and timely intervention, the pilot can quickly adjust his operational attention and reduce potential risks in the flight mission. The system will continue to monitor and repeat the optimization intervention steps when the comprehensive cognitive load data and attention distraction risk data are abnormal to ensure that the pilot maintains the best operating state.

[0131] The present specification provides a human-machine interface adjustment system 400 based on dynamic cognitive load management and risk warning. The human-machine interface includes an information module. Figure 4 The human-machine interface adjustment system 400 based on dynamic cognitive load management and risk warning includes: a cognitive load determination module 410 , an attention allocation and risk determination module 420 , and an interface adjustment module 430 .

[0132] A cognitive load determination module 410 is used to obtain comprehensive cognitive load data of an operator during operation of the human-machine interface;

[0133] An attention allocation and risk determination module 420, used to obtain the attention allocation state score corresponding to the information module and the attention distraction risk data of the operator;

[0134] The interface adjustment module 430 is used to adjust the information output mode of the information module based on the attention allocation state score corresponding to the information module and the information characteristic parameters corresponding to the information module when the comprehensive cognitive load data and / or the attention distraction risk data meet the preset conditions, wherein the information characteristic parameters include at least one of the value parameter of the information module, the occurrence probability parameter of the information module, and the difficulty parameter of obtaining the information module.

[0135] For a detailed description of the human-computer interface adjustment system based on dynamic cognitive load management and risk warning, please refer to the above description of the human-computer interface adjustment method based on dynamic cognitive load management and risk warning, which will not be repeated here.

[0136] In some embodiments, an electronic device is provided, which may be a terminal, and its internal structure diagram may be as follows: Figure 5As shown. The electronic device includes a processor, a memory, a communication interface, a display screen and an input device connected through a system bus. Among them, the processor of the electronic device is used to provide computing and control capabilities. The memory of the electronic device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The communication interface of the electronic device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be achieved through WIFI, an operator network, NFC (near field communication) or other technologies. When the computer program is executed by the processor, a human-computer interface adjustment method based on dynamic cognitive load management and risk warning is implemented. The display screen of the electronic device can be a liquid crystal display screen or an electronic ink display screen, and the input device of the electronic device can be a touch layer covered on the display screen, or a key, trackball or touchpad set on the housing of the electronic device, or an external keyboard, touchpad or mouse.

[0137] Those skilled in the art will understand that Figure 5 The structure shown in the figure is merely a block diagram of a portion of the structure related to the scheme disclosed in this specification, and does not constitute a limitation on the electronic device to which the scheme disclosed in this specification is applied. Specifically, the electronic device may include more or fewer components than shown in the figure, or combine certain components, or have a different arrangement of components.

[0138] An embodiment of the present specification provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the steps of the method in any of the above embodiments are implemented.

[0139] One embodiment of the present specification provides a computer program product, which includes instructions. When the instructions are executed by a processor of a computer device, the computer device can perform the steps of the method of any of the above embodiments.

[0140] It should be noted that the logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing logical functions, and can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection portion with one or more wirings (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing in a suitable manner if necessary, and then stored in a computer memory.

Claims

1. A method for adjusting a human-machine interface based on dynamic cognitive load management and risk warning, wherein the human-machine interface includes an information module, characterized in that: The method comprises: Acquiring comprehensive cognitive load data of an operator during operation of the human-machine interface; Acquire the attention allocation state score corresponding to the information module and the attention distraction risk data of the operator; When the comprehensive cognitive load data and / or the attention distraction risk data meet preset conditions, the information output mode of the information module is adjusted based on the attention allocation state score corresponding to the information module and the information characteristic parameters corresponding to the information module, wherein the information characteristic parameters include at least one of the value parameter of the information module, the occurrence probability parameter of the information module, and the difficulty parameter of obtaining the information module.

2. The method according to claim 1, characterized in that The adjusting the information output mode of the information module based on the attention allocation state score corresponding to the information module and the information feature parameter corresponding to the information module comprises: Determining an adjustment parameter corresponding to the information module based on the attention allocation state score corresponding to the information module and the information feature parameter corresponding to the information module; Based on the adjustment parameter, the information output mode of the information module is adjusted.

3. The method according to claim 2, characterized in that The adjusting the information output mode of the information module based on the adjustment parameter includes at least one of the following: When the adjustment parameter satisfies a preset visual prompt condition, adjusting the information output mode of the information module to a saliency enhancement mode; When the adjustment parameter satisfies the preset voice prompt condition, adjusting the information output mode of the information module to the voice prompt mode; When the adjustment parameter satisfies a preset hiding condition, the information output mode of the information module is adjusted to an information hiding mode.

4. The method according to claim 1, characterized in that: The obtaining of the comprehensive cognitive load data of the operator during the operation of the human-machine interface includes: Based on at least two of the operator's operation cognitive load data on the information module, the human cognitive load data extracted from the EEG signal, the physiological load data extracted from the ECG signal, the environmental load data and the task load data, multi-factor weighted fusion is performed to obtain the comprehensive cognitive load data.

5. The method according to claim 4, characterized in that Get operational cognitive load data, including: Based on the current task, determining information characteristic parameters corresponding to the information module from the preset task information relationship, wherein the information characteristic parameters include at least one of a value parameter of the information module, an occurrence probability parameter of the information module, a significance parameter of the information module, and a difficulty parameter for obtaining the information module; The operation cognitive load data is determined based on the information feature parameters corresponding to the information module.

6. The method according to claim 1, characterized in that The obtaining of the attention allocation state score corresponding to the information module includes: Based on the gaze duration corresponding to the information module and the information characteristic parameters corresponding to the information module, the attention allocation state score corresponding to the information module is determined, wherein the information characteristic parameters include at least one of the value parameter of the information module, the occurrence probability parameter of the information module, the significance parameter of the information module, and the difficulty parameter of obtaining the information module.

7. The method according to claim 1, characterized in that The information module includes a low-value module, which obtains the operator's attention distraction risk data, including: Based on the gaze duration corresponding to the low-value module, attention distraction risk data of the operator is determined.

8. A human-machine interface adjustment system based on dynamic cognitive load management and risk warning, the human-machine interface includes an information module, characterized in that: The system comprises: A cognitive load determination module, used to obtain comprehensive cognitive load data of an operator during operation of the human-machine interface; An attention allocation and risk determination module, used to obtain the attention allocation state score corresponding to the information module and the attention distraction risk data of the operator; An interface adjustment module is used to adjust the information output mode of the information module based on the attention allocation state score corresponding to the information module and the information characteristic parameters corresponding to the information module when the comprehensive cognitive load data and / or the attention distraction risk data meet the preset conditions, wherein the information characteristic parameters include at least one of the value parameter of the information module, the occurrence probability parameter of the information module, and the difficulty parameter of obtaining the information module.

9. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images