Attention assessment method, device, equipment and medium based on multimodal data

Abstract

The present invention relates to the field of attention assessment technology, and in particular to an attention assessment method, device, equipment and medium based on multimodal data. The attention assessment method based on multimodal data includes the following steps: S1: obtaining multimodal data of a user to be assessed, inputting the multimodal data into an action attention assessment formula, and obtaining an action assessment value of the user to be assessed; S2: performing a comprehensive attention test on the user to be assessed, combining the action assessment value of the user to be assessed in the comprehensive attention test, obtaining test-related data, and using the attention comprehensive assessment formula according to the test-related data to obtain a comprehensive assessment value of the user to be assessed. The present invention constructs an action attention assessment formula, quantifies and calculates the action assessment value based on the actual motion data of the user to be assessed during the assessment period, and performs personalized attention scoring on the user to be assessed with different attention accompanying actions.

Description

Technical Field

[0001] The present invention relates to the field of attention assessment technology, and in particular to an attention assessment method, device, equipment and medium based on multimodal data. Background Art

[0002] In recent years, with the popularization of informatization and smart devices, the demand for individual behavior and status monitoring in various fields of society has increased significantly. People often show attention fluctuations, distraction or excessive tension in scenarios such as work, study, and driving. In such complex and changeable application scenarios, a single data source or a single sensor monitoring is difficult to fully reflect the individual's attention state. At present, most methods for evaluating people's attention are based on the collection of human motion data by devices such as motion sensors and accelerometers, combined with traditional data analysis methods to evaluate the state of attention. However, this technology will have a large deviation in the degree of adaptation for different personnel, and cannot fully reflect the multi-dimensional behavioral characteristics of individuals in actual work and life; at the same time, when conducting multi-attention assessments, there is a lack of auxiliary cooperation with motion attention assessments. Since the existing technology cannot fully capture and quantify the attention state of personalized personnel in actual applications, there is a widespread problem of insufficient or distracted attention in enterprises and educational institutions. Summary of the invention

[0003] In order to overcome the shortcomings of ignoring the impact of personalized actions on attention assessment and the impact between multiple attention assessments, the present invention provides an attention assessment method, device, equipment and medium based on multimodal data.

[0004] The technical solution is: an attention evaluation method based on multimodal data, comprising the following steps:

[0005] S1: Acquire multimodal data of a user to be evaluated, input the multimodal data into an action attention evaluation formula, and obtain an action evaluation value of the user to be evaluated;

[0006] S2: Performing a comprehensive attention test on the user to be evaluated, combining the action evaluation value of the user to be evaluated in the comprehensive attention test, obtaining test-related data, and using the comprehensive attention evaluation formula according to the test-related data to obtain a comprehensive evaluation value of the user to be evaluated;

[0007] S3: Judging the attention level of the user to be evaluated according to the comprehensive evaluation value of the user to be evaluated.

[0008] Preferably, the step of acquiring multimodal data of the user to be evaluated, inputting the multimodal data into a motion attention evaluation formula, and obtaining a motion evaluation value of the user to be evaluated comprises: collecting multimodal data of the user to be evaluated, the multimodal data comprising the actual number of movements of each motion part of the user to be evaluated during the evaluation period, the baseline number of each motion part of the user to be evaluated, and the number of body parts to be detected and evaluated of the user to be evaluated, inputting the multimodal data into a motion attention evaluation formula, and obtaining a motion evaluation value of the user to be evaluated, wherein the motion attention evaluation formula is:

[0009] ;

[0010] In the formula, The action evaluation value of the user to be evaluated; For the user to be evaluated The actual number of movements of each movement part during the assessment period; For the user to be evaluated Baseline times for each action part; For the The weight of each action part; The number of body parts to be evaluated for the user to be evaluated.

[0011] Preferably, the For the The weight of each action part includes: obtaining relevant action data of the user to be evaluated, using an accompanying action detection model for the relevant action data to obtain the attention accompanying action of the user to be evaluated, analyzing and obtaining the frequency of the attention accompanying action, and according to the frequency of the attention accompanying action, using a weight adjustment formula to obtain the weight of the attention accompanying action, wherein the weight adjustment formula is:

[0012] ;

[0013] In the formula, is the weight of attention accompanying action; is the coefficient adjustment factor; For attention accompanies the frequency of action; The baseline frequency of attention-accompanying actions; is the error tolerance value of attention-action frequency.

[0014] Preferably, the obtaining of relevant action data of the user to be evaluated, using an accompanying action detection model for the relevant action data, and obtaining the attention accompanying action of the user to be evaluated, comprises: obtaining relevant action data of the user to be evaluated, wherein the relevant action data comprises action data and action mode of the user to be evaluated, wherein the action data is the number of actions, frequency and duration of actions within a preset time period; the action mode is the triggering time, interval, duration, speed and acceleration of each action; and performing data preprocessing of normalization, time alignment and feature extraction on the relevant action data to obtain preprocessed data, and inputting the preprocessed data into an accompanying action detection model trained by LSTM to obtain the attention accompanying action of the user to be evaluated; the attention accompanying action is an action that habitually assists the user to be evaluated in concentrating his attention when the user is concentrating his attention.

[0015] Preferably, a comprehensive attention test is performed on the user to be evaluated, and test-related data is obtained in combination with the action evaluation value of the user to be evaluated in the comprehensive attention test, and a comprehensive evaluation formula for attention is used according to the test-related data to obtain a comprehensive evaluation value of the user to be evaluated, including: performing a comprehensive attention test on the user to be evaluated, the comprehensive attention test includes a selective attention test, a continuous attention test, a distributed attention test and an alternating attention test, and test-related data in the comprehensive attention test is obtained, the test-related data includes the test duration of each attention test, the evaluation value of each attention test, the interval duration of each attention test and the action evaluation value of the user to be evaluated during each attention test, and the test-related data is input into the comprehensive attention evaluation formula to obtain the comprehensive evaluation value of the user to be evaluated.

[0016] Preferably, the step of inputting the test-related data into a comprehensive attention evaluation formula to obtain a comprehensive evaluation value of the user to be evaluated comprises: wherein the comprehensive attention evaluation formula is:

[0017] ;

[0018] In the formula, is the comprehensive evaluation value of the user to be evaluated; For the user to be evaluated Attention test assessment value; For the user to be evaluated The duration of the attention test; For the user to be evaluated The action evaluation value during the attention test; For the user to be evaluated The attention test and The interval between attention trials affects the duration function; For the The weight coefficient of the attention test; is the adjustment factor.

[0019] Preferably, the interval impact duration function includes: the interval impact duration function is:

[0020] ;

[0021] In the formula, For the user to be evaluated The attention test and The interval between attention trials affects the duration function; To adjust the parameters; For the user to be evaluated The attention test and The length of time between attention tests; is the preset function center value.

[0022] The present application embodiment provides an attention assessment device, comprising:

[0023] A data acquisition module, used to acquire multimodal data of the user to be evaluated, relevant action data of the user to be evaluated, and test-related data in the comprehensive attention test;

[0024] An action evaluation value acquisition module, used to input the multimodal data into an action attention evaluation formula to obtain an action evaluation value of the user to be evaluated;

[0025] An attention accompanying action acquisition module is used to acquire relevant action data of the user to be evaluated, and use an accompanying action detection model on the relevant action data to obtain the attention accompanying action of the user to be evaluated;

[0026] A weight adjustment module, used to obtain the weight of the attention-accompanying action using a weight adjustment formula according to the frequency of the attention-accompanying action;

[0027] A comprehensive evaluation value acquisition module, used to obtain a comprehensive evaluation value of the user to be evaluated using an attention comprehensive evaluation formula according to the test-related data;

[0028] An interval influence function adjustment module is used to adjust the comprehensive evaluation value using the interval duration of each attention test according to the interval influence duration function;

[0029] The attention level judging module is used to judge the attention level of the user to be evaluated according to the comprehensive evaluation value of the user to be evaluated.

[0030] An embodiment of the present application provides an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, the attention assessment method based on multimodal data as described in any one of the above technical solutions is implemented.

[0031] Preferably, the computer-readable storage medium comprises: the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the attention assessment method based on multimodal data as described in any one of the above technical solutions.

[0032] Beneficial Effects

[0033] 1. The present invention improves the evaluation accuracy by integrating multimodal data, and comprehensively considers the relevant action data of the user to be evaluated and the test-related data in the comprehensive attention test to make the evaluation result more comprehensive and accurate;

[0034] 2. The present invention constructs an action attention evaluation formula, quantifies and calculates the action evaluation value based on the actual motion data of the user to be evaluated during the evaluation period, and performs personalized attention scoring for the user to be evaluated with different attention accompanying actions, so that the attention evaluation is more accurate;

[0035] 3. The present invention combines comprehensive attention test to enhance the reliability of evaluation. While conducting selective attention test, sustained attention test, distributed attention test and alternating attention test, comprehensive test evaluation is conducted in combination with action evaluation value, thereby further improving the comprehensiveness and objectivity of evaluation.

[0036] 4. The present invention introduces an interval impact duration function when conducting a comprehensive attention test to prevent mutual influence between different attention tests when attention is continuously evaluated, and dynamically adjusts the evaluation weight, so as to be more in line with the law of attention changes in real scenes and improve the real-time adaptability of the evaluation system. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flow chart of the attention assessment method based on multimodal data of the present invention;

[0038] Figure 2 It is a schematic diagram of the structure of the attention assessment system based on multimodal data of the present invention. DETAILED DESCRIPTION

[0039] The above scheme is further described below in conjunction with specific examples. It should be understood that these examples are used to illustrate the present application and are not limited to the scope of the present application. The implementation conditions adopted in the examples can be further adjusted according to the conditions of the specific manufacturer, and the implementation conditions not specified are usually the conditions in conventional experiments.

[0040] Embodiment 1: Attention evaluation method based on multimodal data, such as Figure 1 As shown, the following steps are included:

[0041] S1: Acquire multimodal data of a user to be evaluated, input the multimodal data into an action attention evaluation formula, and obtain an action evaluation value of the user to be evaluated;

[0042] Collect multimodal data of the user to be evaluated, wherein the multimodal data includes the actual number of movements of each action part of the user to be evaluated during the evaluation period, the baseline number of each action part of the user to be evaluated, and the number of body parts to be detected and evaluated of the user to be evaluated, and input the multimodal data into the action attention evaluation formula to obtain the action evaluation value of the user to be evaluated, wherein the action attention evaluation formula is:

[0043] ;

[0044] In the formula, The action evaluation value of the user to be evaluated; For the user to be evaluated The actual number of movements of each movement part during the assessment period; For the user to be evaluated Baseline times for each action part; For the The weight of each action part; The number of body parts to be evaluated for the user to be evaluated.

[0045] It should be noted that For the user to be evaluated The actual number of movements of each action part during the assessment period is recorded in real time by using video monitoring, accelerometers, gyroscopes or other wearable devices to record the number of movements of each action part of the user to be assessed during the preset assessment period; For the user to be evaluated The baseline frequency of each action part is obtained by using a pre-set standard or historical data to obtain the movement frequency of each action part when the user to be evaluated is concentrating as a baseline; The number of body parts to be tested and evaluated for the user to be evaluated is determined according to the evaluation requirements, ensuring that the multimodal data covers multiple key parts of the user to be evaluated. Less than When , it means that the user to be evaluated has a high level of attention and has few fidgeting movements, which indirectly reflects that the user to be evaluated has a high level of attention.

[0046] Obtain relevant action data of the user to be evaluated, use the accompanying action detection model on the relevant action data to obtain the attention accompanying action of the user to be evaluated, analyze and obtain the frequency of the attention accompanying action, and use the weight adjustment formula to obtain the weight of the attention accompanying action according to the frequency of the attention accompanying action, wherein the weight adjustment formula is:

[0047] ;

[0048] In the formula, is the weight of attention accompanying action; is the coefficient adjustment factor; For attention accompanies the frequency of action; It is the baseline frequency of attention-accompanying actions; is the error tolerance value of attention-action frequency.

[0049] It should be noted that by analyzing the detected attention-accompanying actions, the frequency of attention-accompanying actions corresponding to each action part is counted and recorded as ; Obtain the frequency of each action part of the user to be evaluated within a preset time period. When the user to be evaluated is in withdrawal reaction or has side effects of drugs, and the baseline frequency of each attention-accompanying action part of the user to be evaluated is irregular or exceeds the baseline frequency of attention-accompanying action, it reflects the inattention state of the user to be evaluated.

[0050] Relevant action data of the user to be evaluated is obtained, wherein the relevant action data includes the action data and action mode of the user to be evaluated, wherein the action data is the number of actions, frequency and duration of actions within a preset time period; the action mode is the trigger time, interval, duration, speed and acceleration of each action; and the relevant action data is normalized, time-series aligned and feature extracted to obtain preprocessed data, and the preprocessed data is input into the accompanying action detection model trained by LSTM to obtain the attention accompanying action of the user to be evaluated; the attention accompanying action is the action that habitually assists the user to be evaluated in concentrating his attention when he is concentrating his attention.

[0051] It should be noted that multiple sensors are arranged in the activity area of ​​the user to be evaluated, including video cameras, accelerometers, gyroscopes and other wearable devices, to collect relevant action data of the user to be evaluated in real time. The relevant action data consists of action data and action patterns, wherein the action data is recorded in a time period, including the number of actions, action frequency and action duration of each action part. The number of actions reflects the total number of actions actually performed in each action part of the user to be evaluated in the time period; the action frequency is the number of repetitions of the action of the user to be evaluated in the preset time period; the action duration is the specific time length of each action execution; the action pattern is the trigger time of each action, the interval between actions, the action duration, the action speed and the action acceleration. These data provide the necessary time, speed and acceleration information for the subsequent dynamic feature analysis of the user to be evaluated, which is helpful to distinguish the users in different attention states. Action performance; and the relevant action data are: normalized, scaling the data collected by different sensors according to the same dimension to eliminate numerical fluctuations caused by equipment differences or environmental changes; timing alignment: time correction is performed on the collected data to ensure that the data from each sensor is strictly synchronized in time so as to accurately reflect the true timing of the user's action; feature extraction: statistical or deep learning methods are used to extract key features from the normalized and time-aligned data, such as action frequency changes, duration distribution and acceleration curve characteristics, to generate representative feature vectors and form preprocessed data; the preprocessed data is input into the accompanying action detection model trained by LSTM to capture the habitual auxiliary actions performed by the user to be evaluated in the state of concentration, that is, the attention accompanying action; the attention accompanying action is the action habitually triggered by the user to be evaluated to maintain or assist attention when the user is highly concentrated.

[0052] S2: Performing a comprehensive attention test on the user to be evaluated, combining the action evaluation value of the user to be evaluated in the comprehensive attention test, obtaining test-related data, and using the comprehensive attention evaluation formula according to the test-related data to obtain a comprehensive evaluation value of the user to be evaluated;

[0053] A comprehensive attention test is performed on the user to be evaluated, wherein the comprehensive attention test includes a selective attention test, a sustained attention test, a distributed attention test and an alternating attention test, and test-related data in the comprehensive attention test is obtained, wherein the test-related data includes the test duration of each attention test, the evaluation value of each attention test, the interval duration of each attention test and the action evaluation value of the user to be evaluated during each attention test, and the test-related data is input into a comprehensive attention evaluation formula to obtain a comprehensive evaluation value of the user to be evaluated.

[0054] It should be noted that the selective attention test is used to detect the user's ability to screen and concentrate in an environment with interference information; the sustained attention test is used to examine the stability of the user's attention over a long period of time; the distributed attention test is used to evaluate the user's efficiency in allocating attention when handling multiple tasks at the same time; the alternating attention test is used to analyze the user's ability to switch attention when switching quickly between different tasks; in the process of the above attention tests, the action attention evaluation formula will be used for auxiliary observation; in each attention test, data will be collected For the user to be evaluated The test duration of the attention test is used to ensure that the test data has sufficient statistical significance; For the user to be evaluated The evaluation value of each test is calculated using a preset algorithm to reflect the user's specific attention performance in each test; For the user to be evaluated The attention test and The interval effect duration function between attention tests is used to evaluate the impact between multiple attention tests; For the user to be evaluated The action evaluation value of each attention test process is used. During the test, the user's action evaluation value is monitored and recorded at the same time.

[0055] The comprehensive evaluation formula for attention is:

[0056] ;

[0057] In the formula, is the comprehensive evaluation value of the user to be evaluated; For the user to be evaluated Attention test assessment value; For the user to be evaluated The duration of the attention test; For the user to be evaluated The action evaluation value during the attention test; For the user to be evaluated The attention test and The interval between attention trials affects the duration function; For the The weight coefficient of the attention test; is the adjustment factor.

[0058] It should be noted that during the comprehensive attention test, the user's test duration, attention test evaluation value and action evaluation value during the test are synchronously collected. At the same time, the interval between each test is recorded to ensure that the impact of the test interval on the attention state can be correctly reflected in subsequent calculations.

[0059] The interval impact duration function is:

[0060] ;

[0061] In the formula, For the user to be evaluated The attention test and The interval between attention trials affects the duration function; To adjust the parameters; For the user to be evaluated The attention test and The length of time between attention tests; is the preset function center value.

[0062] It should be noted that For the user to be evaluated The attention test and The interval effect duration function between attention tests is used to quantify the impact of the interval duration on the test results; To adjust the parameters, used to adjust the steepness of the logistic function, to ensure that a smooth and reasonable function output can be obtained under different time interval conditions; For the user to be evaluated The attention test and The interval between attention tests; parameters and The two parameters are set reasonably according to the actual test environment and user groups to ensure that under different test conditions, the interval impact duration function can truly reflect the impact of the user's attention state in each attention test interval.

[0063] S3: Judging the attention level of the user to be evaluated according to the comprehensive evaluation value of the user to be evaluated.

[0064] The second embodiment of the present invention provides an attention assessment device, such as Figure 2 As shown, including:

[0065] A data acquisition module, used to acquire multimodal data of the user to be evaluated, relevant action data of the user to be evaluated, and test-related data in the comprehensive attention test;

[0066] An action evaluation value acquisition module, used to input the multimodal data into an action attention evaluation formula to obtain an action evaluation value of the user to be evaluated;

[0067] An attention accompanying action acquisition module is used to acquire relevant action data of the user to be evaluated, and use an accompanying action detection model on the relevant action data to obtain the attention accompanying action of the user to be evaluated;

[0068] A weight adjustment module, used to obtain the weight of the attention-accompanying action using a weight adjustment formula according to the frequency of the attention-accompanying action;

[0069] A comprehensive evaluation value acquisition module, used to obtain a comprehensive evaluation value of the user to be evaluated using an attention comprehensive evaluation formula according to the test-related data;

[0070] An interval influence function adjustment module is used to adjust the comprehensive evaluation value using the interval duration of each attention test according to the interval influence duration function;

[0071] The attention level judging module is used to judge the attention level of the user to be evaluated according to the comprehensive evaluation value of the user to be evaluated.

[0072] A third embodiment of the present invention proposes an electronic device, including a memory and a processor, wherein the memory stores a computer program, and when the computer program is executed by the processor, an attention assessment method based on multimodal data as described in any one of the above technical solutions is implemented.

[0073] The fourth embodiment of the present invention proposes a computer-readable storage medium, which stores a computer program that can be executed by one or more processors to implement an attention assessment method based on multimodal data as described in any of the above technical solutions.

[0074] Although the present invention is described in detail with reference to the above embodiments, it is obvious to those skilled in the art through this disclosure that various changes or modifications may be made to the present invention without departing from the principle and spirit of the present invention defined by the claims. Therefore, the detailed description of the embodiments of the present disclosure is only used to explain, not to limit the present invention, but the scope of protection is limited by the content of the claims.

Claims

1. An attention assessment method based on multimodal data, characterized in that: The following steps are involved: S1: Acquire multimodal data of a user to be evaluated, input the multimodal data into an action attention evaluation formula, and obtain an action evaluation value of the user to be evaluated; S2: Performing a comprehensive attention test on the user to be evaluated, combining the action evaluation value of the user to be evaluated in the comprehensive attention test, obtaining test-related data, and using the comprehensive attention evaluation formula according to the test-related data to obtain a comprehensive evaluation value of the user to be evaluated; S3: judging the attention level of the user to be evaluated according to the comprehensive evaluation value of the user to be evaluated; The step of inputting the test-related data into a comprehensive attention evaluation formula to obtain a comprehensive evaluation value of the user to be evaluated includes: wherein the comprehensive attention evaluation formula is: ； In the formula, is the comprehensive evaluation value of the user to be evaluated; For the user to be evaluated Attention test assessment value; For the user to be evaluated The duration of the attention test; For the user to be evaluated The action evaluation value during the attention test; For the user to be evaluated The attention test and The interval between attention trials affects the duration function; For the The weight coefficient of the attention test; is the adjustment factor; The method of obtaining multimodal data of the user to be evaluated, inputting the multimodal data into a motion attention evaluation formula, and obtaining a motion evaluation value of the user to be evaluated comprises: collecting multimodal data of the user to be evaluated, wherein the multimodal data comprises the actual number of movements of each motion part of the user to be evaluated during the evaluation period, the baseline number of each motion part of the user to be evaluated, and the number of body parts to be detected and evaluated of the user to be evaluated, and inputting the multimodal data into a motion attention evaluation formula to obtain a motion evaluation value of the user to be evaluated, wherein the motion attention evaluation formula is: ； In the formula, The action evaluation value of the user to be evaluated; For the user to be evaluated The actual number of movements of each movement part during the assessment period; For the user to be evaluated The baseline frequency of each action part is obtained by using a pre-set standard or historical data to obtain the movement frequency of each action part when the user to be evaluated is concentrating as a baseline; For the The weight of each action part; The number of body parts to be tested and evaluated for the user to be evaluated; Said For the The weight of each action part includes: obtaining relevant action data of the user to be evaluated, using an accompanying action detection model on the relevant action data to obtain the attention accompanying action of the user to be evaluated, analyzing and obtaining the frequency of the attention accompanying action, and using a weight adjustment formula according to the frequency of the attention accompanying action to obtain the weight of the first action part. The weight of each action part, where the weight adjustment formula is: ； In the formula, For the The weight of each action part; is the coefficient adjustment factor; For attention accompanies the frequency of action; It is the baseline frequency of attention-accompanying actions; is the error tolerance value of attention-action frequency.

2. The attention assessment method based on multimodal data according to claim 1, characterized in that: The method of obtaining relevant action data of the user to be evaluated, applying an accompanying action detection model to the relevant action data, and obtaining the attention accompanying action of the user to be evaluated comprises: obtaining relevant action data of the user to be evaluated, wherein the relevant action data comprises action data and action mode of the user to be evaluated, wherein the action data is the number of actions, frequency and duration of actions within a preset time period; the action mode is the triggering time, interval, duration, speed and acceleration of each action; and performing data preprocessing of normalization, time sequence alignment and feature extraction on the relevant action data to obtain preprocessed data, and inputting the preprocessed data into an accompanying action detection model trained by LSTM to obtain the attention accompanying action of the user to be evaluated; the attention accompanying action is an action that habitually assists the user to be evaluated in concentrating his attention when the user is concentrating his attention.

3. The attention assessment method based on multimodal data according to claim 1, characterized in that: The method comprises: performing a comprehensive attention test on the user to be evaluated, combining the action evaluation value of the user to be evaluated in the comprehensive attention test, obtaining test-related data, and using a comprehensive attention evaluation formula according to the test-related data to obtain a comprehensive evaluation value of the user to be evaluated, including: performing a comprehensive attention test on the user to be evaluated, the comprehensive attention test comprising a selective attention test, a continuous attention test, a distributed attention test, and an alternating attention test, obtaining test-related data in the comprehensive attention test, the test-related data comprising the test duration of each attention test, the evaluation value of each attention test, the interval duration of each attention test, and the action evaluation value of the user to be evaluated during each attention test, inputting the test-related data into the comprehensive attention evaluation formula to obtain the comprehensive evaluation value of the user to be evaluated.

4. The attention assessment method based on multimodal data according to claim 1, characterized in that: The interval impact duration function includes: the interval impact duration function is: ； In the formula, For the user to be evaluated The attention test and The interval between attention trials affects the duration function; To adjust the parameters; For the user to be evaluated The attention test and The length of time between attention tests; is the preset function center value.

5. An attention assessment device, according to the attention assessment method based on multimodal data according to any one of claims 1 to 4, characterized in that: include: A data acquisition module, used to acquire multimodal data of the user to be evaluated, relevant action data of the user to be evaluated, and test-related data in the comprehensive attention test; An action evaluation value acquisition module, used to input the multimodal data into an action attention evaluation formula to obtain an action evaluation value of the user to be evaluated; An attention accompanying action acquisition module is used to acquire relevant action data of the user to be evaluated, and use an accompanying action detection model on the relevant action data to obtain the attention accompanying action of the user to be evaluated; A weight adjustment module is used to obtain the first The weight of each action part; A comprehensive evaluation value acquisition module, used to obtain a comprehensive evaluation value of the user to be evaluated using an attention comprehensive evaluation formula according to the test-related data; An interval influence function adjustment module is used to adjust the comprehensive evaluation value using the interval duration of each attention test according to the interval influence duration function; The attention level judging module is used to judge the attention level of the user to be evaluated according to the comprehensive evaluation value of the user to be evaluated.

6. An electronic device, characterized in that: It comprises a memory and a processor, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the attention assessment method based on multimodal data as described in any one of claims 1 to 4 is implemented.

7. A computer-readable storage medium, characterized in that: The computer program stored in the storage medium can be executed by one or more processors and can be used to implement the attention assessment method based on multimodal data as described in any one of claims 1 to 4.

Images