Team collaboration status assessment system and method based on support vector machine

Abstract

The present invention provides a team collaboration status assessment system and method based on support vector machines, which includes four parts: a data acquisition module, a data interface module, an online assessment module, and an output report module. The data acquisition module collects four parts of team information: team information collection, eye movement device collection, electrocardiogram device collection, and video recording collection. The data interface module receives the information transmitted by the data acquisition module and converts and extracts indicator features. The online assessment module performs online analysis and assessment on the team experimental data. The output report module comprehensively outputs the read-in team information and online assessment results. The system can quickly and effectively determine the team collaboration status of the team in various task environments, which helps to improve the level of team collaboration in ship crews and provide a basis for reducing human errors.

Description

Technical Field

[0001] The present invention relates to a team collaboration status assessment system and method based on support vector machine (SVM). Background Art

[0002] In modern military and industrial work environments, tasks often involve high cognitive demands, requiring close and efficient teamwork to successfully complete. Teamwork refers to the interconnected set of thoughts, actions, and feelings required by each member to form a team. This facilitates coordinated, adaptive performance and mission objectives, resulting in value-added results. The goal of teamwork is to create good team performance, and excellent team performance requires accurate information sharing through teamwork. Therefore, teamwork status is a key indicator of whether a team can accurately and efficiently complete various tasks.

[0003] Modern ships are typical examples of complex human-machine systems. Their characteristics include harsh operating environments during long voyages; complex missions requiring high levels of human collaboration; and a vast amount of information at the human-machine interface, placing a high mental workload on crew members. Common ship accidents include collisions, fires, flooding, and explosions. Statistics show that over 70% of ship accidents involve human error, and poor teamwork is a significant contributing factor to these accidents.

[0004] The assessment methods of team collaboration status are mainly divided into four categories: self-report measures, observational measures, event-based measurement and automated measurement.

[0005] Self-report assessments primarily include questionnaires and rating scales, assessing team synergy at both the individual and team levels, particularly those that are inherently affective (such as trust) and difficult to observe. For example, the Team Climate Inventory (TCI) assesses various team synergy capabilities. Specifically, the TCI has five top-level scales, corresponding to 15 subscales; the five top-level scales assess participation safety, innovation support, team vision, task orientation, and social desirability. Other instruments assess individual factors related to team synergy, such as the Psychological Safety Scale, proposed in 1999, which is one of the most widely used measures of team synergy. A major challenge in using self-reports to assess team synergy is how to aggregate or transform individual responses into team-level characteristics. Even with a detailed theoretical understanding of their respective constructs, various aggregation methods are likely to produce very different results and fail to capture the unique characteristics of individual responses. Furthermore, self-report assessments often produce inflated scores because participants tend to rate themselves more highly than observers do; and self-report assessments are often performed before or after team tasks and fail to capture the dynamic nature of team performance during task execution.

[0006] ②Observational ratings can be collected in real time or using video recordings of team collaboration tasks, thus having the potential to capture the dynamic nature of the team collaboration process. Behaviorally Anchored Rating Scales (BARS) are a typical observational rating technique: for excellent, average, and poor team performance, short behavioral descriptions are used as anchors for observational ratings. These behaviors can be rated at the individual level and summarized into team scores or team grades. BARS is widely used in areas such as team function assessment in medical institutions and non-technical skill systems for assessing pilot behavior in crew environments. Although the specific descriptions of the behaviors provided contribute to more accurate ratings, they may cause observers to focus only on the behaviors on the list.

[0007] ③ Event-Based Measurement (EBAT) is a structured observation technique that is often applied to team simulation scenarios. By systematically introducing events into training exercises, EBAT provides an opportunity to observe specific behaviors of interest, which are then marked as present or absent. For example, in an aviation scenario, situational awareness can be assessed by introducing an event in which the instructor deliberately steers the aircraft off course. The target behavioral response is "check navigation," and the result is a successful or failed detection. Using this method, learning objectives can be clearly quantified, and specific focuses for team collaboration training can be set. However, due to the scenario limitations of specific events, EBAT can only be used in training scenarios and not in real performance scenarios. In addition, the development of EBAT metrics is very time-consuming.

[0008] Automated assessments collect team-related data through computer systems that interact with the team. This has the advantages of reducing interruptions, minimizing measurement errors, and reducing experimental personnel resources. This type of assessment has been most frequently and successfully used to assess team performance and team collaboration.

[0009] Related research on team collaboration status assessment has shown inconsistent reliability and validity test reports, as well as a lack of relevant research and application in the field of ship command and control systems.

[0010] Based on the above considerations, it is necessary to develop a system that can directly and conveniently automatically assess team collaboration online, based on simulated ship team experiments. This system uses computer-generated data collection and processing of physiological and communication data, and analyzes this data using support vector machines (SVMs), a machine learning algorithm. In particular, it is hoped that this assessment system can be applied to ergonomic evaluation and analysis of team collaboration on board ships, thereby providing a basis for improving the state of team collaboration and enhancing the team's collaborative capabilities. Summary of the Invention

[0011] According to one aspect of the present invention, a support vector machine-based team collaboration status assessment system is provided. The system comprises four submodules: a data acquisition module, a data interface module, an online assessment module, and an output report module. The data acquisition module is used to collect four types of team information: team information, eye tracking device data, electrocardiogram data, and video recording data. The data interface module receives information transmitted by the data acquisition module and converts and extracts indicator features. The online assessment module performs online analysis and assessment of team experimental data. The output report module outputs a comprehensive output of the read-in team information and online assessment results.

[0012] According to another aspect of the present invention, a method for assessing team collaboration status is provided, which is characterized by including: an initial interface before the assessment operation, for collecting team information; a data collection and processing interface during the assessment operation, for collecting and processing data; an online assessment interface during the assessment operation, for performing online analysis and assessment of team experimental data; and an output result interface after the assessment operation, for comprehensively outputting the read-in team information and online assessment results.

[0013] The beneficial effects of the present invention are:

[0014] The present invention overcomes the shortcomings of current online assessment technology for team collaboration status. Focusing on the team collaboration status during ship experiments, the present invention connects eye tracker equipment and electrocardiogram equipment through an interface module to read team physiological data in real time, and connects video recording and analysis equipment to collect team communication data in real time, thereby realizing online assessment of team collaboration status. It can quickly determine the team collaboration status of the team in various task environments, thereby ensuring effective team collaboration and providing a basis for reducing human errors.

[0015] Advantages of the present invention include:

[0016] (1) An automated team collaboration status assessment system is provided. It does not rely on tedious questionnaires and post-analysis evaluations. It can automatically assess the team collaboration status online by simply collecting eye movement data, electrocardiogram data, and video recordings during the team collaboration experiment. The system is simple and feasible to use, and provides an optimized user interface and usage process. It is particularly suitable for evaluating and analyzing the team collaboration status of ship crews, which helps to improve the team collaboration status of ship crews.

[0017] (2) Provide an objective assessment method for team collaboration status. Based on support vector machines, it only needs to collect physiological data and communication data of team collaboration experiments to directly, conveniently and automatically assess the level of team collaboration status. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 This is an architecture diagram of a team collaboration status assessment system based on a support vector machine according to an embodiment of the present invention;

[0019] Figure 2 This is a flow chart of a method for operating a user interface of a team collaboration status assessment system based on a support vector machine according to an embodiment of the present invention;

[0020] Figures 3(a) to 3(d) 1 is a schematic diagram of a user interface of a support vector machine-based team collaboration status assessment system in various operating states according to an embodiment of the present invention. DETAILED DESCRIPTION

[0021] 1. Overall situation

[0022] Aiming at the real-time collection of applications and different usage requirements of different scenarios and / or different devices, the present invention proposes an online assessment system and method for team collaboration status based on the eye movements, electrocardiogram and communication characteristics of team members during a simulated ship team experiment. The physiological and communication data collected and sorted during the experiment are classified based on a support vector machine to assess the level of team collaboration status.

[0023] 2 Design of Team Collaboration Status Assessment System Architecture Based on Support Vector Machine

[0024] like Figure 1 As shown, the team collaboration status assessment system based on support vector machines according to one embodiment of the present invention includes four modules: a data acquisition module, a data interface module, an online assessment module, and an output report module.

[0025] (1) Data acquisition module

[0026] This module collects four types of team data: team information, eye tracking device, ECG device, and video recording. ① Team information collection includes the team test number, which is entered through the user interface. ② Eye tracking device collection uses the eye tracking device to read raw eye movement data in real time, such as the original spatial coordinates of eye movements, blink marks, gaze marks, and saccade marks. ③ ECG device collection uses the ECG device to read raw ECG data in real time, such as heart rate and raw ECG waveform data. ④ Video recording records the entire activity (team collaboration) and includes both images and audio.

[0027] (2)Data interface module

[0028] This module is used to receive the information transmitted by the data acquisition module and convert and extract indicator features. It includes three main modules: data transceiver interface module, data storage module, and data conversion module. Among them:

[0029] ① Data transceiver interface module: First, set the UDP port number and host address of the eye tracking device and ECG device. Then, based on the C++ platform, establish UDP data communication between the data interface module and the eye tracking device and ECG device through the data transceiver interface. Read the raw data collected by the eye tracking device at a frequency of 60Hz, and read the raw ECG data at a frequency of 1Hz. Use audio analysis software to extract the communication text from the audio in the team experiment.

[0030] ②Data storage module: stores the real-time collected eye movement and ECG raw data of different frequencies and the extracted communication text in the memory space;

[0031] ③ Data conversion module: In one embodiment, the system converts the raw data of eye movement and electrocardiogram into indicators based on the entire activity process and calculates the average value, converting them into eye movement characteristic indicator data such as gaze rate, scan rate, pupil diameter, average gaze time, and electrocardiogram characteristic indicator data such as average heart rate, average RR interval, standardized low-frequency power, and standardized high-frequency power.

[0032] For communication texts, text analysis software is used to perform text analysis, and the frequency of communication codes and the frequency data of communication keywords are counted to obtain communication text feature data.

[0033] (3) Online assessment module

[0034] This module is mainly used for online analysis and evaluation of team experimental data, and includes four modules: eye movement feature preprocessing module, electrocardiogram feature preprocessing module, communication feature preprocessing module, and online evaluation classifier module. Among them:

[0035] ① Eye movement feature preprocessing module, which reads the eye movement feature indicators (fixation rate, saccade rate, pupil diameter, average fixation time, etc.) sent by the data interface module and performs normalization processing such as removing null values ​​or outliers, as well as preprocessing operations such as standardization processing;

[0036] ② ECG feature preprocessing module, which reads the ECG feature indicators (average heart rate, average RR interval, standardized low-frequency power, standardized high-frequency power, etc.) sent from the data interface module, and performs normalization processing such as removing null values ​​or abnormal values, as well as preprocessing operations such as standardization processing.

[0037] ③ The communication feature preprocessing module reads the communication text feature indicators (communication coding frequency, communication keyword frequency, etc.) sent by the data interface module and performs normalization processing such as removing null values ​​or abnormal values, as well as preprocessing operations such as standardization processing;

[0038] ④ The online assessment classifier module reads in standardized and normalized eye movement, electrocardiogram, and communication feature data and transmits them to three one-against-one (OAO) support vector machines for online calculations: SVM1 assigns vectors corresponding to high team collaboration as positive sets (+1) and vectors corresponding to medium team collaboration as negative sets (-1); SVM2 assigns vectors corresponding to high team collaboration as positive sets (+1) and vectors corresponding to low team collaboration as negative sets (-1); and SVM3 assigns vectors corresponding to medium team collaboration as positive sets (+1) and vectors corresponding to low team collaboration as negative sets (-1). These three SVMs perform calculations separately, yielding three classification results: Classification 1, Classification 2, and Classification 3. A voting method is used to determine the final classification for team collaboration level; the result that appears most frequently among the three classifications is the final classification. For example, if SVM1 is judged to be in a high team collaboration state, SVM2 is judged to be in a high team collaboration state, and SVM3 is judged to be in a medium team collaboration state, then the final classification result is a high team collaboration state, thereby realizing online assessment of different team collaboration state levels.

[0039] (4) Output report module

[0040] The team information and online assessment results read in the above module are comprehensively output, where the assessment results are the final classification of the team collaboration status level, and the data list can be saved and exported by selecting a specified path.

[0041] 3. Operation method of user interaction interface of team collaboration status assessment system based on support vector machine

[0042] like Figure 2 As shown in the flowchart of the user interface operation method of the team collaboration status assessment system based on support vector machine according to one embodiment of the present invention, it includes four interfaces in sequence: before assessment operation (initial interface), during assessment operation (data collection and processing interface), during assessment operation (online assessment interface), and after assessment operation (output result interface), which correspond to Figures 3(a) to 3(d) . Specific instructions are as follows.

[0043] like Figures 3(a) to 3(d) FIG. 1 is a schematic diagram of a user interface of a team collaboration status assessment system based on a support vector machine according to an embodiment of the present invention.

[0044] After entering the support vector machine-based team collaboration status assessment system, the initial interface before the assessment operation is shown in Figure 3(a). The system interface consists of four parts: title, input, status, and assessment results. The title section reads "Team Collaboration Status Online Assessment System"; the input section includes a test number input field and a "Start Assessment" button; the status section displays the system's operating status; and the assessment results section includes an output field for the team collaboration status level and assessment results, as well as three buttons: "Save Data," "Export Results," and "Close System."

[0045] At the beginning of the experiment, the user enters the test number in the test number column and clicks the "Start Evaluation" button to start data collection and sorting. During the experiment, the status section displays "Data collection and processing...", as shown in Figure 3(b), which shows the evaluation operation (data collection and processing interface).

[0046] After the experiment, the status section displays “Data collection and processing completed, online assessment in progress…”, indicating that the collected data has begun to be analyzed online, and the team collaboration status level has been assessed based on support vector machine classification. As shown in Figure 3(c), the assessment is in progress (online assessment interface).

[0047] After the online assessment is complete, the status section displays "Online Assessment Completed," and the assessment results section displays the team collaboration status and assessment results. Click the "Save Data" button to save the data and assessment results. Click the "Export Results" button to export the test number and corresponding test results. Click "Close System" to close the online assessment system for team collaboration status. Figure 3(d) shows the post-assessment interface (output results interface).

Claims

1. A team collaboration status assessment system based on support vector machine, characterized by include: The data collection module is used to collect team data information, including collecting team information through user input, collecting team eye movement data through eye tracking equipment, collecting team electrocardiogram data through electrocardiogram equipment, and collecting team video recording data through activity recording, including: Team information includes team test number, The team's eye movement data includes raw eye movement data read in real time, The team's ECG data includes real-time raw ECG data, The team's video recording data includes full video and / or recorded video, including images and audio, The data interface module is used to receive data from the data acquisition module and convert and extract indicator features. The data interface module includes: The data transceiver interface module is used to first set the UDP port number and host address of the eye tracking device and the ECG device. Then, based on the C++ platform, it establishes UDP data communication between the data interface module and the eye tracking device and the ECG device through the data transceiver interface. It reads the raw eye tracking data collected by the eye tracking device, reads the raw ECG data, and extracts the communication text from the team collaboration communication audio. A data storage module is used to store the real-time collected raw eye movement data, raw ECG data and extracted communication text in a memory space; The data conversion module is used to convert the raw eye movement data and the raw ECG data into indicators and calculate the average value. The indicator conversion includes converting the raw eye movement data into eye movement characteristic indicator data including fixation rate, saccade rate, pupil diameter, and average fixation time, and converting the raw ECG data into ECG characteristic indicator data including average heart rate, average RR interval, normalized low-frequency power, and normalized high-frequency power. The data interface module is also used to perform text analysis using text analysis software, and to collect statistics on the frequency of communication codes and the frequency data of communication keywords in the communication text to obtain communication text feature data. An online assessment module is used to perform online analysis and assessment of the team's eye movement data, the team's electrocardiogram data, and the team's communication text feature data. The online assessment module includes: An eye movement feature preprocessing module is used to read the eye movement feature index data sent by the data interface module and perform normalization processing including removal of null values ​​and / or abnormal values ​​and preprocessing operations including standardization processing; The ECG feature preprocessing module is used to read the ECG feature index data sent by the data interface module and perform normalization processing including removal of null values ​​and / or abnormal values, and preprocessing operations including standardization processing; The communication feature preprocessing module is used to read the communication text feature index data sent by the data interface module and perform normalization processing including removal of null values ​​and / or abnormal values, and preprocessing operations including standardization processing; The online assessment classifier module is used to read the standardized and normalized eye movement feature index data, electrocardiogram feature index data and communication text feature index data, and transmit the eye movement feature index data, electrocardiogram feature index data and communication text feature index data to three OAO support vector machines for online calculation: The first OAO support vector machine SVM1 takes the vector corresponding to the high team collaboration state as the positive set +1, and the vector corresponding to the medium team collaboration state as the negative set -1; The second OAO support vector machine SVM2 takes the vector corresponding to the high team collaboration state as the positive set +1, and the vector corresponding to the low team collaboration state as the negative set -1; The third OAO support vector machine SVM3 takes the vector corresponding to the medium team collaboration state as the positive set +1, and the vector corresponding to the low team collaboration state as the negative set -1. SVM1, SVM2 and SVM3 are calculated respectively to obtain three classification results: classification 1, classification 2, and classification 3. The voting method is used to determine the final classification result of the team collaboration status level, that is, the result that appears most frequently among the three classification results is taken as the final classification result. The output report module is used to comprehensively output the above team information and final classification results.

2. The team collaboration status assessment system based on support vector machine according to claim 1 is characterized by: Enter team information through the user interface. Eye tracking devices include wearable eye tracking devices, The original eye movement data includes the original spatial position coordinates of the eye movement, blink marks, fixation marks, and saccade marks. ECG devices include wearable ECG devices, The original ECG data includes heart rate and original ECG waveform data.

3. The team collaboration status assessment system based on support vector machine according to claim 1 is characterized in that: In the online assessment classifier module, if SVM1 judges it to be in a high team collaboration state, SVM2 judges it to be in a high team collaboration state, and SVM3 judges it to be in a medium team collaboration state, then the final classification result is in a high team collaboration state.

4. The team collaboration status assessment system based on support vector machine according to claim 1 is characterized by: The data transceiver interface module reads raw eye movement data at a frequency of 60 Hz and raw electrocardiogram data at a frequency of 1 Hz, and uses audio analysis software to extract communication text from the communication audio in the team experiment.

5. The team collaboration status assessment method based on support vector machine is characterized by The following steps are involved: A) Collecting team data information, including collecting team information through user input, collecting team eye movement data through eye tracking equipment, collecting team electrocardiogram data through electrocardiogram equipment, and collecting team video data through activity recording, including: Team information includes team test number, The team's eye movement data includes raw eye movement data read in real time, The team's ECG data includes real-time raw ECG data, The team's video recording data includes full video and / or recorded video, including images and audio, B) Receive data from the data acquisition module and convert and extract indicator features, including: B1) First, set the UDP port number and host address of the eye tracking device and ECG device. Then, based on the C++ platform, establish UDP data communication between the data interface module and the eye tracking device and ECG device through the data transceiver interface. Read the raw eye movement data collected by the eye tracking device and the raw ECG data, and extract the communication text from the team collaboration audio. B2) storing the real-time collected raw eye movement data, raw ECG data, and extracted communication text in a memory space; B3) converting the raw eye movement data and the raw ECG data into indicators and calculating the average value, wherein the indicator conversion includes converting the raw eye movement data into eye movement characteristic indicator data including fixation rate, saccade rate, pupil diameter, and average fixation time, and converting the raw ECG data into ECG characteristic indicator data including average heart rate, average RR interval, normalized low-frequency power, and normalized high-frequency power. C) Perform text analysis using text analysis software to collect statistics on the frequency of communication codes and the frequency of communication keywords in the communication text to obtain communication text feature data. D) Conduct online analysis and assessment of the team's eye movement data, ECG data, and communication text feature data, including: D1) reading the eye movement characteristic index data sent by the data interface module, and performing normalization processing including removing null values ​​and / or abnormal values ​​and pre-processing operations including standardization processing; D2) reading in the ECG characteristic index data sent from the data interface module, and performing normalization processing including removal of null values ​​and / or abnormal values, and pre-processing operations including standardization processing; D3) reading the communication text characteristic index data sent from the data interface module, and performing normalization processing including removing null values ​​and / or abnormal values, and pre-processing operations including standardization processing; D4) Reading in the standardized and normalized eye movement feature index data, electrocardiogram feature index data, and communication text feature index data, and transmitting the eye movement feature index data, electrocardiogram feature index data, and communication text feature index data to three OAO support vector machines for online calculation: wherein: The first OAO support vector machine SVM1 takes the vector corresponding to the high team collaboration state as the positive set +1, and the vector corresponding to the medium team collaboration state as the negative set -1; The second OAO support vector machine SVM2 takes the vector corresponding to the high team collaboration state as the positive set +1, and the vector corresponding to the low team collaboration state as the negative set -1; The third OAO support vector machine SVM3 takes the vector corresponding to the medium team collaboration state as the positive set +1, and the vector corresponding to the low team collaboration state as the negative set -1. D5) SVM1, SVM2 and SVM3 are used to calculate and obtain three classification results: classification 1, classification 2 and classification 3. D6) Use voting to determine the final classification result of the team collaboration status level, and take the result that appears most frequently among the three classification results as the final classification result. E) Comprehensively output the above team information and final classification results.

6. The method for evaluating team collaboration status based on support vector machines according to claim 5, characterized in that: Enter team information through the user interface. Eye tracking devices include wearable eye tracking devices, The original eye movement data includes the original spatial position coordinates of the eye movement, blink marks, fixation marks, and saccade marks. ECG devices include wearable ECG devices, The original ECG data includes heart rate and original ECG waveform data.

7. The method for assessing team collaboration status based on support vector machines according to claim 5, characterized in that: In step D4), if SVM1 is judged to be in a high team collaboration state, SVM2 is judged to be in a high team collaboration state, and SVM3 is judged to be in a medium team collaboration state, the final classification result is a high team collaboration state.

8. The method for evaluating team collaboration status based on support vector machines according to claim 5, characterized in that: In step B1), the original eye movement data is read at a frequency of 60 Hz, and the original electrocardiogram data is read at a frequency of 1 Hz. The communication text is extracted from the communication audio in the team experiment using audio analysis software.

9. A computer-readable storage medium storing a computer program, wherein the computer program enables a processor to execute the method for evaluating team collaboration status based on a support vector machine according to any one of claims 5 to 8.

Images