Multi-sensory channel riding navigation interaction control system and method based on eye movement data

Abstract

The application discloses a multi-sensory channel riding navigation interaction control system based on eye movement data, which comprises a data acquisition module, a data processing module and a navigation interaction module; wherein the data acquisition module comprises a sensor mounted on a vehicle-mounted handle and an external device, and is used for collecting physiological signals of a rider and vehicle control data; the data processing module is used for standardizing the data obtained by the data acquisition module, and classifying a real-time riding cognitive load level through a support vector machine; the navigation interaction module utilizes a vibrator mounted on the vehicle-mounted handle to remind the state of the rider according to the real-time riding cognitive load level obtained through classification. The application also discloses an interaction control method realized by using the above-mentioned interaction control system, which obtains the cognitive load level state of the rider when riding and gives feedback reminders.

Description

Technical Field

[0001] The present invention belongs to the technical field of ergonomics and relates to a multi-sensory channel cycling navigation interactive control system and an interactive control method based on cyclist's eye movement data. Background Art

[0002] Because human cognitive resources are limited in human-vehicle interactions during navigation, multi-sensory interaction (vision, hearing, and touch) can effectively allocate these resources, ensuring the completion of the primary driving task and improving driving safety. Therefore, multi-sensory interaction is a common approach to addressing multi-tasking situations by applying multi-resource theory.

[0003] Visual channels are the primary means by which people acquire information. Beyond the primary task of cycling, only 30%-50% of visual attention remains available for tasks other than cycling. Existing navigation maps primarily use voice guidance and require users to frequently check the navigation map. However, this interaction method has several limitations in cycling scenarios: Looking down at the navigation map can be dangerous if users are caught in windy, rainy, or snowy conditions with limited vision, or in other critical road conditions where navigation is difficult to view. When using voice guidance in noisy environments with heavy traffic, navigation instructions are difficult to hear clearly, especially for those with hearing impairments. This necessitates constant, inconvenient glances down at the navigation map.

[0004] Within a given navigation route, existing navigation maps only provide a fixed prompt mode from beginning to end. However, during actual cycling, due to changes in road conditions and environment, the user's cognitive resource allocation is in a dynamic state. The existing navigation interaction mode cannot detect changes in the rider's status in real time, and it is difficult to dynamically adjust the method and timing of navigation prompts. Because existing cycling navigation is not suitable for the restrictive scenarios mentioned above, in such cases it will increase the rider's cognitive load and reduce cycling efficiency and experience.

[0005] Therefore, a method is needed to improve the interactive navigation experience during cycling, to detect the rider's cognitive load level in real time, and to flexibly intervene in navigation feedback at the appropriate time, thereby dynamically adjusting the rider's cognitive load level.

[0006] Compared to other interaction channels, the tactile channel is less constrained in receiving information and more resistant to interference. For example, in situations with strong background noise, the tactile channel experiences much less information loss than the auditory channel. Therefore, tactile perception is more adaptable to the noisy environments of urban cycling. Compared to vision and hearing, it can reduce the user's cognitive burden, improving interaction efficiency and experience. Furthermore, during cycling, hand contact with the handlebars is a necessary and stable interaction. Summary of the Invention

[0007] In order to address the shortcomings of the existing technology, the purpose of the present invention is to provide a multi-sensory channel cycling navigation interactive control system based on eye movement data. The system can effectively adjust the cognitive load level under changing road conditions during cycling, and make dynamic navigation mode adjustments according to the real-time status of the rider, providing new ideas for subsequent cycling vehicle navigation design.

[0008] Therefore, the present invention uses touch as the main interactive channel for cycling navigation feedback and transmits information through the tactile channel via the handlebars.

[0009] To understand the allocation of cyclists' cognitive resources and monitor their cognitive load in real time during cycling, this paper collects cyclist physiological data, including eye movement indicators, and vehicle operation data. Based on this data, a qualitative comprehensive evaluation model for the cognitive load level during cycling is constructed. The navigation feedback mode is dynamically adjusted according to the cyclist's real-time status changes, thereby improving the interactive experience of cycling navigation, reducing the user's cognitive load level, improving cycling traffic safety, and promoting physical and mental enjoyment during cycling.

[0010] The present invention proposes a multi-sensory channel cycling navigation interactive control system based on eye movement data, the system includes a data acquisition module, a data processing module and a navigation interaction module;

[0011] The data acquisition module includes sensors mounted on the vehicle handlebars and external devices for collecting the rider's physiological signals and vehicle control data. The physiological signals include eye movement indicators and physiological data, and the vehicle control data includes turning reaction time and wheel speed. The vehicle handlebars are detachable and can be fixed to the vehicle's handlebar connecting tube via a fixing buckle.

[0012] The vehicle-mounted handle is generally in the shape of a hexagonal prism, which can amplify the rider's interaction with the handle;

[0013] The data processing module is used to standardize the data obtained by the data acquisition module and classify the real-time cycling cognitive load level through a support vector machine;

[0014] The navigation interaction module uses a vibrator mounted on the vehicle handlebar to remind the rider of his / her status according to the real-time riding cognitive load level obtained through classification.

[0015] In the data acquisition module, the sensors include a heart rate detector and a gyroscope, and the external devices include a head-mounted eye tracker and a Hall sensor;

[0016] Among the sensors, the heart rate detector is installed on the outside of the vehicle handlebars, at the contact point between the fingers and the handlebars, and includes a heart rate measurement function for detecting the rider's heart rate; the heart rate data is transmitted to the CPU processor via an external Arduino development board;

[0017] The gyroscope is installed inside the vehicle handlebars to detect the actual turn start time, compare the actual turn start time with the ideal turn time in the navigation route, calculate the difference, and transmit the difference data between the actual and ideal turn times to the CPU processor via an external Arduino development board;

[0018] The handlebars also include an LED light module, a magnetic charging module, a positioning module, a battery, a battery cover, an emergency charging port, a power button, and a housing cover. The LED light module provides lighting effects or indication functions, such as a headlight or taillight for nighttime riding, or indicates the device's operating status. The magnetic charging module facilitates charging, typically using a magnetic connection to connect the charger and device to simplify the charging process and enhance the user experience. The positioning module determines the device's location. For example, a Global Positioning System (GPS) module uses satellite positioning to obtain precise longitude and latitude information for navigation, location tracking, and other functions. The battery provides power to the device, enabling independent operation, typically through recharging or battery replacement. The battery cover protects and secures the battery, facilitating battery replacement or maintenance. The emergency charging port allows the device to be connected to an external power source for charging when the internal battery is depleted, allowing continued use. The power button controls the device's on / off state; users can turn the device on or off by pressing or toggling the button. Shell cover: used to protect the various modules and circuits inside the device, while providing aesthetic appearance and mechanical protection.

[0019] Among the external devices, the head-mounted eye tracker is worn on the user's head in the form of glasses to collect eye movement indicators. The eye movement indicators that can be obtained include gaze time, number of gazes, NNI (nearest neighbor index), Sis (saccadic intrusions), scan frequency, scan duration, scan amplitude, scan speed, blink frequency, blink duration, blink interval, etc.

[0020] The fixation time refers to the duration of fixation during the experiment, which reflects the difficulty of the subjects in extracting information and their attention allocation.

[0021] The number of fixations refers to the number of fixations that occurred during the experiment. This indicator reflects the subject's familiarity with the object of fixation and the complexity of the object, and is often used to measure the psychological load on the subject.

[0022] The NNI reflects the randomness of the distribution of fixations;

[0023] Sis refers to the fact that the eyeball does not stop moving during fixation. The fixation process is accompanied by subtle eye movements, which causes the image position of the object on the retina to continuously change to avoid visual disappearance due to retinal adaptation to stimulation. This subtle change in the fixation point is called fixation eye movement, which includes tremor, drift, microsaccades, and saccadic intrusions.

[0024] The scanning frequency refers to the number of times scanning activities occur per unit time;

[0025] The saccade duration is the time required for the eyes to move from one fixation point to the next;

[0026] The sweep amplitude refers to the range covered by the sweep process from the beginning to the end, usually expressed in visual angles;

[0027] The saccade speed refers to the ratio of saccade amplitude to saccade duration;

[0028] The blink frequency refers to the number of blinks that occur per unit time;

[0029] The blink duration refers to the duration of a single blink activity;

[0030] The blink interval refers to the time interval between two adjacent blink activities.

[0031] In the external device, the Hall sensor is installed on the outside of the pedal wheel shaft to detect the wheel speed. The obtained wheel speed data is transmitted to the CPU processor through the external Arduino development board.

[0032] The interactive control system also includes an Arduino prototype platform based on easy-to-use hardware and software and a CPU processor; the Arduino prototype platform includes an Arduino development board and an Arduino integrated development environment; the data obtained by the sensors and the external devices are transmitted to the CPU processor through the Arduino development board, and the classification results obtained after CPU processing send vibration instructions to the vibrator (vibration module) on the vehicle handle through the Arduino development board.

[0033] The present invention also proposes a multi-sensory channel cycling navigation interactive control method based on eye movement data implemented by the above-mentioned interactive control system, the method comprising the following steps:

[0034] Step 1: Using sensors on the handlebars and external devices, collect the rider's physiological signals and vehicle control data, which are subsequently used to determine the rider's cognitive load level;

[0035] Step 2: normalizing the cyclist's physiological signals and vehicle control data collected in Step 1;

[0036] Step 3: Use support vector machine (SVM) to classify the standardized data;

[0037] Step 4: Based on the classified cognitive load level, a vibrator provided on the handlebar is used to output real-time vibration feedback to remind the rider to adjust the cognitive load state.

[0038] In step 1, the sensors include a heart rate monitor and a gyroscope, the external device includes a head-mounted eye tracker and a Hall sensor; the physiological signals include eye movement indicators and physiological data, and the vehicle control data includes turning reaction time and wheel speed;

[0039] In step 2, the StandardScaler class in Python was used to normalize the cyclist's physiological signals and vehicle control data, converting the data into a standard normal distribution with a mean of 0 and a variance of 1. By subtracting the mean of each feature from the value and then dividing it by the standard deviation, the values ​​of different features can be compared on the same scale, preventing the model weights from being biased towards certain features due to numerical differences between the features.

[0040] In step three, we use the support vector machine (SVM) and the "one vs rest" strategy to train multiple SVM classifiers to transform the multi-category problem into a set of multiple two-category problems. Each problem is to distinguish one category from other categories, and the output of each classifier is used as the judgment criterion to determine the category of the test sample.

[0041] In step 3, a three-class SVM classification model for the cyclist's cognitive load level is finally established. Three classifiers, SVM1, SVM2, and SVM3, are used to perform two-class classification at each layer. The two-class classification results at each layer are as follows:

[0042] The first layer of two-class classification: SVM1,

[0043] Category 0 (moderate level) vs. other categories (higher level, overload level);

[0044] The second layer of two-class classification: SVM2,

[0045] Category 1 (higher level) vs. other categories (moderate level, overload level);

[0046] The second-class classification of the third layer: SVM3,

[0047] Category 2 (overload level) vs. other categories (moderate level, high level);

[0048] By using three binary classifiers to form a SVM hierarchical combination classifier, a three-level classification of cognitive load levels is achieved, with levels ranging from {0} to {2}, representing moderate cognitive load, high cognitive load, and overloaded cognitive load, respectively. A decision equation is used to calculate the current cognitive load status of the rider, and the result is compared with the set threshold.

[0049] Training the model: A support vector machine (SVM) model is trained using a labeled dataset. This includes input features (such as cyclist physiological signals and vehicle handling data) and corresponding labels (indicating different levels of cognitive load). The training process learns the model's parameters and thresholds.

[0050] Decision function and threshold: After training, each binary classifier (SVM1, SVM2, SVM3) will have a decision function. The decision function calculates the classification score or distance of the test sample and divides it into different categories based on the threshold.

[0051] Classification Result Evaluation: For a given test sample, input it into the decision function of each binary classifier to obtain a set of decision results. These results can be classification scores, confidence levels, or other metrics. Based on the threshold definition, this set of decision results is compared with the threshold.

[0052] Determine cognitive load level: Based on the threshold comparison results, we can determine which cognitive load level the test sample belongs to. The specific determination method may vary depending on the model design, for example:

[0053] If the decision function output value is greater than the threshold, the cyclist's cognitive load level is considered to be overloaded.

[0054] If the decision function output value is equal to the threshold, the cyclist's cognitive load level is considered to be high.

[0055] If the decision function output value is less than the threshold, the cyclist's cognitive load level is considered to be moderate.

[0056] The decision equation is as follows:

[0057] f(x)=w T x+b

[0058] Among them, w represents the weight of the feature vector, that is, the decision boundary learned during the training process; x represents the feature vector of the data to be classified, that is, a multidimensional vector composed of physiological data and eye movement indicators, whose specific values ​​will affect the classification results; b represents the offset, which can be regarded as the threshold of the classifier, used to adjust the sensitivity and specificity of the classification and determine the position of the sample interface.

[0059] In step 4, a vibration prompt is performed based on the classification result of the support vector machine (SVM); the vibration prompt includes three dimensions: vibration intensity, duration, and triggering timing;

[0060] When the SVM classification result is {2}, that is, the cognitive load level is in an overload state, the vibration feedback range of the vehicle handle is between 1-255, and the vibration feedback with the highest intensity value of 255 is continuously output, and no vibration stop time is set;

[0061] When the SVM classification result is {1}, that is, when the cognitive load level is high, vibration feedback is given according to the upcoming riding state. When the front is about to enter a left turn, vibration feedback is output on the left handlebar (vibration intensity value is 255, duration is 3 seconds); when the front is about to enter a right turn, vibration feedback is output on the right handlebar (vibration intensity value is 255, duration is 3 seconds); when the front is about to enter a straight intersection, vibration feedback is output on both handlebars at the same time (vibration intensity value is 150, number of vibrations is 2, single duration is 0.3 seconds, and the interval between two vibrations is 0.3 seconds);

[0062] When the SVM classification result is {0}, that is, the cognitive load level is moderate, route-based vibration reminders are performed at 50m and 0m away from the intersection: when the front is about to enter a turning intersection, vibration feedback is output on the vehicle handle on the corresponding turning side (vibration intensity value is 255, duration is 3 seconds); when the front is about to enter a straight intersection, vibration feedback is output on the vehicle handles on both sides at the same time (vibration intensity value is 150, number of vibrations is 2 times, single duration is 0.3 seconds, and the interval between two vibrations is 0.3 seconds).

[0063] The present invention also proposes the application of the above-mentioned interactive control system or interactive control method in multi-sensory channel cycling navigation interactive control.

[0064] The beneficial effects of the present invention include:

[0065] 1. This invention uses a head-mounted eye tracker to detect the cyclist's eye movements in real time, employing eye movement data to intuitively reflect the cyclist's cognitive resource allocation. This is supplemented by sensors monitoring heart rate, turning reaction time, and wheel speed, providing real-time insight into the cyclist's cognitive load during cycling.

[0066] 2. The present invention adopts multi-sensory channels as the feedback method for cycling navigation interaction. Navigation information is transmitted through multi-sensory channels, which improves the efficiency of information transmission with an applicable interactive method and effectively reduces the cognitive load level of users in situations where road conditions change during cycling.

[0067] 3. This invention utilizes a flexible mode of dynamic navigation feedback. Based on real-time cognitive load detection during cycling, it provides tiered vibration feedback for route changes. The timing of navigation feedback varies with the rider's cognitive load, flexibly adapting to various cycling scenarios. When a rider experiences high cognitive load, the navigation feedback designed in this invention can adjust the level to help the rider return to a normal cognitive load.

[0068] 4. The present invention uses the SVM support vector machine model to evaluate and calculate the cognitive load level. This technology overcomes the shortcomings of traditional subjective evaluation methods, such as poor real-time performance, and enables the cognitive load level to be calculated and fed back in real time, thereby enhancing the real-time detection of the cyclist's riding status.

[0069] 5. The present invention achieves a combination of software and hardware interaction. The multi-sensory channel cycling navigation interaction control system based on eye movement data is installed in a cycling vehicle grip product in a modular design. It provides a systematic solution combining software and hardware for improving the cycling navigation interaction system, and can provide methodological reference and reference for navigation interaction research in other fields. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without paying any creative work.

[0071] Figure 1 This is a schematic diagram of the multi-sensory channel cycling navigation interactive control system and process based on eye movement data of the present invention.

[0072] Figure 2a 、 2b This is a design rendering of the vehicle-mounted handle and the hand-held handle of the present invention.

[0073] Figure 3 These are the top view and side view of the single-sided vehicle-mounted handle of the present invention.

[0074] Figure 4 This is a schematic diagram of the internal structure of the vehicle-mounted handle of the present invention.

[0075] Figure 5 Schematic diagram of the method for collecting various data by the vehicle handle of the present invention.

[0076] Figure 6 This is a step diagram of the classification results of different SVM real-time cognitive load levels of the present invention.

[0077] Figure 7 Schematic diagram of the virtual simulation interactive environment used in the experiment of the present invention.

[0078] Figure 8 It is an actual cycling route map in an embodiment of the present invention.

[0079] Figure 9 It is a virtual map route design diagram in an embodiment of the present invention.

[0080] Figure 10 This is a schematic diagram of navigation positioning synchronized with the first-person perspective position in an embodiment of the present invention.

[0081] Figure 11 Schematic diagram of the Yesoul M1 exercise bike used in an embodiment of the present invention.

[0082] Figure 12 This is a diagram of the experimental material structure in an embodiment of the present invention, a circuit diagram of the Hall sensor connected to the wheel, and a simulation experiment scene diagram.

[0083] Figure 13 This is a schematic diagram showing the impact of three types of vibration feedback on user experience in an embodiment of the present invention. In the figure, lower scores represent better user experience.

[0084] Figure 14 Schematic diagram of the effects of three types of vibration feedback on scene memory in an embodiment of the present invention. The figure shows that the navigation mode of advance vibration improves the level of scene memory (1, 2, and 3 correspond to the no vibration group, the current vibration group, and the advance vibration group, respectively).

[0085] Figure 15 This figure shows the effects of pre-vibration experiments with and without visual feedback on scene memory (number 3 represents the pre-vibration group without visual feedback, and number 6 represents the pre-vibration group with visual feedback) in an embodiment of the present invention. The figure shows that scene memory scores for the group with visual guidance were slightly lower than those without visual guidance.

[0086] Figure 16 Schematic diagram of the effect of presence or absence of visual feedback on interaction performance in an embodiment of the present invention (1 represents a group without visual feedback, and 2 represents a group with visual feedback).

[0087] Figure 17 Schematic diagram of the effect of vibration feedback on interaction performance in an embodiment of the present invention (1 represents no vibration, 2 represents current vibration, and 3 represents early vibration).

[0088] Figure 18 Schematic diagram of the impact of various navigation feedback methods on interaction performance in an embodiment of the present invention.

[0089] Figure 19 3 is a schematic diagram of the effect of the presence or absence of visual feedback on the total gaze time of the landscape area in an embodiment of the present invention. In the figure, 1 and 2 correspond to the "non-visual feedback group" and the "visual feedback group", respectively.

[0090] Figure 20 3 is a schematic diagram of the effect of the presence or absence of visual feedback on the total gaze time of the landscape area in an embodiment of the present invention. In the figure, 1 and 2 correspond to the "non-visual feedback group" and the "visual feedback group", respectively.

[0091] Figure 21 3 is a schematic diagram of the effect of the presence or absence of visual feedback on RV (ratio of AOI fixation duration of the road area to the landscape area) in an embodiment of the present invention. In the figure, 1 and 2 correspond to the "non-visual feedback group" and the "visual feedback group", respectively.

[0092] Figure 22 This is a schematic diagram of the impact of different types of "vibration feedback" on the "AOI visit count ratio of the road area to the landscape area" (RV) in an embodiment of the present invention (1 indicates no vibration, 2 indicates current vibration, and 3 indicates early vibration).

[0093] Figure 23 This is a typical road segment eye movement heatmap from an embodiment of the present invention. From left to right, the first, second, and third columns correspond to the beginning, middle, and end of the road segment, respectively; from top to bottom, the first, second, and third rows correspond to the three experimental conditions of "no vibration," "current vibration," and "pre-vibration," respectively. DETAILED DESCRIPTION

[0094] The present invention is further described in detail with reference to the following specific examples and accompanying drawings. The processes, conditions, experimental methods, etc. for implementing the present invention, except for those specifically mentioned below, are common knowledge and common common sense in the art and are not particularly limited by the present invention.

[0095] The present invention provides a multi-sensory channel cycling navigation interactive control system based on eye movement data. The present invention assesses and calculates the rider's cognitive load level based on eye movement and physiological data, providing real-time understanding of the rider's current cognitive load level. The present invention classifies the collected data based on a support vector machine multi-classification model to obtain a three-category qualitative assessment model for cognitive load. Based on the rider's real-time status detected by the model (moderate cognitive load level, high cognitive load level, overloaded cognitive load level), a targeted dynamic navigation interaction mode is implemented to assist in returning the rider's cognitive load level to a moderate state, thereby improving the cycling experience and the flexibility of the navigation interaction system.

[0096] Specifically, in the present invention, a moderate cognitive load level means that the rider's cognitive load is at a normal level, and the rider can pay attention to road conditions and navigation instructions at the same time without being distracted or fatigued due to too light or too heavy cognitive load; a high cognitive load level means that the rider's cognitive load is high, requiring more attention to process road conditions and navigation instructions, but still able to maintain a relatively stable riding state; an overloaded cognitive load level means that the rider's cognitive load exceeds the limit of his or her processing capacity, resulting in serious distraction or fatigue, and the rider needs to reduce the cognitive load and return to a moderate state to ensure riding safety and comfort.

[0097] See also Figure 1 The multi-sensory cycling navigation interactive control system based on eye movement data provided by the present invention consists of three modules: a data acquisition module, a data processing module, and a navigation interaction module. The data acquisition module is primarily supported by various sensor hardware installed on the vehicle's handlebars, as well as external devices such as a heart rate monitor and gyroscope. These external devices include a head-mounted eye tracker and Hall effect sensors. These devices can acquire physiological signals from the rider, including eye movement indicators and physiological data, as well as vehicle control data, including turning reaction time and wheel speed.

[0098] The vehicle-mounted handle is also provided with an LED light module, a magnetic charging module, a positioning module, a battery, a battery flip cover, an emergency charging port, a power button, and a shell flip cover.

[0099] By utilizing the above-mentioned multi-sensory cycling navigation interactive control system based on eye movement data, the present invention solves the problem in the prior art that the cognitive load level of cyclists is easily increased when the road conditions and environment change. The technical solution adopted includes the following steps:

[0100] Step 1: Data Collection

[0101] The data acquisition module is responsible for collecting data, including cyclist physiological signals and vehicle operation data. Physiological signals include eye movement indicators such as fixation duration, number of fixations, NNI, SIs, saccade frequency, saccade duration, saccade amplitude, saccade speed, blink frequency, blink duration, and inter-blink interval. Physiological data primarily includes cardiac activity indicators, including heart rate and heart rate variability.

[0102] Vehicle control data includes turning reaction time and wheel speed. The specific definitions of each indicator are shown in Table 1:

[0103] Table 1 Definition of data indicators of cyclist's physiological signals and vehicle operation data

[0104]

[0105] Among them, fixation duration, number of fixations, NNI, and SIs are fixation indicators. Fixation refers to the fovea focusing on an object for a period of time exceeding a certain limit, during which the object is mapped onto the fovea, forming a clear image. This time threshold is usually set at 200ms. That is, if the fovea focuses on an object for more than 200ms, the eye movement behavior can be defined as fixation. Saccadic frequency, saccade duration, saccade amplitude, and saccade speed are saccadic indicators. Saccadic eye movement refers to the rapid movement of the eye from one fixation point to the next. The duration of a saccadic eye movement at a point is typically 20 to 200ms, with a transfer speed of at least 50 degrees per second. Saccades can obtain spatiotemporal information but cannot form a relatively clear image. Their function is to shift the fixation target and move the new fixation target into the central visual range. Blink frequency, blink duration, and blink interval are blink indicators. Blinking refers to the phenomenon of upper and lower eyelids touching each other. Blinking is closely related to human psychological activities. Studies have found that when subjects look at objects of interest or handle more complex tasks, they will allocate more attention to task-related stimuli, thereby inhibiting the occurrence of blinking activities. This phenomenon is called blink inhibition.

[0106] The values ​​described in the present invention are all based on the time from the current collection moment to 10 seconds before the current collection moment during continuous riding, and the total value and average value within the 10 seconds are calculated.

[0107] The data involved in the above table are collected by the vehicle handle with sensors and some peripherals (such as head-mounted eye tracker, Hall sensor, etc.). The vehicle handle is one of the hardware carriers of the navigation interaction system in the present invention. Its component design is as follows: Figure 2a 、 2b and Figure 3 shown.

[0108] In the present invention, the gripping handle is in the shape of a hexagonal prism. Considering the human-machine relationship, compared with a cylinder, the contact surface of the prism surface is larger and the friction is greater, and the protruding edges of the prism surface can amplify the tactile sensation caused by vibration, so this shape design is adopted.

[0109] The vehicle handle and peripherals collect various data in the following ways: Figure 5 The specific description is as follows:

[0110] Eye movement indicators: Eye movement behavior is captured by an eye tracker. Currently, the eye tracker is not installed on the vehicle handlebar. A wearable eye tracker is used, which is worn on the user's head in the form of glasses. In the future, it can also be considered to be integrated into a suitable position on the vehicle handlebar.

[0111] The eye movement indicators obtained by the eye tracker include fixation time, number of fixations, NNI, SIs, saccade frequency, saccade duration, saccade amplitude, saccade speed, blink frequency, blink duration, and blink interval.

[0112] Heart rate: This is based on a heart rate monitor installed on the outside of the handlebars, where the fingers meet the handlebars. The heart rate monitor uses its heart rate detection function to measure the rider's heart rate. The heart rate data is transmitted to the CPU via an Arduino development board.

[0113] Wheel speed: The wheel speed is detected by a Hall sensor installed on the outside of the pedal axle; the obtained wheel speed data is transmitted to the CPU processor through the Arduino development board.

[0114] Turn reaction time: The actual turn start time is detected by a gyroscope installed inside the vehicle's handlebars. The actual turn start time is compared with the ideal turn time in the navigation route, and the difference is calculated. The difference between the actual and ideal turn times is transmitted to the CPU processor via the Arduino development board.

[0115] Arduino is an open-source prototyping platform based on easy-to-use hardware and software. It consists of a programmable circuit board (microcontroller) and readily available software called the Arduino IDE (Integrated Development Environment) for writing and uploading computer code to the physical board. Arduino provides a standard form factor that packages the functionality of a microcontroller into an easier-to-use software package. The Arduino development board can read analog or digital input signals from various sensors and convert them into outputs. The data transmission in this invention is performed by the Arduino; the Arduino development board is currently connected to the sensor via a wired connection, serving as the data transmission hub.

[0116] A vibrator is also installed at the armrest position of the vehicle handle. The vibrator refers to the vibration motor module, which can be directly controlled through the digital port of Arduino. The vibration intensity of the motor can be controlled by PWM. This module can easily complete the conversion of electrical signals to mechanical vibrations. It is suitable for the production of vibration interactive products. When its input is high level, the motor will vibrate, and when it is low level, the motor remains stationary.

[0117] PWM (Pulse Width Modulation) is a technique used to control analog signals by varying the width of the signal's pulses to control the average value of the output level. The PWM signal has a fixed period, but the pulse width can be adjusted as needed.

[0118] In Arduino, PWM signals are generated through digital pins. Specific pins on the Arduino development board are marked with a "~" symbol (called baud rate pins) that support PWM output. The Arduino's PWM output pins produce an analog effect, allowing us to control the output voltage level by varying the pulse width.

[0119] In the present invention, the Arduino is used to control the input and output modules as follows:

[0120] (1)Arduino control process:

[0121] Table 2 Variables involved in the Arduino control process

[0122]

[0123] Define USE_ARDUINO_INTERRUPTS to true

[0124] Include the PulseSensorPlayground library and create the PulseSensorPlayground object pulseSensor

[0125] Define constants VIB_PIN_R, VIB_PIN_L, BTN_PIN_R, BTN_PIN_L, HALL_PIN, PULSE_PIN, PULSE_LED, and PULSE_THRES for vibration motor and button pins

[0126] Define debounce time constants DEBOUNCE_INT and RPM_DIV

[0127] Define variables intensity, duration, vib_lr, vib_ts, buf, and buf_idx

[0128] Defines state constants STATE_IDLE, STATE_CMD, STATE_VIB_RDY, and STATE_VIB, and state variables state, btn_state_l, btn_state_r, btn_last_state_l, btn_last_state_r, hall_last_ts, and state_hall

[0129] Create a function arr2int() to convert a character array to an integer

[0130] Create the hallISR() function, which is triggered by the interrupt and sets the state_hall state variable to 1

[0131] In the setup() function:

[0132] -Open the serial port and set the baud rate to 115200

[0133] - Initialize the modes for the vibration motor, button, and pulse sensor pins

[0134] - Set the properties of the pulse sensor object to the sensor pin, pulse LED pin, and threshold

[0135] - Start the pulse sensor object

[0136] In the loop() function:

[0137] - Check if there is data available on the serial port, and read the data if there is

[0138] - If a newline character is read, reset the character variable c and buf_idx variable to 0 and return

[0139] - If 'L' or 'R' is read, set the state to STATE_CMD, set the vib_lr variable to VIB_PIN_R or VIB_PIN_L, reset the character variable c to 0, and return

[0140] - If the state is STATE_CMD and a valid character is read, the character is added to the buf array. When the buf array size is 7, it is converted into intensity and duration variables. Then the state is set to STATE_VIB_RDY, buf_idx is reset to 0, and 'OK' is sent to the serial port.

[0141] - If the state is STATE_VIB_RDY, set vib_ts to the current number of milliseconds, output the intensity value on the vib_lr pin, and set the state to STATE_VIB

[0142] - If the state is STATE_VIB, calculate the difference between the current time and vib_ts. If it is greater than or equal to the duration variable, set the vib_lr pin to 0, set the state to STATE_IDLE, and then return. - Read the state of the button pin and check if there is a state change. If so, set the debounce_l or debounce_r variable to the current milliseconds.

[0143] - If the difference between the value of the debounce_l or debounce_r variable and the current time is greater than or equal to the DEBOUNCE_INT constant, it means that the debounce time has passed, then the value of the btn_state_l or btn_state_r variable is updated to the current state value. If the state value is LOW, "BL" or "BR" is sent to the serial port.

[0144] - Update the values ​​of btn_last_state_l and btn_last_state_r variables to the current button state

[0145] - If state_hall is 1, send "HALL" to the serial port and set state_hall to 0

[0146] - Detect whether the pulse sensor object detects the start of the pulse. If detected, get the current heart rate value and send it to the serial port

[0147] The basic idea behind this program is to receive control commands from the vibration motor via serial communication and control the intensity and duration of the vibration accordingly. The program also detects button status and heart rate data from the pulse sensor and sends this information to the serial port for interaction with other devices.

[0148] The process of classification training using support vector machine SVM is as follows:

[0149] (2) SVM classification process:

[0150] -Import necessary Python libraries

[0151] -Read Excel data files and convert the data into pandas DataFrame format

[0152] -Select the required feature data (heart rate, speed, scanning angle, number of blinks, etc.) from the DataFrame and perform normalization

[0153] - Use the train_test_split function to divide the data into training and test sets

[0154] -Create a support vector machine classifier object clf, use the radial basis function as the kernel function, and train it using the training set

[0155] -Test the accuracy of the model on the test set

[0156] -Use the predict function to predict the data in the test set and store the prediction results in the pred_target variable

[0157] Step 2: Data Preprocessing

[0158] Since various data indicators reflect the cyclist's status in different dimensions within different numerical ranges, the numerical ranges and units of each indicator are different, and the interactive relationships between individual indicators and other related indicators are also different, the present invention analyzes and calculates data from these multi-sensory dimensions to quantify and output a unified cycling cognitive load level.

[0159] In the data processing module, the received eye movement indicators and physiological data must first be standardized and the values ​​​​are unified in terms of dimension. The calculation function code is as follows, where X is the set of the above eye movement indicators and physiological data:

[0160] X=StandardScaler().fit_transform(X)

[0161] This line of code uses the `StandardScaler` class to normalize the selected feature data `X`, transforming it into a standard normal distribution with mean 0 and variance 1. Specifically, it subtracts the feature's mean from each feature value and divides it by its standard deviation. For each feature, the following calculation is performed: $x' = \frac{x-\mu}{\sigma}$, where $x$ is the original data, $\mu$ is the feature's mean, and $\sigma$ is the feature's standard deviation. This data processing allows comparisons between different features on the same scale, preventing model weights from being biased due to numerical differences between features.

[0162] Step 3: Calculate cognitive load level based on support vector machine

[0163] Support vector machine (SVM) is a two-class classification model. The two-class classification decision equation is:

[0164] f(x)=w T x+b

[0165] In the present invention, a support vector machine is used to classify physiological data and eye movement indicators on a mobile device in order to classify the cyclist's cognitive load level.

[0166] -w represents the weight of the feature vector, that is, the decision boundary learned during training;

[0167] -x represents the feature vector of the data to be classified, that is, a multidimensional vector composed of physiological data and eye movement indicators. Their specific values ​​will affect the classification results;

[0168] -b represents the offset, which can be regarded as the threshold of the classifier. It is used to adjust the sensitivity and specificity of the classification and determine the position of the sample interface.

[0169] In the application of the present invention, w, x, and b represent the weight of the feature vector, the feature vector of the data to be classified, and the offset of the classifier, respectively. These information have an important impact on the final classification result.

[0170] The model is trained using the previously collected data as training data. Here, the Gaussian kernel function is used for training, and the "one vs rest" multi-layer combination classification algorithm in the multi-classification method is used to perform label matching of the cognitive load level.

[0171] The Gaussian kernel function, also known as the radial basis function (RBF), is a kernel function in the support vector machine (SVM). Its function is to map the original data into a high-dimensional space, thereby making linearly inseparable data linearly separable, making it easier to classify. In the present invention, a Gaussian kernel function is used to perform nonlinear mapping on the input physiological data and eye movement indicators, thereby converting the data into a form that can be processed by the support vector machine. At the same time, when performing multi-classification tasks, the "one vs rest" strategy is used to treat each category as a binary classification problem, thereby achieving multi-category classification.

[0172] "One vs rest" is a common multi-class classification method, also known as "one vs all" or "one vs others". It converts a multi-class problem into a collection of multiple two-class problems, each of which is to distinguish one class from the others.

[0173] In this approach, a classifier is first built for each category. For the i-th classifier, the i-th category is labeled as a positive example and all other categories are labeled as negative examples. A binary classification model is then trained to distinguish between the two categories. The output of the classifier is a probability or score indicating the degree to which a given example is a positive example.

[0174] When making a prediction, the test sample is fed into all classifiers, and each classifier outputs a probability or score for belonging to that category. These outputs are then used to determine which category the test sample belongs to. Specifically, the output of each classifier is compared to a threshold; if it exceeds the threshold, the test sample is classified as belonging to that category.

[0175] In the present application, a "one vs rest" multi-layer combined classification algorithm is used to identify the label of cognitive load level. This method is based on support vector machine (SVM) and through training multiple SVM classifiers, the output of each classifier is used as the judgment standard to judge the class of test samples. In this way, accurate identification of multiple cognitive load levels is achieved.

[0176] The following code describes the process of training and testing the accuracy of the classification algorithm using the data set.

[0177] Xtrain, Xtest, Ytrain, Ytest = train_test_split(X, Y, test_size=0.3, random_state=500)

[0178] clf = SVC(kernel="rbf", gamma="auto", degree=1, decision_function_shape='ovr').fit(Xtrain, Ytrain)

[0179] print("The accuracy is%f"%(clf.score(Xtest, Ytest)))

[0180] This code mainly implements the following functions:

[0181] 1. Use the train_test_split function to divide the data set X and Y into training set Xtrain and Ytrain, test set Xtest and Ytest, where the test set accounts for 0.3, and the random number seed is 500, to achieve random division of the data set.

[0182] 2. Use the SVM model to train the training set Xtrain and Ytrain, where the kernel function used is the Gaussian kernel function, and the decision function uses the "one vs rest" multi-layer combined classification algorithm.

[0183] 3. Use the trained model clf to predict the test set Xtest and calculate the accuracy.

[0184] Where the meanings of the parameters are:

[0185] -X: input data set, containing multiple features.

[0186] -Y: input data set, containing corresponding labels.

[0187] -test_size: test set proportion.

[0188] -random_state: random number seed.

[0189] -kernel: kernel function type.

[0190] -gamma: kernel function coefficient.

[0191] -degree: The degree of the polynomial kernel function.

[0192] -decision_function_shape: multi-classification strategy.

[0193] -Xtrain: training set features after division.

[0194] -Ytrain: The training set label after division.

[0195] -Xtest: Test set characteristics after division.

[0196] -Ytest: The test set label after division.

[0197] -clf: trained SVM classifier.

[0198] -score: Calculates the accuracy of the classifier.

[0199] For the standardized data, a three-class SVM model for cyclists' cognitive load levels was constructed. Three classifiers, SVM1, SVM2, and SVM3, were used for binary classification at each layer. A hierarchical SVM combined classifier, constructed using three binary classifiers, achieved a three-class classification of cognitive load levels. Levels ranged from {0} to {2}, respectively, indicating moderate cognitive load, high cognitive load, and excessive cognitive load. Calculations were performed using the decision equation, and the results were compared with a set threshold to determine the cyclist's current cognitive load status.

[0200] The trained support vector machine model is used to perform predictive analysis on the cyclist's real-time data, and the SVC.predict() function is used to match the current cognitive load level in {0}{1}{2}.

[0201] The `SVC.predict()` method is a method of the Support Vector Machine Classifier (SVC) object that is used to predict the classification of new data samples. Its principle is based on the trained support vector machine model for prediction.

[0202] A support vector machine (SVM) is a supervised learning algorithm used to solve binary and multi-classification problems. Its core concept is to separate samples of different categories by constructing a hyperplane. During the training phase, the SVM learns a decision boundary that maximizes the separation between samples of two different categories and finds support vectors (sample points closest to the decision boundary). SVMs can use different kernel functions (such as linear, polynomial, and Gaussian kernels) to process different types of data.

[0203] In the prediction phase, the `SVC.predict()` method takes in new data samples and then uses the trained support vector machine model to perform classification predictions. The specific prediction principle is as follows:

[0204] 1. For the new data sample input, it will go through the same feature preprocessing process (such as normalization) as the training data.

[0205] 2. Next, the method maps the new data samples into the feature space of the support vector machine model.

[0206] 3. Based on the decision boundaries and hyperplanes learned by the support vector machine model, the `SVC.predict()` method compares the new data samples with these decision boundaries.

[0207] 4. Finally, the `SVC.predict()` method assigns the new data sample to a specific class label based on its position on the decision boundary, thus completing the classification prediction.

[0208] The prediction process of the support vector machine model relies on the decision boundary and support vector information learned during the model training phase. By mapping the features of input data samples and comparing them with the decision boundary, the support vector machine model can assign new data samples to different categories and achieve classification prediction.

[0209] Step 4: Dynamic Navigation Interaction

[0210] Based on the existing navigation system's fixed-distance reminder trigger mechanism, the method described in this paper detects changes in the rider's real-time cognitive load and optimizes the existing trigger mechanism. In the navigation interaction module, a vibrator module in the onboard handlebars provides real-time vibration feedback to the rider, with vibration prompts divided into different levels, differentiated by vibration intensity, duration, and trigger timing.

[0211] The vibrator module is installed at the armrest position of the vehicle handle.

[0212] When the SVM classification result is {2}, the cognitive load level is in an overload state, and the rider is in an overload state. This state is not enough to continue to maintain safe riding behavior, and the rider has realistic conditions such as fatigue riding, health damage, or visual obstruction. The navigation gives a warning reminder, and the vibration feedback range of the vehicle-mounted handle is between 1-255, at this time the highest intensity 255 value vibration feedback is continuously output, and the vibration stop time is not set, reminding the rider to stop and rest or to stop the riding process to ensure safety.

[0213] When the SVM classification result is {1}, the cognitive load level is high, and the rider is in a state of high concentration of attention. This state is easy to cause tunneling of attention, and the rider will focus all attention on distinguishing road conditions and routes, or be attracted by a particular landscape for a long time, easily ignoring other landscapes and road events, and being in a tense high-energy state, which is difficult to maintain for a long time. If the rider is within the road condition reminder distance (within 200m of the intersection), it is judged whether to enter this state, and if the cognitive load level is high, the navigation gives a vibration reminder: the vibration reminder is based on the route; if the current front is about to enter a left turn intersection, vibration feedback is output on the left vehicle-mounted handle (vibration intensity value is 255, duration is 3 seconds); if the current front is about to enter a right turn intersection, vibration feedback is output on the right vehicle-mounted handle (vibration intensity value is 255, duration is 3 seconds); if the current front is about to enter a straight intersection, vibration feedback is output on both vehicle-mounted handles (vibration intensity value is 150, vibration times is 2, single duration is 0.3 seconds, and interval time between two vibrations is 0.3 seconds).

[0214] When the SVM classification result is {0}, the cognitive load level is moderate, and the rider is in the best balanced state. Its attention allocation is sufficient to maintain a safe riding process, and has enough remaining cognitive resources to appreciate the landscape, thereby maintaining a leisurely attitude and good sports experience of riding. In this state, the navigation gives a fixed feedback, and vibration reminders based on the route are given at distances of 50m and 0m from the intersection: if the current front is about to enter a turn intersection, vibration feedback is output on the corresponding turn side vehicle-mounted handle (vibration intensity value is 255, duration is 3 seconds); if the current front is about to enter a straight intersection, vibration feedback is output on both vehicle-mounted handles (vibration intensity value is 150, vibration times is 2, single duration is 0.3 seconds, and interval time between two vibrations is 0.3 seconds).

[0215] Fifth step, interactive performance verification

[0216] The dynamic feedback mode of the navigation has a positive effect on the interactive performance of the rider, and the judgment index is the cognitive load level and the turn reaction time.

[0217] First, the navigation interaction module in the present invention outputs targeted interactive feedback based on the real-time data in the data processing module. Its interactive goal is to maintain the rider's cognitive load at a moderate level as much as possible to ensure traffic safety and experience comfort. Figure 6 As shown in the figure, when a rider's cognitive load is "high" or "overloaded," the corresponding graded navigation feedback is promptly adjusted back to a moderate level. This method solves the problem of existing navigation systems that ignore changes in the rider's status and provide fixed reminders, achieving real-time system perception of the rider and a collaborative human-machine interaction.

[0218] Second, the turning reaction time indicator indirectly verifies the interactive performance. This paper verifies the interactive performance through a virtual environment simulation experiment. The subjects were divided into groups, including Group A without navigation feedback, Group B with a single fixed navigation, Group C with multiple fixed navigations, and Group D with dynamic navigation interaction. The differences in turning reaction time of the subjects in the continuous riding state were compared. The various indicators are shown in Table 3 below:

[0219] Table 3 Turning reaction timetable for different riding conditions

[0220]

[0221] Subsequent experimental data shows that the multi-sensory cycling navigation interactive control system based on eye movement data described in the present invention has significantly improved the rider's cognitive load, improved cycling experience and navigation flexibility compared to traditional navigation.

[0222] Example

[0223] Experimental methods and experimental plan

[0224] Experimental design

[0225] This experiment used a 2×3 (2 factors, 3 levels) mixed experimental design, with the two factors being "visual feedback" and "vibration feedback." "Visual feedback" was divided into two levels: visual navigation and non-visual navigation, serving as the within-group factor; "vibration feedback" was divided into three levels: no vibration, immediate vibration, and anticipated vibration, corresponding to 0, 1, and 3 vibration sensations, respectively, serving as the between-group factor.

[0226] Forty-eight participants were divided equally into three groups (no vibration, immediate vibration, and pre-vibration), with 16 participants in each group. Each participant participated in two experimental tests, completing a cycling task in two scenarios: one with visual navigation and the other without. This ensured that there were at least 16 participants at each level. Using a combination of eye movement experiments and traditional qualitative methods, quantitative data was collected through eye movement experiments. A questionnaire was then used to assess the participants' spatial cognition experience. The accuracy of the eye movement data was tested, and conclusions were drawn through comprehensive analysis.

[0227] Participants and testing tools

[0228] This study involved 48 undergraduate and graduate students, including n male and n female subjects. All participants were required to have normal visual acuity (either uncorrected or corrected) of 1.0 or higher, be free of color blindness or color deficiency, and have not participated in similar eye movement experiments. The experiment used a Tobii Pro Nano eye tracker with a sampling frequency of 60 Hz. The monitor used a 23-inch Dell desktop computer with a resolution of 1920 × 1080 pixels and Windows 11. Unity software was used to create the virtual map and interactive scene, an Arduino development board was used to create the interactive hardware, and the Arduino IDE was used to program the hardware control program.

[0229] Experimental material preparation

[0230] The experiment refers to the cycling routes within 2 km of Shanghai Houtan Park (such as Figure 8 ), according to the complexity of the road conditions, a certain degree of simplification was carried out, and the basic route features, such as distance features, turning features, landscape features, etc., were retained and extracted as the route design of the virtual map (such as Figure 9 ). Therefore, the route of the virtual map is divided into three straight sections: long, medium and short, with a total of 6 turning intersections, covering both left and right turning operations. The road in the virtual map is a two-way two-lane road, and its surrounding landscape covers the characteristics of urban and suburban landscapes. The map was produced in Unity software. After the scene is run, the computer screen presents the first-person perspective of the cyclist, with a height of 1.6m. The riding position is the rightmost lane of the road. The riding speed simulates the average bicycle speed and moves forward at a constant speed. The total riding time from the starting point to the end point is 2 minutes and 47 seconds. In addition, this experiment is for a visual navigation group to record mobile phone navigation videos, and its positioning is synchronized with the first-person perspective position (such as Figure 10 ).

[0231] The experiments in the examples were conducted using a Yesoul M1 fitness bike (e.g. Figure 11), the cycling damping is fixed to keep the level of a half of a roll. In the experiment, the vibration sensor is pasted on the wrists of the subjects to output the vibration feedback. In addition, buttons are placed on the left and right handles of the exercise bike to record the turning operations of the subjects; a heart rate sensor is placed on the left handle to record the heart rate of the subjects when cycling; a Hall sensor and a magnet piece are placed on the wheel bearing to record the cycling speed of the subjects; the above accessories are connected by the Arduino UNO development board, and the data is transmitted to the computer and recorded and saved to the background of Unity (as shown in Figure 12 ); Figure 12 From left to right are respectively the experimental material structure diagram in the embodiment of the application, the Hall sensor circuit diagram connected at the wheel, and the simulation experiment scene diagram.

[0232] Experimental process

[0233] In the cycling process, the eye tracker is used to record the eye movement trajectory of the subjects watching the first-person perspective of the screen. Through the analysis of the heat map and eye movement data, combined with the operation data and questionnaire results of the subjects, the interactive situation of the subjects under each type of navigation mode is comprehensively analyzed, so as to obtain the influence of visual and vibration feedback on the spatial cognitive experience of the subjects.

[0234] Firstly, the main tester introduces the principle and precautions of the eye tracker to the subjects. After calibration, the formal experiment is started. The experimental tasks include: (1) operating the turning button according to the navigation feedback, and completing the cycling section; (2) answering questions, and completing a test related to the cycling experience.

[0235] Experimental results and analysis

[0236] Questionnaire scale analysis

[0237] In this experiment, the questionnaire scale is compiled according to the cycling process and the characteristics of the map. According to the overall experience level, the scene recognition and memory level, the sense of direction, the sense of distance, the sense of overall control, and other dimensions, the spatial cognitive experience of the subjects in the cycling process is comprehensively investigated. Taking "visual feedback" and "vibration feedback" as the independent variables, and taking the spatial cognitive experience investigated in the questionnaire scale as the dependent variable, the influence of the independent variables on the dependent variable is tested.

[0238] From the overall experience level, the vibration feedback brings a better experience, and the early vibration significantly improves the overall experience. As shown in Table 4, the "visual feedback" has no significant effect on the experience, while the "vibration feedback" has a significant effect on the experience, and there is no interaction between the two. Further one-way ANOVA analysis on the three levels of "vibration feedback" shows that, as shown in Table 5, the early vibration group has a significant effect on the experience compared with the no vibration group. Figure 13The lower the mid-value, the better the experience, indicating that the vibration feedback has a better improvement on the experience, especially in the pre-vibration group, the overall experience is the best.

[0239] Table 4 Test table of inter-subject effect of "visual feedback" and "vibration feedback"

[0240]

[0241] Table 5 Multiple comparison table between "vibration feedback"

[0242]

[0243]

[0244] *. The significance level of the mean difference is 0.05.

[0245] Note: Vibration feedback 1, 2, 3 corresponds to no vibration group, current vibration group, pre-vibration group

[0246] From the perspective of spatial cognition, pre-vibration helps to improve scene memory, but under the same conditions, visual navigation reduces the level of scene memory. Two-factor variance analysis is performed on "visual feedback" and "vibration feedback", as shown in Table 6, "vibration feedback" has a significant effect on scene memory. Further single-factor variance analysis is performed on the three levels in "vibration feedback", as shown in Table 7, pre-vibration group has a significant effect on scene memory, but there is no significant difference between current vibration group and no vibration group. Combined with Figure 14 illustration, pre-vibration navigation mode improves the level of scene memory. Descriptive analysis is performed on different "visual feedback" in pre-vibration level Figure 15 ), it is found that the scene memory score with visual navigation is slightly lower than that without visual navigation. It can be seen that visual navigation has caused a certain degree of interference to scene memory.

[0247] Table 6 Test table of inter-subject effect of "vibration feedback"

[0248]

[0249] Table 7 Multiple comparison table between "vibration feedback"

[0250]

[0251] *. The significance level of the mean difference is 0.05.

[0252] Note: Vibration feedback 1, 2, 3 corresponds to no vibration group, current vibration group, pre-vibration group

[0253] Behavior data analysis

[0254] The experiment records the operation record of the subjects pressing the button. Here, the total number of times the subjects pressed the button during the cycling process is selected for statistics. Since the cycling route contains a total of 6 intersections, the subjects only need to make a turn operation at each intersection, so the subjects press the button 6 times to complete the turn of the whole route. In this study, the difference between the total number of times the subjects press the button and the correct number of times (6 times) reflects the interaction performance.

[0255] According to the descriptive statistical results( Figure 16 ), the button times with visual navigation are more accurate and the interaction performance is higher, and the button times of the advance vibration group are the most, and the interaction efficiency is lower. The two-factor variance analysis of "visual feedback" and "vibration feedback" found that "vibration feedback" had a significant effect on interaction performance, and there was no interaction effect (Table 8: "Visual feedback" and "vibration feedback" effect on interaction performance). The single-factor variance analysis of "vibration feedback" found that the advance vibration group and the current vibration group had significant differences (Table 9). Figure 17 The figure shows the effect of vibration feedback on interaction performance (1 represents no vibration, 2 represents current vibration, and 3 represents advance vibration). Figure 18 The figure shows the effect of each navigation feedback method on interaction performance. 1, 2, and 3 are the no visual feedback group, corresponding to no vibration, current vibration, and advance vibration feedback, respectively. 4, 5, and 6 are the visual feedback group, corresponding to no vibration, current vibration, and advance vibration feedback, respectively. distance is the difference between the total number of times the subjects press the button and the correct number of times (6 times), used to reflect the interaction performance, the lower the value, the higher the interaction performance. And from Figure 18 It can be seen from the figure that the current vibration group with visual navigation has the highest interaction performance. The reason may be that visual navigation provides node information for turning at intersections, and the only vibration feedback at the intersection becomes a reminder to reinforce the turning operation behavior, and the double reminders enable the subjects to achieve the turning operation with the highest efficiency.

[0256] Table 8 Test table of main effect

[0257]

[0258] Table 9 Effect of different types of "vibration feedback" on interaction performance

[0259]

[0260] 1, 2, and 3 correspond to no vibration, current vibration, and advance vibration feedback, respectively

[0261] Eye movement data analysis

[0262] The duration of fixation on an AOI is the total time from entering to exiting the area, and the number of visits is the number of times the AOI has been visited. Both AOI fixation duration and AOI visit count can be used as indicators of interest and mental processing. Eye movement heat maps can show how participants' attention is distributed across the stimulus material. This experiment used landscape areas (non-road areas) as AOIs, and used AOI fixation duration and AOI visit count as measurement indicators to explore participants' attention and processing of the landscape areas. Eye movement heat maps were also used to analyze changes in participants' attention during cycling.

[0263] AOI fixation duration

[0264] Using "visual feedback" and "vibration feedback" as independent variables and AOI fixation duration as the dependent variable, we examined the impact of the independent variables on the dependent variable. It can be seen that "visual feedback" has a significant impact on the total fixation duration of the landscape area. The total fixation duration of the area of ​​interest under the visual navigation condition is lower than that under the non-visual navigation condition. The fixation duration of the area of ​​interest under the vibration feedback condition is not much different (Table 10, Figure 19 ), Table 10 reflects the effects of “visual feedback” and “vibration feedback” on the AOI fixation duration.

[0265] To further explore the differences, this experiment calculated the AOI fixation duration ratio (RV) of the road area to the landscape area. It can be seen that "visual feedback" has a significant impact on this ratio. Among them, visual navigation increases this ratio, that is, the proportion of time the subjects spend looking at the scenery decreases and the proportion of time looking at the road increases ( Figure 20 , Figure 21 This suggests that visual navigation reduced participants' interest in and mental processing of the landscape. Interestingly, however, different navigation methods did not significantly affect the duration of fixations on the road surface. The independent variable "visual feedback" only affected the landscape, while the independent variable "vibration feedback" had no significant effect on the duration of fixations on either the road surface or the landscape.

[0266] Table 10 Test table of inter-subject effects

[0267]

[0268] AOI visits

[0269] Using "visual feedback" and "vibration feedback" as independent variables and the number of AOI visits as the dependent variable, we examined the impact of the independent variables on the dependent variable. Table 11 shows the impact of "visual feedback" on the number of AOI visits, confirming the above argument. Taking the ratio of the number of AOI visits to the road area to the number of AOI visits to the landscape area as a ratio, we see that "vibration feedback" has a significant impact on this ratio. Further analysis reveals that this ratio increases with increasing vibration feedback (Table 12, Figure 22 Combined with the previous conclusion that "vibration feedback" had no significant effect on the duration of fixation on the road and landscape areas, it can be seen that the time spent focusing on the landscape area remained unchanged, but the experimental condition of pre-vibration caused the subjects to visit the road area more frequently, without affecting their viewing experience, but at the same time, they paid more attention to road conditions. This may be the reason why the overall experience was the highest under the experimental condition of pre-vibration.

[0270] Table 11 The impact of “visual feedback” and “vibration feedback” on the number of AOI visits

[0271]

[0272]

[0273] Table 12 The impact of different types of “vibration feedback” on the “ratio of AOI visits to the road surface area to the landscape area” (RV)

[0274]

[0275] *.The significance level of the mean difference is 0.05.

[0276] Eye movement heat map

[0277] This study selected typical road sections for eye movement heat map analysis (such as Figure 23 ), dividing the road section into the beginning, middle, and end, and capturing eye movement heatmaps. The middle stage of the road section is an important period for leisure and sightseeing. As shown in the heatmap, under the non-vibration experiment, the subjects' eye movement heatmaps exhibited a "scatter-gather-scatter" pattern, meaning they looked around at the intersection, while in the middle stage, they focused on the road and rarely engaged in leisure and sightseeing. Under the vibration experiment, whether during or before the vibration, the subjects' eye movement heatmaps exhibited a "gather-scatter-gather" pattern, meaning they focused on the road and turned at the intersection, while in the middle stage, they looked around and enjoyed leisure and sightseeing. This suggests that vibration feedback promotes leisure and sightseeing.

[0278] When the eye movement heatmaps were dispersed, the pre-vibration condition was compared to the no-vibration condition. Under the pre-vibration condition, the heatmap trend diverged evenly and regularly from the center of the road to the surrounding areas, while under the no-vibration condition, this divergence was chaotic. When the eye movement heatmaps were focused, the pre-vibration condition was compared to the mid-vibration condition. The pre-vibration condition remained relatively dispersed. This suggests that pre-vibration enabled participants to focus on the scenery in a regular and relaxed manner, allowing them to maintain a relatively relaxed state of sightseeing even when focusing on the road to make a turn.

[0279] The protection content of the present invention is not limited to the above embodiments. Without departing from the spirit and scope of the present invention, changes and advantages that can be thought of by those skilled in the art are included in the present invention and are protected by the appended claims.

Claims

1. A multi-sensory channel cycling navigation interactive control system based on eye movement data, characterized in that: The interactive control system includes: a data acquisition module, a data processing module and a navigation interaction module; wherein, The data acquisition module includes: sensors mounted on the vehicle handlebars and external devices for collecting the rider's physiological signals and vehicle control data; the physiological signals include: eye movement indicators and physiological data, and the vehicle control data includes: turning reaction time and wheel speed; The data processing module is used to standardize the data obtained by the data acquisition module and classify the real-time cycling cognitive load level using a support vector machine. The StandardScaler class in Python is used to standardize the rider's physiological signals and vehicle control data, converting the data into a standard normal distribution with a mean of 0 and a variance of 1. The navigation interaction module is used to remind the rider of his / her status based on the real-time cycling cognitive load level obtained through classification using a vibrator mounted on the handlebars; The sensors include a heart rate detector and a gyroscope. The heart rate detector is mounted on the outside of the handlebars, at the point where the fingers touch the handlebars, to detect the rider's heart rate. The gyroscope is mounted on the inside of the handlebars to detect the actual turn start time, compare the actual turn start time with the ideal turn time in the navigation route, and calculate the difference. The external device includes: a head-mounted eye tracker and a Hall sensor; the head-mounted eye tracker is worn on the user's head in the form of glasses and is used to collect eye movement indicators, which include gaze time, number of gazes, nearest neighbor index, eye saccade intrusion, scan frequency, scan duration, scan amplitude, scan speed, blink frequency, blink duration, and blink interval; the Hall sensor is installed on the outside of the pedal wheel shaft and is used to detect the wheel speed.

2. The interactive control system according to claim 1, wherein: The vehicle handle is also provided with an LED light module, a magnetic charging module, a positioning module, a battery, a battery flip cover, an emergency charging port, a power button, a shell flip cover, and a vibrator; wherein, The LED light module is used to provide lighting effects or indication functions; The magnetic charging module is used to support and provide a magnetic charging function; The positioning module is used to determine the location information of the device and realize functions including navigation and positioning tracking; The battery is used to provide power to the device, enabling the device to work independently; The battery cover is used to protect and fix the battery, making maintenance operations easier; The emergency charging port is used to connect an external power source for charging when the internal battery of the device is exhausted so that the device can continue to be used; The power button is used to control the on / off state of the device, and the device can be turned on or off by switching the button; The shell flip cover is used to protect various modules and circuits inside the device, while providing aesthetic appearance and mechanical protection.

3. The interactive control system according to claim 1, wherein: The interactive control system also includes: an Arduino prototype platform based on easy-to-use hardware and software and a CPU processor; the Arduino prototype platform includes an Arduino development board and an Arduino integrated development environment; the data obtained by the sensor and the external device is transmitted to the CPU processor through the Arduino development board, and the classification results obtained after CPU processing send vibration instructions to the vibrator on the vehicle handle through the Arduino development board.

4. An interactive control method implemented using the interactive control system according to any one of claims 1 to 3, characterized in that: The interactive control method includes: Step 1: Using sensors on the handlebars and external devices, collect the rider's physiological signals and vehicle control data, which are subsequently used to determine the rider's cognitive load level; Step 2: normalizing the cyclist's physiological signals and vehicle control data collected in Step 1; Step 3: Use support vector machine to classify the standardized data; Step 4: Based on the classified cognitive load level, a vibrator provided on the handlebar is used to output real-time vibration feedback to remind the rider to adjust the cognitive load state.

5. The interactive control method according to claim 4, wherein: In step three, a support vector machine (SVM) three-classification model of the cyclist's cognitive load level was established using the "one vs. rest" strategy. Three classifiers, SVM1, SVM2, and SVM3, were used to perform binary classification at each layer. A three-classification SVM hierarchical combination classifier was constructed using three binary classifiers to achieve three-classification of cognitive load level. The levels ranged from {0} to {2}, respectively, indicating moderate cognitive load level, high cognitive load level, and overloaded cognitive load level. The results of the SVM decision equation were used to determine the current cognitive load level of the cyclist.

6. The interactive control method according to claim 5, wherein: The decision equation is as follows: f(x)=w T x+b, Where w represents the weight of the feature vector, that is, the decision boundary learned during the training process; x represents the feature vector of the data to be classified, that is, the multidimensional vector composed of physiological data and eye movement indicators; b represents the offset, which serves as the threshold of the classifier and is used to adjust the sensitivity and specificity of the classification and determine the position of the sample interface.

7. The interactive control method according to claim 4, wherein: According to the classification results of the support vector machine, a vibration prompt is performed; the vibration prompt includes three dimensions: vibration intensity, duration and triggering time; When the SVM classification result is {2}, that is, the cognitive load level is in an overload state, the vibration feedback range of the vehicle handle is between 1-255, and the vibration feedback with the highest intensity value of 255 is continuously output, and no vibration stop time is set; When the SVM classification result is {1}, that is, when the cognitive load level is high, different vibration feedback is given according to the upcoming riding state; When the SVM classification result is {0}, that is, the cognitive load level is moderate, a route-based vibration reminder is performed at 50m and 0m away from the intersection.

Images