A multimodal attention oscillation testing device and method

Abstract

The application discloses a kind of multimodal attention oscillation test device and method, belong to attention oscillation test device field.The device includes pressure sensor, light sensor, reset button, host computer, single-chip microcomputer and sound delivery system.Reset button is placed in mute isolation unit, surface is attached with the pressure sensor of acquisition pressure signal in button process;Host computer has GUI display control module, for in screen display interface according to preset attention oscillation research paradigm different interface patterns are presented in turn;Light sensor is used to capture the appearance moment of auxiliary mark pattern;Sound delivery system can randomly play sound stimulation in each test time window of acoustic stimulation to the subject person.The device can realize continuous pressure dynamic acquisition, visual stimulation presentation moment accurate detection and mute operation function by the same equipment, and output data stream with uniform timestamp is used to support multimodal attention oscillation research, while it can support to interface external physiological signal.

Description

Technical Field

[0001] This invention belongs to the field of attention oscillation testing devices, specifically relating to a multimodal attention oscillation testing device and method. Background Technology

[0002] In daily life, our ability to hear others' conversations clearly in noisy environments or accurately locate our own items from a pile of objects relies on a core brain capability—attention. Attention refers to the process of continuously focusing limited cognitive resources on specific information while filtering out or ignoring irrelevant information. It can be understood as a kind of "mental spotlight," which the brain uses to focus on important information while ignoring irrelevant distractions.

[0003] Modern research has found that attention, this "mental spotlight," does not shine continuously and steadily, but rather flickers like a light bulb with a specific rhythm. This periodic fluctuation of attention is called attentional oscillation. The "phase" refers to the specific point in time within this periodic fluctuation; for example, the "peak" (the "brightest" moment) is when the brain's processing capacity is strongest, while the "trough" (the "darkest" moment) is when the brain's processing capacity is weakest.

[0004] When an important or unexpected event occurs (such as a sudden visual stimulus or an active key press), the brain immediately "resets" the rhythm of this "mental spotlight," starting it from a completely new starting point. This process is called resetting. By "resetting" the rhythm of attention, the brain ensures that it is in the best possible state to receive and process this newly emerging crucial information.

[0005] Psychological and cognitive neuroscience research often uses intensive sampling behavioral experiments to capture these oscillations. For example, after a reset event, weak stimuli are presented repeatedly at different time points (usually within 3 seconds of the reset event), and the rhythmic pattern of attentional oscillations is inferred from the changes in the detection rate reported by the subjects. These reset events can be divided into two categories:

[0006] 1) Exogenous stimulus reset: specific visual events (such as the sudden appearance or disappearance of visual stimuli).

[0007] 2) Endogenous action reset: autonomous motor control (such as button pressing, grasping).

[0008] Both types of reset events can reset the attentional oscillation phase, making subsequent attentional fluctuations more regular and predictable, acting like a "metronome" and significantly improving information processing efficiency. However, due to the uncertainty of voluntary button pressing (the timing of a subject's voluntary button pressing is unpredictable), existing research paradigms can only capture attentional oscillations through intensive sampling after the button press, completely failing to detect the dynamics of attentional oscillations before the button press, as well as the specific reset effects and dynamics of attentional oscillations throughout the entire button press process (from initial pressing to final release, lasting hundreds of milliseconds). This results in a lack of research on attentional oscillations before and during the formation of the button press intention.

[0009] Computers are widely used to generate and control the measurement of human reaction time to various stimuli. A standard computer keyboard can be used to measure the reaction delay of experimental subjects to stimuli, i.e., the time from perceiving the stimulus to making a response. Utility model patent application CN201120422259.7 proposes a microcontroller-based device for measuring human reaction time to external stimuli, including a photosensitive sensor, a microcontroller, buttons, and a computer (host computer). The photosensitive sensor is connected to the microcontroller, and the microcontroller is connected to the host computer. This device uses an internal timer of the microcontroller for timing measurement. Its limitations are: firstly, the timing method is difficult to synchronize with external devices; secondly, the device can only capture discrete moments of button presses and cannot collect and analyze continuous dynamic pressure changes throughout the entire button press process (including pressing, holding, and releasing phases). Utility model patent application CN202323024150.X provides a multi-mode, multi-modal fusion interpersonal interaction experimental device and system, which includes two microcontrollers, buttons, a host computer, a sound transmission system, and two pressure sensors. The pressure sensor in this device is only used to identify the trigger moment of the button action (discrete event), and it fails to distinguish and quantify the continuous dynamic changes in pressure throughout the entire button press process (including the pressing, holding and releasing phases).

[0010] Overall, studying the fine dynamics of attentional oscillations using traditional keyboards and screens faces significant challenges:

[0011] 1. Inability to accurately capture the reset time: Traditional keyboards can only record the instant the "key press" action occurs, and cannot capture the continuous dynamics of the entire key press action (from the initial press to the final release, lasting hundreds of milliseconds) (such as the change of pressure over time). This makes it difficult to accurately determine the precise moment when the phase reset triggered by the key itself occurs.

[0012] 2. Inaccurate visual presentation timing: Although experimental software (such as PsychoPy and Psychtoolbox) can preset the presentation time of visual stimuli, due to hardware limitations such as screen size, there is often a small but crucial deviation between the actual time the image appears and the preset time. This makes it difficult to accurately pinpoint the true time of the visual stimulus as a reset event.

[0013] 3. Keyboard noise interference: The mechanical noise generated by traditional keyboard operation and the resonance sound formed with the desktop are difficult to isolate effectively. This sound may interfere with the state of the test subjects, become an additional variable in the experiment, and affect the purity of the data and the reliability of the conclusions.

[0014] 4. Lack of integrated measurement: Existing devices cannot simultaneously and accurately capture the actual moment of visual stimulus presentation and the continuous dynamic details of key presses, nor can they provide an effective silent operating environment to significantly reduce noise interference. This makes it difficult for researchers to distinguish or integrate the two key mechanisms of visual reset and key reset, thus posing a challenge to accurately testing the attentional oscillation phenomenon in subjects. Summary of the Invention

[0015] The purpose of this invention is to solve the problems existing in the prior art and to provide a multimodal attention oscillation testing device and method.

[0016] The specific technical solution adopted in this invention is as follows:

[0017] In a first aspect, the present invention provides a multimodal attentional oscillation testing device, which includes a pressure sensor, a light sensor, a reset button, a host computer, a microcontroller, and a sound transmission system;

[0018] The reset button is placed in a silent isolation unit to isolate the noise generated during the pressing of the button; a pressure sensor is attached to the surface of the reset button to collect pressure signals during the process of the test subject pressing the reset button.

[0019] The host computer has a GUI display and control module, which is used to present different interface patterns in sequence on the screen display interface according to the preset attention oscillation research paradigm. The interface patterns that appear in sequence in each trial are: a visual stimulus pattern and an auxiliary marker pattern that appear simultaneously, an inquiry interface that asks the subject to judge whether they have heard the sound stimulus and asks them to input the judgment result, and a prompt interface that prompts whether the timing of pressing the reset button in this trial is accurate.

[0020] The optical sensor is attached to the screen of the host computer, and its sensing area corresponds to and only corresponds to the display area of ​​the auxiliary marker pattern, used to capture the time when the auxiliary marker pattern appears;

[0021] The microcontroller is communicatively connected to the optical sensor, the host computer, and the pressure sensor on the reset button. The electrical signals detected by the optical sensor and the pressure sensor are all collected by the microcontroller in real time and transmitted to the host computer.

[0022] The sound transmission system is connected to a host computer and is used to randomly play sound stimuli to the subject within the sound stimulus time window of each trial according to the instructions issued by the host computer. The sound stimulus time window is a time window centered on the standard moment when the reset button is pressed.

[0023] As a preferred embodiment of the first aspect, the soundproofing unit includes a latex pad and a soundproof cotton cover, which are assembled to form a sealed enclosure that only allows the hand to enter and exit, and the surface of the latex pad is covered with a mesh fabric.

[0024] As a preferred embodiment of the first aspect mentioned above, the host computer is also connected to a keyboard, and the judgment result is input through two different keys on the keyboard.

[0025] As a preferred embodiment of the first aspect, the optical sensor employs a photodiode with an integrated signal amplifier. The current output by the photodiode is converted into a voltage signal by the signal amplifier and then sent to the microcontroller.

[0026] As a preferred embodiment of the first aspect, the light sensor is attached and fixed to the edge of the screen of the host computer, the display area of ​​the auxiliary marking pattern is located at the corresponding light sensor attachment position, and the visual stimulation pattern is presented in the central area of ​​the screen.

[0027] As a preferred embodiment of the first aspect, the pressure sensor is a piezoresistive force-sensitive resistor. The resistance change of the piezoresistive force-sensitive resistor is converted into a linear voltage signal by a signal amplifier and then sent to the microcontroller.

[0028] As a preferred embodiment of the first aspect, the host computer is also connected to an external physiological signal monitoring device, and the monitored physiological signals include electromyography or respiration.

[0029] As a preferred embodiment of the first aspect, in the GUI display and control module, the duration of a single trial is set to be no less than 4 seconds. A gray screen with a duration of 1 second is first displayed. The visual stimulus pattern and auxiliary marker pattern appear synchronously after the gray screen ends. The standard time for pressing the reset button is 1.5 seconds after the visual stimulus pattern is presented. The sound stimulus time window is from 0.3 seconds to 2.7 seconds after the visual stimulus pattern is presented. The inquiry interface appears 3 seconds after the visual stimulus pattern is presented. The prompt interface appears immediately after the moment the subject completes the judgment result input. The single trial ends after the prompt interface disappears.

[0030] In a second aspect, the present invention provides a method for acquiring multimodal attention oscillation test data, the method being implemented based on the multimodal attention oscillation test apparatus as described in any of the first aspects above, comprising:

[0031] S1. After the host computer receives the start command, the GUI display and control module starts to execute a new trial. First, a gray screen appears on the screen display interface. After the gray screen ends, a visual stimulus pattern and an auxiliary marker pattern are simultaneously displayed on the screen display interface. The microcontroller collects the electrical signal output by the light sensor in real time and transmits it to the host computer to identify and record the first moment when the auxiliary marker pattern appears on the screen.

[0032] S2. Randomly select a time point within the sound stimulation time window, and play sound stimulation to the subject according to a predetermined sound stimulation paradigm. The sound stimulation paradigm includes randomly selected trials with sound and trials without sound. The intensity of the sound stimulation in the trials with sound is controlled at the subject's hearing threshold level. The microcontroller collects the electrical signal output by the pressure sensor on the reset button in real time and transmits it to the host computer to identify and record the second moment when the subject presses the reset button.

[0033] S3. After the sound stimulus time window ends, an inquiry interface is displayed on the screen, asking the subject to judge whether the sound stimulus was heard in the current trial. The microcontroller collects the input judgment result in real time and transmits it to the host computer for recording. Based on the actual record of the sound stimulus played in the current trial, the microcontroller gives the subject the correct or incorrect result of whether the sound stimulus was heard in the current trial.

[0034] S4. Based on the second moment recorded in the current trial when the subject pressed the reset button and the standard moment when the reset button was pressed, a prompt interface is displayed on the screen to indicate to the subject whether the timing of pressing the reset button in this trial was accurate, until the next trial.

[0035] S5. Continuously execute S1~S4 until all trials are completed. Then, the host computer records the data collected in all trials for multimodal attention oscillation analysis.

[0036] As a preferred embodiment of the second aspect above, in the host computer, the first moment or the second moment is taken as the zero point of time. The time difference between the actual playback time of the sound stimulus in each trial and the zero point of time is calculated. Then, the data are grouped according to the time difference, and the sound detection rate corresponding to different time differences is statistically calculated based on the correctness results to obtain a sound detection rate sequence, so as to reflect the oscillation characteristics of auditory attention before and after the occurrence of different reset events.

[0037] Compared with the prior art, the present invention has the following advantages:

[0038] On the one hand, the multimodal attention oscillation testing device of this invention has a series of hardware breakthroughs. Specifically, in the continuous pressure data acquisition structure, this invention integrates a pressure sensor (FSR) above the button contact, outputting a pressure-time curve (from initial contact, pressure change to final release), overcoming the limitations of discrete data in traditional switch-type buttons. In the visual stimulus presentation time calibration mechanism, this invention uses a high-speed photodiode to directly capture the instantaneous changes in pixel brightness in the target area of ​​the screen, effectively overcoming the asynchrony between the software preset time and the actual hardware output, achieving hardware-level synchronous detection of the actual presentation time of the visual stimulus, thus providing a precise time zero point for subsequent analysis. Simultaneously, this invention introduces a layered composite silent isolation unit consisting of mesh fabric, latex buffer layer, mesh fabric, button, pressure sensor, protective film, and sound insulation cotton. Through a dual-effect noise reduction mechanism, the latex layer efficiently absorbs the kinetic energy of the button impact to suppress vibration noise sources, while the enclosed sound insulation cotton shell blocks the propagation path of mechanical noise (including resonance conduction). Therefore, this device is the first to achieve three core functions in the same experimental setup: "continuous pressure dynamic acquisition", "precise detection of visual stimulus presentation time" and "silent operation", and outputs a data stream with a unified timestamp, which can support precise time-domain alignment with external physiological signals (such as electromyography, respiration, etc.).

[0039] On the other hand, the multimodal attentional oscillation test data acquisition method of this invention has a series of structural innovations. Specifically, this invention introduces attentional oscillation detection before button presses. It presents visual stimuli as behavioral cues (requiring the subject to press the reset button during this period), and randomly presents auditory stimuli within a time window after the visual stimuli appear. This ensures that some auditory stimuli occur before the subject presses the reset button. This approach achieves, for the first time, intensive sampling before the button press to detect attentional oscillations, accurately detecting the attentional oscillation characteristics before and during the formation of the button press intention. Furthermore, the combined design of the button and pressure sensor allows for the acquisition of continuous pressure sensing data, enabling precise location of the phase reset start time of the button press (e.g., the moment of pressure abrupt change, the moment the pressure reaches its peak, or the moment it returns to the baseline), and analysis of the oscillation rhythm (e.g., the theta band) of the subject's sound detection rate throughout the entire button operation cycle (covering the press-hold-release phase). Finally, by introducing a dual time reference system of the visual stimulus initiation time (T1) and the key point of the key pressure event (T2), the reset effects caused by exogenous visual stimuli and endogenous key operation were effectively separated, realizing independent analysis and comprehensive examination of the two types of mechanisms. Attached Figure Description

[0040] Figure 1 This is a schematic diagram of the multimodal attention oscillation testing device;

[0041] Figure 2This is a schematic diagram of the structure of the silent isolation unit in an embodiment of the present invention;

[0042] Figure 3 This is one paradigm for studying attentional oscillations in this embodiment of the invention;

[0043] Figure 4 This is the overall process for acquiring multimodal experimental data using the test equipment in this embodiment of the invention;

[0044] Figure 5 This is the average sound detection rate sequence of all subjects before button reset, with visual stimulus (T1) as the zero point in this embodiment of the invention;

[0045] Figure 6 According to the embodiments of the present invention Figure 5 The power spectrum obtained from the sound detection rate sequence is shown in the figure through permutation test (n=1000) and multiple comparison correction based on the maximum value, which compares and shows its 95% significance threshold.

[0046] Figure 7 The figure shows the average sound detection rate sequence of all subjects with the key event (T2) as the time zero point in this embodiment of the invention. The sequence is marked by dashed lines as two parts: before the key press and after the key press.

[0047] Figure 8 According to the embodiments of the present invention Figure 7 The power spectrum obtained from the sound detection rate sequence before the key press is shown in the figure through permutation test (n=1000) and multiple comparison correction based on the maximum value, which compares and shows its 95% significance threshold.

[0048] Figure 9 As described in the embodiments of the present invention Figure 7 The power spectrum obtained from the sound detection rate sequence after the key press is shown in the figure. The 95% significance threshold is compared by permutation test (n=1000) and multiple comparison correction based on the maximum value.

[0049] Figure 10 The curves showing the changes in screen brightness and key pressure over time for a single trial in this embodiment of the invention, along with their corresponding zero-point reset time.

[0050] Figure 11 This is a record of the amplitude of the sound generated by button operation when the mute isolation unit is canceled;

[0051] Figure 12 This is a record of the amplitude of the sound generated by button operation when a mute isolation unit is added.

[0052] The attached diagram is labeled as follows: 1. Silent isolation unit; 2. Pressure sensor; 3. Light sensor; 4. Reset button; 5. Host computer; 6. Microcontroller; 7. Sound transmission system; 8. Protective film; 9. ADC amplifier; 10. Keyboard. Detailed Implementation

[0053] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be practiced in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below. Technical features in various embodiments of the present invention can be combined accordingly without mutual conflict.

[0054] In the description of this invention, it should be understood that when an element is considered to be "connected" to another element, it can be a direct connection to the other element or an indirect connection, i.e., there is an intermediate element. Conversely, when an element is said to be "directly" connected to another element, there is no intermediate element.

[0055] In the description of this invention, it should be understood that the terms "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature.

[0056] This invention provides a multimodal attentional oscillation testing device. This device achieves synchronous integration of multimodal data, enabling three core functions—continuous pressure dynamic acquisition, precise detection of visual stimulus presentation time, and silent operation—all through a single experimental setup. It outputs a data stream with a unified timestamp for multimodal attentional oscillation analysis. Therefore, this invention effectively overcomes the technical limitations of traditional keyboard and screen time detection in psychology and cognitive neuroscience experiments, and is particularly suitable for simultaneously detecting the attentional oscillation phase reset effect triggered by visual stimuli and keystrokes.

[0057] The specific structure of the above-mentioned multimodal attention oscillation test device is described in detail below.

[0058] like Figure 1 As shown, in a preferred embodiment of the present invention, the components of the above-mentioned multimodal attention oscillation testing device include a noise isolation unit 1, a pressure sensor 2, a light sensor 3, a reset button 4, a host computer 5, a microcontroller 6, a sound transmission system 7, a protective film 8, an ADC amplifier 9, and a keyboard 10.

[0059] The reset button 4 is placed in the noise isolation unit 1 to isolate the noise generated during the pressing of the reset button 4. A pressure sensor 2 is attached to the surface of the reset button 4 to collect pressure signals during the pressing of the reset button 4 by the test subject. Based on this combination design of button and pressure sensor, continuous pressure signal data can be collected, and the phase reset start time of the reset button 4 pressing can be accurately located (such as the moment of pressure initiation change, the moment when the pressure reaches the peak or returns to the baseline). In subsequent analysis, the oscillation rhythm (such as the theta band) of the test subject's sound detection rate can be analyzed throughout the entire 200-500 ms button operation cycle (covering the press-hold-release phase).

[0060] Since absolute quiet is required during the study of oscillations, this invention designs a silent isolation unit 1 that can suppress mechanical key noise and resonance interference, thereby avoiding additional interference to the cognitive decision-making process of the test subjects.

[0061] In embodiments of the present invention, such as Figure 2 As shown, the noise isolation unit 1 includes a latex pad 102 and a sound-absorbing cotton cover 104. The sound-absorbing cotton cover 104 is a bottomless cylindrical shape, while the latex pad 102 is slightly larger than the bottom surface of the sound-absorbing cotton cover 104, allowing them to form a sealed enclosure when assembled. Additionally, the side wall of the sound-absorbing cotton cover 104, near the bottom, has a passage for the subject's hands to enter and exit. Except for the hand passage, the entire sealed enclosure remains soundproof. Furthermore, the noise isolation unit 1 includes a lower mesh fabric 101 and an upper mesh fabric 103, which are respectively attached to the upper and lower surfaces of the latex pad 102. The latex pad 102, wrapped in mesh fabric, absorbs the impact energy when the button is pressed, while the sealed sound-absorbing cotton cover 104 blocks the transmission path of mechanical noise, significantly reducing operating noise. Simultaneously, the mesh fabric provides physical isolation, preventing the latex from adhering to the skin and causing interference.

[0062] Furthermore, in multimodal attentional oscillation studies, different visual and auditory stimuli need to be presented to the subjects, and these stimuli are controlled by the host computer 5. Therefore, the host computer 5 needs to have a built-in GUI display module to sequentially present different interface patterns on the screen according to the preset attentional oscillation study paradigm. The GUI display module is a software module in the form of a computer control program, whose program control function is to sequentially display different patterns and information on the screen according to preset logic. In the embodiments of the present invention, in order to achieve the necessary data acquisition for multimodal attentional oscillation studies, the interface patterns that appear sequentially in each trial are divided into three categories: the first step is the appearance of a visual stimulus pattern and an auxiliary marker pattern, and the visual stimulus pattern and the auxiliary marker pattern need to appear synchronously; the second step is an inquiry interface that asks the subjects to judge whether they have heard the auditory stimulus and to input the judgment result; the third step is a prompt interface that prompts whether the timing of pressing the reset button 4 in this trial is accurate.

[0063] The aforementioned auxiliary marker pattern serves to record the precise time when the visual stimulus pattern is actually displayed on the screen. Although the display time of the visual stimulus pattern can be found through the built-in program of the host computer 5, there is an inevitable deviation between the software preset time and the actual hardware output display time, making it difficult to accurately pinpoint the zero point of time. This invention establishes a hardware solution for directly detecting the physical light signal of the screen, which can accurately capture the actual presentation time of the visual stimulus. This solution is achieved through the cooperation of the auxiliary marker pattern and the light sensor 3. The light sensor 3 is attached to the screen of the host computer 5, and its sensing area corresponds to and only corresponds to the display area of ​​the auxiliary marker pattern. Since the visual stimulus pattern and the auxiliary marker pattern are controlled to be displayed simultaneously at the software level, the time delay from software to hardware display is the same. By capturing the appearance time of the auxiliary marker pattern through the light sensor 3, it can be used as the precise time when the visual stimulus pattern is actually displayed on the screen, eliminating the time error from software output to hardware presentation of the visual stimulus and improving the accuracy of multimodal attention oscillation test results.

[0064] In addition, since the auxiliary marker pattern and the visual stimulus pattern are displayed at the same time, in order to avoid the appearance of the auxiliary marker pattern affecting the subject's attention to the visual stimulus pattern, the display area of ​​the auxiliary marker pattern should be located at the edge of the screen, and the light sensor 3 is also attached and fixed to the edge of the screen of the host computer 5. The display area of ​​the auxiliary marker pattern corresponds exactly to the attachment position of the light sensor 3, while the visual stimulus pattern is presented in the central area of ​​the screen.

[0065] The aforementioned units need to establish communication connections with each other. Specifically, the microcontroller 6 communicates with the light sensor 3, the host computer 5, and the pressure sensor 2 on the reset button 4. The electrical signals detected by the light sensor 3 and the pressure sensor 2 are collected in real-time by the microcontroller 6 and transmitted to the host computer 5. Additionally, the sound transmission system 7 communicates with the host computer 5 and is used to randomly play sound stimuli to the subject within the sound stimulation time window of each trial, based on instructions issued by the host computer 5. This sound stimulation time window is a time window centered on the standard moment when the reset button 4 is pressed.

[0066] To ensure the real-time performance and reliability of the communication connection, all units, including the microcontroller 6, optical sensor 3, host computer 5, pressure sensor 2, and sound transmission system 7, use a wired direct connection transmission method for communication.

[0067] In an embodiment of the present invention, the sound transmission system 7 may consist of a sound card and headphones. The host computer 5 drives the headphones to produce sound through the sound card, and the headphones are preferably noise-isolating headphones.

[0068] It should be noted that the standard time for pressing the reset button 4 in this invention refers to the target time in a trial where the subject is expected to press the reset button 4. This standard time can be flexibly adjusted according to the actual trial cycle and experimental requirements.

[0069] In a preferred embodiment of the present invention, a specific paradigm for studying attentional oscillations is provided. For example... Figure 3 As shown, in this attention oscillation research paradigm, the duration of a single trial in the GUI display module is set to be no less than 4 seconds. First, a gray screen with a duration of 1 second is displayed. The visual stimulus pattern and auxiliary marker pattern appear synchronously after the gray screen ends (the presentation duration of the visual stimulus pattern and auxiliary marker pattern is 50 ms). The standard time for pressing the reset button 4 is 1.5 seconds after the visual stimulus pattern is presented. The acoustic stimulus time window is from 0.3 seconds to 2.7 seconds after the visual stimulus pattern is presented. The inquiry interface appears 3 seconds after the visual stimulus pattern. The prompt interface appears immediately after the moment the subject completes the judgment result input. The single trial ends after the prompt interface disappears.

[0070] Furthermore, when the inquiry interface appears, the subject is required to determine whether they have heard a sound stimulus and to input a judgment result, i.e., a binary judgment result representing "heard sound" and "did not hear sound". Theoretically, this judgment result can be input via keyboard and mouse hardware, or via other voice or touch methods. In an embodiment of the present invention, a keyboard 10 is preferably connected to the host computer 5, and two different keys on the keyboard 10 are designated to represent "heard sound" and "did not hear sound" respectively. The judgment result can be input using these two different keys. It should be noted that, unlike the reset button 4, this keyboard does not need to be placed in the silent isolation unit 1.

[0071] In this embodiment, for ease of description, the reset button 4 is referred to as the 'K' key, the key on the keyboard that represents "hearing a sound" is referred to as the 'A' key, and the key on the keyboard that represents "not hearing a sound" is referred to as the 'F' key.

[0072] Therefore, please continue to see Figure 3 As shown, the oscillation study paradigm can be further designed as follows: After the trial begins, a 1-second gray screen is first displayed; after the gray screen ends, a visual stimulus pattern and an auxiliary marker pattern are simultaneously displayed. The visual stimulus pattern can be a green circle displayed in the center of the screen for 50 ms; the auxiliary marker pattern can be a white circle displayed at the edge of the screen simultaneously with the visual stimulus pattern for 50 ms. The 1.5-second mark after the visual stimulus is presented is set as the standard time for the subject to press the reset button 4; the sound stimulus can be played at any time between 0.3 and 2.7 seconds after the visual stimulus appears; at the 3-second mark after the visual stimulus is presented, a query interface will pop up on the screen, displaying the following text: Did you hear a sound? Yes (press 'A'); No (press 'F'). Once the subject completes the judgment, feedback will be immediately given through a prompt interface, informing them whether the button press timing was correct. The trial ends after the prompt interface disappears. In this attention oscillation study paradigm, the prompt interface displays the following text to inform the subject whether the timing of pressing the 'K' key was appropriate: "Pressed just right," "Pressed too fast," "Pressed too slow," and "No 'K' key pressed." The appropriate prompt is displayed based on the actual timing of the press. It should be noted that although the standard time for pressing the reset button 4 is a specific point in time (1.5 seconds after the visual stimulus pattern is presented), actual pressing cannot be so precise. Therefore, this attention oscillation study paradigm sets a tolerance range. If the actual pressing time falls within 1.5 seconds ± 0.5 seconds after the visual stimulus pattern is presented, the prompt interface displays "Pressed just right"; if it is before this range, it displays "Pressed too fast"; if it is after this range, it displays "Pressed too slow"; and if no pressing of the reset button 4 is detected, it displays "No 'K' key pressed."

[0073] The sensors in this invention can be selected according to actual needs to achieve high sensitivity and high accuracy. In the embodiments of this invention, the light sensor 3 uses a photodiode with an integrated signal amplifier, preferably the OPT101 model. The photodiode can directly detect changes in light intensity, and its output current is converted into a voltage signal by the built-in ADC amplifier before being sent to the microcontroller 6. The pressure sensor 2 uses a piezoresistive force resistor (FSR), preferably the RP-C10-ST model. The resistance change of the piezoresistive force resistor is converted into a linear voltage signal by the ADC amplifier 9 before being sent to the microcontroller 6. The microcontroller 6 is recommended to be an Arduino Uno R3 or an Arduino Due. The Arduino Uno R3 reads the signal from the pressure sensing unit (V... ADC1 ) and visual sensing unit (V ADC2 The voltage signal (0-5V) is used. To ensure time accuracy, an interrupt service routine is used to directly read the raw value V from the ADC register at a fixed sampling rate of 1 kHz. ADC1 and V ADC2 The Arduino Uno R3 reads the raw value V via a wired connection. ADC1 and V ADC2 Real-time transmission to the host computer can use the following data packet format: [Check bit 1 | FSR voltage signal value | photodiode voltage signal value | Check bit 2]. This format embeds check bytes at the beginning and end to ensure precise clock synchronization and data accuracy of the multimodal data stream.

[0074] In addition, software such as MATLAB and Lab Streaming Layer can be installed on the host computer 5 to facilitate data analysis and storage. MATLAB is responsible for real-time data analysis, while Lab Streaming Layer is responsible for persistent data storage. Of course, as an integrated expansion solution, the host computer 5 in the above-mentioned testing device of this invention can also support the connection of external physiological signal monitoring devices. The monitored physiological signals include electromyography or respiration. By aligning and outputting unified timestamp data, it can support precise time-domain alignment between multimodal attention oscillation test data and external physiological signal data, expanding the application range of the device. Therefore, Figure 4 The overall process of multimodal experimental data acquisition in this test device is demonstrated, and all data can be persistently stored through the Lab Streaming Layer.

[0075] In an embodiment of the present invention, a method for acquiring multimodal attention oscillation test data is also provided. This acquisition method is implemented based on the aforementioned multimodal attention oscillation test device, and the specific steps of the method include:

[0076] S1. After receiving the start command, the host computer 5 starts executing a new trial. First, a gray screen with a duration of 1 second is displayed on the screen. After the gray screen ends, visual stimulus patterns and auxiliary marker patterns are displayed on the screen simultaneously. The microcontroller 6 collects the electrical signal output by the light sensor 3 in real time and transmits it to the host computer 5 to identify and record the first moment when the auxiliary marker pattern appears on the screen.

[0077] S2. Randomly select a time point within the sound stimulation time window, and play sound stimulation to the subject according to a predetermined sound stimulation paradigm. The sound stimulation paradigm includes randomly selected trials with sound and trials without sound. The intensity of the sound stimulation emitted in the trials with sound is controlled at the subject's hearing threshold level. The microcontroller 6 collects the electrical signal output by the pressure sensor 2 on the reset button 4 in real time and transmits it to the host computer 5 to identify and record the second moment when the subject presses the reset button 4.

[0078] S3. After the sound stimulus time window ends, an inquiry interface is displayed on the screen, asking the subject to judge whether the sound stimulus was heard in the current trial. The keyboard 10 collects the judgment result (pressing the 'A' key or the 'F' key) entered by the subject in real time and transmits it to the host computer 5 for recording. Based on the actual record of the sound stimulus played in the current trial, the correct result of whether the subject heard the sound stimulus in the current trial is given.

[0079] S4. Based on the second moment recorded in the current trial when the subject pressed the reset button 4 and the standard moment when the reset button 4 was pressed, a prompt interface is displayed on the screen to prompt the subject whether the timing of pressing the reset button 4 in this trial is accurate, until the next trial.

[0080] S5. Continuously execute S1~S4 until all trials are completed. The data collected in all trials is recorded by the host computer 5, including the first moment, the second moment, the correctness result, the time domain signal of the pressure sensor when the reset button 4 is pressed, and the actual playback time of the sound stimulus, for multimodal attention oscillation analysis.

[0081] It should be noted that the specific multimodal attention oscillation analysis method is not limited in principle and can be designed according to actual research needs. In the embodiments of the present invention, in the host computer 5, either the first moment or the second moment can be taken as the time zero point. The time difference between the actual playback time of the sound stimulus in each trial and the time zero point is calculated. Then, the data is grouped according to the time difference, and the sound detection rate corresponding to different time difference groups is statistically calculated based on the correctness results. This yields a sound detection rate sequence with the time difference as the axis, which reflects the oscillation characteristics of auditory attention before and after the occurrence of different reset events. When the first moment is taken as the time zero point, the sound detection rate sequence reflects the oscillation characteristics of auditory attention after the occurrence of the exogenous visual stimulus event (i.e., the appearance of the green dot). When the second moment is taken as the time zero point, the sound detection rate sequence reflects the oscillation characteristics of auditory attention before and after the occurrence of the endogenous key operation event (i.e., pressing the reset button).

[0082] To better demonstrate how the present invention, based on the aforementioned multimodal attention oscillation testing device, specifically implements the multimodal attention oscillation test data acquisition method, a specific experimental paradigm for attention oscillation research is given below through an embodiment.

[0083] Example 1

[0084] In this embodiment, data were collected from 10 participants. All participants had normal vision (including corrected vision) and hearing, and no history of brain injury or neurological disease. All participants signed written informed consent forms before participating. The data collection process strictly adhered to relevant ethical guidelines, and all data was anonymized to protect the privacy of the participants.

[0085] In this embodiment, based on Figure 1 and Figure 2 The multimodal attention oscillation test apparatus shown includes the following steps in its multimodal attention oscillation test data acquisition method:

[0086] Step 1: Experimental Paradigm Design

[0087] use Figure 3 The paradigm for studying attentional oscillations, as shown, uses psychological experiment control plugins such as Psychtoolbox or PsychoPy to control the presentation of visual and auditory stimuli on a host computer. The specific process is as follows:

[0088] (1) Presentation of visual stimulus: First, a gray screen with a duration of 1 second is presented. After the gray screen ends, a green circular visual stimulus is presented in the center of the screen for 50 ms. Simultaneously, a white circle is presented at the edge of the screen. This white circle does not participate in the experimental task and is only used by the photodiode to accurately capture the first moment when the visual stimulus actually appears.

[0089] (2) Button response task: After seeing the green circle, the subject needs to wait for about 1.5 seconds (1.5 seconds ± 0.5 seconds, within the range of 1 to 2 seconds) and then press the reset button 'K' on the keypad;

[0090] (3) Auditory stimulus presentation (probabilistic): The acoustic stimulus time window was from 0.3 s to 2.7 s after the green circle was presented. Within this time window, a pure tone with a frequency of 1000 Hz was presented to both ears of the subjects in a random order for 10 ms. The intensity of the pure tone was controlled at the level of the individual's auditory threshold, so that they needed to maintain focus to hear it. The acoustic stimulus time window was set based on the characteristic that the attention reset effect gradually weakens over time, so the total length was controlled at 2.4 s. The acoustic stimulus presentation time was sampled in 10 ms increments within the interval of 0.3 s to 2.7 s after the visual stimulus, including a total of 241 time points. The sound was repeated 12 times at each time point, for a total of 2892 sound trials. However, the time point at which the pure tone appeared in each sound trial was random.

[0091] In addition, 100 silent trials were inserted as a control, for a total of 2992 trials. To reduce subject fatigue, data collection was conducted in two sessions, morning and afternoon, with 1496 trials performed in each session.

[0092] (4) Auditory detection judgment: After the green circle appears for 3 seconds, ask the subject through the inquiry interface whether he / she heard the pure tone: if he / she heard it, press the designated key (such as the 'A' key); if he / she did not hear it, press another designated key (such as the 'F' key).

[0093] (5) Button Response Feedback: After the subject completes the auditory sound judgment, the screen will display text feedback through a prompt interface, informing the subject whether the timing of pressing the 'K' button was appropriate. The prompt types include four categories: "Button pressed just right," "Button pressed too fast," "Button pressed too slow," and "No 'K' button pressed." If the actual button pressing time falls within the range of 1.5 s ± 0.5 s after the start of the trial, the prompt interface will display "Button pressed just right." If it is before this range, it will display "Button pressed too fast." If it is after this range, it will display "Button pressed too slow." If the reset button 4 is not detected, it will display "No 'K' button pressed." The prompt interface will display for 0.5 s before proceeding to the next trial.

[0094] Step 2: Data Collection

[0095] During the experiment, the following data were collected from the subjects simultaneously in each trial:

[0096] (1) The first moment when the visual stimulus actually begins to be presented (recorded by photodiodes).

[0097] (2) The change of pressure over time during button press (recorded by a pressure sensor, which can be used to identify the second moment when the reset button 4 is pressed).

[0098] (3) The timing of the auditory stimulus (i.e. the timing of the pure tone), the correctness of the judgment on whether the pure tone was heard, etc.

[0099] The core time zero point can be defined in two ways: first zero point and second zero point.

[0100] (1) First zero point (T1): Based on the first moment when the visual stimulus recorded by the photodiode is actually presented, the starting point when the Arduino voltage signal corresponding to the appearance of the green circle changes significantly is defined as the first reference time zero point T1.

[0101] (2) Second Zero Point (T2): Based on the pressure data in the time domain collected during the pressing of the reset button, a second reference time zero point T2 is dynamically defined. The definition method includes, but is not limited to, any of the following:

[0102] (a) The moment when the pressure begins to change significantly (abruptly) during a single key reset;

[0103] (b) The moment when the pressure reaches its maximum value during a single key press reset;

[0104] (c) The moment when the pressure is released and the system returns to the baseline during a single button reset (this is the moment when the brain relaxes).

[0105] This invention allows for the parallel establishment of two time zeros within the same experimental paradigm: time zero T1 corresponds to the moment of visual stimulus presentation (an exogenous reset event), while time zero T2 corresponds to the moment of triggering the reset button (an endogenous reset event). This invention enables independent and integrated analysis of the reset effects induced by exogenous visual stimuli and endogenous button operations, allowing for the simultaneous exploration of the visual-motor dual-pathway oscillation mechanism in subjects (previous studies could only analyze one type of attentional reset effect).

[0106] Step 3: Attention Oscillation Analysis Based on Time Zero

[0107] (1) Using T1 or T2 as the defined time reference, calculate the time difference between the time of auditory stimulus presentation and the time reference in each trial (since the sound may be presented before the button reset, the time difference includes positive and negative values); then, group the different time differences, and statistically analyze the sound detection rate corresponding to different time differences based on the correct and incorrect results, thereby obtaining the sound detection rate sequence; use a Gaussian sliding window with a standard deviation of 15 ms and a window width of 50 ms, and perform a sliding average on the sound detection rate sequence of each subject with a step size of 10 ms to obtain the time-aligned sound detection rate sequence;

[0108] (2) Analyze the sound detection rate sequence aligned with T1 (the actual start time of the visual stimulus) as the time zero point to reveal the oscillation characteristics of the subjects' auditory attention after the visual stimulus event;

[0109] (3) Analyze the sound detection rate sequence aligned with T2 (key point of key pressure event) as the time zero point to reveal the oscillation characteristics of the subjects' auditory attention before and after the key reset event (pressure event);

[0110] (4) For oscillation feature analysis, the time-aligned detection rate sequence is subjected to subsequent processing (including but not limited to signal averaging, detrending, windowing, spectrum transformation and statistical significance evaluation, etc.) to finally obtain significant frequency components that characterize attention oscillations.

[0111] In this embodiment, the analysis of attentional oscillation data based on the zero-point of time aims to examine whether the sound detection rate exhibits rhythmic oscillations (i.e., attentional oscillations) when different reset events are used as time bases. The specific process is as follows:

[0112] 3.1) Data preprocessing and time alignment

[0113] Using the actual start time of the visual stimulus (T1) and the key pressure point of the button (T2) as the time zero points, the time difference between the presentation time of the auditory stimulus and the time zero point was calculated for each trial (the time difference can be positive or negative since the sound may occur before or after the button press). A Gaussian sliding window with a standard deviation of 15 ms and a window width of 50 ms was used to perform a moving average on the sound detection rate data for each participant with a step size of 10 ms. The averaged individual data were then aligned according to the time points to finally calculate the average detection rate time series for all participants.

[0114] 3.2) Oscillation Analysis After Visual Reset (T1 Time Zero)

[0115] Using T1 as the baseline, we analyzed the sound detection rate data after the visual stimulus appeared and before the button reset occurred. To determine the significance of oscillations, a permutation test was used: the actual detection rate time series was randomly shuffled 1000 times along the time axis to construct a "random fluctuation" distribution. Fourier transforms were performed on the actual data and all 1000 shuffled data to calculate their power spectra in the frequency range of 1-15 Hz. The power spectra of the actual data were compared with the random distribution generated by the shuffled data, and after multiple comparison correction based on the maximum value, oscillation frequencies with power significantly higher than the random level (p < 0.05) were identified.

[0116] 3.3) Oscillation analysis before and after button reset (at time zero of T2)

[0117] Using the key pressure point (T2) as the zero point of time, the sound detection rate data before and after the key press were analyzed. To control for the influence of the "sensory attenuation" effect (i.e., the perceived intensity of self-evident stimuli is weaker than that of external stimuli), data intervals affected by this effect were excluded from the analysis. The average detection rate within the time window of -1 s to -0.5 s before the key press was used as the baseline level, and effective time periods were selected before and after the key press.

[0118] In the post-key phase, the first time point where the detection rate is consistently higher than the baseline is selected, and a time series is constructed from this point.

[0119] Before the button is pressed, the end time point that is closest to the zero time point (T2) and whose detection rate is consistently higher than the baseline is selected, and a time series is constructed with this as the endpoint.

[0120] Based on the selected effective time period, the average detection rate sequence before and after the key press is calculated, and the permutation test and frequency domain analysis consistent with step 3.2) are performed to determine whether there is a significant oscillation frequency component.

[0121] In this embodiment, with the visual stimulus (T1) as the zero point, the average sound detection rate of all subjects before the button reset is as follows: Figure 5 As shown. Based on this sound detection rate sequence, the power spectrum is further calculated as follows. Figure 6 As shown, the dashed line represents the 95% significance threshold obtained after 1000 permutation tests and multiple comparison corrections.

[0122] In addition, the average sound detection rate sequence for all subjects, with the key event (T2) as the zero point, is as follows: Figure 7 As shown, based on the baseline level, the dashed lines divide the sequence into two time periods: before the button press (-1.1 s to -0.26 s) and after the button press (0.26 s to 1.04 s). Power spectrum analysis was performed on the sound detection rate sequences for these two intervals, and the corresponding results are shown below. Figure 8 (Before pressing the button) and Figure 9 (After pressing the button) As shown; the dashed lines in both figures represent the 95% significance threshold obtained using the same permutation test and correction method.

[0123] Analysis of the above results shows that this invention introduces the detection of pre-key press attentional oscillations, overcoming the shortcomings of existing paradigms. By performing intensive sampling before the key reset occurs, it accurately detects whether attentional oscillations exist before and during the formation of the key press intention. Furthermore, by introducing a dual time reference system of the visual stimulus initiation time (T1) and the key point of the key press pressure event (T2), it effectively separates the reset effects caused by exogenous visual stimuli and endogenous key press operations, respectively, achieving independent analysis and comprehensive examination of the two types of mechanisms. Specifically, from Figure 5 and Figure 6 As can be seen, after the visual stimulus is presented (but before the button reset), auditory attention exhibits significant oscillations in the theta band; while from Figure 7 , Figure 8 and Figure 9 As can be seen, no significant attentional oscillations were detected before the key was pressed, while significant theta-band oscillations were observed in auditory attention after the key was pressed. This result indicates that the modulatory effects of exogenous visual stimuli and endogenous key presses on attentional oscillations are temporally separable. Visual stimuli can independently trigger attentional reset and induce subsequent oscillations, while the key-triggered reset and its subsequent oscillation effects only occur after the key press is executed. In summary, the above results verify that this invention can effectively reveal the oscillatory characteristics of auditory attention before and after a reset event, providing a direct testing device and method for studying the mechanisms of exogenous and endogenous attention regulation.

[0124] In addition, the changes in screen brightness and key pressure in a single test are as follows: Figure 10 As shown, the key pressure event (time zero T2) locates the moment when the key pressure reaches its peak.

[0125] Example 2

[0126] In this embodiment, based on Figure 1 The multimodal attentional oscillation test apparatus shown was used to compare and test the amplitude recording of sound generated by 20 consecutive key presses with and without a mute isolation unit. Specifically, key press sounds were recorded in a soundproof room using a Yamaha UR22 sound card microphone set, and data was acquired using Lab Streaming Layer software. When the mute isolation unit 1 was removed, the test subject directly pressed the reset button 4, which has a pressure sensor 2 attached to its surface. The amplitude recording of the sound generated by the key press is as follows: Figure 11 As shown; and after adding the noise isolation unit 1, the amplitude of the sound produced by the same button operation is recorded as follows. Figure 12 As shown in the figure. The comparison results show that after using the silent isolation unit 1 designed in this invention, the sound amplitude generated by the button operation is significantly reduced, effectively suppressing noise and verifying the feasibility and superiority of the silent structure design. It can greatly reduce the interference on the multimodal attention oscillation test results.

[0127] The embodiments described above are merely some preferred implementations of the present invention and are not intended to limit the invention. Those skilled in the art can make various changes and modifications without departing from the spirit and scope of the invention. Therefore, all technical solutions obtained through equivalent substitution or transformation fall within the protection scope of the present invention.

Claims

1. A multimodal attention oscillation testing device, characterized in that, It includes a pressure sensor (2), a light sensor (3), a reset button (4), a host computer (5), a microcontroller (6), and a sound transmission system (7); The reset button (4) is placed in the silent isolation unit (1) to isolate the noise generated during the pressing of the button (4); a pressure sensor (2) is attached to the surface of the reset button (4) to collect pressure signals during the process of the test subject pressing the reset button (4); The host computer (5) has a GUI display and control module, which is used to display different interface patterns in the screen display interface according to the preset attention oscillation research paradigm. The interface patterns that appear in each trial are: a visual stimulus pattern and an auxiliary marker pattern that appear synchronously, an inquiry interface that asks the subject to judge whether they hear the sound stimulus and asks them to input the judgment result, and a prompt interface that prompts whether the timing of pressing the reset button (4) in this trial is accurate. In the GUI display and control module, the duration of a single trial is set to be no less than 4 seconds. A gray screen with a duration of 1 second is first presented. The visual stimulus pattern and the auxiliary marker pattern appear synchronously after the gray screen ends. The standard time for pressing the reset button (4) is 1.5 seconds after the visual stimulus pattern is presented. The sound stimulus time window is from 0.3 seconds to 2.7 seconds after the visual stimulus pattern is presented. The inquiry interface appears 3 seconds after the visual stimulus pattern. The light sensor (3) is attached to the screen of the host computer (5), and its sensing area corresponds to and only corresponds to the display area of ​​the auxiliary mark pattern, for capturing the time of appearance of the auxiliary mark pattern; The microcontroller (6) is connected to the light sensor (3), the host computer (5) and the pressure sensor (2) on the reset button (4) respectively. The electrical signals detected by the light sensor (3) and the pressure sensor (2) are collected by the microcontroller (6) in real time and transmitted to the host computer (5). The sound transmission system (7) is connected to the host computer (5) for randomly playing sound stimuli to the subject within the sound stimulation time window of each trial according to the instructions issued by the host computer (5). The sound stimulation time window is a time window centered on the standard moment when the reset button (4) is pressed.

2. The multimodal attention oscillation testing device as described in claim 1, characterized in that, The silent isolation unit (1) includes a latex pad (102) and a sound insulation cotton cover (104), which are assembled to form a closed enclosure that only allows the hand to enter and exit, and the surface of the latex pad (102) is covered with a mesh cloth.

3. The multimodal attention oscillation testing device as described in claim 1, characterized in that, The host computer (5) is also connected to a keyboard (10), and the judgment result is input through two different keys on the keyboard (10).

4. The multimodal attention oscillation testing device as described in claim 1, characterized in that, The optical sensor (3) uses a photodiode with an integrated signal amplifier. The current output by the photodiode is converted into a voltage signal by the signal amplifier and then sent to the microcontroller (6).

5. The multimodal attention oscillation testing device as described in claim 1, characterized in that, The light sensor (3) is attached and fixed to the edge of the screen of the host computer (5), the display area of ​​the auxiliary marking pattern is located at the corresponding attachment position of the light sensor (3), and the visual stimulation pattern is presented in the central area of ​​the screen.

6. The multimodal attention oscillation testing device as described in claim 1, characterized in that, The pressure sensor (2) is a piezoresistive force-sensitive resistor. The resistance change of the piezoresistive force-sensitive resistor is converted into a linear voltage signal by a signal amplifier and then sent to the microcontroller (6).

7. The multimodal attention oscillation testing device as described in claim 1, characterized in that, The host computer (5) is also connected to an external physiological signal monitoring device, and the monitored physiological signals include electromyography or respiration.

8. The multimodal attention oscillation testing device as described in claim 1, characterized in that, In the GUI display and control module, the prompt interface appears immediately after the moment the subject completes the input of the judgment result, and the single trial ends after the prompt interface disappears.

9. A method for acquiring multimodal attention oscillation test data, characterized in that, This acquisition method is based on the multimodal attention oscillation testing device as described in claim 1, and includes: S1. After the host computer (5) receives the start command, the GUI display control module starts to execute a new trial. First, a gray screen is displayed. After the gray screen ends, visual stimulation patterns and auxiliary marking patterns are displayed synchronously on the screen display interface. The microcontroller (6) collects the electrical signal output by the light sensor (3) in real time and transmits it to the host computer (5) to identify and record the first moment when the auxiliary marking pattern appears on the screen. S2. Randomly select a time point within the sound stimulation time window, and play sound stimulation to the subject according to the predetermined sound stimulation paradigm. The sound stimulation paradigm includes randomly selected sound trials and no sound trials. The intensity of the sound stimulation emitted in the sound trials is controlled at the subject's hearing threshold level. The microcontroller (6) collects the electrical signal output by the pressure sensor (2) on the reset button (4) in real time and transmits it to the host computer (5) to identify and record the second moment when the subject presses the reset button (4). S3. After the sound stimulus time window ends, an inquiry interface is displayed on the screen, asking the test subject to judge whether the sound stimulus was heard in the current trial. The microcontroller (6) collects the input judgment result in real time and transmits it to the host computer (5) for recording. Then, based on the actual record of the sound stimulus played in the current trial, the microcontroller gives the test subject the correct or incorrect result of whether the sound stimulus was heard in the current trial. S4. Based on the second moment recorded in the current trial when the subject pressed the reset button (4) and the standard moment when the reset button (4) was pressed, a prompt interface is displayed on the screen to prompt the subject whether the timing of pressing the reset button (4) in this trial is accurate, until the next trial. S5. S1~S4 are continuously executed in a loop until all trials are completed. The host computer (5) records the data collected in all trials for multimodal attention oscillation analysis. The host computer (5) uses the first time or the second time as the zero point of time to calculate the time difference between the actual playback time of the sound stimulus in each trial and the zero point of time. Then, the data is grouped according to the time difference and the sound detection rate corresponding to different time differences is statistically calculated based on the correct and incorrect results to obtain the sound detection rate sequence, so as to reflect the oscillation characteristics of auditory attention before and after different reset events.

Images