Method and system for marking sound events of evoked brain electrical activity for general purpose computers

Abstract

This invention provides a method and system for labeling sound events that induce electroencephalograms (EEGs) using a general-purpose computer, applied in the field of EEG processing technology. The method includes: in response to receiving an EEG acquisition command, constructing N target sound data sequences corresponding to N test trials, wherein the first target sound data segment in the N target sound data sequences has different audio data characteristics from the other target sound data segments; based on a callback function, generating and sending an event labeling command to an EEG acquisition device according to multiple target sound data segments in the N target sound data sequences, so as to acquire N sets of EEG signals corresponding to the N test trials generated by the test subject based on the stimulus audio obtained from the N target sound data sequences; and based on the event identifiers in the N sets of EEG signals, performing time axis alignment and superposition averaging processing on the N sets of EEG signals to obtain target-related potential signals.

Description

Technical Field

[0001] This invention relates to the field of electroencephalogram (EEG) processing technology, and more specifically to a method and system for labeling sound events that induce EEG in a general-purpose computer. Background Technology

[0002] Electroencephalography (EEG) signals are important electrophysiological indicators used to reflect the brain activity of a target subject. Evoked potentials (EPPs) can include spontaneous potentials (SVPs) and evoked potentials (EPPs). Evoked potentials typically require predetermined stimulation of the target subject's nervous system to obtain evoked potentials with a relatively strict time-locked relationship with the stimulus. For example, auditory evoked potentials can be obtained by stimulating the target subject with sound.

[0003] In the process of realizing the above-mentioned inventive concept, it was found that: in the process of acquiring EEG signals containing auditory evoked potentials, the related technology has a relatively complex multi-threaded operation mode due to the platform environment of sound scheduling and processing. While scheduling and processing various types of tasks, the audio thread for sound stimulation of the target object is easily preempted, resulting in unsmooth audio playback and a large offset error between the time of EEG signal tagging and the time of audio stimulation, thus making it difficult to align multiple sets of EEG signals. Summary of the Invention

[0004] In view of the above problems, the present invention provides a method and system for labeling sound events that induce electroencephalograms (EEGs) using a general-purpose computer.

[0005] According to a first aspect of the present invention, a method for labeling sound events that induce electroencephalograms (EEGs) in a general-purpose computer is provided, comprising: in response to receiving an EEG acquisition command, constructing N target sound data sequences corresponding to N test trials, wherein each of the N target sound data sequences includes multiple target sound data segments, and the first target sound data segment in the N target sound data sequences has different audio data characteristics from the other target sound data segments; generating and sending an event labeling command to an EEG acquisition device based on a callback function, according to the multiple target sound data segments in the N target sound data sequences, to acquire N sets of EEG signals corresponding to the N test trials generated by the test subject based on the stimulus audio obtained from the N target sound data sequences; and performing time axis alignment and superposition averaging on the N sets of EEG signals based on the event identifiers in the N sets of EEG signals to obtain target-related potential signals.

[0006] A second aspect of the present invention provides a system for labeling sound events that induce electroencephalograms (EEGs) for a general-purpose computer, comprising: a processor configured to, in response to receiving an EEG acquisition command, construct N target sound data sequences corresponding to N test trials; sequentially play N stimulus audios to a test subject based on multiple target sound data segments in the N target sound data sequences, simultaneously generating an event labeling command and sending it to an EEG acquisition device; upon receiving N sets of EEG signals, performing time axis alignment and superposition averaging processing on the N sets of EEG signals based on event identifiers in the N sets of EEG signals to obtain target-related potential signals; and an EEG acquisition device electrically connected to the processor, which, in response to receiving the event labeling command, performs labeling processing on the acquired EEG signals to obtain N sets of EEG signals and sends them to the processor.

[0007] A third aspect of the present invention provides an electronic device comprising: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors perform the method described above.

[0008] A fourth aspect of the present invention also provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, cause the processor to perform the methods described above.

[0009] A fifth aspect of the invention also provides a computer program product, including a computer program that, when executed by a processor, implements the above-described method.

[0010] The method and system for labeling evoked EEG sound events for a general-purpose computer according to the present invention, based on the application scenario of a general-purpose computer system, responds to receiving an EEG acquisition command, and after establishing relevant communication and initialization settings between the computer and the EEG acquisition device, constructs N target sound data sequences to facilitate systematic audio stimulation of the test subject. Then, based on multiple target sound data segments in the target sound data sequences, it is determined whether the stimulation audio played to the test subject is at a valid playback time, thereby sending an event labeling command to the EEG acquisition device to perform high-accuracy and low-latency labeling processing on the EEG signal corresponding to the valid playback time. This achieves high-precision labeling processing of multiple sets of EEG signals under multiple test trials using a low-resource-consumption hardware event-driven callback mode, while ensuring high stability, high reliability, low latency (close to millisecond-level latency), and low jitter.

[0011] Furthermore, by using callback functions in the hardware event-driven callback mode, and while minimizing reliance on the host computer's performance (i.e., minimizing brief stutters or performance fluctuations due to multi-threaded operation), it is possible to independently trigger event markers and label EEG signals with high temporal precision. This facilitates the alignment and subsequent processing of N sets of EEG signals with event markers after N test trials. It is compatible with current mainstream operating systems while maintaining low resource consumption, is easily expandable, and possesses high application value. Attached Figure Description

[0012] The above-described features, other objects, and advantages of the present invention will become clearer from the following description of embodiments of the invention with reference to the accompanying drawings, in which:

[0013] Figure 1 The illustration shows an application scenario of a method for labeling sound events that induce electroencephalograms for a general-purpose computer according to an embodiment of the present invention.

[0014] Figure 2 A flowchart is shown of a method for tagging sound events that induce electroencephalograms for a general-purpose computer according to an embodiment of the present invention;

[0015] Figure 3 A flowchart illustrating the generation of the nth set of stimulus audio according to an embodiment of the present invention is shown;

[0016] Figure 4 A flowchart illustrating the playback of multiple target sound data segments according to an embodiment of the present invention is shown;

[0017] Figure 5 A schematic diagram of a method for tagging sound events that induce electroencephalograms according to an embodiment of the present invention is shown;

[0018] Figure 6 A schematic diagram comparing the labeling of EEG signals using the method of the present invention and the labeling of EEG signals using conventional methods is shown in an embodiment of the present invention.

[0019] Figure 7 A schematic diagram of a system for tagging sound events that induce electroencephalograms for a general-purpose computer according to an embodiment of the present invention is shown;

[0020] Figure 8 A block diagram of an electronic device for a method of tagging sound events that induce electroencephalograms for a general-purpose computer according to an embodiment of the present invention is shown. Detailed Implementation

[0021] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. In the following detailed description, numerous specific details are set forth to provide a thorough understanding of the embodiments of the invention for ease of explanation. However, it will be apparent that one or more embodiments may be practiced without these specific details. Furthermore, descriptions of well-known structures and techniques are omitted in the following description to avoid unnecessarily obscuring the concept of the invention.

[0022] The terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the invention. The terms “comprising,” “including,” etc., as used herein indicate the presence of the stated features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0023] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art, unless otherwise defined. It should be noted that the terms used herein are to be interpreted in a manner consistent with the context of this specification, and not in an idealized or overly rigid way.

[0024] When using expressions such as "at least one of A, B and C", they should generally be interpreted in accordance with the meaning that is commonly understood by those skilled in the art (e.g., "a system having at least one of A, B and C" should include, but is not limited to, a system having A alone, a system having B alone, a system having C alone, a system having A and B, a system having A and C, a system having B and C, and / or a system having A, B and C, etc.).

[0025] In the technical solution of this invention, the user information (including but not limited to user personal information, user image information, user device information, such as location information) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, storage, use, processing, transmission, provision, disclosure, and application of related data all comply with relevant laws, regulations, and standards, take necessary confidentiality measures, do not violate public order and good morals, and provide corresponding operation entry points for users to choose to authorize or refuse.

[0026] Electroencephalogram (EEG) is a signal used to record brain activity using electrophysiological indicators. EEG reflects the electrical wave changes resulting from the sum of postsynaptic potentials of numerous neurons in the cerebral cortex during brain activity. EEG signals can be categorized into two types: spontaneous potentials and evoked potentials. Spontaneous potentials exhibit irregular, random fluctuations over a short period; while evoked potentials refer to the bioelectrical responses produced by the nervous system in response to specific stimuli or when the brain processes information from stimuli (positive or negative), exhibiting a strictly time-locked relationship and fixed phase.

[0027] Event-related potentials (ERPs) are a special type of evoked brain electrical activity. ERPs utilize the electrical potential changes in the brain caused by multiple or diverse stimuli, which assign specific meaning to those stimuli. ERPs can be used to reveal the subtle voltage fluctuations that occur during cognitive processing such as perception, attention, and memory. ERPs provide millisecond-level temporal resolution, accurately revealing the dynamic time process of information processing in the brain. There are various classification methods related to ERPs. Based on the stimulation channel, they can be divided into auditory evoked potentials (AEPs), visual evoked potentials, and somatosensory evoked potentials; based on latency, they can be divided into early latency evoked potentials, middle latency evoked potentials, late (long) latency evoked potentials, and slow waves.

[0028] Auditory evoked potentials (AEPs) refer to the activation of the auditory nervous system through specific sound stimuli, which in turn triggers a detectable bioelectrical response in the central nervous system. When a sound stimulus acts on the auditory pathway, it originates in the cochlea, travels through the auditory nerve, and finally reaches the cerebral cortex. During this process, sound is gradually converted into electrical signals. Different neural structures generate waveforms with specific characteristics when transmitting these signals. For example, a short click sound, a common stimulus signal, is generated by sending a square wave with a duration of 50-200µs to a speaker or headphones. It has a wide spectrum, with energy concentrated in the 2-4kHz range, effectively synchronizing neural output and producing a clear response wave. By precisely placing electrodes on the scalp, these waveforms can be captured, and their latency, amplitude, and other key parameters can be analyzed in depth to accurately determine the functional state of the auditory system and the location of lesions. Auditory EEG-evoked AEPs can be further categorized into early latency (0-10 ms, such as auditory brainstem response, ABR), intermediate latency (10-50 ms, such as 40 Hz event-related potentials), and long latency responses (>50 ms, such as P300) based on their latency. Each type of response plays an irreplaceable role in clinical diagnosis and neuroscience research.

[0029] Auditory evoked potentials (AEPs) have been widely used in scientific research, clinical diagnosis, and assessment due to their outstanding advantages, including high temporal resolution at the millisecond level, non-invasiveness, relatively low cost, and rich component indicators. Their applications extend beyond assessing the integrity of the auditory pathway to the diagnosis, monitoring, and prognosis of various diseases, including mental disorders, attention deficit hyperactivity disorder (ADHD), consciousness disorders, neurological diseases, and age-related brain dysfunction. They are an indispensable tool for exploring higher cognitive functions of the brain and for clinical neuropsychological assessment.

[0030] However, if the time offset of event labeling and EEG signal recording is unstable, the various EEG signal segments are difficult to align when all EEG signals are superimposed, leading to a weakening of the amplitude of event-related potential components and a shift in latency, thus affecting research conclusions. During the development process, researchers discovered that in acquiring EEG signals containing auditory evoked potentials, the platform environment for sound scheduling and processing has a complex multi-threaded operation. While scheduling and processing various types of tasks, the audio thread for stimulating the target object is easily preempted, resulting in choppy audio playback and a large offset error between the timing of EEG signal labeling and the timing of audio stimulation, thus creating a technical problem where multiple sets of EEG signals are difficult to align.

[0031] In view of this, embodiments of the present invention provide a method for labeling sound events that induce electroencephalograms (EEGs) using a general-purpose computer, comprising: in response to receiving an EEG acquisition command, constructing N target sound data sequences corresponding to N test trials, wherein each of the N target sound data sequences includes multiple target sound data segments, and the first target sound data segment in the N target sound data sequences has different audio data characteristics from the other target sound data segments; based on a callback function, generating and sending an event labeling command to an EEG acquisition device according to the multiple target sound data segments in the N target sound data sequences, so as to acquire N sets of EEG signals corresponding to the N test trials generated by the test subject based on the stimulus audio obtained from the N target sound data sequences; and based on the event identifiers in the N sets of EEG signals, performing time axis alignment and superposition averaging on the N sets of EEG signals to obtain target-related potential signals.

[0032] Figure 1 The diagram illustrates an application scenario of a method for labeling sound events that induce electroencephalograms for a general-purpose computer, according to an embodiment of the present invention.

[0033] like Figure 1 As shown, the application scenario according to this embodiment may include a general-purpose computer system comprising a processor 101, a target object 102, and a acquisition device 103. The processor 101 is used to send stimulus audio to the target object 102.

[0034] Users can use the processor 101 to interact with the target object 102 and the acquisition device 103 to receive or send signals, etc.

[0035] The acquisition device 103 can be a device for acquiring the electroencephalogram (EEG) signals of the target object 102, such as acquiring and processing the EEG signals generated by the target object 102 (for example only). The acquisition device 103 can also send the acquired EEG signals to the processor 101. The processor 101 can process the EEG signals.

[0036] It should be noted that the method for marking evoked EEG sound events for a general-purpose computer provided in this embodiment of the invention can generally be executed by processor 101. Correspondingly, the apparatus for marking evoked EEG sound events for a general-purpose computer provided in this embodiment of the invention can generally be located in processor 101. The method for marking evoked EEG sound events for a general-purpose computer provided in this embodiment of the invention can also be executed by a processor cluster different from processor 101 and capable of communicating with acquisition device 103 and / or acquisition device 103. Correspondingly, the apparatus for marking evoked EEG sound events for a general-purpose computer provided in this embodiment of the invention can also be located in a processor cluster different from processor 101 and capable of communicating with acquisition device 103 and / or acquisition device 103.

[0037] It should be understood that Figure 1 The number of processors, target objects, and acquisition devices shown is merely illustrative. Depending on implementation needs, any number of processors, target objects, and acquisition devices can be used.

[0038] The following will be based on Figure 1 The described scene, through Figures 2-7 A method for tagging sound events that induce electroencephalograms for a general-purpose computer, according to a disclosed embodiment, will be described in detail.

[0039] Figure 2 A flowchart is shown of a method for tagging sound events that induce electroencephalograms for a general-purpose computer according to an embodiment of the present invention.

[0040] like Figure 2 As shown, the method for labeling sound events that induce EEG in this embodiment includes operations S210 to S230.

[0041] In operation S210, in response to receiving the EEG acquisition command, N target sound data sequences corresponding to N test trials are constructed.

[0042] Each of the N target sound data sequences can include multiple target sound data segments, and the first target sound data segment in the N target sound data sequences has different audio data features from the other target sound data segments, where N≥1.

[0043] First, parameters can be preset and initialized for the computer equipment and EEG acquisition device. After the initialization of all parameters is completed, in response to the received EEG acquisition command, a communication connection is established between the audio playback device and the EEG acquisition device.

[0044] Based on a predetermined number of test trials and the original audio data corresponding to each test trial, each original audio data is processed separately to construct N target sound data sequences corresponding to the N test trials.

[0045] In operation S220, based on the callback function, an event marking instruction is generated and sent to the EEG acquisition device according to multiple target sound data segments in the N target sound data sequences, so as to obtain N sets of EEG signals corresponding to the N test trials generated by the test subject based on the stimulation audio obtained from the N target sound data sequences.

[0046] Having obtained the target sound data sequence corresponding to each test trial, each target sound data segment in the target sound data sequence is called sequentially, and each target sound data segment is subjected to audio conversion processing, thereby obtaining the stimulus audio corresponding to each target sound data sequence and playing it to the test subject.

[0047] Meanwhile, during the audio conversion process of each target sound data segment, an event marking instruction is triggered and sent to the EEG acquisition device at a predetermined time according to the different target sound data segments. In conjunction with the playback of the stimulus audio, each group of EEG signals acquired in real time by the EEG acquisition device is accurately marked with small error at the predetermined time to obtain N groups of EEG signals with event labels.

[0048] In operation S230, based on the event identifiers in the N sets of EEG signals, the N sets of EEG signals are time-axis aligned and superimposed and averaged to obtain the target-related potential signal.

[0049] After labeling, the EEG signals corresponding to each test trial are obtained, and event markers are added to the EEG signal locations corresponding to the start of the truly effective stimulus audio playback. Based on the event markers in each group of EEG signals, each group of EEG signals is aligned, superimposed, and averaged to obtain the target-related potential signal.

[0050] According to an embodiment of the present invention, it should be noted that the target-related potential signal obtained here is only used as reference information for medical staff to make diagnoses, and is not used as a direct diagnostic result.

[0051] According to an embodiment of the present invention, in an application scenario relying on a general-purpose computer system, in response to receiving an EEG acquisition command, after establishing relevant communication and initialization settings between the computer and the EEG acquisition device, N target sound data sequences are constructed to facilitate systematic audio stimulation of the test subject. Then, based on multiple target sound data segments in the target sound data sequences, it is determined whether the stimulation audio played to the test subject is at a valid playback moment, thereby sending an event marking command to the EEG acquisition device to perform high-accuracy and low-latency marking processing on the EEG signal corresponding to the valid playback moment. This achieves high-precision marking processing of multiple sets of EEG signals under multiple test trials using a low-resource-consumption hardware event-driven callback mode, while ensuring high stability, high reliability, low latency (close to millisecond-level latency), and low jitter.

[0052] Furthermore, by using callback functions in the hardware event-driven callback mode, and while minimizing reliance on the host computer's performance (i.e., minimizing brief stutters or performance fluctuations due to multi-threaded operation), it is possible to independently trigger event markers and label EEG signals with high temporal precision. This facilitates the alignment and subsequent processing of N sets of EEG signals with event markers after N test trials. It is compatible with current mainstream operating systems while maintaining low resource consumption, is easily expandable, and possesses high application value.

[0053] It should be noted that the target-related potential signal obtained by the present invention is only an intermediate result and cannot be used to directly derive diagnostic results or health status from the target-related potential signal obtained by the method according to the present invention.

[0054] Figure 3 A flowchart illustrating the generation of the nth set of stimulus audios according to an embodiment of the present invention is shown.

[0055] like Figure 3 As shown, the method of this embodiment includes operations S310 to S330.

[0056] In operation S310, in response to the audio application interface retrieving the first target sound data segment corresponding to the nth target sound data sequence from the output buffer, a callback function is triggered and used to retrieve the next target sound data segment corresponding to the nth target sound data sequence.

[0057] In operation S320, the next target sound data segment is identified and judged to obtain the nth judgment result.

[0058] In operation S330, if the nth judgment result indicates that the data segment identifier of the next target sound data segment is correct, the next target sound data segment is written into the input buffer so that the audio application interface can retrieve it and generate the nth set of stimulus audio corresponding to the nth target sound data sequence.

[0059] This invention can be applied only to general-purpose computer systems (general-purpose operating systems). General-purpose operating systems typically provide audio application interfaces (APIs) that delve into the hardware level, but they do not include the development and runtime environment for complex programming software such as FPGAs. For example, Windows provides WASAPI, Linux provides ALSA, and macOS provides Core Audio. Combined with the audio application interface of a general-purpose operating system, a hardware event-driven callback mode (callback functions) is provided, allowing for more precise control of audio playback timing and the completion of low-latency operations.

[0060] The input buffer can be characterized as a first write buffer corresponding to the target audio data segment, and the output buffer can be characterized as a second write buffer corresponding to the target audio data segment.

[0061] Identification judgment can be characterized as the judgment of the correlation between the next target audio data segment and the first target audio data segment.

[0062] Through the dual buffering of input and output buffers, a short buffer time is provided for high-accuracy and low-latency tagging of the EEG signal corresponding to the effective playback moment during the process of retrieving any specific target sound data segment from multiple target sound data segments and being retrieved by the audio application interface in the general operating system. This allows for high-precision alignment and adaptation of the EEG signal with the event tag to the effective stimulus audio.

[0063] If the nth judgment result indicates that the data segment identifier of the next target sound data segment is correct and it is the second target sound data segment, the target sound data segment is written into the input buffer so that it can be retrieved by the audio application interface, thereby being converted into a small segment of stimulus audio corresponding to the target sound data segment and played to the test object.

[0064] According to an embodiment of the present invention, after retrieving the first target sound data segment in the sequence, a callback function is triggered. The callback function is used to retrieve the next target sound data segment, and the next target sound data segment is judged. If the data segment identifier of the next target sound data segment is determined to be correct (the second target sound data segment), it is written into the input buffer for retrieval by the audio application interface in the general operating system. Thus, the above-described cyclical operation is performed on multiple target sound data segments within the target sound data sequence to generate high-quality stimulus audio corresponding to the nth test trial through near-uninterrupted continuous generation of multiple small audio segments through parsing, conversion, and playback with extremely low latency.

[0065] According to an embodiment of the present invention, the method further includes: while writing the next target sound data segment into the input buffer, determining the nth event identifier for labeling the nth group of EEG signals based on the sequence identifier of the nth target sound data sequence.

[0066] According to an embodiment of the present invention, an event tagging instruction corresponding to the nth target sound data sequence is generated based on the nth event identifier and sent to the EEG acquisition device so that the EEG acquisition device can perform tagging processing on the acquired EEG signals according to the event tagging instruction.

[0067] Event identifiers can be characterized as identifiers for display in EEG signals, and can correspond to the nth target sound data sequence to indicate that the EEG signals were collected in the nth test.

[0068] According to an embodiment of the present invention, after retrieving the first target sound data segment in the sequence, a callback function is triggered to determine the next target sound data segment retrieved by the callback function. It is determined whether the stimulus audio to be played to the test subject is in the valid playback audio stage (i.e., the second target sound data segment represents the beginning stage of valid playback audio). An event marking instruction is then sent to the EEG acquisition device to perform high-accuracy and low-latency marking processing on the EEG signal corresponding to the start time of the valid playback audio. This achieves high-precision marking processing of multiple sets of EEG signals across multiple test trials using a low-resource-consumption hardware event-driven callback mode, ensuring high stability, high reliability, low latency (close to millisecond-level latency), and low jitter.

[0069] According to an embodiment of the present invention, the process of identifying and determining the next target sound data segment to obtain the nth determination result may specifically include the following operations.

[0070] According to an embodiment of the present invention, the data segment identifier of the second target sound data segment in the nth target sound data sequence is obtained.

[0071] According to an embodiment of the present invention, the data segment identifier of the next target sound data segment is identified based on the data segment identifier of the second target sound data segment and the data segment identifier of the first target sound data segment, and the nth identification result is obtained.

[0072] In cases where the data segment identifier of the next target sound data segment is the same as that of the second target sound data segment, and is arranged in order with the data segment identifier of the first target sound data segment, the nth judgment result is confirmed to indicate that the data segment identifier of the next target sound data segment is correct.

[0073] For example, the data segment identifier of the next target sound data segment itself can be 2, the data segment identifier of the second target sound data segment in the nth target sound data sequence can be 2, and the data segment identifier of the first target sound data segment can be 1. An auditory evoked procedure is used to determine the identifier of the next target sound data segment. If it is confirmed that the data segment identifier of the next target sound data segment is the same as that of the second target sound data segment, and the next target sound data segment is a continuous data segment with the first target sound data segment, then the determination result indicates that the data segment identifier is correct.

[0074] According to an embodiment of the present invention, by identifying the next target sound data segment based on the data segment identifier of the next target sound data segment, it is possible to determine whether the stimulus audio to be played to the test subject is the first valid playback audio. This allows for the sending of event marking instructions to the EEG acquisition device with high time accuracy, while sequentially performing corresponding conversion processing on subsequent target sound data segments to play stimulus audio to the test subject.

[0075] Furthermore, based on the above judgment, if the judgment result indicates an abnormal data segment (the data segment identifier of the next target audio data segment is different from that of the second target audio data segment, or the data segment identifier is not in order with that of the first target audio data segment), then for the next target audio data segment that is a valid audio data segment but not the second target audio data segment, the callback function is re-triggered to search and retrieve from multiple target audio data segments. If the re-retrieved target audio data segment passes the judgment, the subsequent operations proceed normally. If the re-retrieved target audio data segment still fails the judgment, the relevant operators are notified for warning, the current test session is closed, and the test jumps to the (n+1)th test session.

[0076] Figure 4 A flowchart illustrating the playback of multiple target sound data segments according to an embodiment of the present invention is shown.

[0077] like Figure 4 As shown, the method of this embodiment includes operations S410 to S430.

[0078] In operation S410, if the nth target sound data sequence includes M target sound data segments, in response to the audio application interface, the (m-1)th copied sound data segment corresponding to the (m-1)th target sound data segment is retrieved from the output buffer, the mth target sound data segment is copied, and the mth copied sound data segment is obtained and sent to the output buffer.

[0079] In operation S420, during the process of sending the m-th copied audio data segment to the output buffer, a callback function is triggered, and the callback function is used to retrieve the (m+1)-th target audio data segment from the M target audio data segments.

[0080] In operation S430, the (m+1)th target sound data segment is written to the input buffer, and the above copy operation, fetch operation and write operation are repeated until the Mth target sound data segment is written to the input buffer, generating the nth group of stimulus audio corresponding to the nth target sound data sequence.

[0081] Where M≥3, m≥2.

[0082] The nth target sound data sequence can include multiple target sound data segments. With input and output buffers, by repeating the above iterative operation on each target sound data segment, the nth set of stimulus audio corresponding to the nth target sound data sequence can be generated.

[0083] According to an embodiment of the present invention, in response to the call status of the audio application interface within a general-purpose operating system, a callback function is used to cyclically retrieve the remaining target sound data segments, write them into the input buffer, and copy them to the output buffer at the appropriate time for retrieval by the audio application interface. Thus, by parsing, converting, and playing multiple small audio segments, high-quality stimulus audio corresponding to the nth test can be generated almost uninterruptedly with extremely low latency.

[0084] According to an embodiment of the present invention, in response to receiving an EEG acquisition command, constructing N target sound data sequences corresponding to N test trials may include the following operations.

[0085] According to an embodiment of the present invention, based on the capacity information of the input buffer, the N valid sound data corresponding to the N test trials are segmented to obtain N valid sound data sequences.

[0086] Each valid sound data sequence includes multiple valid sound data segments;

[0087] According to an embodiment of the present invention, N debugging sound data segments are generated based on predetermined debugging rules and input buffer capacity information.

[0088] According to an embodiment of the present invention, N debugging audio data segments are all used as the first target audio data segments and inserted into N valid audio data sequences respectively to obtain N target audio data sequences.

[0089] Based on the capacity information of the input buffer, the size of each valid audio data segment in the resulting N valid audio data sequences can correspond to (be the same as) the capacity of the input buffer.

[0090] Predefined debugging rules can be characterized as feature rules used to generate audio data within a debugging audio data segment. For example, according to the predetermined debugging rules, the features of the audio data generated within the debugging audio data segment can all be 1, or they can be audio features with a regular pattern of 0101.

[0091] After generating the debug audio data segment, it is inserted before multiple valid audio data segments in any valid audio data sequence (equivalent to serving as the first target audio data segment), thereby obtaining the final target audio data sequence.

[0092] According to an embodiment of the present invention, by segmenting N valid sound data corresponding to N test trials based on the capacity information of the input buffer or output buffer, and generating N debug sound data segments, which are then inserted, N target sound data sequences can be constructed. This achieves the generation of the first target sound data segment with specific symbolic meaning, as well as the accurate generation of the final N target sound data sequences. Simultaneously, by adding debug sound data segments before the original stimulus audio to assist in determining the generation of event labeling instructions and the precise timing of labeling the EEG signals, a low-resource-consumption hardware event-driven callback mode can be utilized to perform high-precision labeling processing on multiple sets of EEG signals under multiple test trials while ensuring high stability, high reliability, low latency (close to millisecond-level latency), and low jitter.

[0093] According to an embodiment of the present invention, based on the capacity information of the input buffer, the N valid sound data corresponding to the N test trials are segmented to obtain N valid sound data sequences, which may include the following operations.

[0094] According to an embodiment of the present invention, the duration information of each valid audio data segment is determined based on the capacity information of the input buffer and the audio sampling rate.

[0095] According to an embodiment of the present invention, based on the duration information of the valid audio data segments, N valid audio data are segmented to obtain N initial audio data sequences. Each initial audio data sequence includes multiple initial audio data segments.

[0096] According to an embodiment of the present invention, if the duration of the last initial sound data segment in N initial sound data sequences is equal to the duration of the effective sound data segment, the N initial sound data sequences are determined as N effective sound data sequences.

[0097] Input and output buffers can typically have the same capacity. Specifically, both the input and output buffers can have a size of 2. N (e.g., 64, 128, 256, etc.).

[0098] For example, the size of both the input buffer and the output buffer can be 64. When the sound sampling rate is 48000Hz, the duration of the sound data corresponding to the input buffer and the output buffer can be 64÷48000=1.333ms.

[0099] Based on the capacity information of the input buffer and the duration information corresponding to the input buffer, the valid audio data corresponding to each test can be segmented to obtain multiple valid audio data segments with the same capacity information (and the same duration information) as the input buffer. According to the order of segmentation, multiple valid audio data sequences can be obtained.

[0100] According to embodiments of the present invention, the above method may further include the following operations.

[0101] According to an embodiment of the present invention, when the duration of the last initial sound data segment in any initial sound data sequence is not equal to the duration of the effective sound data segment, the audio data in the last initial sound data segment is padded based on a predetermined data rule and the duration of the effective sound data segment to obtain N processed sound data sequences.

[0102] According to an embodiment of the present invention, N processed sound data sequences are determined as N valid sound data sequences.

[0103] For valid audio data segments (usually the last audio data segment) that do not meet the same capacity information as the input buffer after segmentation, pre-defined audio data can be added after the valid audio data segment to make it the same as the capacity information of the input buffer. The pre-defined audio data can be audio 0 data.

[0104] According to an embodiment of the present invention, the first target sound data segment is a sound data segment in which all audio data features are 0.

[0105] Figure 5 A schematic diagram of a method for labeling sound events that induce electroencephalograms according to an embodiment of the present invention is shown.

[0106] like Figure 5 As shown, based on the operating background of a general-purpose operating system, the subject can undergo N trials of EEG signal acquisition. There is a stimulation interval between two adjacent trials. During the sound stimulation phase of each trial, the following operations are performed: the first target sound data segment corresponding to audio 0 is written to the input buffer, and then the first target sound data segment corresponding to audio 0 is copied from the input buffer to the output buffer. The audio application interface in the general-purpose operating system retrieves the first target sound data segment corresponding to audio 0 from the input buffer, and at the same time triggers a callback function to retrieve the second target sound data segment corresponding to audio 1. The second target sound data segment corresponding to audio 1 is judged based on the degree of auditory evokedness. If the judgment is successful, the second target sound data segment corresponding to audio 1 is written into the input buffer, and an event tagging signal is sent to the EEG acquisition device so that the EEG signal is marked with an event identifier at that moment. Then, the second target sound data segment corresponding to audio 1 is copied from the input buffer to the output buffer. The audio application interface retrieves the second target sound data segment corresponding to audio 1 from the input buffer, and at the same time triggers the callback function to retrieve the third target sound data segment corresponding to audio 2. This process continues until all target sound data segments in this trial have been retrieved and processed in the above manner. The sound stimulation phase of one trial is completed. After the stimulation interval ends, the sound stimulation phase of the next trial begins.

[0107] Figure 6 A schematic diagram comparing the labeling of EEG signals using the method of the present invention and the labeling of EEG signals using conventional methods is shown in an embodiment of the present invention.

[0108] like Figure 6 As shown, the sound signal is used as one channel of the EEG acquisition device, recorded synchronously with event markers, and all event markers and their corresponding sound signals are extracted and plotted, as shown below. Figure 6As shown in the figure, each curve represents a single trial of the sound signal (frequency 100Hz), with the event marker at time 0. It can be observed from the figure that the sound signal generated using the method of this invention has good synchronization, with an average delay of 1.76±0.34ms with the event marker in the EEG signal, and the sound signal curves of each trial almost overlap. In contrast, the sound signal generated using conventional methods has an average delay of 9.61±4.54ms with the event marker in the EEG signal, which is not only higher but also exhibits greater delay fluctuations, with the sound signal curves of each trial intersecting.

[0109] Figure 7 A schematic diagram of a system for tagging sound events that induce electroencephalograms for a general-purpose computer according to an embodiment of the present invention is shown.

[0110] like Figure 7 As shown, the system for tagging sound events that induce EEG in this embodiment includes a processor 710 and an EEG acquisition device 720.

[0111] The processor 710 is configured to, in response to receiving an EEG acquisition command, construct N target sound data sequences corresponding to N test trials; based on multiple target sound data segments in the N target sound data sequences, sequentially play N stimulus audios to the test subject, simultaneously generate an event marker command, and send it to the EEG acquisition device; upon receiving N sets of EEG signals, based on the event markers in the N sets of EEG signals, perform time axis alignment and superposition averaging processing on the N sets of EEG signals to obtain target-related potential signals.

[0112] The EEG acquisition device 720 is electrically connected to the processor. In response to receiving an event marking instruction, it marks the acquired EEG signals to obtain N sets of EEG signals and sends them to the processor.

[0113] Figure 8 A block diagram of an electronic device for a method of tagging sound events that induce electroencephalograms for a general-purpose computer according to an embodiment of the present invention is shown.

[0114] like Figure 8As shown, an electronic device according to an embodiment of the present invention includes a processor 801, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 802 or a program loaded from a storage portion 808 into a random access memory (RAM) 803. The processor 801 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or an associated chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 801 may also include onboard memory for caching purposes. The processor 801 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present invention.

[0115] RAM 803 stores various programs and data required for the operation of the electronic device. Processor 801, ROM 802, and RAM 803 are interconnected via bus 804. Processor 801 executes various operations of the method flow according to embodiments of the present invention by executing programs in ROM 802 and / or RAM 803. It should be noted that the programs may also be stored in one or more memories other than ROM 802 and RAM 803. Processor 801 may also execute various operations of the method flow according to embodiments of the present invention by executing programs stored in said one or more memories.

[0116] According to embodiments of the present invention, the electronic device may further include an input / output (I / O) interface 805, which is also connected to a bus 804. The electronic device may also include one or more of the following components connected to the I / O interface 805: an input section 806 including a keyboard, mouse, etc.; an output section 807 including a cathode ray tube (CRT), liquid crystal display (LCD), etc., and a speaker, etc.; a storage section 808 including a hard disk, etc.; and a communication section 809 including a network interface card such as a LAN card, modem, etc. The communication section 809 performs communication processing via a network such as the Internet. A drive 810 is also connected to the I / O interface 805 as needed. A removable medium 811, such as a disk, optical disk, magneto-optical disk, semiconductor memory, etc., is installed on the drive 810 as needed so that computer programs read from it can be installed into the storage section 808 as needed.

[0117] The present invention also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments; or it may exist independently and not assembled into the device / apparatus / system. The computer-readable storage medium carries one or more programs, which, when executed, implement the method according to the embodiments of the present invention.

[0118] According to embodiments of the present invention, a computer-readable storage medium may be a non-volatile computer-readable storage medium, such as including, but not limited to: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In the present invention, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. For example, according to embodiments of the present invention, a computer-readable storage medium may include ROM 802 and / or RAM 803 and / or one or more memories other than ROM 802 and RAM 803 described above.

[0119] Embodiments of the present invention also include a computer program product comprising a computer program containing program code for performing the methods shown in the flowchart. When the computer program product is run on a computer system, the program code is used to cause the computer system to implement the method for labeling auditory events that induce electroencephalograms provided in the embodiments of the present invention.

[0120] When the computer program is executed by the processor 801, it performs the functions defined in the system / apparatus of this invention. According to embodiments of the invention, the systems, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0121] In one embodiment, the computer program may rely on a tangible storage medium such as an optical storage device or a magnetic storage device. In another embodiment, the computer program may also be transmitted and distributed in the form of signals over a network medium, and may be downloaded and installed via the communication section 809, and / or installed from a removable medium 811. The program code contained in the computer program can be transmitted using any suitable network medium, including but not limited to: wireless, wired, etc., or any suitable combination thereof.

[0122] In such an embodiment, the computer program can be downloaded and installed from a network via communication section 809, and / or installed from removable medium 811. When the computer program is executed by processor 801, it performs the functions defined in the system of this embodiment of the invention. According to embodiments of the invention, the systems, devices, apparatuses, modules, units, etc., described above can be implemented by computer program modules.

[0123] Those skilled in the art will understand that the features described in the various embodiments and / or claims of the present invention can be combined or combined in various ways, even if such combinations or combinations are not explicitly described in the present invention. In particular, the features described in the various embodiments and / or claims of the present invention can be combined or combined in various ways without departing from the spirit and teachings of the present invention. All such combinations and / or combinations fall within the scope of the present invention.

Claims

1. A method for labeling auditory events that evoke electroencephalograms (EEGs) using a general-purpose computer, characterized in that, include: In response to receiving an EEG acquisition command, N target sound data sequences corresponding to N test trials are constructed, wherein each of the N target sound data sequences includes multiple target sound data segments, and the first target sound data segment in the N target sound data sequences has different audio data features from the other target sound data segments; Based on the callback function, an event marking instruction is generated and sent to the EEG acquisition device according to the multiple target sound data segments in the N target sound data sequences, so as to obtain N sets of EEG signals corresponding to the N test trials generated by the test subject based on the stimulation audio obtained from the N target sound data sequences through the EEG acquisition device. Based on the event identifiers in the N sets of EEG signals, the N sets of EEG signals are time-axis aligned and superimposed and averaged to obtain target-related potential signals.

2. The method according to claim 1, characterized in that, The step of generating and sending an event marking instruction to the EEG acquisition device based on the plurality of target sound data segments from the N target sound data sequences includes: In response to the audio application interface retrieving the first target sound data segment corresponding to the nth target sound data sequence from the output buffer, the callback function is triggered and used to retrieve the next target sound data segment corresponding to the nth target sound data sequence; The next target sound data segment is identified and judged to obtain the nth judgment result; If the nth judgment result indicates that the data segment identifier of the next target sound data segment is correct, the next target sound data segment is written into the input buffer so that the audio application interface can retrieve it and generate the nth set of stimulus audio corresponding to the nth target sound data sequence.

3. The method according to claim 2, characterized in that, The method further includes: While writing the next target sound data segment into the input buffer, the nth event identifier for labeling the nth group of EEG signals is determined according to the sequence identifier of the nth target sound data sequence. Based on the nth event identifier, an event tagging instruction corresponding to the nth target sound data sequence is generated and sent to the EEG acquisition device, so that the EEG acquisition device can tag the acquired EEG signals according to the event tagging instruction.

4. The method according to claim 2, characterized in that, The step of identifying and judging the next target sound data segment to obtain the nth judgment result includes: Obtain the data segment identifier of the second target sound data segment in the nth target sound data sequence; Based on the data segment identifier of the second target sound data segment and the data segment identifier of the first target sound data segment, the data segment identifier of the next target sound data segment is identified to obtain the nth identification result; wherein, if the data segment identifier of the next target sound data segment is the same as the data segment identifier of the second target sound data segment and is arranged in order with the data segment identifier of the first target sound data segment, the nth identification result is confirmed to indicate that the data segment identifier of the next target sound data segment is correct.

5. The method according to claim 2, characterized in that, When the nth judgment result indicates that the next target sound data segment is correctly identified, the next target sound data segment is written into the input buffer for retrieval by the audio application interface, including: In the case that the nth target sound data sequence includes M target sound data segments, In response to the audio application interface retrieving the (m-1)th copied audio data segment corresponding to the (m-1)th target audio data segment from the output buffer, copying the m-th target audio data segment, and obtaining and sending the m-th copied audio data segment to the output buffer; During the process of sending the m-th copied audio data segment to the output buffer, the callback function is triggered, and the callback function is used to retrieve the (m+1)-th target audio data segment from the M target audio data segments; Write the (m+1)th target sound data segment into the input buffer, and repeat the above copy, fetch and write operations until the Mth target sound data segment is written into the input buffer, generating the nth set of stimulus audio corresponding to the nth target sound data sequence.

6. The method according to claim 1, characterized in that, The step of responding to the received EEG acquisition command by constructing N target sound data sequences corresponding to N test trials includes: Based on the capacity information of the input buffer, the N valid sound data corresponding to the N test trials are segmented to obtain N valid sound data sequences, wherein each valid sound data sequence includes multiple valid sound data segments; Based on the predetermined debugging rules and the capacity information of the input buffer, N debugging sound data segments are generated; Each of the N debug audio data segments is used as the first target audio data segment and inserted into the N valid audio data sequences to obtain the N target audio data sequences.

7. The method according to claim 6, characterized in that, The step involves segmenting the N valid audio data corresponding to the N test trials based on the capacity information of the input buffer to obtain N valid audio data sequences, including: Based on the capacity information of the input buffer and the audio sampling rate, the duration information of each valid sound data segment is determined; Based on the duration information of the effective sound data segments, the N effective sound data are segmented to obtain N initial sound data sequences, wherein each initial sound data sequence includes multiple initial sound data segments. If the duration of the last initial sound data segment in the N initial sound data sequences is equal to the duration of the effective sound data segment, then the N initial sound data sequences are determined as the N effective sound data sequences.

8. The method according to claim 7, characterized in that, The method further includes: If the duration of the last initial sound data segment in any initial sound data sequence is not equal to the duration of the effective sound data segment, the audio data in the last initial sound data segment is padded based on the predetermined data rules and the duration of the effective sound data segment to obtain N processed sound data sequences. The N processed sound data sequences are determined as the N valid sound data sequences.

9. The method according to claim 6, characterized in that, The first target sound data segment is a sound data segment in which all audio data features are 0.

10. A system for labeling sound events that induce electroencephalograms (EEGs) on a general-purpose computer, applied to the method for labeling sound events that induce EEGs on a general-purpose computer as described in any one of claims 1-9, characterized in that, include: The processor is used to construct N target sound data sequences corresponding to N test trials in response to receiving an EEG acquisition command; Based on the multiple target sound data segments in the N target sound data sequences, N stimulus audios are played sequentially to the test subject, while an event marking instruction is generated and sent to the EEG acquisition device. Upon receiving N sets of EEG signals, based on the event identifiers in the N sets of EEG signals, the N sets of EEG signals are time-axis aligned and superimposed and averaged to obtain target-related potential signals. The EEG acquisition device is electrically connected to the processor. In response to receiving the event marking instruction, it performs tagging processing on the acquired EEG signals to obtain the N sets of EEG signals, and sends them to the processor.

Images