Signal acquisition system, marking device and storage medium for fNIRS and EEG combined use
By using high sampling rate marking devices and processing devices in the scenarios of fNIRS and EEG, the problem of fNIRS equipment obtaining incomplete marking operation time information is solved, and a more accurate analysis of brain function status is achieved.
Patent Information
- Application Number
- CN202410712149.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2024-06-04
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2044-06-04
AI Technical Summary
In the scenario where fNIRS and EEG are used together, the fNIRS device cannot obtain complete marking operation time information due to the difference in sampling rate, resulting in inaccurate analysis of brain function status.
A signal acquisition system is adopted, including an fNIRS signal acquisition module and an EEG signal acquisition module. The marking start marking signal is transmitted to both through the marking device, and the marking trigger operation is sampled using a sampling rate higher than the fNIRS sampling rate. The tag timing data is obtained and the timestamps are aligned by the processing device to ensure the accuracy of the analysis.
It is realized that the complete marking trigger operation time information is obtained in the scenario of fNIRS and EEG combined, ensuring that the processing device can accurately analyze changes in brain function status.
Smart Images

Figure CN118568610B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the technical field of physiological signal acquisition, and in particular to a signal acquisition system, a marking device and a storage medium for use in a scenario where fNIRS and EEG are used in combination. Background Art
[0002] fNIRS technology analyzes the changes in blood oxygen in the cerebral cortex by collecting the fNIRS signals of the subjects. EEG technology is a method of recording brain activity using electrophysiological indicators. It records the changes in electric waves during brain activity and is the overall reflection of the electrophysiological activities of brain nerve cells on the cerebral cortex or scalp surface. In order to conduct a comprehensive analysis of the brain function status of the subjects, fNIRS technology and EEG technology can be used together. Since fNIRS technology and EEG technology are two completely different technologies, the working principles and technical parameters of fNIRS devices and EEG devices are different. Therefore, many technical problems will be encountered in the process of combining fNIRS and EEG. In particular, when performing marking operations in the scenario where the two types of devices are used together, the sampling rates of the two types of devices are quite different. The sampling rate of the fNIRS device is much lower than that of the EEG device. Therefore, the acquisition of time information of marking operations by fNIRS devices and EEG devices is quite different. There may be a situation where EEG devices with higher sampling rates can obtain complete time information of all marking operations, while fNIRS devices with lower sampling rates can only obtain time information of part of the marking operations. That is, the fNIRS device may miss labels, resulting in the processor being unable to accurately analyze changes in brain function based on the acquired fNIRS signals, EEG signals and time information of marking operations. Summary of the invention
[0003] This application is proposed to solve the above technical problems in the prior art. This application aims to provide a signal acquisition system, a marking device and a storage medium for the scenario of fNIRS and EEG being used together, which can obtain the complete time information of each marking trigger operation for analysis or display in the scenario of fNIRS and EEG being used together, and can determine the corresponding relationship between the start time of the marking trigger operation and the acquisition time of the fNIRS signal and the EEG signal, so that the processing device can analyze the changes in brain function status more accurately based on this information.
[0004] According to a first solution of the present application, a signal acquisition system for a scenario in which fNIRS and EEG are used together is provided, wherein the signal acquisition system at least comprises an fNIRS signal acquisition module for acquiring fNIRS signals of a subject at a first sampling rate, an EEG signal acquisition module for acquiring EEG signals of a subject at a second sampling rate greater than the first sampling rate, and a marking device used in conjunction with the fNIRS signal acquisition module, the EEG signal acquisition module, and a processing device, wherein the marking device is configured to: at least transmit a marking start mark signal to the fNIRS signal acquisition module and the EEG signal acquisition module, respectively; when the frequency of the marking trigger operation When the sampling rate is greater than the first sampling rate, the first marking trigger signal generated in response to the marking trigger operation is sampled at a third sampling rate to obtain marking timing data containing time information of each event of interest that occurs, wherein the third sampling rate is not less than the frequency of the marking trigger operation; the marking timing data is transmitted to the processing device; the processing device is configured to: obtain the first fNIRS data containing the marking start mark from the fNIRS signal acquisition module, the first EEG data containing the marking start mark from the EEG signal acquisition module, and the marking timing data from the marking device for analysis or display.
[0005] According to a second scheme of the present application, a marking device for use in a scenario where fNIRS and EEG are used together is provided, wherein the marking device is used in conjunction with the fNIRS signal acquisition module, the EEG signal acquisition module, and the processing device, wherein the fNIRS signal acquisition module acquires the fNIRS signal of the subject at a first sampling rate, and the EEG signal acquisition module acquires the EEG signal of the subject at a second sampling rate greater than the first sampling rate; the marking device is configured to: at least transmit a marking start mark signal to the fNIRS signal acquisition module and the EEG signal acquisition module, respectively; when the frequency of the marking trigger operation is greater than the first sampling rate, sample the first marking trigger signal generated in response to the marking trigger operation at a third sampling rate to obtain marking timing data containing time information of each event of interest that has occurred, wherein the third sampling rate is not less than the frequency of the marking trigger operation; and output the marking timing data.
[0006] According to a third embodiment of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processing device, the computer program performs the following steps: obtaining first fNIRS data including a marking start mark from the fNIRS signal acquisition module, first EEG data including a marking start mark from the EEG signal acquisition module, and marking timing data from the marking device, wherein the fNIRS signal acquisition module acquires the fNIRS signal of the subject at a first sampling rate, and the EEG signal acquisition module acquires the fNIRS signal of the subject at a second sampling rate greater than the first sampling rate. The marking timing data is obtained by sampling the first marking trigger signal generated in response to the marking trigger operation by the marking device at a third sampling rate, wherein the third sampling rate is not less than the frequency of the marking trigger operation; displaying the first fNIRS data including the marking start mark and the first EEG data including the marking start mark in the time domain on the display interface, and saving the marking timing data, or displaying the first fNIRS data including the marking start mark and the first EEG data including the marking start mark in the time domain on the display interface, and the mark of the event of interest included in the marking timing data.
[0007] Compared with the prior art, the beneficial effects of the embodiments of the present application are:
[0008] The signal acquisition system for the fNIRS and EEG combined use scenario provided in the embodiment of the present application includes an fNIRS signal acquisition module, an EEG signal acquisition module, a processing device and a marking device. The marking device can transmit a marking start mark signal to the fNIRS signal acquisition module and the EEG signal acquisition module respectively, so that the fNIRS signal acquisition module and the EEG signal acquisition module can determine the time information of the start of marking based on the marking start mark signal. After acquiring the first fNIRS data and the first EEG data containing the marking start mark signal, the processing device can determine the corresponding relationship between the start time of the marking trigger operation and the acquisition time of the fNIRS signal and the EEG signal based on the marking start mark signal, so that the processing device can more accurately analyze the changes in brain function status based on this information. When the frequency of the marking trigger operation is greater than the first sampling rate, that is, when the fNIRS signal acquisition module cannot obtain the complete time information of all marking operations, the marking device samples the first marking trigger signal at a third sampling rate to obtain the marking time series data containing the time information of all events of interest. Since the third sampling rate is not less than the frequency of the marking trigger operation, the marking device can collect the time information of all events of interest. After the processing device aligns the timestamps of the acquired marking time series data, the first fNIRS data, and the first EEG data, the changes in brain function status can be more accurately analyzed based on the marking positions of the time information of each event of interest collected in the first fNIRS data, the first EEG data, and the marking time series data.
[0009] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above description and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] In the drawings, which are not necessarily drawn to scale, the same reference numerals may describe similar parts in different views. Similar reference numerals with letter suffixes or different letter suffixes may represent different examples of similar components. The accompanying drawings generally illustrate various embodiments by way of example and not by way of limitation, and together with the description and claims, are used to illustrate the disclosed embodiments. Such embodiments are illustrative and exemplary and are not intended to be exhaustive or exclusive embodiments of the present apparatus, system, or non-transitory computer-readable medium having instructions for implementing the method.
[0011] Figure 1 A schematic diagram of the structure of a signal acquisition system for the fNIRS and EEG combined scenario according to an embodiment of the present application is shown.
[0012] Figure 2 Another structural schematic diagram of the signal acquisition system for the fNIRS and EEG combined scenario according to the embodiment of the present application is shown.
[0013] Figure 3 Another structural schematic diagram of a signal acquisition system for a scenario where fNIRS and EEG are used together according to an embodiment of the present application is shown.
[0014] Figure 4 A schematic diagram of the structure of a marking device for a scenario in which fNIRS and EEG are used in conjunction with each other according to an embodiment of the present application is shown. DETAILED DESCRIPTION
[0015] In order to enable those skilled in the art to better understand the technical solution of the present application, the present application is described in detail below in conjunction with the accompanying drawings and specific implementation examples. The embodiments of the present application are further described in detail below in conjunction with the accompanying drawings and specific implementation examples, but are not intended to limit the present application.
[0016] The words "first", "second" and similar words used in this application do not indicate any order, quantity or importance, but are only used to distinguish. The words "include" or "comprises" and similar words used in this application mean that the elements before the word include the elements listed after the word, and do not exclude the possibility of covering other elements. In this application, the arrows shown in the figures of each step are only examples of the execution order, not limitations. The technical solution of this application is not limited to the execution order described in the embodiments. The various steps in the execution order can be combined, decomposed, or the order can be changed, as long as the logical relationship of the execution content is not affected.
[0017] All terms (including technical terms or scientific terms) used in this application have the same meaning as those understood by ordinary technicians in the field to which this application belongs, unless otherwise specifically defined. It should also be understood that terms defined in general dictionaries should be interpreted as having a meaning consistent with their meaning in the context of the relevant technology, and should not be interpreted in an idealized or extremely formal sense, unless explicitly defined here. Techniques and equipment known to ordinary technicians in the relevant field may not be discussed in detail, but where appropriate, the techniques and equipment should be considered as part of the specification.
[0018] Figure 1The structural schematic diagram of the signal acquisition system for the fNIRS and EEG combined scenario according to the embodiment of the present application is shown. The signal acquisition system 100 includes at least an fNIRS signal acquisition module 102 for acquiring the fNIRS signal of the subject at a first sampling rate, an EEG signal acquisition module 104 for acquiring the EEG signal of the subject at a second sampling rate greater than the first sampling rate, and a marking device 105 used in conjunction with the fNIRS signal acquisition module 102, the EEG signal acquisition module 104, and the processing device 106. In some embodiments, the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104 can respectively acquire the time information of the subject performing the marking operation at the first sampling rate and the second sampling rate and mark it.
[0019] Among them, the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104 may include physiological signal acquisition components such as electrodes, probes or head caps for sensing the physiological signals of the subject, as well as multiple components such as devices and signal sending devices that are communicatively connected to the processing device 106.
[0020] Specifically, the first sampling rate of the fNIRS signal acquisition module 102 can vary from a few Hz to tens of Hz. Generally speaking, the first sampling rate is between 1 Hz and 15 Hz. For example, the common first sampling rate of the fNIRS signal acquisition module 102 is 11 Hz. The second sampling rate of the EEG signal acquisition module 104 is generally much higher than the first sampling rate. For example, the second sampling rate can be 100 Hz, 250 Hz, 500 Hz or even 1000 Hz and above.
[0021] This is only used as an example and does not constitute a specific limitation on the first sampling rate and the second sampling rate.
[0022] A specific example of the use of the marking device 105 in conjunction with the fNIRS signal acquisition module 102 , the EEG signal acquisition module 104 , and the processing device 106 will be described in detail below.
[0023] The marking device 105 is configured to transmit at least the marking start mark signal to the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104, respectively. Exemplarily, the marking device 105 may include an input device or be used in conjunction with the input device. For example, the input device may be a button, a remote control, a touch key, etc., and the subject may perform related control operations such as a marking trigger operation through the input device. In some embodiments, before the marking starts, the input device may first generate a marking start mark signal based on the marking start operation and transmit it to the marking device 105. For example, a start button may be used to start the marking trigger operation to generate a marking start mark signal. In other embodiments, the input device may also generate a marking start mark signal in response to the first marking trigger operation and transmit it to the marking device 105. The marking device 105 receives the marking start mark signal and transmits it to the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104, respectively.
[0024] Specifically, the marking trigger operation can be triggered manually or automatically by an input device according to a set computer program instruction. For example, the marking trigger operation can be an active response of the subject to a certain stimulation event, such as selecting an answer to a question by manually pressing a button according to an instruction requiring the subject to respond to the stimulation event. It can also be that the input device automatically stimulates the subject according to a set computer program instruction, without the subject having to respond to the stimulation with external operations such as body language or other external operations, but the changes in the subject's physiological condition caused by the stimulation can be analyzed through the collected physiological signals.
[0025] Specifically, the marking device 105 may include an input device or communicate with the input device, the input device is used to generate a first marking trigger signal in response to each marking trigger operation, and the marking device 105 may receive the first marking trigger signal from the input device. The marking start mark signal may be the first first marking trigger signal generated in response to the marking start operation or the first marking operation. During the marking process, the input device may only transmit the marking start mark signal to the marking device 105, or may transmit all first marking trigger signals including the marking start mark signal generated in response to each marking trigger operation to the marking device 105. The marking device 105 receives the marking start mark signal and transmits it to the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104 respectively, so that the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104 can know the start time of the marking trigger operation. In order to facilitate recognition by each signal acquisition module, the marking device 105 can convert the marking start mark signal and the first marking trigger signal sent by the input device into signals that can be recognized by each signal acquisition module, for example, converting the marking trigger signal into a recognizable electrical signal.
[0026] This is only an exemplary description and does not constitute a limitation on the specific scheme.
[0027] That is to say, the marking device 105 can only transmit the marking start mark signal to the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104, without transmitting the first marking trigger signal generated each time to the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104. While collecting fNIRS signals and EEG signals, the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104 can respectively determine the starting time information of the marking trigger operation based on the marking start mark signal, and save the marking start mark signal in the first fNIRS data and the first EEG data.
[0028] In other embodiments, the marking device 105 may also transmit only the marking start mark signal to one of the signal acquisition modules, and transmit all the marking trigger signals received by it including the marking start mark signal to the other signal acquisition module. For example, only the marking start mark signal is transmitted to the fNIRS signal acquisition module 102, and all the marking trigger signals received by it are transmitted to the EEG signal acquisition module 104 with a higher sampling rate. When the second sampling rate of the EEG signal acquisition module 104 is not less than the frequency of the marking trigger operation, the EEG signal acquisition module 104 can obtain each marking trigger signal while collecting the EEG signal, that is, it can realize real-time marking while collecting the EEG signal, and there will be no missing of the mark, so that in the subsequent analysis, it is possible to directly analyze how the brain electrophysiological activity represented by the EEG signal changes at the moment when each marking trigger operation occurs, without parsing the marking time series data obtained based on the marking device 105, aligning it with the EEG signal data for timestamps, and marking each marking trigger operation on the EEG signal data, etc., which can save computing power and reduce errors.
[0029] In some embodiments, the marking device block 105 may also monitor the frequency of the marking trigger operation performed by the subject, and when it is detected that the frequency of the marking trigger operation is greater than the first sampling rate, the first marking trigger signal generated in response to the marking trigger operation is sampled at a third sampling rate to obtain the marking timing data containing the time information of each event of interest, wherein the third sampling rate is not less than the frequency of the marking trigger operation. Specifically, the marking device 105 may receive the first marking trigger signal generated in response to the marking trigger operation, and the first marking trigger signal may be a pulse signal with a short and sudden change in time, which carries the time information corresponding to when the subject performs the marking trigger operation.
[0030] When the frequency of the marking trigger operation is greater than the first sampling rate, it means that the subject performs the marking trigger operation at a higher frequency, and the fNIRS signal acquisition module 102 cannot obtain all the time information of executing the marking trigger operation based on the first sampling rate. If the marking device 105 samples the first marking trigger signal at the first sampling rate or even a lower sampling rate, it is also impossible to collect all the time information of executing the marking trigger operation. However, the marking device 105 samples the first marking trigger signal at a third sampling rate that is not less than the frequency of the marking trigger operation, shortening the sampling period, and can collect the time information of all the events of interest that occur when collecting the fNIRS signal and EEG signal of the subject, and mark the time information of all the events of interest to obtain the marked time series data containing the time information of each event of interest.
[0031] The event of interest may be, for example, an active response of the subject to a certain stimulation event, such as selecting an answer to a question by manually pressing a button according to the subject's instruction to respond to the stimulation event, or it may be that a stimulation device stimulates the subject without the subject having to respond to the stimulation with external operations such as body language or other physical actions. For example, a display may present a set stimulation picture to the subject at a certain moment according to set computer program instructions, and the effect of the stimulation picture on changes in the subject's physiological condition may be analyzed by collecting the subject's physiological signals.
[0032] The marking device 105 transmits the marking timing data to the processing device 106, wherein the processing device 106 may be a processor, and the processor may be a processing device including one or more general processing devices, such as a microprocessor, a central processing unit (CPU), a graphics processing unit (GPU), etc. More specifically, the processor may be a complex instruction set computing (CISC) microprocessor, a reduced instruction set computing (RISC) microprocessor, a very long instruction word (VLIW) microprocessor, a processor running other instruction sets, or a processor running a combination of instruction sets. The processor may also be one or more special processing devices, such as an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), a digital signal processor (DSP), a system on a chip (SoC), etc.
[0033] The marking device 105 is communicatively connected to the processing device 106 in a wireless or wired communication mode. The marking device 105 can encode the time information of each event of interest in the marking timing data. The processing device 106 obtains the marking timing data from the marking device 105 and decodes it to obtain the time information of each event of interest.
[0034] This is only an example and does not constitute a limitation on the specific solution.
[0035] The processing device 106 is configured to obtain the first fNIRS data containing the marking start mark from the fNIRS signal acquisition module 102, the first EEG data containing the marking start mark from the EEG signal acquisition module 104, and the marking timing data from the marking device 105 for analysis or display. Among them, the first fNIRS data and the first EEG data may not contain the time information of the event of interest, but the first fNIRS data and the first EEG data both contain the marking start mark. In some embodiments of the present application, the marking device 105 transmits the marking start mark signal to the processing device 106, and the processing device 106 can determine the corresponding relationship between the start time of the marking trigger operation and the acquisition time of the fNIRS signal and the EEG signal based on the received marking start mark signal, so that the processing device 106 can analyze the changes in brain function status more accurately based on this information.
[0036] Specifically, after aligning the timestamps of the acquired marked time series data, the first fNIRS data, and the first EEG data, the processing device 106 compares the time information of all events of interest in the marked time series data, and marks the time information of each event of interest in the time domain of the first fNIRS data and / or the first EEG data. In this way, the changes in brain function status can be more accurately analyzed based on the marked positions of the time information of each event of interest collected in the first fNIRS data, the first EEG data, and the marked time series data. Alternatively, in other embodiments, if the first fNIRS data already contains the marks of the time information of the event of interest acquired during the collection process, the processing device 106 can also fill in the missing marks in the time domain of the first fNIRS data. The processing device 106 can display both the first fNIRS data and the first EEG data after the fill-in marks on the display, so that the user can analyze the physiological condition of the subject. In other embodiments, the processing device 106 may also display only the first fNIRS data and the first EEG data after the timestamps are aligned on the display, and when the user needs to analyze the area of interest in the first fNIRS data, after selecting the area of interest, the processing device 106 automatically analyzes and matches the marked position of the time information of the event of interest contained in the area of interest based on the marked time series data to analyze the changes in brain function status.
[0037] This is merely an exemplary description and does not constitute a specific limitation on the manner in which the processing device 106 performs analysis or display.
[0038] In this embodiment, the time information of all events of interest occurring in the process of collecting the fNIRS signal and EEG signal of the subject can be obtained by using the marking device 105 in coordination with the fNIRS signal acquisition module 102, the EEG signal acquisition module 104, and the processing device 106. At the same time, based on the marking start mark contained in the first fNIRS data and the first EEG data, the corresponding relationship between the start time of the marking trigger operation and the acquisition time of the fNIRS signal and the EEG signal can be determined, so that the processing device 106 can analyze the changes in brain function status more accurately based on this information.
[0039] In some embodiments of the present application, the marking device 105 is further configured to transmit the second marking trigger signal generated based on the first marking trigger signal to the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104, respectively, to generate the first fNIRS data containing the time information of each event of interest and the first EEG data containing the time information of each event of interest. Specifically, the marking device 105 may convert the first marking trigger signal containing the marking start mark signal into the second marking trigger signal that can be recognized by each signal acquisition module and transmit it to the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104 respectively. In response to the first marking trigger signal obtained by the marking trigger operation, a second marking trigger signal can be obtained through related equipment such as a signal conversion circuit or a control system. The second marking trigger signal can be directly recognized by each signal acquisition module, and the second marking trigger signal is transmitted to the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104 respectively, so that the fNIRS signal acquisition module 102 can acquire the second marking trigger signal while acquiring the fNIRS signal at the first sampling rate, thereby realizing real-time marking during the fNIRS signal acquisition process, and the EEG signal acquisition module 104 can acquire the second marking trigger signal while acquiring the EEG signal at the second sampling rate, thereby realizing real-time marking during the EEG signal acquisition process. At the same time, the marking device 105 samples the first marking trigger signal at a third sampling rate to obtain marking timing data containing time information of each event of interest that has occurred.
[0040] In this embodiment, since the first sampling rate of the fNIRS signal acquisition module 102 is low and the sampling period is long, when acquiring the second marking trigger signal, it is impossible to acquire the time information of all events of interest based on the second marking trigger signal, resulting in the first fNIRS data may only have the mark of the time information of some events of interest, and the mark of the time information of some events of interest is missing. Among them, since the second sampling rate of the EEG signal acquisition module 104 is much greater than the first sampling rate, in general, the second sampling rate can be as high as 1000Hz or even have a higher sampling rate. When the EEG signal acquisition module 104 acquires the second marking trigger signal, it can usually acquire the time information of all events of interest and mark them in real time. In some other embodiments, the third sampling rate is the same as the second sampling rate, so that it is convenient to align the marked time series data with the first EEG data for timestamp alignment, and the time information of each event of interest can be better aligned with the collected EEG signal data, with less deviation or even no deviation, which is conducive to more accurate analysis of the changes in the electrophysiological activity of the brain when each event of interest occurs.
[0041] In some embodiments of the present application, Figure 2 As shown, the marking device 105 is also configured to not sample the first marking trigger signal generated in response to the marking trigger operation when the frequency of the marking trigger operation is not greater than the first sampling rate; and transmit the second marking trigger signal generated based on the first marking trigger signal to the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104, respectively, to generate second fNIRS data containing time information of each event of interest that occurs and second EEG data containing time information of each event of interest that occurs, respectively.
[0042] That is to say, when the frequency of the marking trigger operation performed by the subject is not greater than the first sampling rate, at this time, the fNIRS signal acquisition module 102 collects fNIRS signals at the first sampling rate and simultaneously obtains the second marking trigger signal transmitted by the marking device 105, and marks the acquired time information of the event of interest to obtain the second fNIRS data. Since the frequency of the marking trigger operation performed by the subject is not greater than the first sampling rate, the second fNIRS data includes the time information of all events of interest. Similarly, the EEG signal acquisition module 104 collects EEG signals at a second sampling rate higher than the first sampling rate and simultaneously obtains the second marking trigger signal transmitted by the marking device 105, so it can also obtain the time information of all events of interest and mark them to obtain the second EEG data.
[0043] Since the second fNIRS data and the second EEG data can both contain the time information of all the events of interest that have occurred, the marking device 105 does not need to sample the first marking trigger signal when the frequency of the marking trigger operation is not greater than the first sampling rate. In this case, after the processing device 106 obtains the second fNIRS data and the second EEG data, it can directly align the timestamps of the second fNIRS data and the second EEG data based on the marking start mark, so that the physiological condition of the brain function of the subject can also be accurately analyzed. Alternatively, the processing device 106 can also display the second fNIRS data and the second EEG data after the timestamp alignment on the display, so that the doctor can intuitively refer to the marking of each event of interest, combined with the second fNIRS data and the second EEG data, to analyze the physiological function activity of the subject in different cognitive and behavioral tasks.
[0044] like Figure 3 As shown, the signal acquisition system 100 further includes an input device 107, and the input device 107 is configured to transmit a first marking trigger signal to the marking device 105 in response to a marking trigger operation. The input device 107 may be a button, a keyboard, a remote controller, a sensor, or an electronic device with a touch screen, etc. The input device 107 and the marking device 105 may be communicatively connected or integrated into the marking device 105. Taking the input device 107 as a button as an example, after the subject presses the button, the button transmits the first marking trigger signal to the marking device 105 in a wireless or wired communication connection manner.
[0045] The marking device 105 is further configured to generate a marking trigger electrical signal based on the received first marking trigger signal, and transmit it as the second marking trigger signal to the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104 respectively. Specifically, the marking device 105 may be configured with a signal conversion module or a signal conversion circuit, and when the first marking trigger signal is received, the first marking trigger signal is converted into a marking trigger electrical signal. Alternatively, the marking device 105 may be equipped with a control system, and when the first marking trigger signal is detected, the control system generates a marking trigger electrical signal according to a preset logic and algorithm. This is only used as an exemplary description and does not constitute a limitation on the specific scheme.
[0046] Based on the first marking trigger signal received, a marking trigger electrical signal is generated, and the time information of the event of interest of the first marking trigger signal can be transmitted to the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104 in the form of an electrical signal, so that the fNIRS signal acquisition module 102 and the EEG signal acquisition module 104 can identify the marking trigger electrical signal and obtain the time information of the event of interest based on the marking trigger electrical signal.
[0047] In some embodiments of the present application, the signal acquisition system 100 further includes a storage module (not shown), and the marking device 105 is further configured to transmit the marking timing data to the storage module when the distance between the marking device 105 and the processing device 106 is greater than a preset distance, and / or the marking device 105 and the processing device 106 cannot establish a communication connection. Specifically, if the distance between the marking device 105 and the processing device 106 is far, the communication connection between the marking device 105 and the processing device 106 may be interrupted, or the communication connection cannot be established. Alternatively, under the influence of an interference signal, the marking device 105 and the processing device 106 cannot establish a communication connection, so that the marking timing data obtained by the marking device 105 sampling the first marking trigger signal cannot be transmitted to the processing device 106.
[0048] At this time, the marking device 105 can transmit the marking timing data to the storage module, and the storage module can be a portable storage device, a network storage device, a solid state hard disk, etc., which is not limited. For example, the marking timing data can be stored in a portable storage device. When the user wants to analyze and process the marking timing data, the processing device 106 can be used to read the stored marking timing data based on the portable storage device for analyzing the marking timing data, thereby improving the flexibility of storing and processing the marking timing data.
[0049] In some embodiments of the present application, the marking device 105 is further configured to generate a second marking trigger signal containing the marking type of the event of interest based on the received first marking trigger signal. The marking type may be an identifier, a numerical value, a string, etc., which is not limited. For example, the subject can perform the marking trigger operation in two ways, the first being sound and the second being pressing a button. When the subject speaks, the input device may generate a first marking trigger signal in response to the subject's voice, and the marking device 105 receives the first marking trigger signal to generate a second marking trigger signal containing the marking type A of the event of interest, namely speaking. When the subject presses a button, the input device may generate a first marking trigger signal in response to pressing a button, and the marking device 105 receives the first marking trigger signal to generate a second marking trigger signal containing the marking type B of the event of interest, namely pressing a button. In this way, different events of interest can be distinguished by marking type.
[0050] The above description is merely an exemplary description and does not constitute a limitation on the specific solution.
[0051] Figure 4 The structural schematic diagram of the marking device for the fNIRS and EEG combined scenario according to the embodiment of the present application is shown. The marking device 400 is used in conjunction with the fNIRS signal acquisition module 402, the EEG signal acquisition module 404, and the processing device 405, wherein the fNIRS signal acquisition module 402 acquires the fNIRS signal of the subject at a first sampling rate, and the EEG signal acquisition module 404 acquires the EEG signal of the subject at a second sampling rate greater than the first sampling rate; the marking device 400 is configured to at least transmit the marking start mark signal to the fNIRS signal acquisition module 402 and the EEG signal acquisition module 404 respectively; when the frequency of the marking trigger operation is greater than the first sampling rate, the first marking trigger signal generated in response to the marking trigger operation is sampled at a third sampling rate to obtain the marking time series data containing the time information of each event of interest, wherein the third sampling rate is not less than the frequency of the marking trigger operation; and the marking time series data is output. In this way, the marking device 400 can sample the time information of each event of interest in the first marking trigger signal at a third sampling rate. Since the third sampling rate is high and the sampling period is short, the obtained marking timing data can include the time information of all events of interest that occur.
[0052] In some embodiments of the present application, the marking device 400 is further configured to transmit each second marking trigger signal generated based on the first marking trigger signal to the fNIRS signal acquisition module 402 and the EEG signal acquisition module 404, respectively, to generate first fNIRS data containing time information of each event of interest and first EEG data containing time information of each event of interest. In this way, the first fNIRS data and the first EEG data can be time-stamp aligned based on the marking start mark, and the missing marks in the first fNIRS data or the first EEG data can be determined by referring to the marks of each event of interest in the marking time series data, so as to fill in the missing marks in the time domain of the first fNIRS data or the first EEG data.
[0053] In some embodiments of the present application, the marking device 400 is also configured to not sample the first marking trigger signal generated in response to the marking trigger operation when the frequency of the marking trigger operation is not greater than the first sampling rate; and transmit the second marking trigger signal generated based on the first marking trigger signal to the fNIRS signal acquisition module 402 and the EEG signal acquisition module 404, respectively, to generate second fNIRS data containing time information of each event of interest that occurs and second EEG data containing time information of each event of interest that occurs, respectively.
[0054] This application describes various operations or functions that may be implemented as software codes or instructions or defined as software codes or instructions. Such content may be source code or differential code ("incremental" or "patch" code) that can be directly executed ("object" or "executable" form). Software codes or instructions may be stored in a computer-readable storage medium, and when executed, may cause a machine to perform the described functions or operations, and include any mechanism for storing information in a form accessible to a machine (e.g., a computing device, an electronic system, etc.), such as recordable or non-recordable media (e.g., read-only memory (ROM), random access memory (RAM), magnetic disk storage media, optical storage media, flash memory devices, etc.).
[0055] The exemplary methods described herein may be at least partially machine or computer implemented. In some embodiments, a computer-readable storage medium stores computer program instructions, which, when executed by a processing device, are used to obtain first fNIRS data containing a marking start mark from the fNIRS signal acquisition module, first EEG data containing a marking start mark from the EEG signal acquisition module, and marking timing data from the marking device, wherein the fNIRS signal acquisition module acquires the fNIRS signal of the subject at a first sampling rate, and the EEG signal acquisition module acquires the EEG signal of the subject at a second sampling rate greater than the first sampling rate, and the marking timing data is obtained by sampling the first marking trigger signal generated in response to the marking trigger operation by the marking device at a third sampling rate, wherein the third sampling rate is not less than the frequency of the marking trigger operation; displaying the first fNIRS data containing the marking start mark and the first EEG data containing the marking start mark in the time domain on a display interface, or displaying the first fNIRS data containing the marking start mark and the first EEG data containing the marking start mark in the time domain and the mark of the event of interest contained in the marking timing data on the display interface.
[0056] All the steps performed by the processing device in the above embodiments can be combined here and implemented by the processing device running computer program instructions, which will not be described in detail here.
[0057] The implementation of such a method may include software code, such as microcode, assembly language code, high-level language code, etc. Various software programming techniques may be used to create various programs or program modules. For example, a program portion or program module may be designed with or by means of Java, Python, C, C++, assembly language, or any known programming language. One or more of such software portions or modules may be integrated into a computer system and / or a computer-readable medium. Such software code may include computer-readable instructions for executing various methods. The software code may form a part of a computer program product or a computer program module. In addition, in an example, the software code may be tangibly stored on one or more volatile, non-temporary or non-volatile tangible computer-readable media, such as during execution or at other times. Examples of these tangible computer-readable media may include, but are not limited to, hard disks, removable disks, removable optical disks (such as optical disks and digital video disks), cassettes, memory cards or memory sticks, random access memories (RAMs), read-only memories (ROMs), etc.
[0058] In addition, although exemplary embodiments have been described herein, the scope includes any and all embodiments based on the present application with equivalent elements, modifications, omissions, combinations (e.g., various embodiments intersecting schemes), adaptations or changes. The elements in the claims will be interpreted broadly based on the language adopted in the claims, and are not limited to the examples described in this specification or during the implementation of the application, and the examples will be interpreted as non-exclusive. Therefore, this specification and examples are intended to be considered as examples only, and the true scope and spirit are indicated by the following claims and the full scope of their equivalents.
[0059] The above description is intended to be illustrative rather than restrictive. For example, the above examples (or one or more of them) can be used in combination with each other. For example, those of ordinary skill in the art can use other embodiments when reading the above description. In addition, in the above-mentioned specific embodiments, various features can be grouped together to simplify the application. This should not be interpreted as an intention that a disclosed feature that does not require protection is necessary for any claim. On the contrary, the subject matter of the present application may be less than all the features of a specific disclosed embodiment. Thus, the claims are incorporated into the specific embodiments as examples or embodiments, wherein each claim is independently used as a separate embodiment, and it is considered that these embodiments can be combined with each other in various combinations or arrangements. The scope of the present application should be determined with reference to the full scope of the equivalent forms of the attached claims and these claims.
[0060] The above embodiments are only exemplary embodiments of the present application and are not intended to limit the present application. The protection scope of the present application is defined by the claims. Those skilled in the art may make various modifications or equivalent substitutions to the present application within the essence and protection scope of the present application, and such modifications or equivalent substitutions shall also be deemed to fall within the protection scope of the present application.
Claims
1. A signal acquisition system for fNIRS and EEG combined use, characterized in that: The signal acquisition system at least comprises an fNIRS signal acquisition module for acquiring fNIRS signals of a subject at a first sampling rate, an EEG signal acquisition module for acquiring EEG signals of a subject at a second sampling rate greater than the first sampling rate, and a marking device used in conjunction with the fNIRS signal acquisition module, the EEG signal acquisition module and the processing device. The marking device is configured to: transmit at least a marking start mark signal to the fNIRS signal acquisition module and the EEG signal acquisition module respectively; When the frequency of the marking trigger operation is greater than the first sampling rate, sampling the first marking trigger signal generated in response to the marking trigger operation at a third sampling rate to obtain marking timing data including time information of each event of interest that occurs, wherein the third sampling rate is not less than the frequency of the marking trigger operation; transmitting the marked timing data to the processing device; The processing device is configured as follows: The first fNIRS data including the marking start mark from the fNIRS signal acquisition module, the first EEG data including the marking start mark from the EEG signal acquisition module, and the marking timing data from the marking device are acquired for analysis or display.
2. The signal acquisition system according to claim 1, characterized in that: The marking device is configured to transmit a second marking trigger signal generated based on the first marking trigger signal to the fNIRS signal acquisition module and the EEG signal acquisition module, respectively, so as to respectively generate first fNIRS data containing time information of each event of interest that occurs and first EEG data containing time information of each event of interest that occurs.
3. The signal acquisition system according to claim 1, characterized in that: The marking device is further configured to: when the frequency of the marking trigger operation is not greater than the first sampling rate, not sample the first marking trigger signal generated in response to the marking trigger operation; The second marking trigger signal generated based on the first marking trigger signal is transmitted to the fNIRS signal acquisition module and the EEG signal acquisition module respectively, so as to respectively generate second fNIRS data containing time information of each event of interest and second EEG data containing time information of each event of interest.
4. The signal acquisition system according to claim 2 or 3, characterized in that: The signal acquisition system further includes an input device, wherein the input device is configured to: transmit a first marking trigger signal to the marking device in response to a marking trigger operation; The marking device is further configured as follows: Based on the received first marking trigger signal, a marking trigger electrical signal is generated, and the marking trigger electrical signal is transmitted to the fNIRS signal acquisition module and the EEG signal acquisition module respectively as the second marking trigger signal.
5. The signal acquisition system according to any one of claims 1 to 3, characterized in that: The third sampling rate is the same as the second sampling rate.
6. The signal acquisition system according to any one of claims 1 to 3, characterized in that: The marking device is further configured to: transmit the marking start mark signal to the processing device; The processing device is further configured to: Based on the received marking start mark signal, the first fNIRS data and the first EEG data are time stamp aligned.
7. The signal acquisition system according to claim 2 or 3, characterized in that: The marking device is further configured to generate a second marking trigger signal including a mark type of the event of interest based on the first marking trigger signal.
8. A marking device for fNIRS and EEG combined scenarios, characterized in that: The marking device is used in coordination with the fNIRS signal acquisition module, the EEG signal acquisition module, and the processing device, wherein the fNIRS signal acquisition module acquires the fNIRS signal of the subject at a first sampling rate, and the EEG signal acquisition module acquires the EEG signal of the subject at a second sampling rate greater than the first sampling rate; The marking device is configured as follows: At least transmitting the marking start mark signal to the fNIRS signal acquisition module and the EEG signal acquisition module respectively; When the frequency of the marking trigger operation is greater than the first sampling rate, sampling the first marking trigger signal generated in response to the marking trigger operation at a third sampling rate to obtain marking timing data containing time information of each event of interest that occurs, wherein the third sampling rate is not less than the frequency of the marking trigger operation; and The marked timing data is output.
9. The marking device according to claim 8, characterized in that: The marking device is further configured as follows: The respective second marking trigger signals generated based on the first marking trigger signal are transmitted to the fNIRS signal acquisition module and the EEG signal acquisition module respectively, so as to respectively generate the first fNIRS data containing the time information of the respective events of interest and the first EEG data containing the time information of the respective events of interest.
10. The marking device according to claim 8 or 9, characterized in that: The marking device is further configured to: when the frequency of the marking trigger operation is not greater than the first sampling rate, not sample the first marking trigger signal generated in response to the marking trigger operation; The second marking trigger signal generated based on the first marking trigger signal is transmitted to the fNIRS signal acquisition module and the EEG signal acquisition module respectively, so as to respectively generate second fNIRS data containing time information of each event of interest and second EEG data containing time information of each event of interest.
11. A computer-readable storage medium, characterized in that: The computer readable storage medium stores a computer program, which, when executed by a processing device, performs: Acquire first fNIRS data including a marking start mark from an fNIRS signal acquisition module, first EEG data including a marking start mark from an EEG signal acquisition module, and marking timing data from a marking device, wherein the fNIRS signal acquisition module acquires the fNIRS signal of the subject at a first sampling rate, the EEG signal acquisition module acquires the EEG signal of the subject at a second sampling rate greater than the first sampling rate, and the marking timing data is obtained by the marking device sampling a first marking trigger signal generated in response to the marking trigger operation at a third sampling rate, wherein the third sampling rate is not less than the frequency of the marking trigger operation; The first fNIRS data including the marking start mark and the first EEG data including the marking start mark in the time domain are displayed on the display interface, or the first fNIRS data including the marking start mark and the first EEG data including the marking start mark in the time domain and the mark of the event of interest included in the marking time series data are displayed on the display interface.
