Multi-modal signal processing method and system based on functional ultrasound and electroencephalogram

Abstract

The invention discloses a multi-mode signal processing method and system based on functional ultrasound and electroencephalogram, relates to the technical field of signal processing, and solves the technical problems that ultrasonic signals and electroencephalogram signals cannot be monitored synchronously and single-mode information of an existing brain function imaging technology is incomplete. According to the technical scheme, the method is characterized in that synchronous processing and conjoint analysis are carried out on collected ultrasonic signals and electroencephalogram signals, signal alignment is achieved through a unified reference, hemodynamic characteristics and nerve electrical activity characteristics are extracted, a coupling model is constructed, brain function task decoding and brain region activation evaluation are achieved, and the brain function task is analyzed. The integrity and accuracy of brain function evaluation are improved, and the stability of cerebral blood flow imaging is improved.

Description

Technical Field

[0001] This application relates to the field of signal processing technology, and in particular to a multimodal signal processing method and system based on functional ultrasound and electroencephalography. Background Technology

[0002] Functional brain imaging is an important research tool for studying brain neural activity. Commonly used functional brain imaging techniques include functional magnetic resonance imaging (fMRI), functional near-infrared imaging (fNIRS), and electroencephalography (EEG). Among them, fMRI has high spatial resolution, but the equipment is bulky and expensive, making it difficult to achieve bedside or mobile monitoring; fNIRS equipment is portable, but the signal penetration depth is limited, and it can only detect superficial cortical activity; EEG has high temporal resolution, but its spatial localization ability is insufficient, making it difficult to comprehensively reflect brain functional activity.

[0003] Functional ultrasound (fUS) is an emerging neurological imaging method that indirectly reflects neural activity by detecting changes in cerebral blood flow or volume using ultrafast ultrasound. Compared to other brain functional imaging modalities, fUS offers both high temporal and spatial resolution, along with advantages such as large imaging depth and high sensitivity for blood flow detection. However, due to the strong attenuation and distortion of ultrasound signals by the human skull, the signal quality of transcranial fUS is limited, and it is currently mostly used in animal experiments or research under craniotomy conditions.

[0004] Handheld ultrasound devices, characterized by their small size, low cost, and portability, have been widely used in bedside examinations and intraoperative monitoring. Combining handheld ultrasound devices with functional ultrasound imaging allows for the detection of hemodynamics in localized brain regions within the bone window area of ​​post-craniotomy patients, providing a new approach for clinical brain function assessment.

[0005] The question remains to be solved on how to use handheld ultrasound devices for functional ultrasound imaging and how to combine them with EEG devices to achieve synchronous monitoring of EEG and ultrasound signals. Summary of the Invention

[0006] This application provides a multimodal signal processing method and system based on functional ultrasound and electroencephalography. The technical objective is to reduce the size of the brain functional imaging system by introducing a handheld ultrasound device, and to perform cross-modal joint analysis of neural electrical activity characteristics and hemodynamic characteristics, so as to improve the integrity and accuracy of brain functional imaging and improve the stability of cerebral blood flow imaging.

[0007] The above-mentioned technical objective of this application is achieved through the following technical solution: A multimodal signal processing method based on functional ultrasound and electroencephalography includes: Acquisition of ultrasound signal data stream and electroencephalogram (EEG) signal data stream; The ultrasound signal data stream is processed to obtain hemodynamic characteristics; The characteristics of neural electrical activity are obtained by processing the EEG signal data stream; The neural electrical activity features and hemodynamic features are time-aligned and synchronized based on a unified time reference. After time alignment, a coupling model between the neural electrical activity features and the hemodynamic features is constructed. The coupling model estimates the correlation strength and time delay relationship between changes in neural electrical activity and changes in blood flow response through correlation analysis, regression analysis, or model fitting, thereby obtaining the neural-blood flow coupling characteristics of the target brain region. Based on the neural-blood flow coupling characteristics, a joint analysis of brain function reflecting the functional state, task-related activation patterns, and neural regulation characteristics of the target brain region is obtained.

[0008] Furthermore, the processing of the EEG signal data stream to obtain neural electrical activity characteristics includes: The EEG signal data stream is preprocessed using reference reconstruction to obtain a first EEG signal, represented as: ; in, This indicates the first EEG signal after rereference. Indicates the first EEG signal data stream of one EEG channel Indicates the first EEG signal data stream of one EEG channel Indicates the number of effective EEG channels; The non-brain-derived artifact components in the first EEG signal are identified and suppressed to obtain the second EEG signal, which is represented as follows: ; in, This indicates the second EEG signal after inhibition. This represents the artifact components estimated through artifact modeling or separation methods. Time-domain analysis, frequency-domain analysis, or time-frequency-domain analysis are performed on the second EEG signal to obtain neural electrical activity characteristics; among which, the power spectrum characteristics obtained through frequency-domain analysis are expressed as follows: ; in, This indicates that the second EEG signal is the first Individual EEG channel signals in the preset frequency band The power spectrum characteristics within, i.e., the characteristics of neural electrical activity; for The frequency domain representation, This indicates that the second EEG signal is the first One EEG channel signal.

[0009] Furthermore, the processing of the ultrasound signal data stream to obtain hemodynamic characteristics includes: Based on the ultrasound signal data stream, the corresponding channel-level slow time-domain complex signal is obtained, and low-rank sparse decomposition processing is performed on the channel-level slow time-domain complex signal to separate the low-rank component representing the tissue background from the sparse component representing the blood flow signal. During brain function monitoring, beamforming is performed based on the low-rank components to obtain tissue structure imaging results; simultaneously, beamforming is performed based on the sparse components to obtain preliminary blood flow imaging results characterizing the microvascular blood flow distribution in the target brain region; motion estimation and motion correction are performed on the preliminary blood flow imaging results based on the tissue structure imaging results to obtain compensated blood flow imaging results; and power Doppler signals are calculated based on the compensated blood flow imaging results to obtain functional ultrasound imaging results reflecting the microvascular blood flow distribution and changes in the target brain region. The functional ultrasound imaging results were baseline normalized to obtain hemodynamic characteristics.

[0010] Furthermore, the channel-level slow time-domain complex signal is represented as: ; in, , Represents the Hilbert transform. This represents the radio frequency signal corresponding to the ultrasound signal data stream. Indicates the receiver element number. ; Represents discrete-time sampling points. Indicates the index of the slow time frame. ; The low-rank sparse decomposition process is represented as follows: ; in, Indicates the low-rank component. Represents sparse components; The low-rank sparse decomposition process is solved by an optimization problem, which is expressed as: ; in, Represents the nuclear norm. express Norm, These are the regularization weight coefficients; The tissue structure imaging results are expressed as follows: ; in, Represents low-rank components Fourier transform from the space-time domain to the frequency-wavenumber domain This represents the fk transition operator. This represents the inverse Fourier transform from the frequency-wavenumber domain to the spatial domain; The preliminary blood flow imaging results are expressed as follows: ; in, This indicates the preliminary blood flow imaging results. Represents the angular coherence factor, and , To prevent stable terms with a denominator of zero; This represents the blood flow imaging results at different incident angles; The compensated blood flow imaging results are expressed as follows: ; in, This indicates the results of blood flow imaging after compensation. Represents the structural images of two adjacent frames and The displacement field; The functional ultrasound imaging results are expressed as follows: ; in, This indicates the results of functional ultrasound imaging. This indicates the blood flow imaging results after compensation.

[0011] Furthermore, the hemodynamic characteristics are expressed as follows: ; in, This represents the baseline blood flow signal corresponding to the resting phase. This represents the average blood flow signal within the brain region of interest. , Indicates brain regions of interest The number of pixels within, Indicates time index At any pixel location within the target brain region The power Doppler signal; This indicates hemodynamic characteristics.

[0012] Furthermore, the processing of the ultrasound signal data stream also includes: When performing spatial registration between the ultrasound imaging region and the target brain region, beamforming is performed based on the low-rank component to generate tissue structure imaging results for registration with B-mode ultrasound images or magnetic resonance images, and then spatial registration is performed based on the tissue structure imaging results.

[0013] Furthermore, the coupling model is expressed as: ; in, Represents time series of hemodynamic characteristics; Represents a time series of characteristics of neural electrical activity; Represents the neural-blood flow response function. Represents the residual term. This represents the convolution operation; The correlation index of the correlation analysis or regression analysis is expressed as follows: ; in, Describing covariance, The standard deviation representing the characteristics of neural electrical activity The standard deviation representing the characteristics of cerebral hemodynamics.

[0014] A multimodal signal processing system based on functional ultrasound and electroencephalography (EEG), the multimodal signal processing system being used to implement the aforementioned multimodal signal processing method, the multimodal signal processing system comprising: A handheld ultrasound device is used to emit ultrasound waves to a target brain region and receive echo signals. The echo signals are sampled and digitally processed to generate an ultrasound signal data stream arranged in chronological order. EEG equipment collects multi-channel EEG signals from the target brain region, timestamps and encapsulates the collected EEG signals, and generates an EEG signal data stream with additional timestamps. The ultrasound processing unit processes the ultrasound signal data stream to obtain hemodynamic characteristics; The EEG processing unit processes the EEG signal data stream to obtain neural electrical activity characteristics; The joint analysis module performs time alignment and synchronization processing on the neural electrical activity features and the hemodynamic features based on a unified time reference. After time alignment, a coupling model between the neural electrical activity features and the hemodynamic features is constructed. The coupling model estimates the correlation strength and time delay relationship between changes in neural electrical activity and changes in blood flow response through correlation analysis, regression analysis, or model fitting, thereby obtaining the neural-blood flow coupling characteristics of the target brain region. Based on the neural-blood flow coupling characteristics, the joint analysis results of brain function reflecting the functional state, task-related activation patterns, and neural regulation characteristics of the target brain region are obtained.

[0015] A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the multimodal signal processing method.

[0016] A computer storage medium storing a computer program, which, when executed by a processor, implements the steps of the multimodal signal processing method.

[0017] The above technical solution can achieve at least some of the following technical effects: (1) Achieve portable brain functional imaging: By introducing handheld ultrasound devices, functional ultrasound imaging has the advantages of small size, low cost and easy deployment, and is suitable for bedside monitoring and postoperative application scenarios. (2) Achieve multimodal synchronous monitoring: By using a unified time reference, realize the synchronous acquisition and alignment analysis of EEG signals and functional ultrasound signals, and improve the integrity and accuracy of brain function assessment; (3) Improve the stability of cerebral blood flow imaging: By separating tissue and blood flow signals and using a motion correction mechanism based on tissue imaging, the influence of motion interference on blood flow imaging results can be effectively reduced; (4) Enhance brain function analysis capabilities: By establishing the coupling relationship between neural electrical activity and cerebral hemodynamic response, new technical means are provided for brain function research, postoperative evaluation and neuromodulation analysis. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the multimodal signal processing system based on functional ultrasound and electroencephalography described in the embodiments of this application; Figure 2 This is a flowchart of the multimodal signal processing method based on functional ultrasound and electroencephalography described in the embodiments of this application; Figure 3 This is a flowchart of the ultrasonic signal processing in an embodiment of this application. Detailed Implementation

[0019] The technical solution of this application will be described in detail below with reference to the accompanying drawings.

[0020] like Figure 1 As shown, the multimodal signal processing system based on functional ultrasound and electroencephalography (EEG) described in this application includes a handheld ultrasound device 2, an EEG device 3, and a host computer terminal 1. All three are connected via wired or wireless communication to form a data acquisition and analysis network.

[0021] In this embodiment, the host computer terminal 1 is used for system control, data synchronization, signal processing, and multimodal joint analysis; the handheld ultrasound device 2 is used for transmitting, receiving, and initially digitizing ultrasound signals; and the electroencephalogram (EEG) device 3 is used for synchronous acquisition and time stamping of EEG signals. Through a unified time reference, the above three components work together to achieve synchronous acquisition and analysis of cerebral hemodynamic information and neural electrical activity information. The unified time reference is established through a synchronization trigger command issued by the host computer terminal, and both the ultrasound signal data stream and the EEG signal data stream are timestamped based on this reference.

[0022] like Figure 1 As shown, the handheld ultrasound device 2 includes an ultrasound probe 21, a transmission and reception control module 22, a data acquisition module 23, a first time synchronization module 24, and a first data transmission module 25.

[0023] The ultrasound probe 21 is used to emit ultrasound waves to the target brain region and receive echo signals. In this embodiment, the ultrasound probe 21 adopts a linear array structure with 128 array elements, a center frequency of 5.208 MHz, and an element spacing of 0.298 mm, which is suitable for local brain region ultrasound imaging in the bone window area after craniotomy.

[0024] The transmission and reception control module 22 is used to control the transmission and reception process of the ultrasonic probe 21. Its specific functions include configuring ultrasonic transmission waveform parameters, transmission channel combination mode, transmission timing, and reception channel opening sequence. In this embodiment, the transmission and reception control module 22 can control the ultrasonic probe 21 to transmit ultrasound in a preset multi-angle plane wave mode according to the control instructions issued by the host computer terminal 1, and simultaneously open the corresponding reception channel to obtain the echo signal.

[0025] During ultrasonic emission, the first The ultrasonic signal emitted by each array element can be represented as: ; in, For the transmitted pulse waveform, For the first The transmission delay corresponding to each array element These are emission weighting coefficients used to control the emission aperture and sound field distribution. Different emission delays are applied to different array elements. It can form ultrasonic waves at different angles within the imaging area. (Emission delay) Angle of incidence of plane wave It has the following relationship: ; in, For the first The horizontal position coordinates of each array element relative to the array center The angle of incidence of the ultrasonic plane wave relative to the array normal direction is denoted as . The speed at which ultrasound propagates in the medium is denoted as .

[0026] The data acquisition module 23 is used to perform analog front-end processing and digital processing on the echo signal received by the ultrasound probe 21. Specifically, the data acquisition module 23 includes a preamplifier unit, an analog-to-digital converter unit, and a channel data buffer unit. The preamplifier unit is used to amplify weak echo signals, and the analog-to-digital converter unit is used to sample analog signals according to a preset sampling rate and convert them into digital signals.

[0027] In this embodiment, the ultrasonic sampling rate of the data acquisition module 23 is preferably 40MHz to meet the time-domain resolution requirements of high-frequency ultrasonic echo signals. After analog-to-digital conversion, multi-channel radio frequency signal data arranged in the order of receiving channels and time is formed, which can be represented as: , , ; in, Indicates the receiver element number. ; This represents a discrete sampling time point.

[0028] The first time synchronization module 24 is used to time-stamp the ultrasound radio frequency signal data to achieve precise time alignment with the electroencephalogram (EEG) signal. Specifically, the first time synchronization module 24 can add timestamp information to each frame of ultrasound data or each segment of radio frequency data based on a unified time reference signal sent by the host computer terminal 1 or based on an external clock source. The timestamp information can adopt absolute time stamping or relative time counting to ensure the consistency of ultrasound data and EEG data on the time axis.

[0029] The first data transmission module 25 is used to transmit the digitized ultrasound data output by the data acquisition module 23 to the host computer terminal 1 in real time. Preferably, the first data transmission module 25 uses a high-speed serial communication protocol to realize data interaction, such as USB 3.0, USB 3.1 or an equivalent high-speed wired communication interface, to meet the requirements of high bandwidth, high data throughput and low transmission latency in functional ultrasound imaging.

[0030] With the above-described structure, the handheld ultrasound device 2 can complete the entire process from ultrasound signal transmission, echo reception, digital sampling, time synchronization marking to data transmission, providing a high-quality, time-consistent raw data foundation for subsequent ultrasound signal processing, blood flow imaging, and multimodal joint analysis.

[0031] like Figure 1 As shown, the EEG device 3 includes EEG electrodes 31, an EEG signal acquisition module 32, a second time synchronization module 33, and a second data transmission module 34. The EEG electrodes 31 are used to acquire EEG signals in the target brain region. The EEG electrodes 31 can be standard Ag / AgCl electrodes or dry electrodes, and their arrangement can be determined according to the international 10–20 system or the needs of local brain region research.

[0032] Data acquisition module 32 is used for pre-amplification, filtering, and analog-to-digital conversion of EEG signals. Its functions include: (1) Differential amplification of weak EEG signals to improve the signal-to-noise ratio; (2) Bandpass filtering is used to suppress power frequency interference and low frequency drift, with the preferred filtering range being 0.1–100 Hz; (3) Convert the analog signal into a digital signal according to a preset sampling rate (preferably 1–10 kHz), and the digitized EEG signal is represented as: , ; in, Indicates the brainwave channel number, Represents discrete sampling time points. This indicates the total number of EEG acquisition channels.

[0033] The second time synchronization module 33 is used to add a timestamp consistent with the ultrasound data to each channel of EEG data, so as to achieve precise synchronization between EEG signals and ultrasound signals. It can be based on a time reference uniformly issued by the host computer terminal 1 or an external trigger signal.

[0034] The second data transmission module 34 is used to transmit digitized EEG signals to the host computer terminal 1 in real time. It preferably uses a high-speed serial communication protocol (such as USB 3.0 / 3.1) or an equivalent wireless communication protocol to ensure low-latency data transmission under high sampling rate of multiple channels.

[0035] Through the above structural setup, the EEG device 3 can achieve multi-channel, high-precision, and low-latency EEG signal acquisition, and can acquire signals synchronously with the handheld ultrasound device 2, providing a reliable technical foundation for multimodal imaging and joint analysis of brain function.

[0036] like Figure 1 As shown, the host computer terminal 1 includes a task prompting module 11, a region registration module 12, a signal processing module 13, a joint analysis module 14, and a data storage module 15.

[0037] Specifically, the task prompting module 11 is used to output task prompting information to the target object according to a preset brain function experimental paradigm, and generate event time information corresponding to the task prompting information. The task prompting information can be presented through visual cues, auditory cues, or a combination thereof, and corresponding event time information is generated at the start of the task, the start of the task, the start of the task, or the end of the task, so as to achieve time alignment of multimodal data.

[0038] The region registration module 12 is used to determine the spatial correspondence between the ultrasound imaging region and the target brain region based on the B-mode ultrasound images acquired by the handheld ultrasound device 2 and the pre-acquired MRI or fMRI brain structure / functional imaging data. Specifically, the region registration module 12 can map the ultrasound imaging coordinate system to the brain anatomical coordinate system through an image registration algorithm, thereby determining the brain region of interest for ultrasound blood flow analysis and electroencephalogram (EEG) analysis.

[0039] The signal processing module 13 includes an ultrasound processing unit 131 and an electroencephalogram (EEG) processing unit 132. The ultrasound processing unit 131 processes the ultrasound signal data stream to generate functional ultrasound imaging results characterizing the distribution and dynamic changes of microvascular blood flow in the target brain region. The EEG processing unit 132 processes the electroencephalogram (EEG) signal data stream to obtain EEG analysis results characterizing the state of neural electrical activity in the target brain region.

[0040] The co-analysis module 14 is used to time-align functional ultrasound imaging results and electroencephalogram (EEG) analysis results based on a unified time reference and to perform cross-modal co-analysis. Specifically, the co-analysis module 14 can establish a coupling model between neural electrical activity characteristics and cerebral hemodynamic characteristics, and evaluate the driving effect of neural activity changes on blood flow response and its temporal relationship through correlation analysis, regression analysis, or model fitting, and output co-analysis results reflecting the functional state of the target brain region, task-related activation patterns, or neural modulation characteristics.

[0041] The data storage module 15 is used to store the collected raw data, processed data and joint analysis results, and supports retrieval according to time information or task conditions for subsequent function evaluation or data review.

[0042] Through the above structural setup, the host computer terminal 1 can send synchronous acquisition commands to the handheld ultrasound device 2 and the EEG device 3, establish a unified time reference, and realize task control, data synchronization, signal processing and multimodal joint analysis, providing complete control and data processing capabilities for the brain functional imaging system.

[0043] like Figure 2 As shown, the flowchart of the multimodal signal processing method based on functional ultrasound and electroencephalography in this embodiment includes: 100: Acquire ultrasound signal data stream and electroencephalogram (EEG) signal data stream.

[0044] In this embodiment of the application, as described above, the ultrasound signal data stream is acquired by the handheld ultrasound device 2, and the brainwave signal data stream is acquired by the brainwave device 3.

[0045] 101: The ultrasound signal data stream is processed to obtain hemodynamic characteristics.

[0046] like Figure 3 As shown in this embodiment, the ultrasonic signal processing flow is executed by the ultrasonic processing unit 131 in the host computer terminal 1, and its overall processing includes the following steps: (1) Obtain the corresponding channel-level slow time domain complex signal based on the ultrasound signal data stream, and perform low-rank-sparse decomposition processing on the channel-level slow time domain complex signal to separate the low-rank component representing the tissue background from the sparse component representing the blood flow signal.

[0047] First, the multi-channel radio frequency signals uploaded by the handheld ultrasound device 2 are processed. Let the number of receiving array elements be... The number of frames in slow time is The original radio frequency signal obtained is represented as: , ; ; in, Indicates the receiver element number. Represents discrete-time sampling points. This indicates the index of the slow time frame.

[0048] By demodulating and performing Hilbert transform on the radio frequency signal, a channel-level slow-time domain complex signal matrix is ​​constructed, represented as: ; in, , This represents the Hilbert transform, used to obtain complex envelope signals.

[0049] Secondly, in order to suppress tissue background echo and enhance microvascular blood flow signals, the channel-level slow time-domain complex signal matrix was modified. The mathematical model for performing low-rank sparse decomposition is as follows: ; in, Represents the low-rank component, used to characterize the tissue background and stable echo structure; This represents the sparse component, used to characterize blood flow-related scattering changes.

[0050] Preferably, the low-rank sparse decomposition process is solved by an optimization problem, which is expressed as: ; in, Represents the nuclear norm. express Norm, This is the regularization weight coefficient.

[0051] (2) After completing the low-rank-sparse decomposition process, the processing task judgment is entered to determine the task type corresponding to the current processing flow. The task type includes spatial registration processing task or brain functional imaging processing task.

[0052] (2.1) When performing spatial registration between the ultrasound imaging region and the target brain region, beamforming is performed based on the low-rank component to generate tissue structure imaging results for registration with B-mode ultrasound images or magnetic resonance images.

[0053] Specifically, when performing the spatial registration task, the low-rank component is selected as the input signal, and beamforming processing based on fk migration is performed on the low-rank component to generate tissue structure imaging results. The tissue structure imaging results are then expressed as follows: ; in, Represents low-rank components Fourier transform from the space-time domain to the frequency-wavenumber domain This represents the fk transition operator. This represents the inverse Fourier transform from the frequency-wavenumber domain to the spatial domain.

[0054] The generated tissue structure imaging results are used to spatially register with B-mode ultrasound images or magnetic resonance structural or functional images, thereby determining the correspondence between the ultrasound imaging coordinate system and the brain anatomical coordinate system.

[0055] During spatial registration, motion estimation and correction are first performed based on tissue structure imaging results obtained at adjacent time points to eliminate displacement errors introduced by factors such as probe micro-motion and subject physiological movement. Let two adjacent tissue structure images be... and Then its displacement field is estimated as follows: ; Based on the above displacement field, spatial compensation is performed on the tissue structure imaging results to obtain the motion-corrected compensated tissue structure imaging results, which are expressed as follows: .

[0056] Compensated tissue structure imaging results Image registration is performed with pre-acquired brain structural or functional images. This image registration can be achieved using rigid or non-rigid transformation models, and its spatial mapping relationship is expressed as follows: ; in, Let be the spatial transformation matrix. It is a translation vector.

[0057] The above registration process determines the spatial correspondence between the ultrasound imaging coordinate system and the target brain region anatomical coordinate system, and the correspondence is used for the spatial mapping of subsequent brain functional imaging results.

[0058] (2.2) During brain function monitoring, beamforming is performed based on the low-rank components to obtain the tissue structure imaging results, which is the same as the low-rank component beamforming process described above. Simultaneously, beamforming is performed based on the sparse components to obtain preliminary blood flow imaging results characterizing the microvascular blood flow distribution in the target brain region; wherein, during the beamforming process based on the sparse components, signals from different transmission angles and receiving channels are weighted and superimposed according to local signal-to-noise ratio, coherence factor, or statistical characteristics.

[0059] Specifically, when performing brain function monitoring, sparse components are selected. As the input signal, it undergoes fk migration-based beamforming to obtain blood flow imaging results at different incident angles, as follows: .

[0060] Subsequently, an angular coherence factor was introduced into the angular domain to weight and composite the blood flow imaging results at different incident angles, resulting in angular coherence composite blood flow imaging results, i.e., preliminary blood flow imaging results, expressed as: ; in, This indicates the preliminary blood flow imaging results. Represents the angular coherence factor, and , To prevent stable terms with a denominator of zero.

[0061] (3) Based on the tissue structure imaging results, perform motion estimation and motion correction on the preliminary blood flow imaging results to compensate for the spatial drift caused by probe micro-displacement, physiological motion or tissue deformation, and obtain the compensated blood flow imaging results.

[0062] Specifically, based on low-rank components The generated tissue structure imaging results are used to perform motion estimation and motion correction processing on the preliminary blood flow imaging results, that is, through the displacement field between two adjacent tissue structure images. Spatial compensation is performed on the preliminary blood flow imaging results to obtain the compensated blood flow imaging results, which are represented as follows: .

[0063] (4) Based on the compensated blood flow imaging results, the power Doppler signal is calculated to obtain functional ultrasound imaging results reflecting the distribution and changes of microvascular blood flow in the target brain region, as follows: ; in, This indicates the results of functional ultrasound imaging. This indicates the blood flow imaging results after compensation.

[0064] Preferably, the functional ultrasound imaging results are subjected to baseline normalization to obtain hemodynamic characteristics, expressed as follows: ; in, This represents the baseline blood flow signal corresponding to the resting phase. This represents the average blood flow signal within the region of interest. , Indicates brain regions of interest The number of pixels within, Indicates time index At any pixel location within the target brain region The power Doppler signal; This indicates hemodynamic characteristics.

[0065] The hemodynamic features include at least one of the following: the magnitude of change in blood flow signal in the target brain region relative to the baseline, peak blood flow response, time delay characteristics of the blood flow response, and the response curve of the blood flow signal over time. These hemodynamic features serve as characterizations of neural activity responses from functional ultrasound imaging results and are used for subsequent joint analysis with electroencephalogram (EEG) signals.

[0066] 102: Process the EEG signal data stream to obtain neural electrical activity characteristics.

[0067] In this embodiment, the EEG processing unit 132 is used to preprocess the multi-channel EEG signals collected by the EEG device 3 to suppress noise and non-brain-source interference, and improve the availability and stability of EEG data.

[0068] Specifically, the EEG processing unit is used for: (1) The EEG signal data stream is subjected to reference reconstruction processing, preferably using an average reference method, to obtain the first EEG signal, which is expressed as: ; in, This indicates the first EEG signal after rereference. Indicates the first EEG signal data stream of one EEG channel Indicates the first EEG signal data stream of one EEG channel This indicates the number of effective brainwave channels.

[0069] (2) To suppress non-brain-derived artifacts such as eye movements, electromyography, or electrode displacement, the non-brain-derived artifacts in the first EEG signal are identified and suppressed to obtain the second EEG signal, which is represented as: ; in, This indicates the second EEG signal after inhibition. This represents the artifact components estimated through artifact modeling or separation methods.

[0070] The preprocessed EEG signals are output while retaining the timestamp information, and are used for subsequent EEG feature extraction and joint analysis with functional ultrasound imaging results.

[0071] (3) Perform time-domain analysis, frequency-domain analysis, or time-frequency-domain analysis on the second EEG signal to obtain neural electrical activity characteristics. Among these, power spectrum characteristics can be obtained through frequency-domain analysis, expressed as: ; in, This indicates that the second EEG signal is the first Individual EEG channel signals in the preset frequency band The power spectrum characteristics within, i.e., the characteristics of neural electrical activity; for The frequency domain representation, This indicates that the second EEG signal is the first One EEG channel signal.

[0072] In task-based analysis, the EEG processing unit 132 can also perform event-locked analysis on the EEG signals based on task event time information to obtain event-related potential or time-frequency energy change characteristics. These neural electrical activity characteristics are used to characterize the neural activity state of the target brain region and are input as neural driving variables to the joint analysis module.

[0073] 103: Based on a unified time reference, the neural electrical activity features and the hemodynamic features are time-aligned and synchronized; after time alignment, a coupling model between the neural electrical activity features and the hemodynamic features is constructed; the coupling model estimates the correlation strength and time delay relationship between changes in neural electrical activity and changes in blood flow response through correlation analysis, regression analysis, or model fitting, to obtain the neural-blood flow coupling characteristics of the target brain region; based on the neural-blood flow coupling characteristics, the joint analysis results of brain function reflecting the functional state, task-related activation patterns, and neural regulation characteristics of the target brain region are obtained.

[0074] Specifically, after extracting hemodynamic features and neural electrical activity features, the joint analysis module 14 performs time alignment processing on the two types of features under a unified time reference to establish the coupling relationship between neural electrical activity and hemodynamic response.

[0075] Let the time series of neural electrical activity characteristics after time alignment be... The time series of hemodynamic characteristics are The nerve-blood flow coupling relationship, i.e., the coupling model, is described by the following model: ; in, Represents time series of hemodynamic characteristics; Indicates characteristics of neural electrical activity; Represents the neural-blood flow response function. Represents the residual term. This represents the convolution operation; The joint analysis module 14 can also quantify the strength of the association between the components through correlation analysis or regression analysis. The correlation index of the correlation analysis or regression analysis is expressed as follows: ; in, Describing covariance, The standard deviation representing the characteristics of neural electrical activity The standard deviation representing the characteristics of cerebral hemodynamics.

[0076] Based on the above neural-blood flow coupling analysis results, the joint analysis module 14 outputs brain function joint analysis results that reflect the functional state of the target brain region, task-related activation patterns, or neural regulation characteristics.

[0077] In summary, through the above-mentioned multimodal signal processing method and system based on functional ultrasound and electroencephalography, this application can realize the synchronous acquisition, processing and joint analysis of handheld ultrasound brain functional images and electroencephalogram signals, taking into account both high spatial resolution and high temporal resolution, and providing a low-cost, portable and real-time multimodal imaging solution for brain function monitoring and neuromodulation research.

[0078] The above are exemplary embodiments of this application, and the scope of protection of this application is defined by the claims and their equivalents.

Claims

1. A multimodal signal processing method based on functional ultrasound and electroencephalography, characterized in that, include: Acquisition of ultrasound signal data stream and electroencephalogram (EEG) signal data stream; The ultrasound signal data stream is processed to obtain hemodynamic characteristics; The characteristics of neural electrical activity are obtained by processing the EEG signal data stream; The neural electrical activity features and the hemodynamic features are time-aligned and synchronized based on a unified time reference. After time alignment is completed, a coupling model between the neural electrical activity characteristics and the hemodynamic characteristics is constructed. The coupling model estimates the correlation strength and time delay relationship between changes in neural electrical activity and changes in blood flow response through correlation analysis, regression analysis, or model fitting, thereby obtaining the neural-blood flow coupling characteristics of the target brain region. Based on the neural-blood flow coupling characteristics, a joint analysis of brain function reflecting the functional state, task-related activation patterns, and neural regulation characteristics of the target brain region is obtained.

2. The multimodal signal processing method as described in claim 1, characterized in that, The processing of the EEG signal data stream to obtain neural electrical activity characteristics includes: The EEG signal data stream is preprocessed using reference reconstruction to obtain a first EEG signal, represented as: ； in, This indicates the first EEG signal after rereference. Indicates the first EEG signal data stream of one EEG channel Indicates the first EEG signal data stream of one EEG channel Indicates the number of effective EEG channels; The non-brain-derived artifact components in the first EEG signal are identified and suppressed to obtain the second EEG signal, which is represented as follows: ； in, This indicates the second EEG signal after inhibition. This represents the artifact components estimated through artifact modeling or separation methods. Time-domain analysis, frequency-domain analysis, or time-frequency-domain analysis are performed on the second EEG signal to obtain neural electrical activity characteristics; among which, the power spectrum characteristics obtained through frequency-domain analysis are expressed as follows: ； in, This indicates that the second EEG signal is the first Individual EEG channel signals in the preset frequency band The power spectrum characteristics within, i.e., the characteristics of neural electrical activity; for The frequency domain representation, This indicates that the second EEG signal is the first One EEG channel signal.

3. The multimodal signal processing method as described in claim 2, characterized in that, The process of processing the ultrasound signal data stream to obtain hemodynamic characteristics includes: Based on the ultrasound signal data stream, the corresponding channel-level slow time-domain complex signal is obtained, and low-rank sparse decomposition processing is performed on the channel-level slow time-domain complex signal to separate the low-rank component representing the tissue background from the sparse component representing the blood flow signal. During brain function monitoring, beamforming is performed based on the low-rank components to obtain tissue structure imaging results; simultaneously, beamforming is performed based on the sparse components to obtain preliminary blood flow imaging results characterizing the microvascular blood flow distribution in the target brain region; motion estimation and motion correction are performed on the preliminary blood flow imaging results based on the tissue structure imaging results to obtain compensated blood flow imaging results; and power Doppler signals are calculated based on the compensated blood flow imaging results to obtain functional ultrasound imaging results reflecting the microvascular blood flow distribution and changes in the target brain region. The functional ultrasound imaging results were baseline normalized to obtain hemodynamic characteristics.

4. The multimodal signal processing method as described in claim 3, characterized in that, The channel-level slow time-domain complex signal is represented as follows: ； in, , Represents the Hilbert transform. This represents the radio frequency signal corresponding to the ultrasound signal data stream. Indicates the receiver element number. ; Represents discrete-time sampling points. Indicates the index of the slow time frame. ; The low-rank sparse decomposition process is represented as follows: ； in, Indicates the low-rank component. Represents sparse components; The low-rank sparse decomposition process is solved by an optimization problem, which is expressed as: ； in, Represents the nuclear norm. express Norm, These are the regularization weight coefficients; The tissue structure imaging results are expressed as follows: ； in, Represents low-rank components Fourier transform from the space-time domain to the frequency-wavenumber domain This represents the fk transition operator. This represents the inverse Fourier transform from the frequency-wavenumber domain to the spatial domain; The preliminary blood flow imaging results are expressed as follows: ； in, This indicates the preliminary blood flow imaging results. Represents the angular coherence factor, and , To prevent stable terms with a denominator of zero; This represents the blood flow imaging results at different incident angles; The compensated blood flow imaging results are expressed as follows: ； in, This indicates the results of blood flow imaging after compensation. Represents the structural images of two adjacent frames and The displacement field; The functional ultrasound imaging results are expressed as follows: ； in, This indicates the results of functional ultrasound imaging. This indicates the blood flow imaging results after compensation.

5. The multimodal signal processing method as described in claim 4, characterized in that, The hemodynamic characteristics are expressed as follows: ； in, This represents the baseline blood flow signal corresponding to the resting phase. This represents the average blood flow signal within the brain region of interest. , Indicates brain regions of interest The number of pixels within, Indicates time index At any pixel location within the target brain region The power Doppler signal; This indicates hemodynamic characteristics.

6. The multimodal signal processing method as described in claim 5, characterized in that, The processing of the ultrasonic signal data stream also includes: When performing spatial registration between the ultrasound imaging region and the target brain region, beamforming is performed based on the low-rank component to generate tissue structure imaging results for registration with B-mode ultrasound images or magnetic resonance images, and then spatial registration is performed based on the tissue structure imaging results.

7. The multimodal signal processing method as described in claim 6, characterized in that, The coupling model is represented as follows: ； in, Represents time series of hemodynamic characteristics; Represents a time series of characteristics of neural electrical activity; Represents the neural-blood flow response function. Represents the residual term. This represents the convolution operation; The correlation index of the correlation analysis or regression analysis is expressed as follows: ； in, Describing covariance, The standard deviation representing the characteristics of neural electrical activity The standard deviation representing the characteristics of cerebral hemodynamics.

8. A multimodal signal processing system based on functional ultrasound and electroencephalography, wherein the multimodal signal processing system is used to implement the multimodal signal processing method according to any one of claims 1-7, characterized in that, The multimodal signal processing system includes: A handheld ultrasound device is used to emit ultrasound waves to a target brain region and receive echo signals. The echo signals are sampled and digitally processed to generate an ultrasound signal data stream arranged in chronological order. EEG equipment collects multi-channel EEG signals from the target brain region, timestamps and encapsulates the collected EEG signals, and generates an EEG signal data stream with additional timestamps. The ultrasound processing unit processes the ultrasound signal data stream to obtain hemodynamic characteristics; The EEG processing unit processes the EEG signal data stream to obtain neural electrical activity characteristics; The joint analysis module performs time alignment and synchronization processing on the neural electrical activity features and the hemodynamic features based on a unified time reference. After time alignment, a coupling model between the neural electrical activity features and the hemodynamic features is constructed. The coupling model estimates the correlation strength and time delay relationship between changes in neural electrical activity and changes in blood flow response through correlation analysis, regression analysis, or model fitting, thereby obtaining the neural-blood flow coupling characteristics of the target brain region. Based on the neural-blood flow coupling characteristics, the joint analysis results of brain function reflecting the functional state, task-related activation patterns, and neural regulation characteristics of the target brain region are obtained.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the multimodal signal processing method as described in any one of claims 1 to 7.

10. A computer storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the multimodal signal processing method as described in any one of claims 1 to 7.

Images