Method and device for denoising multi-modal directed brain network signals, equipment and medium

Abstract

This application provides a method, apparatus, device, and medium for denoising multimodal directed brain network signals. The method includes: acquiring a denoised signal and a multimodal brain network signal at time t, where the denoised signal at time t is either the denoised signal calculated in the previous loop or the initial noise signal; inputting the denoised signal and the multimodal brain network signal at time t into a denoising model, and extracting local and global features between brain regions in the denoised signal and the multimodal brain network signal at time t through the denoising model to obtain a denoised signal and a directed connectivity brain network at time t-1; repeating the above steps until a denoised signal at time 0 is obtained, and then using the corresponding directed connectivity brain network as the target directed connectivity brain network. Through some embodiments of this application, global and local features in multimodal brain network signals can be obtained, thereby improving the denoising effect.

Description

Technical Field

[0001] This application relates to the field of signal denoising, specifically to a method, apparatus, device, and medium for denoising signals in a multimodal directed brain network. Background Technology

[0002] Signal denoising is a crucial step before signal recognition. Common techniques for signal denoising include two approaches: first, using empirical formulas and a small number of parameters to build shallow machine learning models for both denoising and signal recognition; and second, establishing deep nonlinear mappings through multiple convolutional networks. However, both methods only learn local features of the signal, resulting in poor denoising performance and reduced overall denoising effectiveness.

[0003] Therefore, improving the effectiveness of signal denoising is a problem that needs to be solved. Summary of the Invention

[0004] This application provides a method, apparatus, device, and medium for denoising multimodal directed brain network signals. Through some embodiments of this application, at least global and local features in multimodal brain network signals can be obtained, thereby improving the denoising effect.

[0005] In a first aspect, this application provides a method for denoising multimodal brain network signals. The method includes: acquiring a denoised signal at time t and a multimodal brain network signal, where t is an integer greater than or equal to 1, the denoised signal at time t is a denoised signal calculated in the previous loop or an initial noise signal, and the brain network signal is the user's current diffusion tensor magnetic resonance imaging and functional magnetic resonance imaging; inputting the denoised signal at time t and the multimodal brain network signal into a denoising model, extracting local and global features between brain regions in the denoised signal at time t and the multimodal brain network signal through the denoising model, and obtaining a denoised signal at time t-1 and a directed connectivity brain network at time t-1; repeating the above steps until a denoised signal at time 0 is obtained, and then using the corresponding directed connectivity brain network as the target directed connectivity brain network.

[0006] Therefore, unlike the related technologies that only use convolutional kernels in neural network models, the embodiments of this application denoise multimodal brain network signals by iteratively learning the local and global features between different brain regions, which can improve the denoising effect and thus ensure the quality of subsequent tasks.

[0007] In conjunction with the first aspect, in one embodiment of this application, the denoising model includes a phase attention alignment network, a graph spatiotemporal attention module, and a directed neural conduction detection network; the step of extracting local and global features between brain regions in the denoised signal at time t and the multimodal brain network signal through the denoising model to obtain the denoised signal at time t-1 and the directed connectivity brain network at time t-1 includes: fusing the denoised signal at time t and the brain network signal through the phase attention alignment network to obtain a fused signal; extracting local and global features between brain regions through the graph spatiotemporal attention module to obtain the directed connectivity features of the target brain region, wherein the graph spatiotemporal attention module includes spatial multi-head attention and temporal multi-head attention; and obtaining the denoised signal at time t-1 and the directed connectivity brain network at time t-1 through the directed neural conduction detection network.

[0008] Therefore, in this embodiment of the application, after fusing the denoised signal at time t and the multimodal brain network signal, the local and global features between brain regions are extracted through the graph spatiotemporal attention module, which can ensure the denoising effect.

[0009] In conjunction with the first aspect, in one embodiment of this application, the step of extracting local and global features between brain regions from the fused signal through the graph spatiotemporal attention module to obtain directed connectivity features of the target brain region includes: extracting local and global features between brain regions from the fused signal through the graph spatiotemporal attention module, and performing upsampling or downsampling operations to obtain directed connectivity features of the target brain region.

[0010] Therefore, by performing upsampling or downsampling operations during the learning of local and global features, the embodiments of this application can reduce the computational load of each module, thereby improving the denoising speed.

[0011] In conjunction with the first aspect, in one embodiment of this application, any one of the graph spatiotemporal attention modules performs the following steps to extract local and global features between the brain regions: constructing directed connections between multiple brain regions through spatial multi-head attention in the graph spatiotemporal attention module; and constructing temporal sequence features of directed connections in a single brain region through temporal multi-head attention in the graph spatiotemporal attention module; performing graph convolution operations on the directed connections between the multiple brain regions and the temporal sequence features to obtain the local and global features between the brain regions. Therefore, the embodiments of this application, through spatial multi-head attention and temporal multi-head attention, can learn the features of directed connections between brain regions, fully explore the spatiotemporal connection features between brain regions, and thus improve the denoising effect.

[0012] In conjunction with the first aspect, in one embodiment of this application, the step of performing a graph convolution operation on the directed connectivity relationships between the plurality of brain regions and the temporal sequence features to obtain local and global features between the brain regions includes: fusing the directed connectivity relationships between the plurality of brain regions and the temporal sequence features to obtain directed temporal fusion features; and applying the directed temporal fusion features and the structural connectivity matrix through the graph convolution operation to obtain local and global features between the brain regions, wherein the structural connectivity matrix is ​​calculated based on diffusion tensor magnetic resonance imaging (DTI).

[0013] Therefore, by adding a structural connection matrix during graph convolution, the embodiments of this application can ensure that the denoising model learns more brain network structures, thereby enabling the model to learn directional information between arbitrary brain regions in the topological space by utilizing the physical structural connections between brain regions.

[0014] In conjunction with the first aspect, in one embodiment of this application, fusing the denoised signal at time t and the multimodal brain network signal through the phase attention alignment network to obtain a fused signal includes: calculating the query space of the multimodal brain network signal; calculating the key space and value space of the denoised signal at time t; performing a single-layer mapping operation on the query space, the key space, and the value space to obtain mapping features; and summing the mapping features with the denoised signal at time t to obtain the fused signal.

[0015] Therefore, by performing single-layer mapping operations on the query space, key space, and value space, this application can enhance the matching of similar features, suppress broadband random noise, and improve the alignment effect of effective features of multi-source brain regions through phase attention mechanism by performing conjugate projection of multi-source spatial features.

[0016] In conjunction with the first aspect, in one embodiment of this application, before obtaining the denoised signal and the multimodal brain network signal at time t, the method further includes: mapping clean samples to Gaussian noise samples by adding noise; and training the denoised model and the anomaly classifier to be trained using the Gaussian noise samples and the brain network signal samples to obtain the denoised model.

[0017] Therefore, in the training process, this application embodiment improves the denoising capability of the denoising model by adding noise.

[0018] In conjunction with the first aspect, in one embodiment of this application, the denoising model to be trained includes a directed economic regularization loss function; wherein, the directed economic regularization loss function controls the sparsity of the directed connected brain network through hyperparameters, and the larger the value, the sparser the directed connected brain network.

[0019] Therefore, the embodiments of this application propose a directed economic regularization loss function, which weakens the characteristics of global directed information flow, mines locally efficient bidirectional neural transmission information, improves the overall economy of information transmission in brain networks, and enhances the representation effect of directed connected brain networks.

[0020] Secondly, this application provides a device for denoising brain network signals. The device includes: a signal acquisition module configured to acquire a denoised signal at time t and a multimodal brain network signal, wherein t is an integer greater than or equal to 1, the denoised signal at time t is a denoised signal calculated in the previous loop or an initial noise signal, and the multimodal brain network signal is the user's current diffusion tensor magnetic resonance imaging and functional magnetic resonance imaging; a feature calculation module configured to input the denoised signal at time t and the multimodal brain network signal into a denoising model, and extract local and global features between brain regions in the denoised signal at time t and the multimodal brain network signal through the denoising model to obtain a denoised signal at time t-1 and a directed connectivity brain network at time t-1; repeating the above steps until a denoised signal at time 0 is obtained, then using the corresponding directed connectivity brain network as the target directed connectivity brain network.

[0021] In conjunction with the second aspect, in one embodiment of this application, the denoising model includes a phase attention alignment network, a graph spatiotemporal attention module, and a directed neural conduction detection network; the feature calculation module is further configured to: fuse the denoised signal at time t and the functional magnetic resonance imaging (fMRI) through the phase attention alignment network to obtain a fused signal; extract local and global features between brain regions from the fused signal and the diffusion tensor magnetic resonance imaging (DTI) through the graph spatiotemporal attention module to obtain directed connectivity features of the target brain region, wherein the graph spatiotemporal attention module includes spatial multi-head attention and temporal multi-head attention; and obtain the denoised signal at time t-1 and the directed connectivity brain network at time t-1 through the directed neural conduction detection network.

[0022] In conjunction with the second aspect, in one embodiment of this application, the feature calculation module is further configured to: extract local and global features between brain regions from the fused signal through the graph spatiotemporal attention module, and perform upsampling or downsampling operations to obtain directed connectivity features of the target brain region.

[0023] In conjunction with the second aspect, in one embodiment of this application, any one of the graph spatiotemporal attention modules performs the following steps to extract local and global features between the brain regions: constructing directed connections between multiple brain regions through spatial multi-head attention in the graph spatiotemporal attention module; and constructing temporal sequence features of directed connections in a single brain region through temporal multi-head attention in the graph spatiotemporal attention module; performing graph convolution operation on the directed connections between the multiple brain regions and the temporal sequence features to obtain local and global features between the brain regions. In conjunction with the second aspect, in one embodiment of this application, the feature calculation module is further configured to: fuse the directed connections between the multiple brain regions and the temporal sequence features to obtain directed temporal fusion features; and perform graph convolution operation on the directed temporal fusion features and the structural connectivity matrix to obtain local and global features between the brain regions, wherein the structural connectivity matrix is ​​calculated based on diffusion tensor magnetic resonance imaging (DTI).

[0024] In conjunction with the second aspect, in one embodiment of this application, the feature calculation module is further configured to: calculate the query space of the functional magnetic resonance imaging; calculate the key space and value space of the denoised signal at time t; perform a single-layer mapping operation on the query space, the key space, and the value space to obtain the mapping features; and sum the mapping features with the denoised signal at time t to obtain the fused signal.

[0025] In conjunction with the second aspect, in one embodiment of this application, the signal acquisition module is further configured to: map clean samples to Gaussian noise samples by adding noise; and train the denoising model and the anomaly classifier to be trained using the Gaussian noise samples and the multimodal brain network signal samples to obtain the denoising model.

[0026] In conjunction with the second aspect, in one embodiment of this application, the denoising model to be trained includes a directed economic regularization loss function; wherein, the directed economic regularization loss function controls the sparsity of the directed connectivity brain network through hyperparameters, and the larger the value, the sparser the directed connectivity brain network.

[0027] Thirdly, this application provides an electronic device, including: a processor, a memory, and a bus; the processor is connected to the memory via the bus, the memory stores a computer program, and the computer program, when executed by the processor, can implement the method as described in any embodiment of the first aspect.

[0028] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed, can perform the methods described in any embodiment of the first aspect. Attached Figure Description

[0029] Figure 1 This is a schematic diagram illustrating the scene composition for brain network signal denoising in an embodiment of this application;

[0030] Figure 2 This is one of the flowcharts illustrating a method for denoising brain network signals according to an embodiment of this application;

[0031] Figure 3 This is a second flowchart illustrating a method for denoising brain network signals according to an embodiment of this application;

[0032] Figure 4 This is the third flowchart illustrating a method for denoising brain network signals according to an embodiment of this application;

[0033] Figure 5 This is the fourth flowchart illustrating a method for denoising brain network signals according to an embodiment of this application;

[0034] Figure 6 This is the fifth flowchart illustrating a method for denoising brain network signals in an embodiment of this application;

[0035] Figure 7 This is the sixth flowchart illustrating a method for denoising brain network signals in an embodiment of this application;

[0036] Figure 8 This is a schematic diagram illustrating the components of a brain network signal denoising device according to an embodiment of this application;

[0037] Figure 9 This is a schematic diagram illustrating the composition of an electronic device according to an embodiment of this application. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. The components of the embodiments of this application described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the detailed description of the embodiments of this application provided in the accompanying drawings is not intended to limit the scope of the claimed application, but merely represents selected embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the protection scope of this application.

[0039] With an aging population and changing lifestyles, the burden of neurological diseases is constantly increasing, making in-depth research on these diseases crucial for improving public and social health. Research on directed connectivity brain networks can help develop more precise treatment methods and provide individualized interventions to improve the quality of life for patients with neurological diseases. In this application, after denoising multimodal brain network signals using a denoising model, the resulting targeted directed connectivity brain network can be used to lay the foundation for intelligent auxiliary classification, pathogenesis analysis, biomarker screening, and targeted repair of neurological diseases. Furthermore, the targeted directed connectivity brain network can also be used to analyze the emotional state, fatigue level, and other states of corresponding users.

[0040] In existing technologies, the input data for constructing directed connectivity brain networks consists of temporal features of brain regions processed by software templates. However, different parameter settings during software use can lead to significant uncertainties and errors in the calculation of these temporal features, causing machine learning models to produce erroneous results when estimating the effective connectivity brain network using these features. Furthermore, this method requires two steps to calculate the effective connectivity brain network, resulting in low efficiency and hindering intelligent analysis in assisted diagnostic systems. To address this problem, this application proposes a Transformer network based on a conditional diffusion model. This model utilizes the spatial locations of anatomically prior brain regions to convert four-dimensional fMRI into coarse temporal features, which are then used as conditional controls for the backdiffusion process, thereby generating accurate temporal features of brain regions and an effective connectivity brain network.

[0041] In existing technologies, methods for constructing directed connectivity brain networks are based on computation and analysis using functional modality data. This approach fails to consider the long-distance topological features between brain regions and neglects the physical basis of interactions between them, resulting in inaccurate causal relationships between the constructed brain regions and reduced performance in subsequent judgments using effective connections. To address this issue, this application incorporates structural connectivity into a conditional diffusion model. It utilizes the physical structural connections between brain regions to guide the model in learning directional information between arbitrary brain regions in the topological space. Furthermore, it combines graph convolution and multi-head attention mechanisms to capture global and local multi-scale connectivity information, fully exploiting the complementary features of structure and function to establish causal relationships between multimodal brain regions.

[0042] This application can be applied to scenarios involving denoising brain network signals. To address the problems in the background art, some embodiments of this application map four-dimensional fMRI images into real temporal features of brain regions and directed connectivity brain networks between brain regions. This module includes multiple steps, each of which utilizes the physical structural connections between brain regions to guide the learning of directional information between arbitrary brain regions in the topological space. It also combines graph convolution and multi-head attention mechanisms to capture global and local multi-scale connectivity information, uncover potential causal relationships between brain regions, and improve the generation quality of directed connectivity brain networks, thereby enhancing the denoising effect.

[0043] The method steps in the embodiments of this application are described in detail below with reference to the accompanying drawings.

[0044] Figure 1 This application provides a scenario for denoising multimodal directed brain network signals in some embodiments, including a brain network signal acquisition device 110 and a server 120. Specifically, the multimodal brain network signal acquisition device 110 acquires the user's multimodal brain network signals and inputs them into the server 120. Simultaneously, an initial noise signal (i.e., Gaussian noise) is input into the server 120. The server 120 denoises the brain network signals and finally outputs the target directed brain network.

[0045] Therefore, this application falls under the category of brain network computing. It proposes a method for constructing directed connectivity brain networks based on multi-scale structure-function Transformer diffusion denoising, and applies it to the early prediction of brain diseases and the prediction of abnormal directed connections. This application can integrate complementary features between structural and functional data, and utilize multi-level Transformer networks to extract global and local directed connectivity features, thereby achieving accurate calculation of directed connections related to brain diseases and the prediction of potential biomarkers.

[0046] The following uses a server as an example to illustrate the implementation of a method for denoising multimodal directed brain network signals provided in some embodiments of this application.

[0047] At least in order to solve the problems in the background technology, such as Figure 2 As shown, some embodiments of this application provide a method for denoising multimodal directed brain network signals. This method includes an application process and a training process. The specific steps of the application process are as follows:

[0048] S210, acquire the denoised signal and multimodal brain network signal at time t.

[0049] It should be noted that the denoised signal at time t is either the denoised signal calculated in the previous loop or the initial noise signal, and the multimodal brain network signal is the user's current diffusion tensor magnetic resonance imaging (DTI) and functional magnetic resonance imaging (fMRI). In other words, since the denoising process in this application is cyclical, the denoised signal output from the previous loop is the denoised signal input to the current loop. It can be understood that during the calculation in the first loop, the input is the initial noise signal. Each loop decrements the value of t by 1, ultimately obtaining the denoised signal at time 0. t represents the time t of the noise signal, with a preset step size between each time point; time 0 is the last time point of the current noise signal.

[0050] As a specific embodiment of this application, such as Figure 3 As shown, the input in the first loop is the initial noise signal X. T After multiple iterations, the denoised signal X at time t is obtained. t , will X t As input to the next iteration, output the denoised signal X at time t-1. t-1 This process is repeated until a clean sample, i.e., the denoised signal at time 0, is obtained, denoised as X0. Each denoised signal is spaced at a certain step size, represented by time. Multiple denoised signals correspond to multiple time points, and each denoised signal is the signal at the current time. In the denoising model, the denoised signal X at time t represents... t Brain network signals H and structural connectivity matrix The denoised signal X at time t-1 is calculated. t-1 Then, the obtained parameter p θ .

[0051] It should be noted that after acquiring the current user's multimodal brain network signal, the server randomly extracts a Gaussian noise signal of the same length and size as the aforementioned brain network signal from the database. The initial noise signal is the signal of the first moment of this Gaussian noise signal.

[0052] S220, input the denoised signal and multimodal brain network signal at time t into the denoising model, and extract the local and global features between brain regions in the denoised signal and multimodal brain network signal at time t through the denoising model to obtain the denoised signal and directed connected brain network at time t-1.

[0053] In one embodiment of this application, the denoising model includes a structure-guided multi-scale functional Transformer denoising process to guide the denoising model from Gaussian noise to obtain clean samples and directed connected brain networks. The inputs are a four-dimensional fMRI and a structural connectivity matrix. The four-dimensional fMRI is dot-producted with a prior mask (m brain regions) of anatomical brain region knowledge to obtain coarse temporal features H of the brain regions. The structural connectivity matrix is ​​calculated based on diffusion tensor imaging (DTI). Simultaneously, a time step t is added to the input to ensure that each denoising step is independent, which is beneficial for estimating noise introduced during the calculation process.

[0054] The above module is a U-shaped structure. First, it uses a phase attention alignment network to fuse coarse samples and noisy samples. Then, it uses a multi-scale graph spatiotemporal attention Transformer network to extract multi-level spatiotemporal orientation features of brain regions. Finally, it uses a directed neural conduction detection network to obtain accurate temporal features of brain regions and directed connectivity brain networks.

[0055] In one embodiment of this application, the denoising model specifically includes a phase attention alignment network, a graph spatiotemporal attention module, and a directed neural conduction detection network. The specific steps for obtaining the denoised signal at time t-1 and the directed connectivity brain network at time t-1 through the denoising model are as follows:

[0056] S2201, the denoised signal at time t and the functional magnetic resonance imaging are fused through a phase attention alignment network to obtain a fused signal.

[0057] It should be noted that the functional magnetic resonance imaging (fMRI) is input into the prior mask and matched with each brain region to obtain the signal H.

[0058] Specifically, the steps to obtain the fused signal include:

[0059] S1: Calculate the query space for functional magnetic resonance imaging;

[0060] S2: Calculate the key space and value space of the denoised signal at time t;

[0061] S3: Perform a single-level mapping operation on the query space, key space, and value space to obtain mapping features;

[0062] S4: Summing the mapped features with the denoised signal at time t yields the fused signal.

[0063] In other words, the phase attention alignment network's role is to fuse the input brain network signal (i.e., coarse sample) H and the denoised signal at time t. A conventional method for fusing multiple features is concatenation, but this significantly increases feature dimensionality and model parameters, hindering stable model training. To overcome this problem, this application introduces an attention mechanism. It calculates the query space Q for the coarse sample H and the key space K and value space V for the denoised signal at time t. LM represents a single-layer mapping, and Norm represents normalization. The calculation formula for the phase attention alignment network is:

[0064] Q=LM1(Norm(H)) K=LM2(Norm(X t V = LM3(Norm(X) t ))

[0065]

[0066] Wherein, the input signal H and the denoised signal X at time t are... t They have the same size m×d, with intermediate variables Q representing the query space, K representing the key space, V representing the value space, and output variables. Indicates the fused signal. The size is m×d, where m represents the number of brain regions, d represents the dimension of the temporal features of each brain region, and softmax represents the normalization function.

[0067] For example, such as Figure 4 As shown, firstly, the signal H is normalized and then subjected to a single-layer mapping operation. Next, the query space values ​​corresponding to the features after the single-layer mapping are calculated. Simultaneously, the signal H is normalized and subjected to a single-layer mapping operation again. Then, the key space and value space corresponding to the features after the single-layer mapping are calculated separately. Next, a matrix multiplication operation is performed between the query space and the key space. The result of the multiplication is normalized, and then the weighted value between the current brain region and other brain regions in the value space vector is calculated using the normalized data. Then, the above weighted value is subjected to matrix multiplication and a single-layer mapping operation with the value space to obtain the multiplied mapping features. Finally, the denoised signal X at time t is... t The fused signal is obtained by summing the multiplication mapping features described above.

[0068] S2202 extracts local and global features between brain regions by using a graph-temporal attention module to obtain directed connectivity features of the target brain region by combining fused signals and diffusion tensor magnetic resonance imaging.

[0069] Understandably, the graph spatiotemporal attention module can also be called the graph spatiotemporal attention Transformer.

[0070] As an embodiment of this application, the specific steps for obtaining the directed connectivity features of the target brain region include: extracting local and global features between brain regions from the fused signal through a graph spatiotemporal attention module, and performing upsampling or downsampling operations to obtain the directed connectivity features of the target brain region.

[0071] This application incorporates multiple graph-temporal attention modules into the denoising module for feature computation, such as... Figure 3 As shown, the fused signal is input into the graph spatiotemporal attention transformer 302. After feature extraction and calculation, it is input into the downsampling module 308 for downsampling. The downsampled features are then input into the graph spatiotemporal attention transformer 303. After feature extraction and calculation, they are input into the downsampling module 309 for further downsampling. The downsampled features are then input into the graph spatiotemporal attention transformer 304. After feature extraction and calculation, they are input into the upsampling module 311 for further upsampling. The upsampled features are then input into the graph spatiotemporal attention transformer 305. Since the output data dimensions of the graph spatiotemporal attention transformer 305 and the graph spatiotemporal attention transformer 303 are the same, the features output by these two modules can be averaged to fuse the features obtained from the two modules. The averaged data is then input into the upsampling module 310 for upsampling. Finally, the upsampled features are input into the graph spatiotemporal attention transformer. In step 306, feature calculation continues. Finally, the average value of the features calculated in graph spatiotemporal attention Transformer 302 and graph spatiotemporal attention Transformer 306 is calculated to obtain the directed connectivity features of the target brain region.

[0072] Furthermore, the graph-temporal attention module includes spatial multi-head attention and temporal multi-head attention. Any graph-temporal attention module performs the following steps to extract local and global features between brain regions:

[0073] S11: Construct directed connections between multiple brain regions through spatial multi-head attention in the graph spatiotemporal attention module.

[0074] S12: Construct temporal sequence features of directed connections in a single brain region through temporal multi-head attention in the graph spatiotemporal attention module.

[0075] S13: Perform graph convolution operations on the directed connectivity and temporal sequence features between multiple brain regions to obtain local and global features between brain regions.

[0076] Furthermore, the specific implementation steps of S13 above include:

[0077] T1: The directed connectivity and temporal sequence features between multiple brain regions are fused to obtain directed temporal fusion features.

[0078] T2: Directed temporal fusion features and structural connectivity matrices are processed through graph convolution to obtain local and global features between brain regions.

[0079] It should be noted that the structural connectivity matrix is ​​calculated based on diffusion tensor magnetic resonance imaging (DTI).

[0080] In other words, such as Figure 5 As shown, taking the graph-temporal attention Transformer 302 as an example, firstly, the fused signal is normalized. Then, the normalized fused signal is input into spatial multi-head attention to construct directed connections between multiple brain regions. Activation functions are then calculated, and the output features are summed with the fused features to obtain spatial features. Next, the spatial features are normalized, and the normalized signal is input into temporal multi-head attention to construct temporal sequence features of directed connections in a single brain region. Activation functions are then calculated, and the output features are fused with the aforementioned spatial features to obtain spatiotemporal features. Finally, the structural connectivity matrix and spatiotemporal features are input into graph convolution. After graph convolution calculation, activation functions are calculated, and then the input is again fed into graph convolution for further activation function calculation. Finally, the spatiotemporal features are summed with the features output by the last activation function to obtain the local and global features between brain regions.

[0081] It is understandable that the specific structure of each graph spatiotemporal attention Transformer is as follows: Figure 5 As shown.

[0082] As a specific embodiment of this application, the graph spatiotemporal attention Transformer learns the directed connectivity features between global and local brain regions in the topological space. Through downsampling and upsampling, these features are concatenated into a U-shaped structure to extract multi-scale directed connectivity features of brain regions. The input fused signal undergoes spatial and temporal multi-head attention to fully extract the spatiotemporal connectivity features between brain regions. Then, after two layers of graph convolution, local and global causal relationships between brain regions are established in the topological space. Finally, the spatiotemporal directed features of brain regions (i.e., local and global features between different brain regions) are output. The calculation formula is as follows:

[0083]

[0084] T2 = ReLU(TMA(Norm(T1))) + T1

[0085]

[0086] Here, SMA represents the spatial multi-head attention layer, which takes temporal features of brain regions as input (i.e., the signal output by the phase attention alignment network or the signal output by the previous module during computation) to construct directed connections between multiple brain regions. TMA represents the temporal multi-head attention layer, which takes feature values ​​of m brain regions at a certain moment as input to construct temporal features of directed connections within a single brain region. ReLU is an activation layer, GCN is a single-layer graph convolutional layer, T1 represents spatial features, and T2 represents spatial-temporal features. This represents the local and global characteristics between different brain regions. This represents the structural connection matrix. The feature sizes of the input and output are the same, i.e., m×d.

[0087] S2203, the directed connectivity features of the target brain region are obtained by using a directed neural conduction detection network to obtain the denoised signal at time t-1 and the directed connectivity brain network at time t-1.

[0088] Specifically, such as Figure 6 As shown, the directed neural conduction detection network calculates the temporal features of the denoised brain regions to obtain directed connectivity brain networks. After passing through a U-shaped graph-spatiotemporal attention Transformer network, the output is the directed connectivity features of the target brain region. The process involves two branches: one is a fine-grained neuronal denoising layer, and the other consists of a spatiotemporal transformation layer and a time-shifted causal projection layer. The fine-grained neuronal denoising layer branch yields precise temporal features of brain regions, calculated using the following formula:

[0089] (like Figure 6 (The fine-grained neuron denoising layer shown)

[0090]

[0091] in, Network models can be used. θ (X t ,t|H,G) indicates that after time step T, the precise temporal features of the brain region X′0 at time 0 are finally obtained, which has the same size as the clean sample X0. Structural connectivity matrix The size is m×m. The output of the next branch is a directed connectivity brain network. It first passes through a spatiotemporal transformation layer, represented as ST, whose structure consists of multiple one-dimensional convolutional kernels with consistent input and output sizes. Then it passes through a time-shifted causal projection layer. Finally, the formula for calculating the directed connectivity brain network E is as follows:

[0092] (like Figure 6 (The time-shifted causal projection layer shown)

[0093]

[0094] (like Figure 6 (The spacetime conversion layer shown)

[0095]

[0096] Where, ε i The random error representing the reconstruction is represented by the output directed connection brain network E, which has a size of m×m. This matrix is ​​asymmetric, with diagonal elements being 0, and element E... ii This indicates a directed connection: brain region i → brain region j, y j Let {y1, y2, ..., y m Any one of}, y i Let {y1, y2, ..., y m Any one of the following.

[0097] S230, repeat the above steps until the denoised signal at time 0 is obtained, then take the corresponding directed connectivity brain network as the target directed connectivity brain network.

[0098] In other words, a directed connectivity brain network and a denoised signal are obtained in each loop until the denoised signal at time 0 is obtained. The directed connectivity brain network corresponding to the denoised signal at time 0 is the target directed connectivity brain network.

[0099] In one embodiment of this application, such as Figure 3 As shown, the target directed connection brain network can be input into the neural abnormality detector, and the probability of various brain diseases can be predicted through the neural abnormality detector.

[0100] The process of model building and training in the embodiments of this application will be described below.

[0101] This application proposes a multi-scale structure-function Transformer diffusion denoising method for constructing directed connectivity brain networks, and applies it to the prediction and early screening of abnormal connections in neurological diseases. This method applies a diffusion model to the generation and analysis of generative models of multimodal brain networks. For patients with brain diseases, it integrates structural connectivity and functional brain imaging to establish a multi-stage, step-by-step diffusion mapping relationship, mining directional information between topological brain regions to gradually generate an effective connectivity brain network. This lays the foundation for intelligent assisted diagnosis, pathogenesis analysis, biomarker screening, and targeted repair therapy for neurological diseases.

[0102] This application addresses the challenges of constructing effectively connected brain networks by fully utilizing complementary information from multimodal data, and the inability to establish a complete framework for generating effective connected brain networks from functional brain imaging. Driven by functional and structural data, the diffusion denoising model learns the topological denoising process of functional brain images under the guidance of the physical structural connectivity of brain regions. Based on multi-scale network structures and multi-head attention mechanisms, it focuses on global and local multi-scale connectivity information, improving the accuracy of noise estimation and the precision of directed connection calculation, thereby enhancing the disease representation effect of multimodal directed connected brain networks.

[0103] Before training the model, this application uses a Markov broadband superposition diffusion process to transform the temporal features of empirical brain regions into Gaussian noise. This module comprises multiple steps, progressively adding a small amount of Gaussian-distributed random noise. Specifically, a clean sample X0 is gradually mapped to Gaussian noise X by adding noise. T The specific process is as follows: Using the brain region temporal features obtained after processing the four-dimensional fMRI samples with the software toolbox, the clean sample X0 is used. Gaussian noise is then progressively added to the clean sample, with each step's Gaussian noise weighted by a specific β. t This means that after T steps (T>1), Gaussian noise X is generated. T For simplicity, noise samples at two adjacent time points can be represented as:

[0104]

[0105]

[0106] Where ∈ represents added Gaussian noise, I represents the multivariate covariance identity matrix, and N represents a Gaussian distribution. β t The range is 0 to 1, and it increases as the step size t increases. Through recursion, the relationship between Gaussian samples and clean samples can be obtained:

[0107]

[0108]

[0109] Where, α t =1-β t ,∈ t The noise sample is sampled from a Gaussian distribution at time t. When T is sufficiently large, the noise sample approximates a standard normal distribution. At each step of the diffusion process, the size of the noise sample is m×d, where m is the number of brain regions and d is the dimension of the temporal features of each brain region.

[0110] In other words, for each subject, both fMRI and DTI modalities were collected, and the actual temporal features (clean sample) X0 and structural connectivity matrix of the brain regions were calculated using a software toolbox. The fMRI, structural connectivity matrix, and temporal features of real brain regions are fed into the model. First, a Markov broadband superposition diffusion process (T steps) is performed to obtain noisy samples Xt at each diffusion time step and the final Gaussian samples. Then, a structure-guided multi-scale functional Transformer denoising process is used to progressively obtain denoised samples Xt' and directed connectivity brain networks E at each time step. After T steps, the final clean samples X0' and directed connectivity brain networks E are obtained. Finally, the directed connectivity brain network E is passed to a classifier to predict the probability of Parkinson's disease. The four loss functions used in this training process are as follows:

[0111] 1. Noise estimation loss function. In the reverse denoising process, noise needs to be estimated at each step before the sample at that time can be denoised. Given the noise sample Xt at time t and the actual noise added during the diffusion process ∈ t Then this loss function can be expressed as:

[0112]

[0113] For L n Backpropagation is performed to guide the parameter updates of the phase attention alignment network and the graph spatiotemporal attention Transformer network.

[0114] 2. Directed propagation reconstruction loss function. After passing through the spatiotemporal transformation layer and the time-shifted causal projection layer, the noise sample Y at time t-1 is reconstructed. t-1 The loss can be defined as:

[0115]

[0116] For L r Backpropagation is performed to guide the parameter updates of the directed neural conduction detection network.

[0117] 3. Directed economic regularization loss function. To constrain the sparsity of directed connected brain networks, it is defined as:

[0118]

[0119] Wherein, ω is a hyperparameter that controls the sparsity of the brain network; the larger the value, the sparser the network.

[0120] 4. Loss Function for Neural Abnormality Detection. A directed neural network E passes through a neural abnormality detector C and outputs a predicted disease category. The difference in distribution between this prediction and the actual disease label B is defined as:

[0121]

[0122] For L cBackpropagation is performed to guide the parameter updates of the classifier network.

[0123] A 10-fold cross-training strategy was used to divide L subjects into groups, with 90% of the subjects used for training the model in each fold and the remaining 10% used for testing. The model converged when the training reached its maximum number of iterations or when the loss function value no longer changed. This model has a strong ability to predict Parkinson's disease and can generate disease-related directed connectivity brain networks.

[0124] During the testing process, such as Figure 7 As shown, after the denoising model is trained, it is input into a four-dimensional fMRI, Gaussian noise samples, and a structural connectivity matrix. This data is then processed through T structure-guided multi-scale functional Transformer networks and a neural abnormality detector, outputting a predicted category of brain disease, precise temporal features of brain regions, and a directed connectivity brain network. This directed connectivity brain network can be used to analyze abnormal directed connectivity features (enhanced and weakened directed connectivity strengths) at different stages of brain diseases, providing potential biomarkers for early prediction of brain diseases. Furthermore, this model can be used for the prediction and analysis of any neurological disease.

[0125] Therefore, the key technical points of this application are as follows:

[0126] (1) This application proposes a directed connectivity brain network computation method based on multi-scale structure-function Transformer diffusion denoising. It constructs a multi-step bidirectional mapping of brain region temporal features from two directions: forward diffusion and reverse denoising. It perceives the information flow characteristics of multi-scale brain regions in topological space, fully integrates effective information of structure and function, improves the accuracy of noise estimation and the accuracy of directed brain network computation, and enhances the disease expression effect of multimodal directed connectivity brain networks.

[0127] (2) This application designs a Markov broadband superposition and diffusion module, which utilizes the multi-band characteristics of Gaussian noise to gradually add Gaussian noise of a certain amplitude to low-frequency samples. After multiple steps of projection superposition, it diffuses into Gaussian noise samples with high flatness. This module can establish an accurate power spectrum noise sample distribution and improve the calculation accuracy of the directed connectivity features in the denoising process.

[0128] (3) This application designs a structure-guided multi-scale functional Transformer denoising module. Based on the physical structural connections between brain regions, a Transformer network with a U-shaped groove structure is designed in the topological space. Through multiple time steps, multi-scale and multi-level brain region connection features are extracted. The progressive growth order method is used to map four-dimensional fMRI images into real brain region temporal features and directed connectivity brain networks, realizing end-to-end mapping from images to directed connectivity brain networks and accurate prediction of neurological diseases.

[0129] (4) This application designs a phase attention alignment network. Through the phase attention mechanism, multi-source spatial features are conjugately projected to enhance similar feature matching, suppress broadband random noise, and improve the alignment effect of effective features of multi-source brain regions.

[0130] (5) This application designs a graph-temporal attention Transformer network to establish the spatial dependence and temporal continuity of brain region temporal features in the topological space. By introducing relevant interferogram convolution, it learns and pays attention to the directional flow features between brain regions, which enhances the complementarity of structure and function and improves the estimation ability of broadband Gaussian noise.

[0131] (6) This application designs a directed neural conduction detection network, which removes noise from noisy samples through distribution matching consistency to obtain accurate temporal features of brain regions. At the same time, it utilizes the spatiotemporal transformation layer and the time-shifted causal projection layer to mine the information conduction features between multiple brain regions, establishes a directional model of neural signal conduction, and improves the generation quality of the directed connection brain network and the prediction performance of brain diseases.

[0132] (7) This application proposes a directed economic regularization loss function, which weakens the characteristics of global directed information flow, mines locally efficient bidirectional neural transmission information, improves the overall economy of information transmission in brain networks, and enhances the representation effect of directed connected brain networks.

[0133] Compared with the prior art, this application has the following main advantages:

[0134] (1) Compared with existing methods for constructing effective connections, the directed connectivity brain network calculation method based on multi-scale structure-function Transformer diffusion denoising proposed in this application can realize one-stop calculation of directed connectivity brain network features from image data, simplify manual preprocessing steps, improve efficiency, reduce the estimation error of directed connectivity brain networks, and has strong clinical application prospects.

[0135] (2) Compared with existing data-driven methods, this application uses the physical structure of brain regions to guide the learning of the directional features of neural conduction from functional images, and combines multi-scale Transformer networks to further explore the directional dependence of brain regions at local and global scales, deeply integrating complementary information between modalities, which is conducive to generating accurate directed connectivity brain networks.

[0136] (3) This application uses structural connectivity, functional imaging and noise data as input to the model, learns the directional connectivity features of neural conduction in brain regions through deep mapping, and further analyzes abnormal directed connectivity, providing potential biomarkers for early prediction and clinical treatment of patients with brain diseases.

[0137] The above describes a specific embodiment of a method for denoising multimodal directed brain network signals. The following describes an apparatus for denoising multimodal directed brain network signals.

[0138] like Figure 8 As shown, some embodiments of this application provide a multimodal directed brain network signal denoising device 800, which includes a signal acquisition module 810 and a feature calculation module 820.

[0139] The signal acquisition module 810 is configured to acquire the denoised signal and the multimodal brain network signal at time t, where t is an integer greater than or equal to 1. The denoised signal at time t is the denoised signal calculated in the previous loop or the initial noise signal, and the multimodal brain network signal is the user's current diffusion tensor magnetic resonance imaging and functional magnetic resonance imaging. The feature calculation module 820 is configured to input the denoised signal and the multimodal brain network signal at time t into a denoising model, and extract the local and global features between brain regions in the denoised signal and the multimodal brain network signal at time t through the denoising model to obtain the denoised signal at time t-1 and the directed connectivity brain network at time t-1. The above steps are repeated until the denoised signal at time 0 is obtained, and the corresponding directed connectivity brain network is taken as the target directed connectivity brain network.

[0140] In one embodiment of this application, the denoising model includes a phase attention alignment network, a graph spatiotemporal attention module, and a directed neural conduction detection network; the feature calculation module 820 is further configured to: fuse the denoised signal at time t and the functional magnetic resonance imaging (fMRI) through the phase attention alignment network to obtain a fused signal; extract local and global features between brain regions from the fused signal and the diffusion tensor magnetic resonance imaging (DTI) through the graph spatiotemporal attention module to obtain directed connectivity features of the target brain region, wherein the graph spatiotemporal attention module includes spatial multi-head attention and temporal multi-head attention; and obtain the denoised signal at time t-1 and the directed connectivity brain network at time t-1 through the directed neural conduction detection network.

[0141] In one embodiment of this application, the feature calculation module 820 is further configured to: extract local and global features between brain regions from the fused signal through the graph spatiotemporal attention module, and perform upsampling or downsampling operations to obtain directed connectivity features of the target brain region.

[0142] In one embodiment of this application, any one of the graph spatiotemporal attention modules performs the following steps to extract local and global features between the brain regions: constructing directed connections between multiple brain regions through spatial multi-head attention in the graph spatiotemporal attention module; and constructing temporal sequence features of directed connections in a single brain region through temporal multi-head attention in the graph spatiotemporal attention module; performing graph convolution operation on the directed connections between the multiple brain regions and the temporal sequence features to obtain local and global features between the brain regions. In conjunction with the second aspect, in one embodiment of this application, the feature calculation module 820 is further configured to: fuse the directed connections between the multiple brain regions and the temporal sequence features to obtain directed temporal fusion features; and perform graph convolution operation on the directed temporal fusion features and the structural connectivity matrix to obtain local and global features between the brain regions, wherein the structural connectivity matrix is ​​calculated based on diffusion tensor magnetic resonance imaging (DTI).

[0143] In one embodiment of this application, the feature calculation module 820 is further configured to: calculate the query space of the functional magnetic resonance imaging; calculate the key space and value space of the denoised signal at time t; perform a single-layer mapping operation on the query space, the key space and the value space to obtain the mapping features; and sum the mapping features with the denoised signal at time t to obtain the fused signal.

[0144] In one embodiment of this application, the signal acquisition module 810 is further configured to: map clean samples to Gaussian noise samples by adding noise; and train the denoising model and the anomaly classifier to be trained using the Gaussian noise samples and brain network signal samples to obtain the denoising model.

[0145] In one embodiment of this application, the denoising model to be trained includes a directed economic regularization loss function; wherein, the directed economic regularization loss function controls the sparsity of the directed connected brain network through hyperparameters, and the larger the value, the sparser the directed connected brain network.

[0146] In the embodiments of this application, Figure 8 The module shown can achieve Figures 1 to 7 Each process in the method embodiment. Figure 8 The operations and / or functions of each module in the document are respectively designed to achieve... Figures 1 to 7 The corresponding processes in the method embodiments are described above. For details, please refer to the descriptions in the above method embodiments; to avoid repetition, detailed descriptions are omitted here.

[0147] like Figure 9As shown, this application provides an electronic device 900, including: a processor 910, a memory 920 and a bus 930. The processor is connected to the memory via the bus. The memory stores computer-readable instructions. When the computer-readable instructions are executed by the processor, they are used to implement the method as described in any one of the above embodiments. For details, please refer to the description in the above method embodiments. To avoid repetition, detailed descriptions are appropriately omitted here.

[0148] The bus is used to enable direct communication between these components. In this embodiment, the processor can be an integrated circuit chip with signal processing capabilities. The processor can be a general-purpose processor, including a Central Processing Unit (CPU), a Network Processor (NP), etc.; it can also be a Digital Signal Processor (DSP), an Application-Specific Integrated Circuit (ASIC), an On-Premises Programmable Gate Array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in this embodiment. The general-purpose processor can be a microprocessor or any conventional processor.

[0149] The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc. The memory stores computer-readable instructions, which, when executed by the processor, can perform the methods described in the above embodiments.

[0150] Understandable. Figure 9 The structure shown is for illustrative purposes only and may include structures larger than [other structures]. Figure 9 The more or fewer components shown, or having the same Figure 9 The different configurations shown. Figure 9 The components shown can be implemented using hardware, software, or a combination thereof.

[0151] This application also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a server, it implements any of the methods described in all the above embodiments. For details, please refer to the descriptions in the above method embodiments. To avoid repetition, detailed descriptions are appropriately omitted here.

[0152] The above description is merely a preferred embodiment of this application and is not intended to limit this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application. It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0153] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for denoising multimodal directed brain network signals, characterized in that, The method includes: Acquire the denoised signal and multimodal brain network signal at time t, where t is an integer greater than or equal to 1, the denoised signal at time t is the denoised signal calculated in the previous loop or the initial noise signal, and the multimodal brain network signal is the user's current diffusion tensor magnetic resonance imaging and functional magnetic resonance imaging. The denoised signal at time t and the multimodal brain network signal are input into the denoising model. The denoising model extracts the local and global features between brain regions in the denoised signal at time t and the multimodal brain network signal to obtain the denoised signal at time t-1 and the directed connectivity brain network at time t-1. Repeat the above steps until the denoised signal at time 0 is obtained, then use the corresponding directed connectivity brain network as the target directed connectivity brain network.

2. The method according to claim 1, characterized in that, The denoising model includes a phase attention alignment network, a graph spatiotemporal attention module, and a directed neural conduction detection network. The step of extracting local and global features between brain regions in the denoised signal at time t and the multimodal brain network signal using the denoising model to obtain the denoised signal at time t-1 and the directed connectivity brain network at time t-1 includes: The denoised signal at time t and the functional magnetic resonance imaging are fused through the phase attention alignment network to obtain a fused signal; The fused signal and the diffusion tensor magnetic resonance imaging are used to extract local and global features between brain regions through the graph spatiotemporal attention module to obtain the directed connectivity features of the target brain region. The graph spatiotemporal attention module includes spatial multi-head attention and temporal multi-head attention. The directed connectivity features of the target brain region are used to obtain the denoised signal at time t-1 and the directed connectivity brain network at time t-1 through the directed neural conduction detection network.

3. The method according to claim 2, characterized in that, The step of extracting local and global features between brain regions from the fused signal through the graph-spatiotemporal attention module to obtain directed connectivity features of the target brain region includes: The fused signal is processed by the spatiotemporal attention module to extract local and global features between brain regions, and upsampling or downsampling operations are performed to obtain the directed connectivity features of the target brain region.

4. The method according to claim 3, characterized in that, Any of the aforementioned spatiotemporal attention modules performs the following steps to extract local and global features between the brain regions: The spatial multi-head attention in the graph spatiotemporal attention module constructs directed connections between multiple brain regions; and the temporal multi-head attention in the graph spatiotemporal attention module constructs temporal sequence features of directed connections between a single brain region. By performing graph convolution operations on the directed connectivity relationships between the multiple brain regions and the temporal sequence features, local and global features between the brain regions are obtained.

5. The method according to claim 4, characterized in that, The step of performing graph convolution operations on the directed connectivity relationships between the multiple brain regions and the temporal sequence features to obtain local and global features between the brain regions includes: The directed connectivity relationships between the multiple brain regions and the temporal sequence features are fused to obtain directed temporal fusion features; The directed temporal fusion features and structural connectivity matrix are processed through the graph convolution operation to obtain local and global features between the brain regions, wherein the structural connectivity matrix is ​​calculated based on the diffusion tensor magnetic resonance imaging.

6. The method according to any one of claims 2-5, characterized in that, The step of fusing the denoised signal at time t and the multimodal brain network signal through the phase attention alignment network to obtain a fused signal includes: Calculate the query space for the functional magnetic resonance imaging; Calculate the key space and value space of the denoised signal at time t; A single-level mapping operation is performed on the query space, the key space, and the value space to obtain the mapping features; The fused signal is obtained by summing the mapping features with the denoised signal at time t.

7. The method according to any one of claims 2-5, characterized in that, Before acquiring the denoised signal and multimodal brain network signal at time t, the method further includes: Clean samples are mapped to Gaussian noise samples by adding noise. The denoising model is obtained by training the denoising model and the anomaly classifier to be trained using the Gaussian noise samples and multimodal brain network signal samples.

8. The method according to claim 7, characterized in that, The denoising model to be trained includes a directed economic regularization loss function; The directed economic regularization loss function controls the sparsity of the directed connected brain network through the hyperparameter ω. The larger ω is, the sparser the directed connected brain network is.

9. A device for denoising multimodal directed brain network signals, characterized in that, The device includes: The signal acquisition module is configured to acquire the denoised signal and the multimodal brain network signal at time t, where t is an integer greater than or equal to 1, the denoised signal at time t is the denoised signal calculated in the previous loop or the initial noise signal, and the multimodal brain network signal is the user's current diffusion tensor magnetic resonance imaging and functional magnetic resonance imaging. The feature calculation module is configured to input the denoised signal at time t and the multimodal brain network signal into the denoising model, and extract the local and global features between brain regions in the denoised signal at time t and the multimodal brain network signal through the denoising model to obtain the denoised signal at time t-1 and the directed connected brain network at time t-1. Repeat the above steps until the denoised signal at time 0 is obtained, then use the corresponding directed connectivity brain network as the target directed connectivity brain network.

10. An electronic device, characterized in that, include: Processor, memory, and bus; The processor is connected to the memory via the bus, and the memory stores a computer program that, when executed by the processor, can implement the method as described in any one of claims 1-8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed, implements the method as described in any one of claims 1-8.

Images