Electric field distribution identification method, model training method, transcranial magnetic stimulation real-time electric field prediction system, equipment and medium
Through the alignment processing and the use of deep learning models, the problem that electric field prediction methods in the prior art are difficult to adapt to multiple devices and multi-scan parameters is solved, and efficient and accurate electric field recognition is achieved, reducing labeling costs, and improving the generalization ability of the model.
Patent Information
- Application Number
- CN202510125700.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-27
- Publication Date
- 2025-05-30
AI Technical Summary
The existing electric field prediction methods are difficult to adapt to the needs of multi-device and multi-scan parameters, and rely on complex finite element analysis, resulting in high data labeling costs and insufficient model generalization capabilities.
Through an electric field distribution recognition method, the target domain image is obtained using MRI technology, and the image is encoded and decoded through the model, and the source domain and target domain images are aligned to determine the electric field distribution image of the target object. This method adopts a domain alignment layer and a multi-task layer to achieve feature alignment across devices and across parameters through deep learning models.
The cost of electric field distribution labeling is significantly reduced, and accurate electric field recognition is achieved for images of different MRI devices and diverse scanning parameters, which improves the generalization ability and scope of application of the model.
Smart Images
Figure CN120071353A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of artificial intelligence technology, and particularly to electric field distribution recognition, model training methods, transcranial magnetic stimulation real-time electric field prediction systems, devices and media. Background Art
[0002] Transcranial Magnetic Stimulation (TMS) is a non-invasive stimulation technology based on electromagnetic fields, which induces an electric field in the cerebral cortex through a time-varying pulsed magnetic field, thereby regulating neuronal electrical activity.
[0003] Currently, existing electric field prediction methods are usually trained based on labeled data of a single type of MRI device and fixed scanning parameters. This method is difficult to meet the requirements of multiple devices and multiple scanning parameters in actual clinical practice. MRI devices from different manufacturers (such as GE, Siemens, Philips) have significant differences in hardware configurations (such as coil design, gradient intensity) and software algorithms. Even when the scanning parameters are the same, the quality and texture features of the generated MRI images may be significantly different. In addition, the acquisition of label data (i.e., electric field distribution) depends on complex finite element analysis (FEM), which requires a large amount of time and computing resources, further limiting the expansion of data scale and the generalization ability of the model. Summary of the Invention
[0004] The purpose of the present application is to provide electric field distribution recognition, model training methods, transcranial magnetic stimulation real-time electric field prediction systems, devices and media.
[0005] According to the first aspect of the embodiments of the present application, an electric field distribution recognition method is provided. The method specifically includes the following steps: obtaining a target domain image of a target object; wherein the target domain image is a cranial magnetic image of the target object acquired by using MRI technology; during the process of encoding the image by using a model, performing alignment processing on the target domain image and a source domain image; wherein the source domain image includes paired original MRI images and electric field distributions; and determining an electric field distribution image of the target object according to the alignment processing result.
[0006] Optionally, obtaining the target domain image of the target object includes: scanning the head of the target object by using a first magnetic resonance device to obtain the target domain image of the target object; wherein the source domain image is an image scanned by using a second magnetic resonance device, and the first magnetic resonance device and the second magnetic resonance device are different devices.
[0007] Optionally, during the process of encoding an image using the model, alignment processing is performed on the target domain image and the source domain image, including: encoding the target domain image and the source domain image using the encoder in the model to extract multi-scale features; decoding the extracted multi-scale features using the decoder in combination with skip connections; identifying the device parameters of the first magnetic resonance device using the decoding result; and performing alignment processing on the target domain image and the source domain image according to the identification result of the device parameters.
[0008] Optionally, the model training method includes: obtaining training samples, where the training samples include: source domain images and target domain images; among them, the target domain images are unlabeled images, and the source domain images are images containing labels; inputting the source domain images and the target domain images into the model to be trained; and training the model to be trained to obtain the model by processing the source domain images and the target domain images using the domain alignment layer and the multi-task layer included in the model to be trained.
[0009] Optionally, the domain alignment layer is used to adjust the gradient during backpropagation through the gradient reversal layer to promote feature alignment between the source domain image and the target domain image.
[0010] Optionally, the multi-task layer includes an electric field prediction branch and a domain classification branch; among them, the electric field prediction branch is used to predict the electric field distribution of the target domain image; and the domain classification branch is used to identify the device parameters of the first magnetic resonance device for the target domain image.
[0011] Optionally, the domain alignment layer dynamically adjusts the value of the gradient scaling factor during training, sets a first scaling factor at the beginning of training, and sets a second scaling factor at the end of training; where the first scaling factor is less than the second scaling factor.
[0012] Optionally, the mean squared error loss function is adopted for the electric field prediction branch in the multi-task layer, and the categorical cross-entropy loss function is adopted for the domain classification branch.
[0013] According to the second aspect of the embodiments of the present application, a model training method is provided, and the method includes: obtaining training samples, where the training samples include: the source domain images and the target domain images; among them, the target domain images are unlabeled cranial magnetic images collected using a first magnetic resonance device, and the source domain images are labeled cranial magnetic images collected using a second magnetic resonance device; inputting the source domain images and the target domain images into the model to be trained; and training the model to be trained to obtain the model by processing the source domain images and the target domain images using the domain alignment layer and the multi-task layer included in the model to be trained, so as to identify the electric field distribution of unlabeled target domain images using the model.
[0014] According to a third aspect of the embodiments of the present application, there is provided a transcranial magnetic stimulation electric field prediction system, characterized in that the system includes: a first magnetic resonance device for scanning a target object to obtain a target domain image of the target object; an electric field recognition device including a source domain image; the source domain image is a paired-annotated original MRI image and electric field distribution collected by a second magnetic resonance, and the first magnetic resonance device and the second magnetic resonance device are different devices; for performing alignment processing on the target domain image and the source domain image during the encoding and decoding process of the target domain image; according to the alignment processing result, predicting an electric field distribution image of the target object so as to perform cranial magnetic stimulation on the target object using the electric field distribution image.
[0015] According to a third aspect of the embodiments of the present application, there is provided an electronic device including a memory and a processor, the memory is used to store a computer program executable by the processor; the processor is used to execute the computer program in the memory to implement the method described in the first aspect.
[0016] According to a fourth aspect of the embodiments of the present application, there is provided a computer-readable storage medium having a computer program stored thereon, characterized in that when the executable computer program in the storage medium is executed by a processor, it can implement the method described in the first aspect.
[0017] Compared with the prior art, the beneficial effects of the present application are as follows: obtaining a target domain image of a target object; wherein, the target domain image is a cranial magnetic image of the target object collected by using MRI technology; during the process of encoding the image by using a model, performing alignment processing on the target domain image and the source domain image; wherein, the source domain image includes a paired-annotated original MRI image and electric field distribution; according to the alignment processing result, determining an electric field distribution image of the target object. By using the domain alignment method to weaken the ability of the domain classifier to distinguish the feature distributions of the source domain image and the target domain image, the feature encoder is prompted to learn the shared features of the source domain image and the target domain image. Utilizing the unannotated MRI data of the target domain image, the feature space mapping is completed through the domain alignment strategy, without the need for additional electric field distribution annotation of the target domain image, significantly reducing the annotation cost. In addition, since the domain alignment method can achieve feature alignment between the source domain image and the target domain image from different MRI devices, this solution can be applied to various types of MRI devices and has a wider applicable effect. Description of the Drawings
[0018] Figure 1 It is a schematic flowchart of a method for identifying an electric field distribution provided by the present disclosure.
[0019] Figure 2 It is a schematic flowchart of a model training method illustrated by the present application.
[0020] Figure 3 This is a schematic diagram of a transcranial magnetic stimulation electric field prediction system proposed in this application.
[0021] Figure 4 This is a schematic diagram of the process of model establishment and electric field identification illustrated in this application.
[0022] Figure 5 This is a schematic diagram of the model training process illustrated in this application.
[0023] Figure 6 This is a block diagram of an electronic device shown according to an exemplary embodiment. Detailed implementation manners
[0024] Unless otherwise defined, the technical terms or scientific terms used in this specification and claims should have the ordinary meanings understood by those of ordinary skill in the technical field to which this invention belongs. The following will describe the detailed implementation manners of the present invention with reference to the accompanying drawings. It should be noted that in the process of the detailed description of these implementation manners, for the sake of concise description, this specification cannot describe all the features of the actual implementation manners in detail. Without departing from the spirit and scope of the present invention, those skilled in the art can modify and replace the implementation manners of the present invention, and the obtained implementation manners are also within the protection scope of the present invention.
[0025] Term explanation:
[0026] Transcranial magnetic stimulation (TMS) is a non-invasive neuromodulation technique that uses pulsed magnetic fields to act on the central nervous system of the brain to regulate the activities of brain neurons. TMS is based on the principles of electromagnetic induction and electromagnetic conversion. It uses a powerful transient current in the stimulation coil to generate a magnetic field, which can penetrate the skull and then be converted into an induced current in the intracranial conductor in the opposite direction to the current direction of the stimulation coil. This endogenous induced current stimulates neurons, generating a series of physiological and biochemical reactions, thereby achieving the purpose of treatment or regulation.
[0027] Magnetic Resonance Imaging (MRI) is an advanced medical imaging technique. MRI uses a strong magnetic field and high-frequency pulses (radio waves) to cause the atomic nuclei (mainly hydrogen atomic nuclei) inside the human body to resonate and generate signals. These signals are received by a receiver and then reconstructed and processed by a computer to finally generate high-definition images.
[0028] In the prior art, electric field prediction methods are usually trained based on labeled data of a single type of MRI device and fixed scanning parameters. This approach is difficult to meet the requirements of multiple devices and multiple scanning parameters in actual clinical practice. MRI devices from different manufacturers (such as GE, Siemens, Philips) have significant differences in hardware configurations (such as coil design, gradient strength) and software algorithms. Even when the scanning parameters are the same, the quality and texture features of the generated MRI images may be significantly different. The current electric field prediction models are mainly based on specific types of MRI devices and fixed scanning parameters, and it is difficult to adapt to MRI images generated by different devices or diverse scanning parameters (such as repetition time TR, echo time TE, flip angle, etc.). This results in insufficient generalization ability of the model, limiting its clinical promotion under multi-center and multi-device conditions.
[0029] In addition, electric field distribution labels usually need to be generated through finite element analysis calculations. This process is computationally complex and time-consuming. Especially when there is a large amount of unlabeled data in the target domain, traditional methods are difficult to achieve efficient adaptation of the unlabeled domain, further increasing the cost and time overhead of data annotation. Moreover, the acquisition of labeled data (i.e., electric field distribution) relies on complex finite element analysis (FEM), which requires a large amount of time and computing resources, further restricting the expansion of data scale and the generalization ability of the model. Therefore, a solution that can accurately identify the electric field of images of various MRI devices at low cost is needed.
[0030] The execution entity of the electric field distribution recognition method in the specific embodiments of the present disclosure can be an electronic device such as a server (including a local server or a cloud server).
[0031] Such as Figure 1 is a schematic flowchart of an electric field distribution recognition method provided by the present disclosure. As can be seen from Figure 1 it, the method includes the following steps:
[0032] Step 101: Obtain a target domain image of a target object; wherein, the target domain image is a transcranial magnetic image of the target object acquired using MRI technology.
[0033] The target object mentioned here can be a person, an animal, etc. that requires magnetoencephalography images. Collecting magnetoencephalography images of the target object using MRI technology can be understood as collecting nuclear magnetic resonance images of the head of a person or an animal. Before image collection, it is necessary to set the MRI device parameters first, such as the repetition time TR, the echo time TE, the flip angle, etc. However, it should be noted that there are differences in the device parameters of MRI devices from different manufacturers, and there may also be differences in the effects of the collected magnetoencephalography images. In order to improve the general recognition effect of magnetoencephalography images, the domain alignment mechanism of the model is used in the present disclosure solution to avoid the problem of inaccurate identification of the image electric field distribution due to differences in MRI devices or parameters.
[0034] Step 102: During the process of encoding the image using the model, perform alignment processing on the target domain image and the source domain image; wherein, the source domain image includes paired labeled original MRI images and electric field distributions.
[0035] The model mentioned here can be a trained machine learning model, such as MDD-UNet. Among them, MDD-UNet combines the U-Net architecture and MDD theory, aiming to achieve domain-adaptive medical image segmentation. U-Net is a U-shaped encoder-decoder network with skip connections, and there is a skip connection structure between the network layers corresponding to the contraction path and the expansion path. MDD is a new method for measuring distribution differences, and strict learning boundaries can be derived based on the scoring function and the margin loss.
[0036] The source domain image mentioned here can be understood as the magnetoencephalography image collected by the MRI device that has completed the annotation of the electric field distribution. When using the MRI device to collect the source domain image, it is generally the MRI image data under fixed scanning parameters (for example, TR = 2530ms, TE = 2ms, flip angle = 7°), and the corresponding electric field distribution label is generated through SimNIBS. In order to obtain a more accurate source domain image, a whole-brain electric field distribution covering 50 individual persons can be generated. The source domain image obtained after annotation includes paired data of "source domain image, coil stimulation parameter - electric field distribution". In order to enable the source domain image to provide more comprehensive and accurate data, it is necessary to perform a labeled data set under a large number of specific scanning parameters.
[0037] The alignment mentioned here refers to aligning the feature distributions of the target domain image and the source domain image. During the alignment process, it is not restricted by the device and can have a good general effect. Even if the MRI device of the target domain image is different from that of the source domain image, the accurate alignment of the feature distributions of the target domain image and the source domain image can be achieved.
[0038] Step 103: Determine the electric field distribution image of the target object according to the alignment processing result.
[0039] After alignment processing, the correspondence between the image features in the target domain image and the coil stimulation parameters - electric field distribution can be known. According to the alignment processing result, the electric field distribution of the target object can be known, and further, an electric field distribution image of the target object can be generated.
[0040] Based on the above processing method, a target domain image of the target object is obtained; wherein, the target domain image is a transcranial magnetic image of the target object collected by using MRI technology; during the process of encoding the image by using the model, alignment processing is performed on the target domain image and the source domain image; wherein, the source domain image includes paired labeled original MRI images and electric field distributions; according to the alignment processing result, an electric field distribution image of the target object is determined. By using the method of domain alignment to weaken the ability of the domain classifier to distinguish the feature distributions of the source domain image and the target domain image, the feature encoder is prompted to learn the shared features of the source domain image and the target domain image. By using the unlabeled MRI data of the target domain image, the feature space mapping is completed through the domain alignment strategy, without additional electric field distribution labeling for the target domain image, significantly reducing the labeling cost. In addition, since the method of domain alignment can achieve feature alignment of the source domain image and the target domain image from different MRI devices, this solution can be applied to various types of MRI devices and has a wider application effect.
[0041] In one or more embodiments of the present application, obtaining the target domain image of the target object includes: scanning the head of the target object by using a first magnetic resonance device to obtain the target domain image of the target object; wherein, the source domain image is an image scanned by using a second magnetic resonance device, and the first magnetic resonance device and the second magnetic resonance device are different devices.
[0042] It should be noted that the first magnetic resonance device and the second magnetic resonance device can be two devices of different brands, or devices of the same brand but different batches, or devices of the same brand but used by different users. The images collected by using two different magnetic resonance devices are also different. When collecting magnetic resonance images, the obtained image parameters are also different. Although different magnetic resonance devices are used for image collection, it is necessary to ensure that the target domain image and the source domain image collected by the two devices have the same or similar imaging effects or image parameters.
[0043] After obtaining the target domain image, it is necessary to preprocess the target domain image obtained by the first magnetic resonance device, including denoising, enhancing contrast, correcting image distortion, etc., to improve the image quality. If there are significant differences in resolution, gray level, etc. between the source domain image and the target domain image, methods such as interpolation and resampling can be used for matching. In actual operation, the influence of individual differences of the target object (such as age, gender, disease type, etc.) on the scanning results needs to be fully considered. For different types of magnetic resonance devices, different calibration methods and domain adaptation algorithms may be required. When introducing new algorithms or technologies, sufficient verification and testing are needed to ensure their safety and reliability.
[0044] Through the implementation of this solution, the image information obtained by scanning different magnetic resonance devices can be effectively utilized, and the accuracy and reliability of medical image segmentation and diagnosis can be improved. At the same time, it also provides the possibility for medical image sharing and analysis across devices and institutions.
[0045] In one or more embodiments of the present application, during the process of encoding the image using the model, aligning the target domain image and the source domain image includes: encoding the target domain image and the source domain image using the encoder in the model to extract multi-scale features; using the decoder in combination with skip connections to decode the extracted multi-scale features; identifying the device parameters of the first magnetic resonance device using the decoding result; and performing alignment processing on the target domain image and the source domain image according to the identification result of the device parameters.
[0046] The solution of the present disclosure extracts features from the target domain image and the source domain image through a deep learning model (including an encoder and a decoder), and uses these features to identify the parameter differences of the magnetic resonance device, so as to achieve alignment processing, in order to reduce or eliminate the influence of device differences in subsequent analyses (such as image segmentation, registration, diagnosis, etc.).
[0047] Specifically, preprocessing the target domain image includes denoising, enhancing contrast, normalizing, etc., to improve the image quality and reduce the difficulty of subsequent processing. Use the encoder in the deep learning model to extract multi-scale features from the preprocessed image. The encoder usually includes multiple convolutional layers and pooling layers, which can capture local and global features in the image. The extracted features should include information such as texture, shape, and edge at different scales for subsequent decoding and identification.
[0048] Use the decoder in combination with skip connections to decode the extracted multi-scale features. Skip connections can combine high-level features and low-level features in the encoder to improve the accuracy and robustness of decoding. The decoding result should be able to reconstruct a feature representation similar to the original image, while eliminating the influence of device differences on image feature alignment.
[0049] Using the feature information in the decoding result, the device parameters of the first magnetic resonance device are identified through a classifier (such as a support vector machine, neural network, etc.). The identified parameters may include magnetic field strength, scan sequence type, slice thickness, slice spacing, etc. The identification result can eliminate the feature changes caused by device differences between the target domain image and the source domain image, thus achieving common feature alignment.
[0050] According to the identification result of the device parameters, the target domain image and the source domain image are aligned using an image registration algorithm (such as affine transformation, non-rigid transformation, etc.). The purpose of alignment is to make the two images consistent in spatial position for subsequent comparison and analysis.
[0051] The alignment process should take into account the possible non-linear differences between different devices and use an appropriate transformation model for compensation. Post-processing is performed on the aligned images, including smoothing, artifact removal, etc., to improve the image quality. The effect of the alignment process is verified through quantitative evaluation (such as mean square error, peak signal-to-noise ratio, etc.) and qualitative evaluation (such as doctor evaluation).
[0052] During the feature extraction and decoding process, it is necessary to ensure the generalization ability of the model and avoid overfitting or underfitting. When identifying device parameters, it is necessary to fully consider the possible similarities and differences between different devices and select appropriate classifiers and feature representation methods. This model also weakens the ability of the domain classifier to distinguish the feature distributions of the source domain and the target domain, prompting the feature encoder to learn the shared features of the source domain and the target domain.
[0053] Through the above solution, the deep learning model can be effectively used to extract and decode features of the target domain image and the source domain image, and accordingly identify the parameter differences of the magnetic resonance device to achieve image alignment processing. This helps to reduce or eliminate the impact of device differences on medical image analysis and improve the accuracy and reliability of diagnosis.
[0054] In one or more embodiments of the present disclosure, the model training method includes: obtaining training samples, the training samples including: the source domain image and the target domain image; wherein, the target domain image is an unlabeled image, and the source domain image is an image containing labels. Inputting the source domain image and the target domain image into the model to be trained. After processing the source domain image and the target domain image using the domain alignment layer and the multi-task layer included in the model to be trained, the model to be trained is trained to obtain the model.
[0055] Collect source domain images and target domain images as training samples. The source domain images should contain rich annotation information to achieve accurate segmentation annotation of the electric field distribution; the target domain images are unannotated images, which may come from different magnetic resonance devices or scanning parameters. Preprocess the source domain images and target domain images, including denoising, enhancing contrast, normalizing, etc., to improve the image quality and reduce the difficulty of subsequent processing.
[0056] Design the model to be trained, which should include an encoder, a domain alignment layer, a multi-task layer, and a decoder (if needed). The encoder is used to extract image features, the domain alignment layer is used to reduce the differences between the source domain and the target domain, the multi-task layer is used to achieve the learning of multiple tasks (such as segmentation, classification, etc.), and the decoder (if needed) is used to reconstruct the image or generate the output.
[0057] The domain alignment layer can adopt adversarial training strategies (such as the domain adversarial network DANN), maximum mean discrepancy MMD, etc. to reduce the domain differences. The multi-task layer can design corresponding loss functions and output layers according to specific tasks.
[0058] Input the preprocessed source domain images and target domain images into the model to be trained. Use the domain alignment layer to reduce the differences between the source domain and the target domain, and at the same time use the multi-task layer to learn the annotation information in the source domain images. Calculate the loss function according to the output of the multi-task layer and the annotation information, and update the model parameters through the backpropagation algorithm. Iteratively train the model until the preset convergence condition or the maximum number of iterations is reached.
[0059] Evaluate the performance of the model on the validation set, including indicators such as segmentation accuracy and classification accuracy. Optimize the model according to the evaluation results, such as adjusting the model structure, adding regularization terms, modifying the loss function, etc. Repeat the training and evaluation process until the model performance reaches the optimal.
[0060] When collecting training samples, it should be ensured that the source domain images and target domain images have sufficient diversity and representativeness to cover different devices and different scanning parameter situations. When designing the model to be trained, the interaction and influence between the domain alignment layer and the multi-task layer should be fully considered to ensure that the model can simultaneously handle the needs of cross-domain feature extraction and task learning. During the model training process, appropriate optimization algorithms and hyperparameters such as learning rates should be selected to accelerate the model convergence and improve the training efficiency. During the model evaluation and optimization process, the generalization ability and robustness of the model should be concerned to avoid overfitting or underfitting phenomena.
[0061] Through the above solution, a model capable of handling image differences of different magnetic resonance devices can be trained. This model can learn effective feature representations using the annotation information in the source domain images, and reduce the differences between the source domain and the target domain through the domain alignment layer, so as to achieve tasks such as accurate segmentation and classification of the target domain images. This helps to improve the accuracy and reliability of medical image analysis.
[0062] In one or more embodiments of the present disclosure, the domain alignment layer is used to adjust the gradient during backpropagation through the gradient reversal layer to promote feature alignment between the source domain image and the target domain image.
[0063] Gradient Reversal Layer (GRL) mechanism: In the domain alignment layer, the gradient reversal layer is the core component. It keeps the data unchanged during the forward propagation process, but during the backpropagation, it reverses the sign of the gradient (i.e., multiplies by -1) and scales it proportionally. This mechanism aims to weaken the ability of the domain classifier to distinguish between the source domain and the target domain, while ensuring that the feature encoder can learn their common features, thus promoting the alignment of the feature distributions of the target domain and the source domain.
[0064] Although the gradient reversal layer always reverses the gradient sign during backpropagation, the scaling ratio of the gradient can be dynamically adjusted according to the training process. This helps the network to focus more on downstream tasks (such as electric field distribution prediction) in the initial stage of training, and pay more attention to domain alignment in the later stage of training.
[0065] In one or more embodiments of the present application, the multi-task layer includes an electric field prediction branch and a domain classification branch; wherein, the electric field prediction branch is used to predict the electric field distribution of the target domain image; the domain classification branch is used to identify the device parameters of the first magnetic resonance device of the target domain image.
[0066] Electric field prediction branch (fc): This branch generates an individualized TMS electric field distribution through a regression task based on the output of the feature decoder. To optimize the prediction accuracy, the mean squared error (MSE) is used as the loss function. The goal of the electric field prediction branch is to provide an accurate electric field distribution prediction for the target domain image, which is crucial for applications such as TMS (transcranial magnetic stimulation).
[0067] Domain classification branch (fa): This branch is also based on the output of the feature decoder, but performs a classification task to identify the device parameters of the first magnetic resonance device for the target domain image. The loss function uses categorical cross-entropy to optimize the accuracy of domain classification. The introduction of the domain classification branch is to guide the feature encoder to learn features that can distinguish different devices or scanning parameters during the training process, but is restricted by the gradient reversal layer to ensure that these features are aligned between the source domain and the target domain.
[0068] In one or more embodiments of the present application, the domain alignment layer dynamically adjusts the value of the gradient scaling factor during the training process, sets a first scaling factor at the beginning of the training, and sets a second scaling factor at the end of the training; wherein the first scaling factor is less than the second scaling factor.
[0069] During the training process, the gradient scaling factor in the domain alignment layer (i.e., the scaling ratio in GRL) should be dynamically adjusted. A smaller scaling factor (such as 0.0001) is set at the beginning to allow the network to mainly focus on downstream tasks (such as electric field prediction). As the training progresses, the scaling factor is gradually increased to make the network pay more attention to domain alignment. This strategy can be achieved through linear growth, exponential growth, or other appropriate strategies.
[0070] In actual training, the effect of domain alignment can be evaluated by monitoring the accuracy of the domain classification branch. If the domain classification accuracy is too high, it indicates insufficient domain alignment; if it is too low, it may indicate that the network pays too much attention to domain alignment and ignores downstream tasks. According to these feedbacks, the value of the gradient scaling factor can be adjusted in a timely manner.
[0071] In one or more embodiments of the present application, the electric field prediction branch in the multi-task layer uses a mean squared error loss function, and the domain classification branch uses a categorical cross-entropy loss function.
[0072] Loss function of the electric field prediction branch: The electric field prediction branch uses minimizing the mean squared error (MSE) as the loss function. This is because the electric field distribution prediction is a regression task, and MSE can measure the deviation between the predicted value and the true value. To optimize the prediction accuracy, the network parameters can be continuously adjusted during the training process to reduce the MSE value.
[0073] Loss function of the domain classification branch: The domain classification branch uses categorical cross-entropy as the loss function. This is because domain classification is a classification task, and cross-entropy can measure the accuracy of the classification result. By optimizing the cross-entropy value, the network can be guided to learn features that can accurately distinguish different devices or scanning parameters.
[0074] During the training process, it is necessary to consider the loss functions of both the electric field prediction branch and the domain classification branch simultaneously. Combine the two through weighted summation or other appropriate strategies to form a unified loss function to guide the training of the network. This can ensure both the accuracy of electric field prediction and achieve the goal of domain alignment.
[0075] Based on the same idea, this application also provides a model training method. As Figure 2 This is a schematic flowchart of a model training method illustrated by this application. The method includes:
[0076] Step 201: Obtain training samples, where the training samples include: the source domain image and the target domain image; among them, the target domain image is an unlabeled magnetoencephalogram image collected by a first magnetic resonance device, and the source domain image is a labeled magnetoencephalogram image collected by a second magnetic resonance device.
[0077] Step 202: Input the source domain image and the target domain image into the model to be trained.
[0078] Step 203: After processing the source domain image and the target domain image using the domain alignment layer and the multi-task layer included in the model to be trained, train the model to be trained to obtain the model, so as to use the model to identify the electric field distribution of the unlabeled target domain image.
[0079] Specifically, collect magnetoencephalogram images from two different magnetic resonance devices (for example, a first magnetic resonance device and a second magnetic resonance device). Ensure that the images collected by the second magnetic resonance device contain detailed annotation information, which usually includes electric field distribution, key anatomical structures, etc. The images collected by the first magnetic resonance device are unlabeled, and these images will be processed as target domain data.
[0080] Perform necessary preprocessing on the source domain image and the target domain image, including denoising, image enhancement, normalization, etc., to ensure the quality and consistency of the data. Verify the annotation information of the source domain image to ensure the accuracy and integrity of the annotation.
[0081] Divide the source domain image into a training set and a test set for model training and evaluation. Since the target domain image has no annotation, all of it will be used for domain alignment and model evaluation during the training process.
[0082] Select or design a deep learning model that includes a domain alignment layer and a multi-task layer. The domain alignment layer is responsible for reducing the feature differences between the source domain and the target domain and achieving cross-domain transfer learning. The multi-task layer is responsible for simultaneously performing electric field prediction and domain classification tasks to improve the generalization ability of the model.
[0083] Input the preprocessed source domain image and target domain image into the corresponding input layers of the model respectively. Ensure that the format, resolution, etc. of the input data meet the requirements of the model.
[0084] To improve the training efficiency, batch processing can be adopted, where multiple images are grouped into a batch for input. In each batch, maintain an appropriate ratio of source domain images and target domain images to balance the training of the two tasks.
[0085] Extract and align the features of the input source domain image and target domain image through the domain alignment layer. Input the aligned features into the multi-task layer to perform electric field prediction and domain classification tasks simultaneously. Loss calculation: Calculate the electric field prediction loss based on the electric field prediction result and the ground truth annotation (only for the source domain image). Calculate the domain classification loss based on the domain classification result and the ground truth domain label (source domain or target domain).
[0086] Combine the two losses and calculate the overall loss value using the joint loss function. Backpropagation and optimization: Update the weights and biases of the model through the backpropagation algorithm according to the calculated loss value. Adopt the dynamic learning rate decay strategy and the gradient scaling factor adjustment strategy of GRL to optimize the training process of the model. Regularly check the training situation of the model, such as the change of the loss value, the performance on the validation set, etc., to adjust the training parameters and strategies.
[0087] During the training process, regularly save the best weights and biases of the model. Evaluate the electric field prediction error of the model on the source domain test set to ensure the accuracy of the model for the source domain data. On the target domain unlabeled data, use the features extracted after domain alignment to test the prediction performance of the model, and analyze the alignment effect of the feature distribution through TSNE visualization.
[0088] As Figure 3 is a schematic diagram of a transcranial magnetic stimulation electric field prediction system proposed in this application. As can be seen from Figure 3 the system includes:
[0089] The first magnetic resonance device 31 is used to scan the target object to obtain the target domain image of the target object.
[0090] The electric field recognition device 32 includes a source domain image; the source domain image is the original MRI image and the electric field distribution that are collected by the second magnetic resonance and paired and labeled. The first magnetic resonance device and the second magnetic resonance device are different devices; it is used to perform alignment processing on the target domain image and the source domain image during the encoding and decoding process of the target domain image; according to the alignment processing result, predict the electric field distribution image of the target object so as to perform transcranial magnetic stimulation on the target object using the electric field distribution image.
[0091] This system can achieve asFigures 1 to 3 For the above-described solution, the specific implementation process can be referred to the above embodiments and will not be repeated here.
[0092] For ease of understanding, the following will illustrate the implementation methods of electric field distribution recognition and model training through specific embodiments.
[0093] As Figure 4 This is a schematic diagram of the model establishment and electric field recognition process illustrated in this application. As Figure 4 shown, the technical route of the present disclosure includes the following four main steps: production of the data set and training samples, model establishment, model training optimization, and use of the model to perform electric field distribution recognition on the target domain image.
[0094] Data set production:
[0095] Source domain data: Collect MRI image data under fixed scanning parameters (TR = 2000ms, TE = 30ms, flip angle = 60°) from GE MRI equipment, and generate corresponding electric field distribution labels through SimNIBS, covering the whole-brain electric field distribution of 50 individuals.
[0096] Target domain data: Collect unlabeled MRI images from different MRI equipment (such as GE, Siemens, Philips), covering the following common scanning parameters:
[0097] Repetition time (TR): 500ms - 3000ms (for example: 500ms, 1000ms, 1500ms, 2000ms, 2500ms, 3000ms).
[0098] Echo time (TE): 10ms - 100ms (for example: 10ms, 30ms, 50ms, 70ms, 100ms).
[0099] Flip angle: 15° - 120° (for example: 15°, 30°, 60°, 90°, 120°).
[0100] By covering the above core parameters, the model can adapt to MRI data under different equipment and scanning parameters, meeting the actual needs in multi-center clinical applications.
[0101] Data preprocessing: Standardize all MRI images (normalize pixel values to [0, 1]), and adjust the resolution to 144×72×24 through interpolation.
[0102] As Figure 5 This is a schematic diagram of the model training process illustrated in this application. From Figure 5As can be seen, the feature encoder (fe) extracts multi-scale features and encodes the spatial information of the input T1 image. It consists of convolutional layers and max pooling layers. The number of output channels of each layer gradually increases (such as 32, 64, 128, 512).
[0103] The feature decoder (fd) combines skip connections to fuse the high-resolution features of the encoder with the upsampled features of the decoder, and uses up-convolution layers to gradually restore the image spatial resolution.
[0104] Domain alignment module GRL: It does not change the data during forward propagation, but during backpropagation, it reverses the sign of the gradient (multiplies by a negative number) and scales it proportionally. The purpose is to weaken the ability of the domain classifier to distinguish between the source domain and the target domain, but at the same time ensure the ability to handle downstream tasks such as electric field distribution prediction. This mechanism prompts the feature encoder to learn their common features during training, aligning the feature distributions of the target domain and the source domain.
[0105] Usually, the value of the proportional scaling is dynamically adjusted during training to gradually increase it from small (for example, using a linear or exponential growth strategy), so that the network focuses more on downstream tasks in the initial stage of training and pays more attention to domain alignment in the later stage of training. Here, it is set to 0.0001 in the initial stage.
[0106] In this framework, the source domain images refer to a large number of labeled datasets under specific scanning parameters, including paired data of "MRI images, coil stimulation parameters - electric field distribution"; the target domain is MRI data from different devices and scanning parameters, and these data are unlabeled. The GRL is used to align the feature distributions of the source domain and the target domain, so as to achieve efficient adaptation and accurate prediction on the target domain.
[0107] In the model of this embodiment, there is a multi-task learning module: Electric field prediction branch (fc): Based on the output of the feature decoder, it generates an individualized TMS electric field distribution through a regression task. The prediction accuracy L reg is optimized using a loss function that minimizes the mean squared error (MSE). Domain classification branch (fa): Based on the output of the feature decoder, it identifies the device or scanning parameters through a classification task. The loss function is the common classification cross-entropy L domain.
[0108] Model training and optimization, combined loss function: Combining the electric field prediction loss and the domain classification loss to optimize the overall network:
[0109] L = L reg + λL domain
[0110] Where λ is a weight parameter used to balance the loss contributions of the two tasks.
[0111] During the model training process, the initial learning rate is set to 0.05, and a dynamic learning rate decay strategy is adopted. The initial value of the gradient scaling factor α of GRL is 0.0001 and is adjusted exponentially. The model optimizes the electric field prediction and domain classification tasks on the source domain image training set, and realizes the collaborative improvement of feature alignment and prediction performance through the joint loss function.
[0112] During the model training, it is also necessary to evaluate the model training effect. Evaluate the electric field prediction error on the source domain image test set to ensure the accuracy of the model's recognition of the source domain images. On the unlabeled data of the target domain images, use the features extracted after domain alignment to test the prediction performance of the model. Visualize and analyze the feature distribution alignment effect under multi-device and multi-parameter conditions through TSNE.
[0113] The real-time electric field prediction method for transcranial magnetic stimulation adapting to multiple scan parameters proposed in this embodiment is based on deep learning technology. By introducing a domain alignment mechanism and a multi-task learning strategy, it realizes the efficient adaptation to different MRI devices and scan parameters and individualized electric field prediction. It has the following advantages: Combining the electric field prediction and domain classification tasks, using the joint loss function to balance the optimization objectives of the two, significantly improving the cross-device generalization ability of the model. Weakening the ability of the domain classifier to distinguish the feature distributions of the source domain images and the target domain images through the GRL module (domain alignment module) in the model, and prompting the feature encoder to learn the shared features of the source domain and the target domain. Using the unlabeled MRI data of the target domain images, completing the feature space mapping through the domain alignment strategy, without additional electric field distribution annotations, significantly reducing the annotation cost.
[0114] Based on any of the above embodiments, the present disclosure also provides an electronic device, which can execute the instruction transmission and data processing methods of any of the above embodiments described in the present disclosure.
[0115] Figure 6 It is a structural schematic block diagram of an electronic device according to an embodiment of the present disclosure.
[0116] The hardware structure of the electronic device 1000 can be implemented using a bus architecture. The bus architecture can include any number of interconnecting buses and bridges, depending on the specific application and overall design constraints of the hardware. The bus 1100 connects various circuits including one or more processors 1200, a memory 1300, and / or hardware modules together. The bus 1100 can also connect various other circuits 1400 such as peripheral devices, voltage regulators, power management circuits, external antennas, etc.
[0117] The bus 1100 can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Component (EISA) bus, or the like. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of representation, only one connecting line is used in this figure, but it does not mean that there is only one bus or one type of bus.
[0118] The present disclosure also provides a readable storage medium storing a computer program, which when executed by a processor is used to implement the above method. The "readable storage medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by or in connection with an instruction execution system, apparatus, or device. More specific examples of the readable storage medium include the following: an electrical connection part (electronic device) having one or more wirings, a portable computer disk cartridge (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable read-only memory (CDROM), etc.
[0119] The present disclosure also provides a computer program product. The method of the present disclosure can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instructions are loaded and executed, the processes or functions of the present disclosure are executed in whole or in part.
[0120] The computer program or instructions can be stored in a readable storage medium, or transmitted from one readable storage medium to another. For example, the computer program or instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired or wireless manner. The readable storage medium can be any available medium that can be accessed or a data storage device such as a server or data center integrating one or more available mediums. The available medium can be a magnetic medium, such as a floppy disk, a hard disk, or a magnetic tape; it can also be an optical medium, such as a digital video disc; or it can be a semiconductor medium, such as a solid state drive. The computer-readable storage medium can be a volatile or non-volatile storage medium, or can include both volatile and non-volatile types of storage media.
[0121] Those skilled in the art should understand that the embodiments of the present disclosure may be provided as a method, a system, or a computer program product. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.
[0122] The present disclosure is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the present disclosure. It should be understood that each flow and / or block in the flowchart and / or block diagram, as well as the combination of flows and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate means for realizing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks.
[0123] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer-readable memory generate a manufactured article including instruction means that realizes the functions specified in Figure 1 one or more of the flows Figure 1 or blocks.
[0124] These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable device provide steps for realizing the functions specified in Figure 1 one or more of the flows Figure 1 or blocks.
[0125] In the description of this specification, the description referring to terms such as "one embodiment / way", "some embodiments / ways", "example", "specific example", or "some examples", etc. means that the specific features, structures, or characteristics described in connection with that embodiment / way or example are included in at least one embodiment / way or example of the present disclosure. In this specification, the schematic expressions of the above terms do not necessarily refer to the same embodiment / way or example. Moreover, the specific features, structures, or characteristics described can be combined in a suitable manner in any one or more embodiments / ways or examples. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments / ways or examples described in this specification and the features of different embodiments / ways or examples.
[0126] In addition, the terms "first" and "second" are used only for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present disclosure, "a plurality" means at least two, such as two, three, etc., unless otherwise specifically defined.
[0127] Those skilled in the art should understand that the above embodiments are merely for clearly explaining the present disclosure and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications can be made on the basis of the above disclosure, and these changes or modifications are still within the scope of the present disclosure.
Claims
1. A method for identifying electric field distribution, characterized in that: The method comprises: Acquire a target domain image of a target object; wherein the target domain image is a cranial magnetic image of the target object acquired by using MRI technology; In the process of encoding the image using the model, the target domain image and the source domain image are aligned; wherein the source domain image includes the original MRI image and the electric field distribution which are annotated in pairs; According to the alignment processing result, the electric field distribution image of the target object is determined.
2. The method according to claim 1, characterized in that The step of acquiring a target domain image of a target object comprises: The target object's head is scanned by a first magnetic resonance device to obtain the target domain image of the target object; wherein the source domain image is an image scanned by a second magnetic resonance device, and the first magnetic resonance device and the second magnetic resonance device are different devices.
3. The method according to claim 2, characterized in that In the process of encoding the image by using the model, aligning the target domain image with the source domain image includes: Using the encoder in the model to encode the target domain image and the source domain image to extract multi-scale features; The decoder is used in combination with skip connections to decode the extracted multi-scale features; identifying device parameters of the first magnetic resonance device using the decoding result; The target domain image and the source domain image are aligned according to the recognition result of the device parameters.
4. The method according to claim 1, characterized in that The model training method includes: Acquire a training sample, wherein the training sample includes: the source domain image and the target domain image; wherein the target domain image is an unlabeled image, and the source domain image is an labeled image; Inputting the source domain image and the target domain image into a model to be trained; After processing the source domain image and the target domain image using the domain alignment layer and the multi-task layer included in the model to be trained, the model to be trained is trained to obtain the model.
5. The method according to claim 4, characterized in that The domain alignment layer is used to adjust the gradient in back propagation through the gradient reversal layer to promote feature alignment between the source domain image and the target domain image.
6. The method according to claim 4 or 5, characterized in that: The multi-task layer includes an electric field prediction branch and a domain classification branch; wherein the electric field prediction branch is used to predict the electric field distribution of the target domain image; and the domain classification branch is used to identify device parameters of a first magnetic resonance device of the target domain image.
7. The method according to claim 4, characterized in that The domain alignment layer dynamically adjusts the value of the gradient scaling factor during the training process, setting a first scaling factor at the beginning of the training and a second scaling factor at the end of the training; The first scaling factor is smaller than the second scaling factor.
8. The method according to claim 4, characterized in that The electric field prediction branch in the multi-task layer adopts the minimization of mean square error loss function, and the domain classification branch adopts the classification cross entropy loss function.
9. A model training method, characterized in that: The method comprises: Acquire training samples, wherein the training samples include: a source domain image and a target domain image; wherein the target domain image is an unlabeled cranial magnetic image acquired by a first magnetic resonance device, and the source domain image is an labeled cranial magnetic image acquired by a second magnetic resonance device; Inputting the source domain image and the target domain image into a model to be trained; After processing the source domain image and the target domain image using the domain alignment layer and the multi-task layer included in the model to be trained, the model to be trained is trained to obtain the model, so as to use the model to identify the electric field distribution of the unlabeled target domain image.
10. A transcranial magnetic stimulation electric field prediction system, characterized in that: The system comprises: A first magnetic resonance device is used to scan a target object to obtain a target domain image of the target object; An electric field recognition device comprises a source domain image; the source domain image is an original MRI image and an electric field distribution that are collected and annotated in pairs by using a second magnetic resonance, and the first magnetic resonance device and the second magnetic resonance device are different devices; the device is used to align the target domain image with the source domain image during encoding and decoding of the target domain image; and according to the alignment result, predict the electric field distribution image of the target object so as to perform cranial magnetic stimulation on the target object by using the electric field distribution image.
11. An electronic device, comprising a processor and a memory, wherein the memory stores at least one instruction, at least one program, a code set or an instruction set, and the at least one instruction, at least one program, a code set or an instruction set is loaded and executed by the processor to implement the method according to any one of claims 1 to 8, or the method according to claim 9.
12. A computer-readable medium having stored thereon at least one instruction, at least one program, code set or instruction set, wherein the at least one instruction, at least one program, code set or instruction set is loaded and executed by a processor to implement the method according to any one of claims 1 to 8, or the method according to claim 9.