A region-aware magnetic localization method applied to tongue movement tracking

Abstract

The application discloses a regional perception magnetic positioning method applied to tongue movement tracking, and comprises the following steps: a magnetic positioning calibration subsystem is established, a magnetic tracker is controlled to move in a sensor array working area according to a predetermined track by a mechanical arm, spatial position and attitude angle data and magnetic field intensity data of the magnetic tracker are synchronously collected to construct a training set; a generation model based on a neural network is established according to the synchronously collected data, a mapping relationship from the position and attitude of the magnetic tracker to the magnetic sensing data is learned, and a sample is synthesized to expand the training set; the change range of the spatial position and the attitude angle of the magnetic tracker in the tongue movement tracking process is divided into a plurality of sub-regions; a regional perception module is constructed based on the neural network, the region where the magnetic tracker is located is estimated according to the magnetic sensing data, and the magnetic tracker is pre-trained; and the output of each local magnetic positioning model which is pre-trained is integrated by weighted average through the pre-trained regional perception module.

Description

Technical Field

[0001] This invention belongs to the field of wireless magnetic positioning technology, and particularly relates to a region-aware magnetic positioning method for tongue movement tracking. Background Technology

[0002] Magnetic positioning technology provides an economical and convenient means for wireless target tracking, and has wide applications in military, medical, geological exploration, and autonomous driving fields. This invention proposes a region-aware magnetic positioning method for tongue movement tracking, used for speech rehabilitation training of patients with speech disorders. Most existing related work uses a magnetic dipole model based on Euler's equations, achieving magnetic positioning through iterative optimization, including the Nelder-Mead algorithm, Powell method, Levenberg-Marquardt (LM) algorithm, particle swarm optimization, and expectation maximization. These methods have two drawbacks in practical applications: 1) Euler's equations simplify the magnetic tracker to a dipole, approximating the magnetic field strength in real-world scenarios, resulting in model bias; 2) the iterative optimization algorithms used for inverting Euler's equations are highly complex and sensitive to initial values. Artificial neural networks have strong representational capabilities, are suitable for modeling complex magnetic fields in real-world scenarios, and do not require iterative optimization during the inference stage, making them a current research hotspot in the field of magnetic positioning. For example, a pioneering method using a multilayer perceptron has achieved acceptable accuracy in magnetic positioning. However, this method requires a 10×10×10cm... 3The existing method collects over 1.7 million samples within a given area for network parameter training, limiting its practicality. To reduce data acquisition, a composite algorithm based on multi-layer perception (MLP) and LM (Magnetic Dipole Model) has been proposed. The MLP is trained only on a simulation dataset, providing initial estimates of the magnetic tracker's orientation and position, while the LM algorithm, responsible for the inverse operation of the magnetic dipole model, is used to refine the initial estimate. This composite algorithm effectively improves the accuracy and speed of LM magnetic positioning. However, in real-world environments, unavoidable system calibration errors degrade the quality of the simulation dataset, leading to biases in the MLP's initial estimate. Therefore, an ensemble model of multiple neural networks is trained using transfer learning, combining simulation and real datasets. This model employs MLP and convolutional neural networks (CNNs) to construct a classifier, performing soft decision-making on the magnetic tracker's location and providing initial estimates and search boundaries for subsequent iterative optimization. However, the iterative optimization algorithm used to improve magnetic positioning accuracy is still based on the magnetic dipole model, and model bias remains. In the latest research, a novel CNN model was designed to estimate the attitude of a toroidal magnet. They first generated a large amount of simulation data based on the magnetic field theory model to pre-train the CNN model, and then fine-tuned the parameters of the model sub-modules using a small number of real samples. This small-sample cross-domain learning method can effectively enhance the generalization performance of the magnetic pole localization model. The above-mentioned magnetic pole localization method based on artificial neural networks has the following two common limitations: 1) In the actual working environment, the magnetic field distribution is complex. When the magnetic tracker moves closer to or further away from the sensor (or changes direction), the measured magnetic field strength data will change significantly. Relying solely on a single neural network model to represent the mapping relationship between the magnetic field strength data and the magnetic tracker state in the entire workspace will inevitably increase the model capacity and affect the generalization ability. Moreover, noise in the training data will affect the learning of all network parameters, increasing the risk of model overfitting. 2) In the actual working environment, the magnetic positioning system is inevitably subject to various electromagnetic noise interferences, which are often non-stationary. The neural network models proposed by existing methods lack mechanisms to resist non-stationary noise, and are prone to overfitting when using a small amount of real-sample data for model training or fine-tuning. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention proposes a region-aware magnetic localization method for tongue movement tracking, which solves the bottleneck problem of existing tongue movement magnetic localization systems relying excessively on the quantity and quality of training samples, and significantly improves magnetic localization accuracy.

[0004] To achieve the above objectives, the present invention provides a region-aware magnetic localization method for tongue movement tracking, comprising:

[0005] A space-angle integrated model based on region perception was constructed to enhance the magnetic positioning model's ability to represent complex magnetic field data, avoid the global impact of measurement noise on model parameters, and improve the robustness of the magnetic positioning model.

[0006] The magnetic field data generation method based on neural networks reduces the risk of overfitting due to insufficient training samples by generating simulation data in a reasonable manner.

[0007] Optional, region-aware space-angle ensemble models include:

[0008] First, a series of sub-region localization (SRL) modules are trained using samples from different spatial regions and different angular regions. Each module is a low-complexity local regressor. Noisy samples can only affect the learning of the SRL module corresponding to their location.

[0009] Secondly, the collected magnetic field data is intelligently analyzed through the spatial-aware (SA) module and the angular-aware (AA) module, and the positioning results of each SRL module are adaptively modulated.

[0010] Finally, the results of spatial sub-region integration and angular sub-region integration are integrated using the attentional feature fusion (AFF) module.

[0011] Optionally, neural network-based methods for generating magnetic field data include:

[0012] First, for each of the 32 sensors, a generative model was built by dividing the space into 8 sub-regions, resulting in a total of 256 magnetic field data generation sub-models. The original dataset D1 was used to train each magnetic field data generation sub-model, and the model parameters were saved.

[0013] Secondly, new location data is synthesized by linear interpolation of the original dataset D1. The synthesized new location data is then input into the trained magnetic field data generation sub-model to generate corresponding magnetic field data.

[0014] Finally, the generated magnetic field data is filled into the original dataset D1 to obtain the expanded training dataset D2.

[0015] Optionally, the spatial integration includes a pre-trained SRL module and a spatial awareness module; the spatial region is divided into eight sub-regions, each with a volume of 2.5 × 3.5 × 2.0 cm. 3Each sub-region has a corresponding local magnetic positioning module; the spatial perception module is used to sense which region the tracker currently belongs to; the spatial perception module and each local positioning module are constructed from gated multilayer sensors.

[0016] Optionally, the neural network training process includes:

[0017] Step 1: Pre-training of SRL modules: There are a total of 8 sub-region modules. The local magnetic localization model corresponding to each sub-region is learned using training samples in each sub-region.

[0018] Step 2: Pre-training of the spatial awareness module: Use the SA module to learn the ensemble weights estimated by SRL;

[0019] Step 3: Pre-training of spatial integration. After pre-training the SRL module and the spatial perception module separately, the parameters of the two modules are retrained together to adjust the parameters of spatial integration.

[0020] Optionally, angle integration includes a pre-trained SRL module and an angle-aware module.

[0021] Optionally, the overall angle distribution is divided into 8 angle sub-regions, from region 1 to region 8; the output of the angle sensing module is the regression weight of the 8 angle regions; the local positioning module consists of 8 separate magnetic local positioning modules, corresponding to the 8 angle sub-regions; the local positioning module can achieve preliminary positioning for each angle sub-region.

[0022] Optionally, the attention feature fusion module includes:

[0023] g i =σ(W g [s i ;a i ]+b g )

[0024] z i =g i ⊙s i +(1―g i )⊙a i

[0025] Among them, s i and a i ∈R 5 These are the outputs of SE and AE, respectively, W g ∈R 5×10 and b g ∈R 5 These are learnable parameters that enable feature interactions across different dimensions and sets. σ(·) is the sigmoid function. Note the weights g. i and 1-g iControlling s i and a i The proportion. i These are the final estimated parameters.

[0026] Technical effects of the invention: The invention discloses a region-aware magnetic localization method for tongue movement tracking, which avoids the global influence of measurement noise on model parameters and effectively improves the robustness of the magnetic localization model; a neural network-based generative model is established, and the training set is expanded by synthesizing samples through learning the mapping relationship from the position and attitude of the magnetic tracker to the magnetic sensing data, so as to improve the generalization ability of the magnetic localization model. Attached Figure Description

[0027] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0028] Figure 1 The left image is a schematic diagram of the angular region division in an embodiment of the present invention, dividing the θ and φ angular planes into 8 sub-regions. Blue dots represent angular pairs in the training set, and red dots represent angular pairs in the test set. The right image is a schematic diagram of the spatial region division in an embodiment of the present invention, dividing a 5×7×4cm... 3 The space is divided into 8 sub-regions;

[0029] Figure 2 This is an overview diagram of the space-angle integrated model of an embodiment of the present invention;

[0030] Figure 3 This is a diagram of the attention feature fusion structure according to an embodiment of the present invention;

[0031] Figure 4 This is a diagram of a gated multilayer sensor architecture with a spatial gate control unit according to an embodiment of the present invention;

[0032] Figure 5 This is a structural diagram of the magnetic positioning experimental device according to an embodiment of the present invention;

[0033] Figure 6 This is a flowchart of the QT host computer program according to an embodiment of the present invention;

[0034] Figure 7 A flowchart illustrating the steps involved in generating a model according to an embodiment of the present invention;

[0035] Figure 8 This is a schematic flowchart of a region-aware magnetic positioning method for tongue movement tracking according to an embodiment of the present invention.

[0036] Figure 9 This invention provides an example of a human subject who had a magnetic tracker attached to the tip of their tongue and was allowed to have a conversation within the working area of ​​the sensor array.

[0037] Figure 10 This is a diagram showing the predicted tongue trajectory when a test subject pronounces the word "sights" in an embodiment of the present invention. The blue line represents the reconstructed trajectory, the red vector represents the estimated direction of the magnetic tracker, and the white circle and green triangle represent the start and end points of the tongue trajectory. Detailed Implementation

[0038] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0039] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0040] like Figure 8 As shown, this embodiment provides a region-aware magnetic localization method for tongue movement tracking, including:

[0041] (1) Establish a magnetic positioning calibration subsystem to control the robotic arm carrying the magnetic tracker to move along a predetermined trajectory within the working area of ​​the sensor array, and simultaneously collect the spatial position, attitude angle data and magnetic field strength data of the magnetic tracker to construct a training set.

[0042] (2) Based on the synchronously collected data, establish a neural network-based generative model, learn the mapping relationship from the position and attitude of the magnetic tracker to the magnetic sensing data, synthesize samples to expand the training set, and reduce the risk of overfitting caused by insufficient actual training samples to the learning of the magnetic positioning model.

[0043] (3) Divide the range of changes in the spatial position and attitude angle of the magnetic tracker during the tongue movement tracking process into several sub-regions, use the training samples in each sub-region to learn the corresponding local magnetic positioning model and perform model pre-training, which not only enhances the magnetic positioning model's ability to represent complex magnetic field data, but also avoids the global impact of measurement noise on model parameters.

[0044] (4) Construct a region perception module based on a neural network, estimate the region where the magnetic tracker is located based on magnetic sensing data, and pre-train it; use the pre-trained region perception module to integrate the outputs of each pre-trained local magnetic positioning model by weighted averaging.

[0045] (5) Construct an attention feature fusion module to adaptively fuse the results of the two sets of partitioned localization integration of spatial position and attitude angle, further improving the robustness of the tongue motion magnetic localization model;

[0046] (6) Download the various local magnetic positioning models, region perception modules and attention feature fusion modules that have been trained offline on the computing server to the host computer system. Attach the magnetic tracker to the subject's tongue and let the subject have a conversation within the working area of ​​the sensor array. Upload the collected magnetic field strength data to the host computer system via the WIFI communication module for digital signal processing. Estimate the spatial position and attitude angle of the magnetic tracker in real time, store the results in the host computer system storage unit, and display its trajectory in the graphical user interface, thereby realizing tongue movement tracking.

[0047] The data collected in the data acquisition and calibration system in step (1) is uploaded to the storage unit of the host computer system via the WIFI communication module; the operations in steps (2), (3), (4), and (5) are all completed on the computing server, and the trained model data is downloaded to the storage unit of the host computer system via the WIFI communication module. The training sample data in the computing server is loaded by the host computer system via the WIFI communication module.

[0048] A space-angle integrated model based on region perception was constructed to enhance the magnetic positioning model's ability to represent complex magnetic field data, avoid the global impact of measurement noise on model parameters, and improve the robustness of the magnetic positioning model.

[0049] The magnetic field data generation method based on neural networks reduces the risk of overfitting due to insufficient training samples by generating simulation data in a reasonable manner.

[0050] This invention constructs a spatial-angular ensemble (SAE) model based on region awareness. First, a series of sub-region localization (SRL) modules are trained using samples from different spatial and angular regions, each module being a low-capacity local regressor. Noisy samples only affect the learning of the SRL module corresponding to their location. Then, the collected magnetic field data is intelligently analyzed by the spatial-aware (SA) and angular-aware (AA) modules, adaptively modulating the localization results of each SRL module. Finally, an attentional feature fusion (AFF) module is used to integrate the results of spatial and angular sub-region ensembles, further improving the robustness of the magnetic localization model. All SRL, SA, and AA modules are composed of gated multilayer perceptrons (gMLPs). The gMLP adaptively suppresses noise through its gating units and reduces computational complexity through a feature grouping strategy.

[0051] This invention proposes a method for generating magnetic field data based on neural networks, which reduces the risk of overfitting due to insufficient training samples by generating simulation data in a reasonable manner. Figure 7 As shown, firstly, for each of the 32 sensors, eight spatial sub-regions were divided to create a generative sub-model, resulting in a total of 256 magnetic field data generation sub-models. The original dataset D1 was used to train each magnetic field data generation sub-model, and the model parameters were saved. Secondly, new location data was synthesized by linear interpolation from the original dataset D1. This synthesized new location data was then input into the trained magnetic field data generation sub-models to generate corresponding magnetic field data. Finally, the generated magnetic field data was added back to the original dataset D1 to obtain the expanded training dataset D2. By learning the features of existing data through generative models and generating new samples similar to the training data, the diversity and quantity of training data are increased, mitigating overfitting caused by insufficient training samples and improving the model's generalization ability and robustness.

[0052] The specific network architecture and steps are as follows:

[0053] I. Data Acquisition

[0054] This invention uses a robotic arm carrying a small magnet (diameter: 6.02 mm, thickness: 1.01 mm, weight: 0.08 g, residual magnetic induction: 18500 G). Within the workspace of a magnetic tracking system, the robotic arm moves along a cubic trajectory, collecting data on changes in the magnet's orientation. This data forms a dataset, which is then used to train and evaluate the accuracy of the neural network in the magnetic positioning system. Simultaneously, the system equips 32 triaxial sensors (MAG3110, Freescale Semiconductors, Austin, Texas) with 12-bit ADCs to record magnetic field strength data. These datasets are collected by the 32 triaxial sensors along a 5x7x4 cm path. 3 The dataset contains magnetic field strength data for continuous trajectories. To better evaluate model performance, the training and testing trajectories have different starting points, meaning their trajectories do not overlap. Each dataset contains approximately 1200 5D states. Uniformly distributed throughout the entire volume, with a resolution of 2.5 millimeters. θ represents the pitch angle. This represents the yaw angle. Due to the limitations of the robotic arm's data acquisition speed, this resolution was chosen to collect a large number of data points within a reasonable timeframe.

[0055] (1) Training and Validation: In this invention, we collected a total of 77 datasets for training and validation, covering a yaw angle of ±50 degrees with a step size of 10 degrees, and a pitch angle of ±90 degrees with a step size of 30 degrees. To ensure that the training and validation data contain information for all angles and to avoid overfitting to the training set, each dataset was randomly split into a training set and a validation set. The training set comprised 80% of each dataset, and the validation set comprised 20% of each dataset.

[0056] (2) Testing: To better evaluate the robustness and positioning accuracy of the method of this invention, we collected a total of 60 sets of data for the test set, covering a yaw angle of ±45 degrees and a step size of 10 degrees, and a pitch angle of ±75 degrees and a step size of 30 degrees. Furthermore, the test set and training set sample locations did not overlap. The angle distribution of the training set and test set was as follows: Figure 1 As shown in the image on the right.

[0057] II. Methods

[0058] A. Pretreatment

[0059] After determining the trajectory of the magnetic tracker, 32 sensors captured the magnetic tracker within a 5×7×4cm area. 3 Magnetic field strength data during spatial movement. Due to environmental interference, this invention records approximately 100 magnetic samples of the background magnetic field under no-load conditions before each data collection to obtain the average value of the background magnetic field strength data. After data collection, the background magnetic field strength is subtracted to obtain the final magnetic field strength data.

[0060] B. Architecture

[0061] To improve the alignment between the localization model and the target area for magnetic pole localization, we... Figure 2 The paper presents a spatial-angle ensemble. This framework includes a localization module for estimating the tracker's state in different spatial and angular regions; then, spatial and angle-aware modules are constructed to selectively combine estimates from the localization module; finally, a method is employed... Figure 3 The attention feature fusion module shown dynamically integrates these localization results. All modules of the spatial-angle integration are implemented using a gated multilayer perceptron, which contains gating units to adaptively suppress noisy features in the model.

[0062] 1) Spatial Ensemble (SE): Because the magnetic field generated by a magnetic tracker has inconsistent effects in different regions of space, the relationship between the magnetic field strength data collected by the sensor and the tracker's position cannot be accurately described by the Euler equation when the tracker is too far or too close to the sensor. To improve the matching accuracy between the localization model and the spatial region of the magnetic positioning target, this invention proposes a spatial sensing module, which consists of a pre-trained spatial sensing module and a local magnetic positioning module. For example... Figure 2 As shown. The spatial region is divided into... Figure 1 The left figure shows eight sub-regions, each with a volume of 2.5 × 3.5 × 2.0 cm. 3 Each sub-region has a corresponding localization module. The spatial awareness module is used to determine which region the tracker currently belongs to. Both the spatial awareness module and each localization module are constructed from gated multilayer perceptrons.

[0063] The neural network was built and trained using PyTorch, with Adam as the optimizer and a learning rate of 0.001. The training process consisted of three steps:

[0064] Step 1: Pre-training of SRL modules: There are a total of 8 sub-region modules. Pre-train each module to minimize the mean squared error.

[0065]

[0066] Where, N s The number of samples located in subregion s; Represents the gMLP network, ω s This represents the learnable parameters to be trained. ξ represents the position and angle of the magnetic tracer in sample i. i ∈R 96 The magnetic field strength data are either measured or obtained through linear interpolation.

[0067] Step 2: Pre-training of the Spatial Awareness Module: The SA module is used to learn the ensemble weights estimated by SRL. We label the samples according to the spatial partition and design the following loss function for pre-training:

[0068]

[0069] Where N is the total number of samples, c i ∈R 8 This is the unique sub-region label for sample i. Let represent the gMLP network, and γ represent the parameters to be trained. If we use the classification cross-entropy cost function, then the softmax activation function will constrain the output of SA or SE to a convex combination of SRL predictions. However, the true location of the magnetic tracer may be outside the convex hull of each SRL prediction. Too many constraints will reduce the model's generalization.

[0070] Step 3: Pre-training of Spatial Integration: After pre-training the SA and SRL modules separately, retrain the parameters of the spatial integration by combining these two modules; the loss function is:

[0071]

[0072] in, express The s-th element.

[0073] 2) Angular ensemble (AE): Since the yaw angle φ directly changes the direction of the magnetic field, it can have a greater impact on the magnetic field strength data measured by the sensor. To improve the matching accuracy between the positioning model and the angular region of the magnetic positioning target, an angular ensemble was constructed. Figure 2 The angle perception module shown in the image is divided into a pre-trained angle perception module and a local localization module.

[0074] The overall angular distribution is divided into eight angular sub-regions, from region 1 to region 8. For example... Figure 1 As shown in the right figure, the output of the angle perception module is the regression weights for each of the eight angular regions. The local localization module consists of eight separate magnetic local localization modules, corresponding to eight angular sub-regions. The local localization module can achieve preliminary localization for each angular sub-region. The loss function of the neural network is defined as follows:

[0075]

[0076]

[0077]

[0078] In the formula, q is the index of the angular region, and β q and d is a learnable parameter for the SRL and AA modules. i The corner regions are labeled. The training process for AE is the same as that for SE.

[0079] 3) Spatial-Angle Integration: Based on the above reasoning, different region-aware modules may have specific improvements on the accuracy of tracker localization. By combining spatial and angle-aware perception, a greater improvement can be achieved in the predictive accuracy of tracker localization. This combination may also improve the matching accuracy between the localization model and the 5-D region of the magnetic positioning target. Based on this, a proposal is... Figure 2 The spatial-angle integration shown in the image consists of parallel spatial perception and angle perception modules. To comprehensively consider the outputs of both modules, the memory retrieval unit was simplified and improved to create attention-based feature fusion, providing better adaptive feature fusion capabilities. An attention-based feature fusion mechanism adaptively fuses the results of spatial perception and angle perception. The structure of the attention-based feature fusion is as follows:

[0080] g i =σ(W g [s i ;a i ]+b g )#(7)

[0081] z i =g i ⊙s i +(1―g i )⊙a i #(8)

[0082] Where s i a i ∈R 5 These are the outputs of SE and AE, respectively, W g ∈R 5×10 and b g ∈R 5 These are learnable parameters that enable feature interactions across different dimensions and sets. σ(·) is the sigmoid function. Note the weights g. i and 1-g i Controlling s i and a i The proportion. i These are the final estimated parameters. The loss function for AFF is:

[0083]

[0084] It is important to note that the parameters of SE and AE are fixed during AFF training to reduce the risk of overfitting.

[0085] 4) Gated Multilayer Perceptron: This part of the network structure can be replaced with other attention mechanism networks, while the neural network in this paper is mainly based on... Figure 4The model is constructed using a gated multilayer perceptron (N=3). It consists of N stacked blocks of identical structure and size. All projection operations are linear, with "×" indicating element-wise multiplication. A layer refers to layer normalization. In the first linear layer, 96-dimensional magnetic field strength data is mapped to a 256-dimensional high-dimensional latent feature space. This 256-dimensional feature space is divided into 8 channels, each with 32 dimensions. This grouping strategy reduces computational complexity within each channel, thereby reducing the overall computational cost of the model. This design strikes a balance between model complexity and computational efficiency. In each gated multilayer perceptron, each channel undergoes two linear mappings (channel projections), allowing information from different channels to interact and fuse. Subsequently, features interact between groups through spatial gating units. The gating operation generates a gate feature mask that controls the flow and fusion of information, thus suppressing noise and redundancy. Spatial gating units introduce inter-group interactions by applying a one-dimensional convolution operation to the features of each channel, followed by feature fusion through element-wise multiplication with the gate features. One-dimensional convolution has 4 input and 4 output channels, a stride of 2, and a kernel size of 5. These interaction mechanisms enable the model to capture information in multiple dimensions, thereby enhancing its expressive power.

[0086] In summary, this model based on a gated multilayer perceptron architecture effectively performs PML tasks through a combination of channel grouping, feature transformation, inter-group and intra-group interactions, and gating mechanisms. This architecture reduces the computational burden to some extent, enhances the model's robustness, and makes it possible to learn effective feature representations from input data, thereby achieving accurate localization.

[0087] Examples of localization results for different networks are shown in Table 1. A comparison of the root mean square error localization performance of tracker position (P) and attitude angles (θ, φ) for different networks is shown in Table 2.

[0088] Table 1

[0089]

[0090] Table 2

[0091]

[0092] C. Generative Model

[0093] In practical data acquisition, data generation strategies are typically employed to reduce the time cost of data collection. Based on the actual sparse raw dataset, generative models are used to generate data to populate the sparse raw dataset, in order to evaluate the impact of generative models on neural network performance under conditions of insufficient sampling.

[0094] Generative models can be simply understood as: by learning the features of existing data, generating new samples similar to the training data, thereby increasing the diversity and quantity of training data, alleviating overfitting caused by insufficient training samples, and improving the model's generalization ability and robustness.

[0095] The specific process of applying the generative model is as follows:

[0096] The first step involves using 32 sensors. For each sensor, eight spatial sub-regions are divided to create a sub-model, resulting in a total of 256 magnetic field data generation sub-models.

[0097] The second step is to use the original dataset D1 to train sub-models for each magnetic field data and save the model parameters.

[0098] The third step involves using linear interpolation to synthesize new location data from the original dataset D1. This synthesized new location data is then input into the trained magnetic field data generation sub-model to generate the corresponding magnetic field data.

[0099] The fourth step is to fill the generated magnetic field data into the original dataset D1 to obtain the expanded training dataset D2.

[0100] Table 3 shows examples of localization results for different networks based on dataset D2 obtained from the generative model. Table 4 shows a comparison of the localization performance of tracker position (P) and orientation (θ, φ) of different networks.

[0101] The comparison results are shown in Tables 2 and 4. The results demonstrate that, in cases of insufficient sampling, using a generative model to supplement training data effectively addresses the problem of insufficient data and improves the model's performance and reliability.

[0102] Table 3

[0103]

[0104] Table 4

[0105]

[0106] System block diagram and software flowchart

[0107] 1. Experimental apparatus for magnetic positioning

[0108] Experimental apparatus for magnetic positioning, such as Figure 5As shown, the system includes: a magnetic tracker, a forward-motion robotic arm, an array of 32 magnetic sensor modules, an STM32 microcontroller control board, and a QT host computer program. The tracker is a small cylindrical permanent magnet mounted at the end of a 20cm white resin rod and positioned by a three-axis forward-motion robotic arm. The three-axis magnetic sensors are grouped into four modules, each with eight sensors. The digitized magnetic field measurements are transmitted to the STM32 microcontroller control board via a serial port for tracker positioning. The sensor modules are designed to capture changes in the magnetic field caused by tongue movements around the cheeks and jaw when a patient uses the system for tongue tracking. Data from all sensors is collected by the STM32 microcontroller control board, which packages the data and sends it to the QT host computer via a Wi-Fi module. The QT host computer receives and stores the data.

[0109] 2. QT host computer program flow

[0110] The QT host computer program flow is as follows: Figure 6 As shown, the system includes a TCP communication module with the microcontroller, a TCP communication module with the positive motion controller, a control module for the positive motion controller, a data acquisition mode setting module, and a module for receiving and saving data uploaded by the microcontroller. A complete data acquisition program flow is as follows: First, establish TCP communication with the microcontroller, then establish TCP communication with the positive motion controller. After successful communication, set the program to save data, then select the serpentine trajectory acquisition mode, and the program starts acquiring data and saving it as a txt text file.

[0111] Experimental results and analysis:

[0112] The gated multilayer perceptron (MLP) demonstrates stronger fitting ability compared to the multilayer perceptron, as shown in Table 2. In experiments, the gated MLP outperforms the multilayer perceptron across multiple angles, specifically improving the prediction accuracy for position, pitch, and yaw by 57.95%, 30.63%, and 13.18%, respectively. Further testing shows that combining the perception mechanism and ensemble learning significantly improves the model's localization accuracy. Compared to the baseline multilayer perceptron, angle ensemble, spatial ensemble, and spatial-angle ensemble improve the average accuracy of position estimation by 64.05%, 65.80%, and 65.80%, respectively; while in angle estimation, the average accuracy of pitch estimation is improved by 31.90%, 34.43%, and 34.43%, respectively, and the average accuracy of yaw estimation is improved by 32.19%, 27.63%, and 29.91%, respectively. This performance improvement is attributed to better alignment of the training and testing datasets. When the distributions of these two factors become more consistent, the overall generalization ability of the model also increases.

[0113] These findings demonstrate the significant potential of neural networks equipped with integrated region-aware modules for magnet localization. Although only eight sub-regions were explored, this was merely to validate our hypotheses. Furthermore, spatial-angular integration, through a unique mechanism, combines the best properties of angular and spatial integration, resulting in outstanding performance across all evaluation metrics.

[0114] Furthermore, generative models can alleviate overfitting caused by insufficient training samples, thereby enhancing the model's generalization ability. Data expansion using generative models has proven highly beneficial in improving the model's localization and accuracy. For example, as shown in Tables 2 and 4, compared to SAEs without generative models, SAEs incorporating generative models significantly improved the prediction accuracy for position, pitch, and yaw by 29.30%, 25.48%, and 39.90%, respectively. The application of generative models further amplifies the advantages of each model, significantly improving localization and accuracy.

[0115] Pilot human trials of tongue motion tracking system

[0116] This systematic study involved human subjects. All experimental procedures and protocols were approved by the Ethics Committee of Guangdong University of Technology under No. 2018A03031330. This systematic study recruited one healthy 21-year-old male non-native English speaker. Figure 9 As shown, a magnetic tracker was attached to the tip of a subject's tongue using tissue adhesive, allowing the subject to converse within the working area of ​​the sensor array. During the experiment, the subject was asked to pronounce the word "sights [saits]". Figure 10 The image shows the reconstructed magnet trajectory using the region-aware magnetic positioning method proposed in this patent. It can be observed that when attempting to pronounce the / s / sound, the tip of the tongue touches the upper alveolar ridge, allowing airflow to pass through both sides of the tongue. When pronouncing the / ai / sound, the tip of the tongue moves downwards as the jaw descends, then rises. When pronouncing the / ts / sound, the tip of the tongue quickly contacts the upper alveolar ridge and then separates, producing a crisp / t / sound. Simultaneously, the airflow passing through both sides of the tongue produces the / s / sound. Figure 10 The red vectors in the middle represent the x, y, and z components of the normalized magnetic moment estimate, i.e. During pronunciation, the red vector represents the estimated direction of the magnetic tracer. This human subject trial demonstrates the potential of the proposed region-aware magnetic localization method and tongue movement tracking system.

[0117] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A region-aware magnetic positioning method for tongue movement tracking, characterized in that, include: Establish a magnetic positioning calibration subsystem to control the robotic arm carrying the magnetic tracker to move along a predetermined trajectory within the working area of ​​the sensor array, and simultaneously collect the spatial position, attitude angle data and magnetic field strength data of the magnetic tracker to construct a training set. Based on the synchronously collected data, a neural network-based generative model is established to learn the mapping relationship from the magnetic tracker position and attitude to the magnetic sensing data, and synthetic samples are used to expand the training set. The range of changes in the spatial position and attitude angle of the magnetic tracker during tongue movement tracking is divided into several sub-regions. Training samples in each sub-region are used to learn the corresponding local magnetic localization model and perform model pre-training. A region perception module is constructed based on a neural network. The region where the magnetic tracker is located is estimated based on magnetic sensing data and pre-trained. The outputs of each pre-trained local magnetic positioning model are integrated by weighted averaging using the pre-trained region perception module. An attention feature fusion module is constructed to adaptively fuse the results of two sets of partitioned localization integration: spatial position and attitude angle. The various local magnetic positioning models, region perception modules, and attention feature fusion modules trained offline on the server are downloaded to the host computer system. The magnetic tracker is attached to the subject's tongue tip, and the subject is allowed to have a conversation within the working area of ​​the sensor array. The collected magnetic field strength data is uploaded to the host computer system via the WIFI communication module for digital signal processing. The spatial position and attitude angle of the magnetic tracker are estimated in real time. The results are stored in the host computer system storage unit, and its trajectory is displayed in the graphical user interface, thereby realizing tongue movement tracking. The region-aware spatial-angle ensemble model includes: First, a series of sub-region localization (SRL) modules are trained using samples from different spatial regions / different angular regions. Each module is a low-complexity local regressor. Noise samples can only affect the learning of the local localization module corresponding to their location. Secondly, the collected magnetic field data is intelligently analyzed by the spatial-aware (SA) module and the angular-aware (AA) module, and the positioning results of each local positioning module are adaptively modulated. Finally, the results of spatial sub-region integration and angular sub-region integration are adaptively integrated by utilizing the attentional feature fusion (AFF) module. The attention feature fusion module includes: in, and These are the outputs of SE and AE, respectively. and These are learnable parameters. For the sigmoid function, To pay attention to the weights, For estimating parameters.

2. The region-aware magnetic positioning method for tongue movement tracking as described in claim 1, characterized in that, Neural network-based methods for generating magnetic field data include: First, for each of the 32 sensors, eight spatial sub-regions were divided to create a generative sub-model, resulting in a total of 256 magnetic field data generation sub-models. The original dataset D1 was used to train each magnetic field data generation sub-model, and the model parameters were saved. Secondly, new location data is synthesized by linear interpolation of the original dataset D1. The synthesized new location data is then input into the trained magnetic field data generation sub-model to generate corresponding magnetic field data. Finally, the generated magnetic field data is filled into the original dataset D1 to obtain the expanded training dataset D2.

3. The region-aware magnetic positioning method for tongue movement tracking as described in claim 1, characterized in that, The spatial integration includes a pre-trained localization module and a spatial perception module; the spatial region is divided into eight sub-regions, each with a volume of 2.5 × 3.5 × 2.0 cm. 3 Each sub-region has a corresponding local magnetic positioning module; the spatial perception module is used to sense which region the tracker currently belongs to; the spatial perception module and each local positioning module are constructed from gated multilayer sensors.

4. The region-aware magnetic positioning method for tongue movement tracking as described in claim 1, characterized in that, The process of training a neural network includes: Step 1: Pre-training of the local localization module. There are a total of 8 sub-region modules. The local magnetic localization model is learned by using the training samples in each sub-region. Step 2: Pre-training of the spatial perception module, using the spatial perception module to learn the set weights estimated by the SRL module; Step 3: Pre-training of spatial integration. After pre-training the local localization module and the spatial perception module respectively, the parameters of the two modules are retrained together to adjust the parameters of spatial integration.

5. The region-aware magnetic positioning method for tongue movement tracking as described in claim 1, characterized in that, The angle integration includes a pre-trained local localization module and an angle perception module.

6. The region-aware magnetic positioning method for tongue movement tracking as described in claim 4, characterized in that, The overall angle distribution is divided into 8 angle sub-regions, from region 1 to region 8; the output of the angle sensing module is the regression weight of the 8 angle regions; the local positioning module consists of 8 separate magnetic local positioning modules, corresponding to the 8 angle sub-regions; the local positioning module achieves preliminary positioning for each angle sub-region.

Images