Lightweight deep learning fingertip positioning method based on CMUT array

Abstract

The invention belongs to the technical field of non-contact sensing, and particularly discloses a lightweight deep learning fingertip positioning method based on a CMUT array. According to the method, accurate fingertip positioning is realized by utilizing the CMUT array, so that the method is not limited by conditions such as illumination and the like, the response speed is relatively high, real-time tracking of the fingertip is realized, a deep learning fingertip positioning model based on a CNN-Transform mixed architecture is established, and the extraction capability of the model on time and space distribution characteristics of fingertip motion signals is improved; meanwhile, an input-perceived dynamic channel pruning mechanism is introduced into the model, so that the calculation amount is remarkably reduced while the positioning precision is ensured; meanwhile, in order to solve the problem that pruning hyper-parameters are difficult to adjust, a three-stage multi-fidelity multi-objective optimization method is constructed, and efficient search and automatic adjustment of the pruning hyper-parameters are achieved. According to the method, the accurate and lightweight fingertip positioning capability is realized.

Description

Technical Field

[0001] This invention belongs to the field of non-contact sensing technology, and specifically relates to a lightweight deep learning fingertip positioning method based on CMUT array, which is particularly suitable for application scenarios such as natural human-computer interaction with high precision and low latency requirements. Background Technology

[0002] With the rapid development of virtual reality (VR), augmented reality (AR), and smart home systems, high-precision, low-latency fingertip positioning technology has become a key element in realizing natural human-computer interaction (HMI). Existing fingertip recognition methods are mainly divided into contact and non-contact types. Contact technologies typically rely on physical contact devices such as capacitive sensors and pressure-sensitive touch units, and are widely used in smartphones, tablets, and wearable devices. Although this type of technology is mature and reliable, its inherent drawbacks are also significant, including limited interaction freedom, strong dependence on direct contact, and susceptibility to wear and mechanical damage. Furthermore, in scenarios such as medical settings and public terminals, contact interaction may also lead to cross-contamination and hygiene risks. Non-contact sensing alternatives typically rely on optical mechanisms to overcome these limitations. While they perform well for finger recognition in controlled environments, their performance degrades significantly due to factors such as ambient light and occlusion, and their reliability is extremely poor in dim lighting or visually complex conditions. Non-contact sensing technology based on humidity changes is gradually gaining attention due to its low cost, low power consumption, and insensitivity to light. However, existing humidity sensors have limited response speed and low spatial resolution, making it difficult to meet the needs of rapid and precise fingertip positioning.

[0003] Capacitive micromachined ultrasonic transducers (CMUTs) fabricated using microelectromechanical systems (MEMS) technology offer the potential for non-contact fingertip recognition due to their high sensitivity, fast response, and ease of integration. However, CMUTs still face several challenges in practical applications, including differences in skin humidity among users and channel interference within the sensor array. Traditional features relying on manual design (such as amplitude or frequency variations) are insufficient to fully characterize the complex spatial relationships of multi-channel humidity fields, thus hindering high-precision positioning. Deep learning offers new possibilities for solving these problems. Deep learning can automatically extract multi-level, multi-scale spatiotemporal features from raw sensor data and has achieved significant results in wearable gesture recognition, activity monitoring, and tactile perception.

[0004] However, existing deep learning models often focus on local temporal features and fail to adequately consider the spatial dependencies within the sensor array, thus limiting their expressive power. Furthermore, high-precision deep learning models typically involve high computational costs and resource consumption, hindering their integrated deployment in portable, human-computer interaction devices. Existing model compression methods, such as static channel pruning, usually rely on permanently removing some channels using global statistics during training. While this reduces computational overhead, its fixed channel selection cannot adapt to different fingertip positions, humidity variations, and user differences, easily leading to the loss of key features in dynamic scenes and resulting in performance degradation. Moreover, pruning strategies often involve multiple coupled hyperparameters, making overall performance sensitive to parameters, and a complete model evaluation is costly, resulting in low parameter tuning efficiency and high search difficulty. Summary of the Invention

[0005] The purpose of this invention is to propose a lightweight deep learning fingertip localization method based on CMUT array. This non-contact fingertip sensing method designs a deep learning model that integrates temporal and spatial features, and introduces a dynamic channel pruning mechanism for input sensing to significantly reduce computational load while ensuring positioning accuracy. At the same time, it constructs a multi-fidelity multi-objective optimization method to achieve efficient search and automatic adjustment of pruning hyperparameters, ultimately achieving accurate and lightweight fingertip localization capabilities.

[0006] To achieve the above objectives, the present invention adopts the following technical solution: A lightweight deep learning-based fingertip localization method based on CMUT arrays includes the following steps: Step 1. Collect frequency change information of each CMUT sensor when the fingertip is at different positions above the CMUT array, preprocess the collected data, and build a model training dataset; Step 2. Build a deep learning fingertip localization model, which includes a dual-branch feature extraction structure consisting of a temporal feature extraction branch and a spatial feature extraction branch, a feature fusion layer, a Transformer encoder, and a fully connected layer; The temporal feature extraction branch includes one-dimensional convolution, dynamic routing controller, and channel attention module; sensor data input is processed sequentially by one-dimensional convolution, dynamic routing controller, and channel attention module to obtain temporal features; The spatial feature extraction branch includes two-dimensional convolution, dynamic routing controller, and spatial attention module; sensor data input is processed sequentially by two-dimensional convolution, dynamic routing controller, and spatial attention module to obtain spatial features; The dynamic routing controller generates channel activation weights based on real-time input features and fuses them with channel information entropy to obtain fused importance weights. Then, it determines the number of channels to be retained based on the pruning ratio and filters channels according to the fused importance weights to achieve dynamic pruning of channels. The feature fusion layer concatenates temporal and spatial features along the channel dimension to obtain the fused features. The fused features are input into the Transformer encoder, which captures the data change trends during the approach and departure of the fingertip through position encoding and multi-head attention; finally, the probability distribution of the spatial position of the fingertip is output through a fully connected layer. Step 3. Train the model based on the training dataset and use the trained model to achieve fingertip localization.

[0007] The present invention has the following advantages: As described above, this invention relates to a lightweight deep learning fingertip localization method based on a CMUT array. This method utilizes the mass load effect of the CMUT array to achieve precise fingertip localization, and is therefore not limited by conditions such as lighting, and has a high response speed, thereby realizing real-time tracking of the fingertip. To address the issue that traditional deep learning algorithms only extract temporal features while ignoring the spatial distribution features of sensors, this invention constructs a deep learning fingertip localization model based on a CNN-Transformer hybrid architecture. This improves the model's ability to extract both the temporal and spatial distribution features of fingertip motion signals. Simultaneously, this invention introduces a dynamic routing controller into the model. This controller combines real-time input features and channel entropy to automatically disable redundant channels for easily distinguishable simple samples and automatically enable more critical channels for samples with significant channel interference. Through joint training using both sparsity and consistency losses, the model reduces inference computational power while retaining the ability to capture weak signals, thereby improving the localization accuracy of the lightweight model. Furthermore, to address the difficulty of manually adjusting parameters such as pruning rate and deep learning training hyperparameters, this invention uses a surrogate model and acquisition function to effectively reduce the number of model training iterations. It can automatically seek optimal parameters at a lower search cost, thus significantly reducing search costs while maintaining localization accuracy. Attached Figure Description

[0008] Figure 1 This is a flowchart of a lightweight deep learning fingertip localization method based on a CMUT array in an embodiment of the present invention; Figure 2 This is a structural block diagram of the CMUT array signal processing system built in an embodiment of the present invention; Figure 3 This is a structural diagram of the oscillation circuit in an embodiment of the present invention; Figure 4This is a structural block diagram of the deep learning fingertip localization model constructed in an embodiment of the present invention; Figure 5 This is a structural block diagram of the dynamic routing controller in an embodiment of the present invention; Figure 6 This is a flowchart of low-fidelity rapid screening in an embodiment of the present invention; Figure 7 This is a flowchart illustrating the optimization of the high-fidelity proxy model in an embodiment of the present invention; Figure 8 This is a flowchart of the high-fidelity verification process in an embodiment of the present invention; Figure 9 This is a comparison chart showing the accuracy of the model in predicting fingertip positions on the test set. Detailed Implementation

[0009] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments: Example 1 This invention describes a non-contact fingertip sensing method based on a CMUT array to overcome the limitations of traditional optical and humidity sensors in terms of response speed, environmental sensitivity, and spatial resolution. Based on this, the invention designs a deep learning model that integrates temporal and spatial features and introduces a dynamic channel pruning mechanism for input sensing to significantly reduce computational load while maintaining positioning accuracy. Simultaneously, a multi-fidelity, multi-objective optimization method is constructed to achieve efficient search and automatic adjustment of pruning hyperparameters. Through these technical solutions, the invention achieves accurate and lightweight fingertip positioning capabilities.

[0010] like Figure 1 As shown, the non-contact fingertip sensing method based on CMUT array in this embodiment includes the following steps: Step 1. Collect frequency change information of each CMUT sensor when the fingertip is at different positions above the CMUT array, preprocess the collected data, and build a model training dataset.

[0011] In this embodiment, the CMUT array, for example, arranges nine CMUT sensors fabricated using MEMS technology into a 3×3 matrix structure, with a distance of 5mm between each CMUT sensor to avoid crosstalk between sensors.

[0012] Before data acquisition, the CMUT array signal processing system in this embodiment is first designed, such as... Figure 2 As shown, the CMUT array signal processing system consists of a power supply module, an oscillation circuit, an FPGA, and a computer.

[0013] The power module provides a 32V DC bias voltage for each CMUT and ±5V power to the amplifiers and FPGA.

[0014] The oscillation circuit is designed based on the Colpitts oscillation principle of operational amplifiers, tracks the resonant frequency of the CMUT, and converts the analog signal into a square wave; it consists of two operational amplifiers, a positive feedback network, a high-pass filter, and a buffer, as follows: Figure 3 As shown.

[0015] The first-stage operational amplifier selected for the oscillation circuit is the OPA637, which... and The gain is set to 5 for initial signal amplification; the second-stage operational amplifier is an OPA699, which... and The gain is set to 4, and it has the function of limiting the amplitude of the output signal. The positive feedback network consists of resistors. and capacitor By connecting them in series, the loop gain of the entire circuit can be adjusted to meet the amplitude balance condition of the Barkhausen criterion. An input resistor is used to reduce the impact of the input current of the preceding operational amplifier on the circuit. Used to provide DC bias and isolate AC signals for CMUT Used to block DC bias and transmit AC signals to the first-stage operational amplifier. The high-pass filter consists of capacitors. and resistance This design not only filters out stray signals from the front-end circuitry but also compensates for the negative phase of the operational amplifier and other circuits, ensuring that the overall circuit phase meets the phase balance condition of the Barkhausen criterion. A MAX4204 buffer is used to match the impedance between the oscillator and the signal acquisition system, preventing high-frequency signal reflection.

[0016] An FPGA consists of an FPGA chip, a crystal oscillator circuit, a download circuit, a reset circuit, and a power supply circuit. It uses equal-precision measurement methods to measure the frequencies of nine sensors and transmits the data to a computer for data visualization processing via a communication interface.

[0017] To achieve simultaneous acquisition of data from multiple channels, an FPGA chip of model XC7A35TFFG484I was selected. The equal-precision frequency measurement method was adopted, which reduced the error caused by counting the signal under test by adaptively setting the gate time to an integer multiple of the period of the signal under test, thus ensuring equal-precision measurement of the acquisition system within the operating frequency band of the CMUT oscillator.

[0018] Using a universal asynchronous transceiver (UART) to achieve data interaction between the FPGA and the PC enables full-duplex communication, meeting the requirements for real-time data transmission. The host computer software, developed based on the PyQt framework, has functions for storing data, enabling programs, and selecting serial ports. It also plots the frequency curves of each channel in real time, facilitating observation of the dynamic frequency response of the CMUT array to fingertip signals.

[0019] The computer collects and preprocesses the frequency change information of each CMUT when the fingertip is at different positions above the CMUT array.

[0020] This invention designs a CMUT array signal processing system, which tracks the resonant frequency signal of each CMUT in the array through an oscillation circuit and acquires it in real time through an FPGA. The system performs data acquisition, preprocessing, and dataset construction on a computer.

[0021] Step 1.1. Data collection.

[0022] The experiment was conducted in a controlled environment (temperature 26±0.5℃, humidity 35±1%RH). Sensor data for a period of time before frequency stabilization was recorded as a reference frequency value. Based on the frequency response differences of CMUT at different fingertip distances, experiments were conducted above each sensor using 3D-printed supports of different heights, where fingertips approached and then moved away.

[0023] Select n different preset height positions, and measure the frequency change information when the fingertip is above each CMUT sensor at each preset height, for a total of r×r×n spatial position information. Where r and n are natural numbers greater than 1.

[0024] In this embodiment, four different heights—5mm, 7mm, 9mm, and 11mm—are selected, i.e., n=4, thus determining a total of 36 spatial position information points. Four users are selected to repeatedly perform the fingertip-down-pause-lift operation at each spatial position, repeating the operation 100 times at each position to enrich the data sample. Since each CMUT has a different frequency, to facilitate comparison of the responses of different sensors, the frequency data of r×r sensors collected within the stable time are averaged, and the average frequency obtained after the average calculation is used as the starting frequency of each CMUT sensor.

[0025] Subtract the corresponding starting frequency from the collected CMUT sensor data containing fingertip position information to obtain the frequency shift of each CMUT over time. The frequency shift was used to analyze the response of the fingertip.

[0026] Step 1.2. Preprocessing.

[0027] The acquired signals undergo data preprocessing, including filtering, baseline processing, and normalization.

[0028] Because baseline drift can occur due to environmental interference during long-term system operation, leading to signal distortion, an averaging filter is first used to smooth and denoise the signal. The point where the frequency just begins to decrease within the sliding window is taken as the baseline point of that window, and the baseline fluctuation curve is estimated using linear interpolation. The baseline is then subtracted from the filtered signal to eliminate the effect of baseline drift. To ensure consistency between channels, the data is standardized by channel, preserving the frequency variation caused by the fingertip while reducing errors due to orders of magnitude.

[0029] Step 1.3. Construct the dataset.

[0030] Using a sliding window, extract data from C channels at T (e.g., 21) sampling time points during the process of the fingertip approaching and leaving, forming a spatial location data sample. .

[0031] Indicates the sampling time point Time Channel The frequency shift is C = r × r, t = 1, 2, ..., T, c = 1, 2, ..., C.

[0032] If k data samples are extracted from each spatial location point, then a total of r×r×n×k data samples are obtained from r×r×n spatial location points to form a training dataset, which is then divided into training set, validation set and test set according to the proportion.

[0033] For example, k is set to 400. Since 36 spatial location points are selected, a total of 14,400 data samples are obtained to form the dataset. For each spatial location, 80% of the dataset is selected as the training set, 10% as the validation set, and 10% as the test set.

[0034] This method detects local humidity field changes caused by fingertips based on the mass load effect of CMUT arrays, avoiding ambient light interference and surface contact errors, and has a high response speed. Combined with oscillation circuits and FPGAs, it enables real-time tracking of fingertip signals.

[0035] Step 2. Build a deep learning fingertip localization model, such as... Figure 4 As shown, this invention employs a hybrid deep learning architecture combining CNN and Transformer, including 1D-CNN for temporal feature extraction, 2D-CNN for spatial feature extraction, channel and spatial attention modules, feature fusion, and Transformer for global dependency modeling, which synergistically improves the accuracy of fingertip localization, and embeds a dynamic routing controller after the activation function of each convolution stage to achieve dynamic pruning.

[0036] Specifically, in this embodiment, the model includes a dual-branch feature extraction structure consisting of a temporal feature extraction branch and a spatial feature extraction branch, a feature fusion layer, a Transformer encoder, and a fully connected layer.

[0037] The temporal feature extraction branch includes one-dimensional convolution, dynamic routing controller, and channel attention module; sensor data input is processed sequentially by one-dimensional convolution, dynamic routing controller, and channel attention module to obtain temporal features.

[0038] The spatial feature extraction branch includes two-dimensional convolution, dynamic routing controller, and spatial attention module; sensor data input is processed sequentially through two-dimensional convolution, dynamic routing controller, and spatial attention module to obtain spatial features.

[0039] The dynamic routing controller generates channel activation weights based on real-time input features, and fuses them with channel information entropy to obtain fused importance weights. Then, it determines the number of channels to be retained based on the pruning ratio, and filters channels according to the fused importance weights to achieve dynamic pruning of channels.

[0040] The feature fusion layer concatenates temporal and spatial features along the channel dimension to obtain the fused features.

[0041] The fused features are input into the Transformer encoder, which captures the data change trends during the approach and departure of the fingertip through position encoding and multi-head attention; finally, the probability distribution of the spatial position of the fingertip is output through a fully connected layer.

[0042] To address the limitations of feature extraction from time-series data, we construct 1D-CNN and 2D-CNN in parallel to extract temporal and spatial features and utilize Transformer to capture long-term dependencies, thereby enhancing the model's feature extraction capability for time-series data and improving the accuracy of fingertip localization.

[0043] The structure and processing flow of each component module of the model in this embodiment will be further described in detail below.

[0044] like Figure 4 As shown, the processing flow of the time feature extraction branch is as follows: For each input sample, a one-dimensional convolution with a kernel size of 3 and a stride of 1 is first used to extract the temporal correlation features of each CMUT sensor channel. After BatchNorm, LeakyReLU activation and max pooling, the feature map is obtained.

[0045] .

[0046] in , , , , These represent the weights, biases, normalization, LeakyReLU activation, and max pooling of a one-dimensional convolution, respectively. This represents the data from T sampling time points in channel c, i.e., a column of data sample X.

[0047] feature map The dynamic routing controller is input to calculate the channel importance score and pruning rate, and the values ​​of redundant channels are set to zero to generate a sparse feature map. Attention weights are calculated using channel attention mechanisms that employ global average pooling and max pooling. Further focus on key channels, this attention weight The formula is expressed as follows: .

[0048] in For average pooling, , This is the weight matrix. Activated for ReLU This is the Sigmoid function.

[0049] The generated sparse feature map With attention weight Multiplying these together yields the output feature map of the time feature extraction branch.

[0050] .

[0051] in The output feature map of the time feature extraction branch. This is an element-wise product.

[0052] like Figure 4 As shown, the processing flow of the spatial feature extraction branch is as follows: A 2D convolutional layer with a kernel size of 3×3 is used to extract spatial features from the sensor data. These features are then activated using BatchNorm and LeakyReLU to obtain the feature map. The formula is expressed as follows: .

[0053] in , , , , These represent the weights, biases, normalization, LeakyReLU activation, and max pooling of a 2D convolution, respectively. This refers to the entire data sample.

[0054] Similarly, feature maps Input to the dynamic routing controller to obtain a sparse feature map. .

[0055] Spatial weights are calculated using 7×7 convolutional layers, average pooling, and max pooling. Focusing on key areas of fingertip signals.

[0056] ; in Indicates average pooling. This represents a 7×7 convolutional layer. This indicates that ReLU is activated.

[0057] The generated sparse feature map Spatial weights Multiplying these together yields the output feature map of the spatial feature extraction branch.

[0058] ; in The output feature map of the spatial feature extraction branch. This represents element-wise product.

[0059] To reduce the computational cost of the model and make it more lightweight, this invention proposes an adaptive channel pruning method based on a dynamic routing controller. The core idea of ​​this pruning method is to dynamically determine the number of channels to be retained in each convolutional layer according to the feature complexity of different input samples, thereby achieving adaptive pruning with input awareness.

[0060] like Figure 5 As shown, the dynamic routing controller includes a global average pooling layer, two fully connected network layers, and an information entropy statistics module.

[0061] First, global average pooling is performed on the input feature map to compress the channels, resulting in channel feature vectors. .

[0062] Then, a multilayer perceptron, i.e., a two-layer fully connected network, is used to process the feature vector. Encode the features to obtain the encoded features. ; .

[0063] in , These represent the weights and biases, respectively. Represent the activation function; and encode the features Dynamic weights are calculated using the Sigmoid and Softmax activation functions respectively. Pruning ratio and fusion weight .

[0064] Meanwhile, the information entropy statistics module performs real-time analysis on the input feature map: First, the feature map is normalized to a probability distribution, and then the channel information entropy of the current batch of features is calculated using the Shannon entropy formula. It also reads the stored historical entropy values ​​and outputs the channel information entropy of the historical training data. .

[0065] And the dynamic weights of the current input features. The two are then combined using a weighted combination based on their fusion weights: .

[0066] in This refers to the importance weight after fusion. , ,make sure ; according to and pruning ratio Calculate the number of channels that need to be retained for each input data matrix sample, retain the top-K most important channels for each sample, and set the features of the unselected channels to zero to obtain a sparse feature map.

[0067] Specifically, the conventional method is to calculate the weights and pruning rate before pruning. First, calculate the number of channels to be retained for the convolutional layer using the pruning rate, and then sort all channels using the fused importance weights to select the number of channels to be retained.

[0068] The resulting sparse feature map is then passed to the subsequent attention module.

[0069] The features output from the temporal feature branch and the features output from the spatial feature branch are concatenated along the channel dimension, and a 1×1 convolutional layer is used to obtain the fused features. The formula is expressed as follows: .

[0070] The fused features The input is a Transformer encoder, which captures the trend of changes in the data during the approach and departure of the fingertip through position encoding and multi-head attention; finally, the output is a probability distribution of the spatial position of the fingertip through a fully connected layer.

[0071] This invention proposes adaptive dynamic channel pruning. By introducing a dynamic router, it can automatically select the number of channels to be enabled in each layer based on the complexity of the real-time input features and by fusing channel information entropy with the current features. This increases sparsity and reduces computation for simple samples, while retaining more key channels for complex samples to ensure accuracy. Step 3. Train the model based on the training dataset and use the trained model to achieve fingertip localization.

[0072] Because there are many parameters during model training, it is quite complex to manually set the hyperparameters to achieve the best balance between accuracy, efficiency and lightweight design under limited computing resources.

[0073] To address the issues of high computational cost and difficulty in balancing hardware constraints when searching high-dimensional spaces, existing hyperparameter optimization algorithms improve upon traditional multi-fidelity optimization frameworks by designing cost-sensitive acquisition functions and hybrid evaluation strategies.

[0074] Specifically, to address the difficulty in adjusting pruning hyperparameters, this invention constructs a three-stage (low-fidelity rapid screening, medium-fidelity surrogate model optimization, and high-fidelity verification) multi-fidelity optimization strategy for optimization, as follows: Phase 1 uses downsampled input data samples and trains for 1 epoch using only 10% of the training data. It adopts the non-dominated sorting strategy of the NSGA-II algorithm to simultaneously optimize three objectives: accuracy, number of parameters, and inference time. After multiple generations of evolution, a Pareto front containing multiple individuals is obtained, which quickly eliminates erroneous parameter combinations.

[0075] Phase 2 trains a random forest surrogate model based on the evaluation data obtained from the mid-fidelity evaluation of multiple individuals selected in Phase 1, and defines a cost-sensitive acquisition function A(x). When generating offspring, the evolutionary algorithm prioritizes individuals with high A(x) values ​​for mid-fidelity evaluation. Using complete data samples, it trains for 4 epochs to efficiently explore the parameter space. After multiple iterations, a fine Pareto front containing multiple elite individuals can be obtained.

[0076] Here, the evaluation data specifically refers to a set of "input-output" data (which can be understood as xy) obtained after the individuals (parameter combinations) selected in the first stage undergo medium-fidelity evaluation. Here, x represents a set of parameter configurations, and y represents the corresponding performance metrics (accuracy, model size, and inference time). This xy evaluation data is used as the initial dataset for the random forest model to establish a mapping relationship between parameter configuration x and model performance y, thereby predicting the performance of unknown parameters.

[0077] Phase 3 is the high-fidelity verification process. Multiple elite individuals are selected from the Pareto front generated in Phase 2. The complete dataset is used for training and fine-tuning for 8 epochs. The selected elite individuals are then trained with high fidelity to determine the optimal combination of hyperparameters.

[0078] The following is in conjunction with the appendix Figure 6 To be continued Figure 8 The process flow for each stage is explained in detail.

[0079] Phase 1 is a low-fidelity rapid screening phase, using only 10% of the training data for one training cycle. Its purpose is to identify obviously poor parameter combinations at low cost. Fifty random hyperparameter combinations are generated, with parameter values ​​evenly distributed within a specified range to ensure population diversity. From the complete dataset, 10% of samples are randomly selected from each category for one training cycle, and accuracy, model parameter count, and inference time per run are calculated.

[0080] like Figure 6 As shown, the processing flow for Phase One is as follows: NSGA-II is used for multi-generation (e.g., 10 generations) evolutionary optimization. Based on three optimization objectives, individuals in the population are stratified according to their dominance relationship. Unassigned individuals are sorted by non-dominance and crowding is calculated. Individuals with high crowding are selected first.

[0081] For the m-th target, the crowding distance for individual i is: ; in Represents an individual Crowded distance, and It is to classify individuals at this layer according to their goals. Individuals after sorting Two adjacent objective function values, and For individuals in this population layer at the target The maximum and minimum values ​​on.

[0082] Since the parameters to be optimized are continuous real variables, in order to maintain population diversity, simulated binary crossover and polynomial mutation are used to generate offspring from the selected parent individuals.

[0083] I. Simulated binary crossover is used to simulate the probability distribution characteristics of single-point binary crossover in the real number domain. The process is as follows: Let the two parent individuals be... , The two offspring individuals generated , The calculation formula is: .

[0084] in Indicates the first Dimensional parameters, , This indicates that the two parent individuals are in the first generation. The numerical value of the dimension parameter, , This indicates that the two generated offspring individuals are in the first... The numerical value of the dimension parameter.

[0085] The distribution factor is determined by random numbers. and cross-distribution index The decision is expressed in the following formula: .

[0086] Distribution index in this embodiment The preferred value is between 10 and 30, for example, 20.

[0087] Polynomial mutation is used to apply small perturbations to individuals in order to escape local optima. The process is as follows: Let the parameters before mutation be Parameters after mutation The calculation formula is:

[0088] ; in , This represents the boundary values ​​for the parameter.

[0089] For the mutation perturbation, it is generated by random numbers. and variation distribution index The decision is expressed in the following formula: .

[0090] In this embodiment, the variation distribution index For example, we can choose 20.

[0091] The parent and offspring populations are merged, and non-dominated sorting and crowding calculations are performed. The top 50% of individuals are selected based on hierarchical sorting and crowding distance to form the next generation population. After 10 generations of evolution, a Pareto front containing 15-20 individuals is obtained.

[0092] This Pareto front represents a parameter combination that achieves a good balance among the three objectives under the current simplified evaluation conditions. This stage can eliminate obviously poor parameter combinations and reduce the computational cost of the evaluation.

[0093] Phase two involves optimizing the medium-fidelity surrogate model. This invention introduces the performance of random forest prediction parameters and designs a cost-sensitive acquisition function to automatically select the evaluation fidelity and search for potential parameter spaces. Fifteen of the most representative individuals from the Pareto front in Phase one are selected as the initial population, and these individuals undergo medium-fidelity evaluation (complete data, 4 training epochs). These evaluation results serve as initial training data for constructing three random forest regression models.

[0094] like Figure 7As shown, the processing flow for Phase Two is as follows: For any input hyperparameter combination vector x, output the mean of the predictions from all decision trees. and the predicted standard deviation The quantification model's cognitive bias towards unknown regions guides the algorithm to search unexplored areas.

[0095] x refers to the combination of individual parameters including learning rate, batch size, etc., which is the solution set of the first stage output.

[0096] To balance performance improvement, exploration of unknowns, and hardware constraints during the search process, a data acquisition function is designed that incorporates expected improvements, prediction uncertainties, and cost assessment. The expression for the data acquisition function is as follows: .

[0097] in For cost-sensitive data acquisition functions, Used to measure the expected improvement relative to the current Pareto frontier. This reflects the uncertainty of forecasts (and encourages exploration of unknown areas). It is limited by a preset hardware constraint threshold to automatically avoid solutions with excessively high computational costs when evaluating computational costs.

[0098] Indicates the exploration weighting coefficient. This represents the cost penalty coefficient. This represents the standard deviation of the forecast.

[0099] , ; in Indicates the first The observed minimum value of each target. This indicates that the proxy model responds to the input. In the The predicted mean over each target; , The cumulative distribution function and probability density function represent the standard normal distribution.

[0100] From the Pareto front of Phase 1, several of the most representative individuals are selected as the initial population for multi-generational evolutionary optimization. After multiple iterations, a refined Pareto front containing multiple individuals can be obtained. From the output of Phase 1, 15 representative individuals are selected as the initial population for 18 generations of evolutionary optimization. The process for each generation is as follows: 20 new candidate individuals are generated through genetic operations; their performance and uncertainty are predicted using a random forest surrogate model, and the acquisition function value is calculated; the candidate individuals are ranked according to the acquisition function value, and only the top 50% of individuals undergo mid-fidelity testing, while the remaining individuals use the surrogate model's prediction value as the evaluation result to significantly reduce computational overhead; the tested data is added to the training set to update the surrogate model, and the parent and child individuals are merged. The NSGA-II mechanism is used to select the optimal 15 individuals to enter the next generation. After 18 iterations, the algorithm achieves a balance between efficient exploration and accurate evaluation, ultimately outputting a refined Pareto front containing 6-10 elite individuals.

[0101] like Figure 8 As shown, Phase 3 is high-fidelity verification, and its processing flow is as follows: The elite individuals selected in the second stage are precisely evaluated to determine the final optimal hyperparameter configuration.

[0102] During training, a joint loss function is constructed to optimize the network parameters and the parameters of the dynamic routing controller. The joint loss function includes the task classification loss. Calculation efficiency loss And routing consistency loss .

[0103] Task classification loss The cross-entropy loss function is used to calculate the loss, and its formula is expressed as follows: .

[0104] in For batch size, Indicates the true label, This represents the predicted probability.

[0105] Calculation efficiency loss The formula used to balance the inference time and the number of parameters in the model is expressed as follows: ; in , For the number of parameters and the target number of parameters, , The inference time and the target inference time are set.

[0106] Routing consistency loss Due to model sparsity loss Stability loss and routing efficiency loss composition: ; in, , as well as They represent , and Weighting coefficients for the three losses.

[0107] .

[0108] in The average sparsity is denoted as .

[0109] .

[0110] in For the sample Routing weight vector For the sample The routing weight vector.

[0111] .

[0112] in Indicates the number of channels. For the sample aisle The mask value.

[0113] Total loss function for: .

[0114] in , , These are the weights for each loss function.

[0115] From the second phase of the Pareto frontier, select multiple elite individuals, train each individual using the complete training dataset for 8 epochs, and evaluate the accuracy of each individual. Parameter quantity and single reasoning time Calculate the overall score .

[0116] ; in, , , These represent the weighting coefficients for accuracy, number of parameters, and inference time, respectively.

[0117] Selecting the optimal individual is... The individual with the largest value is selected, and this optimal solution is used to train and verify the training results on the test set.

[0118] The low-medium-high three-stage multi-objective parameter optimization designed in this invention effectively reduces the number of model training iterations through a surrogate model and acquisition function. It can automatically seek the optimal parameters with low search cost, solve the problem of difficult manual parameter tuning, and effectively achieve a balance between accuracy and model lightweighting.

[0119] After determining the optimal individual, the optimal combination of model parameters is obtained. The model is then deployed. During the real-time inference phase, the system collects real-time frequency data from the CMUT array and preprocesses it. This data is then input into the optimized deep learning model. At this point, the dynamic routing controller inside the model activates only the key channels for convolution calculation based on the current input features, automatically skipping redundant channels. This significantly reduces computation time while maintaining high accuracy, ultimately achieving precise fingertip positioning.

[0120] This invention constructs a deep learning-based fingertip localization method based on the combination of CNN and Transformer, which is used to extract temporal and spatial features of sensor data, and embeds a dynamic routing controller to generate channel activation weights based on real-time input features, and integrates them with static importance indicators such as channel information entropy to achieve dynamic channel pruning and reduce the computational load of the model.

[0121] Of course, the Transformer encoder can also be replaced with other time-series processing models, such as RNN or LSTM.

[0122] Of course, the size of the CMUT array built in this embodiment is not limited to 3×3, and can be further expanded, such as 4×4, 5×5, etc. Increasing the number of CMUTs in the CMUT array can increase more spatial positions to achieve more precise fingertip positioning.

[0123] In addition, to verify the effectiveness of the method proposed in this invention, the following specific experiments are also given.

[0124] In the final training, the optimal parameters were selected as a pruning rate of 49.4%, a learning rate of 0.001, and a batch size of 64. The original model with 0.63M parameters was compressed to 0.42M, and the single inference time was reduced from 5.27ms to 2.36ms.

[0125] The comparison models used in this invention are the one-dimensional convolutional model 1D-CNN and the two-dimensional convolutional model 2D-CNN. The comparison metrics selected are four metrics: accuracy, precision, recall, and F1 score.

[0126] Table 1 Performance comparison between the method of the present invention and the comparative method As shown in Table 1, the model proposed in this invention achieves precision of 96.69%, recall of 96.60%, and F1 score of 96.59%, which are comprehensively superior to 1D-CNN and 2D-CNN models that rely on only a single feature.

[0127] The high precision demonstrates that the dynamic routing mechanism can effectively suppress noise interference from redundant channels and significantly reduce false alarms; while the high recall reflects the deep fusion capability of the CNN-Transformer hybrid architecture for spatiotemporal features, enabling accurate capture even for weak fingertip signals; the improvement in F1 score further verifies that the multi-objective optimization strategy achieves the best balance between recognition accuracy and robustness while ensuring model lightweightness.

[0128] Figure 9 This chart shows a comparison of the model's training accuracy on the test set, with the horizontal axis representing the training batches and the vertical axis representing the model's accuracy on the test set. Figure 9 It can be seen that the fusion model proposed in this invention outperforms the 1D-CNN and 2D-CNN models that rely on only a single feature in the final fingertip position recognition accuracy, proving the effectiveness of the hybrid architecture in extracting spatiotemporal features.

[0129] The non-contact fingertip localization method proposed in this invention accurately infers the fingertip's position in three-dimensional space by constructing a 3×3 CMUT array and monitoring the resonant frequency shift caused by changes in the local humidity field when the fingertip approaches. This scheme does not rely on lighting conditions and is not limited by surface contact, effectively avoiding problems such as ambient light interference, mechanical contact errors, and surface contamination, thus improving the stability and applicability of fingertip localization. Furthermore, this invention also constructs a deep learning fingertip localization model based on CNN and Transformer architectures and proposes an input-aware dynamic channel pruning method. By introducing a lightweight dynamic routing controller into the model, channel activation weights are generated based on real-time input features and fused with the channel information entropy of historical training and the current feature, dynamically selecting the most effective channels for computation. The model activates Top-K channels on demand during forward propagation, achieving efficient inference. By constructing a multi-objective loss that integrates task accuracy, computational cost, and importance consistency, the controller learns a stable and reliable dynamic decision-making strategy, achieving a balance between low computational cost and high accuracy. To address the challenges of adjusting hyperparameters such as pruning rate and learning rate, and the resulting high evaluation costs, this invention constructs a three-stage multi-objective parameter optimization framework: low-fidelity optimization for rapid screening, followed by the introduction of a random forest surrogate model and a cost-sensitive acquisition function for mid-fidelity optimization, and finally, high-fidelity evaluation for final confirmation. This framework simultaneously optimizes three objectives: accuracy, parameter count, and inference time, effectively reducing the number of training iterations and search costs, and resolving the difficulty of parameter tuning.

[0130] Example 2 This embodiment 2 describes a lightweight deep learning fingertip localization system based on a CMUT array. This system is based on the same inventive concept as the lightweight deep learning fingertip localization method based on a CMUT array in embodiment 1.

[0131] The lightweight deep learning fingertip localization system based on CMUT array in this embodiment includes the following modules: The preprocessing module is used to collect frequency change information of each CMUT sensor when the fingertip is at different positions above the CMUT array, preprocess the collected data, and build a model training dataset. The fingertip localization module is used to build a deep learning fingertip localization model. It includes a dual-branch feature extraction structure consisting of a temporal feature extraction branch and a spatial feature extraction branch, a feature fusion layer, a Transformer encoder, and a fully connected layer. The temporal feature extraction branch includes one-dimensional convolution, dynamic routing controller, and channel attention module; sensor data input is processed sequentially by one-dimensional convolution, dynamic routing controller, and channel attention module to obtain temporal features; The spatial feature extraction branch includes two-dimensional convolution, dynamic routing controller, and spatial attention module; sensor data input is processed sequentially by two-dimensional convolution, dynamic routing controller, and spatial attention module to obtain spatial features; The dynamic routing controller generates channel activation weights based on real-time input features and fuses them with channel information entropy to obtain fused importance weights. Then, it determines the number of channels to be retained based on the pruning ratio and filters channels according to the fused importance weights to achieve dynamic pruning of channels. The feature fusion layer concatenates temporal and spatial features along the channel dimension to obtain the fused features. The fused features are input into the Transformer encoder, which captures the data change trends during the approach and departure of the fingertip through position encoding and multi-head attention; finally, the probability distribution of the spatial position of the fingertip is output through a fully connected layer. The model is trained based on the training dataset, and the trained model is used to achieve fingertip localization.

[0132] This invention addresses the shortcomings of existing non-contact fingertip positioning technologies in terms of sensing stability, feature representation capability, computational overhead, and model parameter tuning, and constructs a high-precision, low-power fingertip positioning system suitable for complex environments.

[0133] It should be noted that any content not mentioned in the above-described functional modules of the system described in Embodiment 2 can be referred to the step description of the corresponding method in Embodiment 1 above, and will not be repeated in detail here.

[0134] Example 3 This embodiment 3 describes a computer device including a memory and one or more processors. Executable code is stored in the memory. When the processor executes the executable code, it implements the steps of the lightweight deep learning fingertip localization method based on a CMUT array described in embodiment 1 above.

[0135] In this embodiment, the computer device can be any device or apparatus with data processing capabilities, and will not be described in detail here.

[0136] Example 4 This embodiment 4 describes a computer-readable storage medium storing a program that, when executed by a processor, is used to implement the steps of the lightweight deep learning fingertip localization method based on a CMUT array in embodiment 1 above.

[0137] The computer-readable storage medium can be an internal storage unit of any device or apparatus with data processing capabilities, such as a hard disk or memory, or an external storage device of any device with data processing capabilities, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc.

[0138] Of course, the above description is only a preferred embodiment of the present invention. The present invention is not limited to the above-described embodiments. It should be noted that any equivalent substitutions or obvious modifications made by those skilled in the art under the guidance of this specification fall within the scope of this specification and should be protected by the present invention.

Claims

1. A lightweight deep learning fingertip localization method based on CMUT array, characterized in that, Includes the following steps: Step 1. Collect frequency change information of each CMUT sensor when the fingertip is at different positions above the CMUT array, preprocess the collected data, and build a model training dataset; Step 2. Build a deep learning fingertip localization model, which includes a dual-branch feature extraction structure consisting of a temporal feature extraction branch and a spatial feature extraction branch, a feature fusion layer, a Transformer encoder, and a fully connected layer; The temporal feature extraction branch includes one-dimensional convolution, dynamic routing controller, and channel attention module; sensor data input is processed sequentially by one-dimensional convolution, dynamic routing controller, and channel attention module to obtain temporal features; The spatial feature extraction branch includes two-dimensional convolution, dynamic routing controller, and spatial attention module; sensor data input is processed sequentially by two-dimensional convolution, dynamic routing controller, and spatial attention module to obtain spatial features; The dynamic routing controller generates channel activation weights based on real-time input features and fuses them with channel information entropy to obtain fused importance weights. Then, it determines the number of channels to be retained based on the pruning ratio and filters channels according to the fused importance weights to achieve dynamic pruning of channels. The feature fusion layer concatenates temporal and spatial features along the channel dimension to obtain the fused features. The fused features are input into the Transformer encoder, which captures the data change trends during the approach and departure of the fingertip through position encoding and multi-head attention; finally, the probability distribution of the spatial position of the fingertip is output through a fully connected layer. Step 3. Train the model based on the training dataset and use the trained model to achieve fingertip localization.

2. The lightweight deep learning fingertip localization method based on CMUT array according to claim 1, characterized in that, In step 1, before data acquisition, a CMUT array is built and a CMUT array signal processing system is designed; wherein the CMUT array consists of r×r CMUT sensors arranged in a matrix, where r is a natural number greater than 1; The CMUT array signal processing system includes a power supply module, an oscillator circuit, an FPGA, and a computer; The oscillation circuit is designed based on the Colpitts oscillation principle of operational amplifiers to track the resonant frequency of the CMUT and convert the analog signal into a square wave; the FPGA measures the frequency of each CMUT sensor and transmits it to the computer. The computer collects and preprocesses the frequency change information of each CMUT when the fingertip is at different positions above the CMUT array.

3. The lightweight deep learning fingertip localization method based on CMUT array according to claim 1, characterized in that, Step 1 specifically involves: Step 1.

1. Data Acquisition; Select n different preset height positions, and measure the frequency change information when the fingertip is above each CMUT sensor at each preset height, for a total of r×r×n spatial position information; The data sample is enriched by repeatedly performing the fingertip drop-hold-lift operation at each spatial location; The average frequency data of r×r sensors collected within a stable time period is calculated, and the average frequency obtained after the average calculation is used as the starting frequency of each CMUT sensor. Subtract the corresponding starting frequency from the collected CMUT sensor data containing fingertip position information to obtain the frequency shift of each CMUT over time. Where r and n are both natural numbers greater than 1; Step 1.

2. Perform data preprocessing on the acquired signals; Step 1.

3. Construct the dataset; Using a sliding window, data from C channels at T sampling time points during the process of the fingertip approaching and leaving the area are extracted to form a spatial location data sample. ; Indicates the sampling time point Time Channel The frequency shift, C = r × r, t = 1, 2, ..., T, c = 1, 2, ..., C; If k data samples are extracted from each spatial location point, then a total of r×r×n×k data samples are obtained from r×r×n spatial location points to form a training dataset, which is then divided into training set, validation set and test set according to the proportion.

4. The lightweight deep learning fingertip localization method based on CMUT array according to claim 1, characterized in that, The processing flow of the time feature extraction branch is as follows: For each input sample, a one-dimensional convolution with a kernel size of 3 and a stride of 1 is first used to extract the temporal correlation features of each CMUT sensor channel. After BatchNorm, LeakyReLU activation and max pooling, the feature map is obtained. ； in , , , , These represent the weights, biases, normalization, LeakyReLU activation, and max pooling of a one-dimensional convolution, respectively. Indicates channel Data from T sampling time points; feature map The dynamic routing controller is input to calculate the channel importance score and pruning rate, and the values ​​of redundant channels are set to zero to generate a sparse feature map. ; Attention weights are calculated using channel attention mechanisms that employ global average pooling and max pooling. The formula is expressed as follows: ； in For average pooling, , This is the weight matrix. Activated for ReLU The sigmoid function is used to generate sparse feature maps. With attention weight Multiplying them together yields the output feature map of the time feature extraction branch; ； in The output feature map of the time feature extraction branch. This is an element-wise product.

5. The lightweight deep learning fingertip localization method based on CMUT array according to claim 1, characterized in that, The processing flow of the spatial feature extraction branch is as follows: A 2D convolutional layer with a kernel size of 3×3 is used to extract spatial features from the sensor data. These features are then activated using BatchNorm and LeakyReLU to obtain the feature map. The formula is expressed as follows: ； in , , , , These represent the weights, biases, normalization, LeakyReLU activation, and max pooling of a 2D convolution, respectively. For the entire data sample; Similarly, feature maps Input to the dynamic routing controller to obtain a sparse feature map. ; Spatial weights are calculated using 7×7 convolutional layers, average pooling, and max pooling. Focusing on key areas of fingertip signals; ； in Indicates average pooling. This represents a 7×7 convolutional layer. Indicates ReLU activation; generates sparse feature maps Spatial weights Multiplying these together yields the output feature map of the spatial feature extraction branch. ； in The output feature map of the spatial feature extraction branch. This represents element-wise product.

6. The lightweight deep learning fingertip localization method based on CMUT array according to claim 1, characterized in that, The dynamic routing controller includes a global average pooling layer, a two-layer fully connected network, and an information entropy statistics module. First, global average pooling is performed on the input feature map to compress the channels, resulting in channel feature vectors. ; Then, a multilayer perceptron, i.e., a two-layer fully connected network, is used to process the feature vector. Encode the features to obtain the encoded features. ; ； in , These represent the weights and biases, respectively. Represent the activation function; and encode the features Dynamic weights are calculated using the Sigmoid and Softmax activation functions respectively. Pruning ratio and fusion weight ; Meanwhile, the information entropy statistics module performs real-time analysis on the input feature map: First, the feature map is normalized to a probability distribution, and then the channel information entropy of the current batch of features is calculated using the Shannon entropy formula. ; It also reads the stored historical entropy values ​​and outputs the channel information entropy of the historical training data. ; And the dynamic weights of the current input features. The two are then combined using a weighted combination based on their fusion weights: ； in This refers to the importance weight after fusion. , , ; according to and pruning ratio Calculate the number of channels that need to be retained for each input data matrix sample, retain the top-K most important channels for each sample, and set the features of the unselected channels to zero to obtain a sparse feature map.

7. The lightweight deep learning fingertip localization method based on CMUT array according to claim 1, characterized in that, In step 3, a three-stage multi-fidelity optimization strategy is constructed for optimization, as follows: Phase 1 is a low-fidelity rapid screening process. This process downsamples the input data samples and trains for 1 epoch using only 10% of the training data. It adopts the non-dominated sorting strategy of the NSGA-II algorithm and optimizes three objectives at the same time: accuracy, number of parameters and inference time. After multiple generations of evolution, a Pareto front containing multiple individuals is obtained. Phase 2 is the optimization process of the medium-fidelity surrogate model. Based on the evaluation data obtained from the medium-fidelity evaluation of multiple individuals selected in Phase 1, a random forest surrogate model is trained, and a cost-sensitive acquisition function A(x) is defined. When the evolutionary algorithm generates offspring, it prioritizes individuals with high A(x) values ​​for medium-fidelity evaluation. Using complete data samples, it trains for 4 epochs, thereby efficiently exploring the parameter space. After multiple iterations, a refined Pareto front containing multiple elite individuals can be obtained. Phase 3 is the high-fidelity verification process. Multiple elite individuals are selected from the Pareto front generated in Phase 2. The complete dataset is used for training and fine-tuning for 8 epochs. The selected elite individuals are then trained with high fidelity to determine the optimal combination of hyperparameters.

8. The lightweight deep learning fingertip localization method based on CMUT array according to claim 7, characterized in that, In step 3, the processing flow of stage one is as follows: The NSGA-II algorithm is used for multi-generational evolution optimization. Based on three optimization objectives, the individuals in the population are stratified according to their dominance relationship. Unassigned individuals are sorted in a non-dominated order and their crowding degree is calculated. Individuals with high crowding degree are selected first. For the m-th target, the crowding distance for individual i is: ； in Represents an individual Crowded distance, and It is to classify individuals at this layer according to their goals. Individuals after sorting Two adjacent objective function values, and For this layer of population in the target The maximum and minimum values ​​on; The selected parent individuals are used to generate offspring through simulated binary crossover and polynomial mutation. I. Simulated binary crossover is used to simulate the probability distribution characteristics of single-point binary crossover in the real number domain. The process is as follows: Let the two parent individuals be... , The two offspring individuals generated , The calculation formula is: ； in Indicates the first Dimensional parameters, , This indicates that the two parent individuals are in the first generation. The numerical value of the dimension parameter, , This indicates that the two generated offspring individuals are in the first... The numerical value of the dimension parameter; It is the distribution factor; From random numbers and cross-distribution index The decision is expressed in the following formula: ； Multinomial mutation is used to apply small perturbations to individuals in order to escape local optima. The process is as follows: Let the parameters before mutation be Parameters after mutation The calculation formula is: ； in , These are the boundary values ​​for the parameter; For variation perturbation; From random numbers and variation distribution index The decision is expressed in the following formula: ； The parent and offspring populations are merged, and non-dominated sorting and crowding calculations are performed. The top 50% of individuals are selected based on hierarchical sorting and crowding distance to form the next generation population. After multiple generations of evolution, a Pareto front containing multiple individuals is obtained.

9. The lightweight deep learning fingertip localization method based on CMUT array according to claim 7, characterized in that, In step 3, the processing flow for stage two is as follows: For any input hyperparameter combination vector x, output the mean of the predictions from all decision trees. and the predicted standard deviation The quantitative model's cognitive bias towards unknown regions guides the algorithm to search unexplored areas. Design a data acquisition function that includes expected improvement, prediction uncertainty, and cost assessment. The expression for the data acquisition function is: ； in For cost-sensitive data acquisition functions, Used to measure the expected improvement relative to the current Pareto frontier. Reflecting the uncertainty of forecasts, Limited by a preset hardware constraint threshold; Indicates the exploration weighting coefficient. This represents the cost penalty coefficient. Indicates the standard deviation of the forecast; ， ； in Indicates the first The observed minimum value of each target. This indicates that the proxy model responds to the input. In the The predicted mean over each target; , The cumulative distribution function and probability density function represent the standard normal distribution; Select several of the most representative individuals from the Pareto front in stage one as the initial population and perform multi-generational evolution optimization; after multiple iterations, a refined Pareto front containing multiple individuals can be obtained.

10. The lightweight deep learning fingertip localization method based on CMUT array according to claim 7, characterized in that, In step 3, the processing flow for stage three is as follows: The elite individuals selected in the second stage are precisely evaluated to determine the final optimal hyperparameter configuration; During training, a joint loss function is constructed to optimize the network parameters and the parameters of the dynamic routing controller. The joint loss function includes the task classification loss. Calculation efficiency loss And routing consistency loss ; Task classification loss The cross-entropy loss function is used to calculate the loss, and its formula is expressed as follows: ； in For batch size, Indicates the true label, Indicates the predicted probability; Calculation efficiency loss The formula used to balance the inference time and the number of parameters in the model is expressed as follows: ； in , For the number of parameters and the target number of parameters, , The inference time and the target inference time are set; Routing consistency loss Due to model sparsity loss Stability loss and routing efficiency loss composition: ； in, , as well as They represent , and Weighting coefficients for the three losses; ； in The average sparsity; ； in For the sample Routing weight vector For the sample The routing weight vector; ； in Indicates the number of channels. For the sample aisle The mask value; Total loss function for: ； in , , These are the weights for each loss function; From the second phase of the Pareto frontier, select multiple elite individuals, train each individual using the complete training dataset for 8 epochs, and compare the accuracy of each individual. Parameter quantity and single reasoning time Calculate the overall score ; ； in, , , These represent the weighting coefficients for accuracy, number of parameters, and inference time, respectively. Selecting the optimal individual is... The individual with the largest value is selected, and this optimal solution is used to train and verify the training results on the test set.

Images