A gesture recognition method based on an ultrasonic array and deep learning and a related device

Abstract

The application discloses a gesture recognition method based on an ultrasonic array and deep learning and related devices, comprising: acquiring pressing information of a user through an ultrasonic array, and converting the pressing information into echo signals; preprocessing the echo signals to obtain analog echo signals; converting the analog echo signals into two-dimensional gray-scale images with space-time continuity; inputting the two-dimensional gray-scale images with space-time continuity into a trained recognition model to obtain probabilities of various dynamic gestures, and taking a dynamic gesture with the highest probability as the final result, so that the method and the related devices can stably distinguish complex dynamic gestures.

Description

Technical Field

[0001] This invention belongs to the field of non-contact human-computer interaction and ultrasonic electronics technology, and relates to a gesture recognition method and related device based on ultrasonic array and deep learning. Background Technology

[0002] Gesture recognition is a crucial entry point for natural human-computer interaction and can be used in scenarios such as smart homes, vehicle control, virtual reality, and robot control. Traditional vision solutions are susceptible to lighting and privacy risks; wearable sensing solutions typically rely on human wearing or contact; while millimeter-wave radar solutions offer the advantage of being non-contact, they have high system costs, algorithm complexity, and power consumption.

[0003] Ultrasonic sensing technology offers advantages such as low hardware cost, compact structure, insensitivity to visible light environments, and low privacy risks. However, the angular resolution of a single ultrasonic transducer is limited, and air attenuation further reduces the echo signal-to-noise ratio as the propagation distance increases. Therefore, relying solely on a single-transmitter, single-receiver structure or simple threshold judgment is insufficient to reliably distinguish complex dynamic gestures. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of the prior art and provide a gesture recognition method and related device based on ultrasonic array and deep learning. This method and related device can stably distinguish complex dynamic gestures.

[0005] To achieve the above objectives, this invention discloses a gesture recognition method based on ultrasonic arrays and deep learning, comprising: The user's pressure information is acquired through an ultrasonic array and converted into an echo signal; The echo signal is preprocessed to obtain a simulated echo signal; Convert the analog echo signal into a two-dimensional grayscale image with spatiotemporal continuity; The two-dimensional grayscale image with spatiotemporal continuity is input into the trained recognition model to obtain the probability of various dynamic gestures, and the dynamic gesture with the highest probability is taken as the final result.

[0006] Furthermore, the ultrasonic array includes eight PZT piezoelectric ceramic transmitting transducers and one PZT receiving transducer. The eight PZT piezoelectric ceramic transmitting transducers are evenly distributed along the circumference, and the one PZT receiving transducer is located in the central region. The PZT receiving transducer outputs an analog electrical signal, which is then conditioned to obtain a one-dimensional echo timing signal.

[0007] Furthermore, the categories of the dynamic gestures include moving up, moving down, moving left, moving right, swiping up from left to right, swiping down from left to right, swiping up from right to left, and swiping down from right to left.

[0008] Furthermore, the preprocessing process for the echo signal is as follows: The echo signal is pre-amplified, bandpass filtered near 40kHz, and amplified in the final stage before being output as a conditioned analog echo signal.

[0009] Furthermore, the process of converting the analog echo signal into a two-dimensional grayscale image with spatiotemporal continuity is as follows: the conditioned analog echo signal is subjected to analog-to-digital conversion, amplitude extraction, time window segmentation and normalization processing, and a two-dimensional grayscale image is generated for dynamic gestures to characterize the movement direction, speed or trajectory changes of the gesture in front of the array.

[0010] Furthermore, the recognition model includes convolutional neural networks, residual networks, and lightweight convolutional networks. A transfer learning model based on ResNet-50 is used to classify ultrasound amplitude images. The model convergence is achieved by freezing some base layers, replacing classification layers, and small-sample augmentation training. During training, amplitude scaling, slight rotation, slight translation, noise perturbation, or time clipping are applied to the training samples to achieve data augmentation. The data augmentation is constrained by the array physical coordinates and the spatiotemporal semantics of the image.

[0011] This invention discloses a gesture recognition system based on ultrasonic arrays and deep learning, comprising: An ultrasonic array is used to acquire the user's pressure information and convert the pressure information into an echo signal; A receiving conditioning circuit is used to preprocess the echo signal to obtain a simulated echo signal; Amplitude imaging module, used to convert analog echo signals into two-dimensional grayscale images with spatiotemporal continuity; A microcontroller is used to input the spatiotemporally continuous two-dimensional grayscale image into the trained recognition model to obtain the probability of various dynamic gestures, and to take the dynamic gesture with the highest probability as the identification result. The microcontroller is connected to the ultrasonic array, which is connected to the microcontroller via a receiving conditioning circuit and an amplitude imaging module.

[0012] Furthermore, a transmission boost drive circuit is provided between the microcontroller and the ultrasonic transmitting array.

[0013] The present invention discloses a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the steps of the gesture recognition method based on ultrasonic array and deep learning.

[0014] The present invention discloses a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the gesture recognition method based on ultrasonic array and deep learning.

[0015] The present invention has the following beneficial effects: In specific operation, the gesture recognition method and related device based on ultrasonic array and deep learning described in this invention acquires the user's pressing information through an ultrasonic array, converts the pressing information into an echo signal, and constructs a two-dimensional grayscale image with spatiotemporal continuity. The two-dimensional grayscale image with spatiotemporal continuity is input into the trained recognition model to obtain the probability of various dynamic gestures. The dynamic gesture with the highest probability is taken as the identification result, thereby stably distinguishing complex dynamic gestures. Attached Figure Description

[0016] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0017] Figure 1 This is a schematic diagram of the structure of the present invention; Figure 2 This is a schematic diagram of an eight-transmitter, one-receiver circular ultrasonic array topology. Figure 3 A comparative schematic diagram of beam coverage for single transducer, linear, square, and circular arrays; Figure 4 This is a schematic diagram of the transmitter boost drive circuit; Figure 5 This is a schematic diagram of the receiving conditioning circuit; Figure 6 This is a schematic diagram of the dynamic gesture amplitude imaging process. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] In the description of this invention, it should be understood that the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0020] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0021] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes such combinations. For example, A and / or B can represent three cases: A alone, A and B simultaneously, and B alone. Additionally, the character " / " in this invention generally indicates that the preceding and following objects have an "or" relationship.

[0022] It should be understood that although terms such as first, second, third, etc., may be used in the embodiments of the present invention to describe the preset range, these preset ranges should not be limited to these terms. These terms are only used to distinguish the preset ranges from one another. For example, without departing from the scope of the embodiments of the present invention, the first preset range may also be referred to as the second preset range, and similarly, the second preset range may also be referred to as the first preset range.

[0023] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0025] The accompanying drawings illustrate various structural schematic diagrams according to embodiments disclosed in this invention. These drawings are not to scale, and some details have been enlarged for clarity, and some details may have been omitted. The shapes of the various regions and layers shown in the drawings, as well as their relative sizes and positional relationships, are merely exemplary and may deviate from reality due to manufacturing tolerances or technical limitations. Furthermore, those skilled in the art can design regions / layers with different shapes, sizes, and relative positions as needed.

[0026] Example 1 The gesture recognition method based on ultrasound array and deep learning described in this invention includes: The user's pressure information is acquired through an ultrasonic array and converted into an echo signal; The echo signal is preprocessed to obtain a simulated echo signal; Convert the analog echo signal into a two-dimensional grayscale image with spatiotemporal continuity; The two-dimensional grayscale image with spatiotemporal continuity is input into the trained recognition model to obtain the probability of various dynamic gestures, and the dynamic gesture with the highest probability is taken as the final result.

[0027] In this embodiment, the preprocessing of the echo signal is as follows: the echo signal is pre-amplified, bandpass filtered near 40kHz, and amplified in the final stage before outputting a conditioned analog echo signal.

[0028] In this embodiment, the process of converting the analog echo signal into a two-dimensional grayscale image with spatiotemporal continuity is as follows: the conditioned analog echo signal is subjected to analog-to-digital conversion, amplitude extraction, time window segmentation and normalization processing, and a two-dimensional grayscale image is generated for dynamic gestures to characterize the movement direction, speed or trajectory changes of the gesture in front of the array.

[0029] In this embodiment, the recognition model includes a convolutional neural network, a residual network, and a lightweight convolutional network. A transfer learning model based on ResNet-50 is used to classify ultrasound amplitude images. The model convergence is achieved by freezing some base layers, replacing classification layers, and small sample augmentation training. During training, amplitude scaling, slight rotation, slight translation, noise perturbation, or time clipping are applied to the training samples to achieve data augmentation. The data augmentation is constrained by the array physical coordinates and the spatiotemporal semantics of the image.

[0030] In this embodiment, the dynamic gesture categories include eight basic movements: moving up, moving down, moving left, moving right, swiping up from left to right, swiping down from left to right, swiping up from right to left, and swiping down from right to left. During data acquisition, the subject's body remains parallel to the ultrasound array plane, and the hand performs the movement within the field of view directly in front of the array.

[0031] In one experiment, dynamic gesture data collection recruited three subjects, each performing each gesture 100 times, resulting in 2400 dynamic gesture samples. After removing invalid samples with incorrect movements or severely distorted signals, approximately 2000 valid samples were retained. This dataset was divided into training, validation, and test sets for training and evaluating deep learning classification models.

[0032] The deep learning model preferably uses the ResNet-50 transfer learning network. During training, a ResNet-50 pre-trained on a general image dataset is loaded, its convolutional feature extraction layers are retained and the pre-trained weights are frozen, and only the last fully connected layer and classification layer are replaced to make the number of output nodes consistent with the number of dynamic gesture categories. To accelerate the convergence of task-specific layers, the learning rate factor of the new fully connected layer and classification layer can be increased.

[0033] Because ultrasound dynamic gesture images have clear spatial and temporal semantics, data augmentation should be constrained by physical meaning. Slight rotation within ±1 degree, slight translation within ±2 pixels, amplitude scaling, or noise perturbation can be used to simulate minute angular deviations, hand position shifts, and environmental noise during gesture acquisition; augmentation methods such as horizontal flipping and vertical flipping, which would change the meaning of the action direction, should not be used.

[0034] After training, the test set is input into the model to obtain the confusion matrix. In actual experiments, the overall classification accuracy of the model based on ResNet-50 transfer learning reached 98.39%. The basic orthogonal translation gestures showed high recognition stability, with a small number of misclassifications mainly occurring between oblique gestures with similar motion patterns. These results demonstrate that the hardware link and dynamic amplitude imaging method of this invention can provide effective features for deep learning models.

[0035] In one training implementation, the samples can be divided into training, validation, and test sets in a 7:1.5:1.5 ratio. The optimizer can use stochastic gradient descent with momentum, with an initial learning rate of 0.001. During training, piecewise learning rate decay is used, with a maximum of 15 training epochs, a batch size of 8, and a momentum coefficient of 0.9. The training data is shuffled before each training epoch to reduce the impact of sample order on model convergence.

[0036] The model outputs a probability distribution of eight types of dynamic gestures. The system can use the category with the highest probability as the recognition result and the highest probability as the confidence level. When the highest confidence level is lower than a preset threshold, or when two consecutive recognition results are inconsistent, the system can output an unknown gesture, request resampling, or enter a confirmation state to reduce false triggers.

[0037] The imaging textures of diagonal swipe gestures are relatively similar, making them prone to intra-class variance due to differences in user movement angle and speed. To further improve the stability of diagonal gesture recognition, the diversity of diagonal gesture samples can be increased without changing the core concept of this invention, or a lightweight temporal feature fusion module can be added after ResNet-50 features. However, the basic embodiments of this invention do not rely on 3DCNN-LSTM or complex three-dimensional temporal networks.

[0038] Once trained, the model can be deployed on a host computer, embedded processor, mobile terminal, or edge computing device. If the deployment platform has limited resources, the model can be pruned, quantized, distilled, or fused with operators to reduce storage consumption and inference time. During actual interaction, the system can first monitor candidate actions at a low sampling rate, and then enter a high sampling rate imaging and recognition state after detecting candidate dynamic gestures, thereby reducing power consumption.

[0039] Example 2 refer to Figure 1 and Figure 2 The gesture recognition system based on ultrasonic array and deep learning described in this invention includes a gesture recognition module and a deep learning recognition module. The gesture recognition module includes a microcontroller, a transmit boost drive circuit, an ultrasonic array, a receive conditioning circuit, an analog-to-digital conversion module, and an amplitude imaging module.

[0040] The ultrasonic array adopts an eight-transmitter, one-receiver circular array, specifically including eight PZT piezoelectric ceramic transmitting transducers and one PZT receiving transducer. The eight PZT piezoelectric ceramic transmitting transducers are evenly distributed along the circumference, and the one PZT receiving transducer is located in the central region. The center frequency of each PZT piezoelectric ceramic transmitting transducer is 40kHz, which has a good balance between air propagation attenuation and spatial resolution. Under normal conditions of 25 degrees Celsius, the length of a 40kHz ultrasonic wave in the air is approximately 8.65mm, which can meet the requirements for detecting close-range hand gestures.

[0041] Compared to a single transducer, the array structure expands the effective illumination range; compared to a linear array, a circular array avoids the problem of forming a high sound pressure band only along the array axis; compared to a square array, a circular array has better sound field consistency in the edge direction and the 45-degree deflection direction. Since this embodiment uses a single PZT receiving transducer, it does not rely on the phase difference between multiple receiving channels, but instead utilizes the differences in echo amplitude distribution caused by multiple transmission directions to construct gesture characteristics.

[0042] Eight PZT piezoelectric ceramic transmitting transducers can transmit synchronously to improve echo energy. If space is limited, the PZT piezoelectric ceramic transmitting transducers can also be arranged in an approximately circular, polygonal, or non-completely closed circular pattern, but the receiver still maintains a single receiving channel to ensure a simple system structure.

[0043] The electrical behavior of a PZT piezoelectric ceramic transmitting transducer can be approximated by a Butterworth-Van Dyke equivalent circuit, where the static capacitance represents the inherent capacitance between the piezoelectric ceramic electrodes, and the dynamic inductance, dynamic capacitance, and dynamic resistance represent the mechanical vibration branch. This equivalent model indicates that the transmitting end requires a sufficient pulse drive amplitude to excite resonance, while the receiving end requires high input impedance and a suitable amplification and filtering link to avoid the weak echo being attenuated by the load effect.

[0044] The radius of the circular array can be determined based on the terminal casing size and recognition distance. If the radius is too small, the propagation difference between each transmission channel and the target area will be insufficient, and the directional characteristics in dynamic gesture imaging will be unclear; if the radius is too large, the terminal size will increase, and the mechanical installation error from each transducer to the central receiving transducer will increase. Therefore, in actual products, the circumference radius can be determined jointly through sound field simulation and standard reflective target testing.

[0045] During array installation, the PZT piezoelectric ceramic transmitting transducer and the PZT receiving transducer can be located on the same PCB or on the same mechanical bracket, with the opening at the transmitting end facing the gesture area. The housing openings, protective mesh, or decorative panels should not obstruct the effective receiving direction of the PZT receiving transducer; if a housing structure is necessary, the fixed attenuation caused by this structure can be recorded during the calibration phase and corrected using channel gain compensation.

[0046] According to the principle of ultrasonic time-of-flight, the farther away the hand gesture target is, the later the echo arrives and the smaller its amplitude. The system can estimate the effective time window based on the propagation relationship t=2d / c, where d is the target distance and c is the air speed of sound. For a recognition range of 10cm to 100cm, the intercept position of the effective time window should avoid the transmission ringing and reflections from near-end structures, while covering the main reflection area of ​​the hand.

[0047] The microcontroller's internal timer outputs a PWM pulse sequence with a frequency of 40kHz, an adjustable pulse count (n), and an amplitude of approximately 3.3V. Because the PZT piezoelectric transducer is a capacitive load, direct low-voltage pulse drive results in low emitted sound pressure, making it difficult to stably detect echoes from distant or weakly reflected signals. Therefore, a transmission boost drive circuit is placed between the microcontroller and the ultrasonic transmitting array.

[0048] The transmitter boost drive circuit uses a MAX232 class level converter chip or a boost chip with an equivalent charge pump structure. This circuit is powered by +5V. C10 is used for power supply decoupling, C6 and C7 are flying capacitors for the charge pump, and C8 and C9 are positive and negative voltage storage capacitors, respectively. The chip internally includes a +5V to +10V voltage multiplier unit and a +10V to -10V voltage inverting unit. The microcontroller outputs IN1 and IN2 are connected to T1IN and T2IN, respectively. After conversion by the internal driver, they are output from T1OUT and T2OUT to OUT1 and OUT2, thereby driving the ultrasonic transmitter array or the transmitter channel selection circuit.

[0049] In the experiment, the peak-to-peak value of the boosted transmit pulse reached approximately 13V, nearly four times higher than the original output of the microcontroller. The boosted pulse not only enhanced the mechanical vibration amplitude of the PZT transducer but also increased the amplitude of the reflected echo from the gesture area, making dynamic gesture imaging more stable within a range of 10cm to longer distances. Since the piezoelectric transducer experiences approximately 5 to 10 cycles of ringing after the pulse ends, the number of pulses should not be increased indefinitely. For close-range recognition, the number of pulses should be kept away from allowing non-ideal front-end echoes to spread into the effective echo area. For long-range recognition, the number of pulses can be appropriately increased to enhance the echo energy.

[0050] The weak analog electrical signal output from the PZT receiving transducer first enters the receiving conditioning circuit instead of directly entering the analog-to-digital converter (ADC). The output of the receiving conditioning circuit is used to connect to the ADC. The input terminal IN of the receiving conditioning circuit is connected to the PZT receiving transducer. R1 and C1 form an input current limiting and AC coupling network, which then enters the first-stage inverting preamplifier. The first-stage feedback resistor R6 determines the initial gain.

[0051] The preamplified signal enters the second-stage multiple-feedback active bandpass filter through R2. This stage, composed of R2, R3, R4, C2, and C3, forms a frequency-selective network with a center frequency around 40kHz. It provides high gain at 40kHz while suppressing low-frequency disturbances, out-of-band ambient noise, and high-frequency electromagnetic interference. Compared to simple passive filters, the multiple-feedback active filter structure has a higher Q value and is more suitable for enhancing the effective frequency band of weak echoes in a single receiving channel.

[0052] The filtered signal is AC-coupled to the third-stage inverting amplifier via R5 and C4, with feedback resistor R7 used to set the final stage gain. Through this three-stage cascaded structure, the receiver conditioning circuit can amplify the millivolt-level 40kHz echo into an analog voltage suitable for subsequent sampling. In the experiment, frequency domain analysis showed a significant gain peak at 40kHz, verifying the frequency selection and amplification effect of the receiver conditioning circuit.

[0053] In this embodiment, the transmit boost drive circuit and the receive conditioning circuit work together to ensure the data quality of dynamic gesture amplitude imaging. The former improves the transmitted sound pressure and echo detectability, while the latter enhances the effective echo signal-to-noise ratio under single-receiver channel conditions, providing a stable input for subsequent amplitude extraction, normalization, and deep learning recognition.

[0054] The first-stage amplifier is positioned close to the PZT receiving transducer to shorten the trace for weak signals and reduce external electromagnetic interference. Input coupling capacitor C1 isolates the DC bias, and R1 limits the input current and matches the input impedance. The preamplifier stage should not be set with excessively high gain to prevent near-end echo amplitude saturation; the main frequency selection and gain enhancement can be handled jointly by the second-stage MFB filter and the final stage amplification.

[0055] The frequency of the transmitted pulse and the center frequency of the MFB bandpass filter stage can be finely adjusted based on the measured resonant frequency of the selected PZT receiving transducer. If the nominal frequency of the PZT receiving transducer is 40kHz, but it deviates due to the influence of the casing, mounting adhesive, and temperature and humidity, the frequency of the transmitted pulse can be changed through MCU code. The bandpass filter circuit can adjust the parameters of R2, R3, R4, C2, or C3 to make the gain peak fall near the actual effective frequency. This parameter adjustment is an equivalent implementation of the receiving conditioning circuit of this invention.

[0056] During operation, the microcontroller controls the transmission array by emitting n 40kHz square pulses every 0.1s, where n is an adjustable number of pulses. Increasing the number of pulses enhances the echo amplitude but does not change the basic waveform variation. Experiments show that when the number of pulses is too high, transducer ringing and non-ideal front-end echoes can affect the effective echo zone at close range. Therefore, in scenarios where operation is required at a close range of approximately 10cm, a suitable compromise is to select, for example, a number of 8 pulses. When longer-distance detection is required, the number of pulses can be appropriately increased to enhance the echo signal.

[0057] The system determines the effective echo window based on the speed of sound and a preset recognition distance. The speed of sound in air can be corrected according to the ambient temperature, and the parameter can be calibrated at approximately 346 m / s around 25 degrees Celsius. During imaging, the entire original waveform is not used; instead, the envelope amplitude is extracted within the effective time window, and the differences in flight time and amplitude caused by gesture reflection are converted into image features.

[0058] The core of dynamic gesture imaging is to convert continuous one-dimensional echo time-series signals into two-dimensional grayscale images with spatiotemporal continuity. The imaging algorithm independently extracts the amplitude of each frame of echo data, and then stitches the continuous single-frame grayscale results in chronological order, so that each pixel position corresponds to the gesture action time and sound wave flight time, and each pixel grayscale corresponds to the echo amplitude at the corresponding position. Different dynamic gestures will cause differences in the position, trajectory, texture, and grayscale change patterns of bright spots.

[0059] To reduce the absolute amplitude difference caused by distance, the system performs channel-level normalization on the spectrogram, ensuring that the grayscale values ​​fall within the [0,1] range. This process avoids the deep learning model from overfitting the absolute amplitude at a fixed distance, thus improving the model's generalization ability under different distances and user action amplitudes.

[0060] In a two-dimensional grayscale image, the horizontal coordinate represents the temporal sequence during dynamic gesture capture, while the vertical coordinate represents the distance and position calculated from the time-of-flight. The grayscale value of a pixel is determined by the echo amplitude at the corresponding sampling location. When the hand slides up, down, left, right, or diagonally, the position and texture direction of the echo bright spots in the image differ, thus forming class differences that can be learned by convolutional networks.

[0061] Compared to directly inputting the raw one-dimensional temporal signal into a classifier, amplitude imaging can concentrate the spatial location and temporal variation information of dynamic gestures into a standardized image. This image retains both the distance information from the flight time and the amplitude variation trend during the action, thus allowing the reuse of mature image classification networks and transfer learning strategies.

[0062] During image generation, invalid front-end echoes, reflections from fixed structures, and tail samples exceeding the recognition distance can be cropped. If the ambient noise is strong, background signals can be acquired first in a gesture-free state to establish a background amplitude template. During imaging, the background template can be subtracted to reduce the impact of fixed noise on grayscale texture.

[0063] This invention has the following characteristics: This invention employs a circular eight-transmitter, one-receiver ultrasonic array structure, with eight ultrasonic transmitting transducers distributed along the circumference and one ultrasonic receiving transducer positioned in the central region. Compared to a single transducer, linear arrays, or square arrays, this structure can create more balanced multi-directional ultrasonic coverage within the gesture area. By leveraging multiple transmission directions, echo amplitude information with directional differences can be obtained using only a single receiving transducer, thereby improving the separability of dynamic gesture features and reducing reliance on phase difference measurements from multiple receiving channels.

[0064] This invention incorporates a boost drive circuit between the microcontroller and the ultrasonic transmitting array, converting the low-voltage PWM pulses output by the microcontroller into higher-amplitude transmission drive pulses. Since the acoustic pressure output of the PZT piezoelectric transducer is correlated with the drive voltage, the boosted drive signal effectively enhances the transmitted acoustic pressure, improving the propagation capability of ultrasonic waves in the air and the echo amplitude after reflection from the hand. This design addresses the problems of insufficient transmission power, weak echoes at long distances, and low signal-to-noise ratio in low-voltage direct drive, thus extending the system's effective detection range and providing a more stable raw signal for subsequent amplitude extraction and dynamic gesture imaging.

[0065] This invention employs a three-stage cascaded receiver conditioning circuit to perform pre-amplification, active bandpass filtering, and final-stage amplification on the weak echo signal output from the central receiving transducer. The pre-amplification stage enhances the initial amplitude of the weak echo; the multi-feedback active bandpass filter stage amplifies the effective signal near the transducer's operating frequency and suppresses environmental noise and high-frequency electromagnetic interference; and the final-stage amplification stage further increases the output signal amplitude, making it suitable for subsequent analog-to-digital conversion and amplitude calculation. This receiver conditioning link can improve the echo signal-to-noise ratio and reduce the impact of out-of-band noise on dynamic gesture images under single-channel conditions.

[0066] This invention converts continuous one-dimensional ultrasonic echo time-series signals into two-dimensional grayscale images with spatiotemporal characteristics. Utilizing the correspondence between time-of-flight, sampling time, and echo amplitude, it represents image differences in the location, trajectory direction, texture morphology, and grayscale variations caused by different dynamic gestures. Compared to directly using the original waveform or manually extracting a limited number of features for classification, amplitude imaging can more completely preserve the spatial position changes and temporal evolution of dynamic gestures, enabling deep learning models to automatically learn more discriminative image features.

[0067] This invention combines dynamic gesture amplitude images with a transfer learning model, preferably employing a ResNet-50-based deep learning recognition model. It reuses the feature extraction capabilities of pre-trained networks even with limited sample size, and achieves rapid adaptation to ultrasound gesture images by replacing the classification layer, freezing some base layers, and fine-tuning the training. Since ultrasound gesture images possess clear spatial and temporal semantics, a restricted data augmentation strategy is used during training to avoid augmentation methods that alter the meaning of the action direction, such as flipping. This improves the rationality and stability of model training, reduces the amount of data and computational cost required to train a deep network from scratch, and enhances the model's generalization ability to different users, different action speeds, and different acquisition environments.

[0068] Example 3 A computer device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the steps of the gesture recognition method based on an ultrasonic array and deep learning. For example, the steps include: acquiring user pressure information via an ultrasonic array; converting the pressure information into an echo signal; preprocessing the echo signal to obtain a simulated echo signal; converting the simulated echo signal into a two-dimensional grayscale image with spatiotemporal continuity; inputting the two-dimensional grayscale image with spatiotemporal continuity into a trained recognition model to obtain the probabilities of various dynamic gestures; and selecting the dynamic gesture with the highest probability as the final result. The memory may include main memory, such as high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device. The processor, network interface, and memory are interconnected via an internal bus, which may be an industry-standard architecture bus, a peripheral component interconnection standard bus, an extended industry-standard architecture bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. The memory is used to store the program; specifically, the program may include program code, which includes computer operation instructions. Memory can include main memory and non-volatile memory, and provides instructions and data to the processor.

[0069] Example 4 A computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of the gesture recognition method based on an ultrasonic array and deep learning. For example, the method includes: acquiring user pressure information via an ultrasonic array; converting the pressure information into an echo signal; preprocessing the echo signal to obtain a simulated echo signal; converting the simulated echo signal into a two-dimensional grayscale image with spatiotemporal continuity; inputting the two-dimensional grayscale image with spatiotemporal continuity into a trained recognition model to obtain the probabilities of various dynamic gestures; and selecting the dynamic gesture with the highest probability as the final result. Specifically, the computer-readable storage medium includes, but is not limited to, volatile memory and / or non-volatile memory. The volatile memory may include random access memory (RAM) and / or cache memory, etc. The non-volatile memory may include read-only memory (ROM), hard disk, flash memory, optical disk, magnetic disk, etc.

[0070] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0071] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0072] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0073] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0074] Other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and disclosure of the invention. This application is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of the invention are indicated by the following claims.

[0075] It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

[0076] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A gesture recognition method based on ultrasonic array and deep learning, characterized in that, include: The user's pressure information is acquired through an ultrasonic array and converted into an echo signal; The echo signal is preprocessed to obtain a simulated echo signal; Convert the analog echo signal into a two-dimensional grayscale image with spatiotemporal continuity; The two-dimensional grayscale image with spatiotemporal continuity is input into the trained recognition model to obtain the probability of various dynamic gestures, and the dynamic gesture with the highest probability is taken as the final result.

2. The gesture recognition method based on ultrasonic array and deep learning according to claim 1, characterized in that, The ultrasonic array includes eight PZT piezoelectric ceramic transmitting transducers and one PZT receiving transducer. The eight PZT piezoelectric ceramic transmitting transducers are evenly distributed along the circumference, and the one PZT receiving transducer is located in the central region. The PZT receiving transducer outputs an analog electrical signal, which is then conditioned to obtain a one-dimensional echo timing signal.

3. The gesture recognition method based on ultrasonic array and deep learning according to claim 1, characterized in that, The categories of dynamic gestures include moving up, moving down, moving left, moving right, swiping up from left to right, swiping down from left to right, swiping up from right to left, and swiping down from right to left.

4. The gesture recognition method based on ultrasonic array and deep learning according to claim 1, characterized in that, The preprocessing process for the echo signal is as follows: The echo signal is pre-amplified, bandpass filtered near 40kHz, and amplified in the final stage before being output as a conditioned analog echo signal.

5. The gesture recognition method based on ultrasonic array and deep learning according to claim 1, characterized in that, The process of converting the analog echo signal into a two-dimensional grayscale image with spatiotemporal continuity is as follows: the conditioned analog echo signal is subjected to analog-to-digital conversion, amplitude extraction, time window segmentation and normalization processing, and a two-dimensional grayscale image is generated for dynamic gestures to characterize the movement direction, speed or trajectory changes of the gesture in front of the array.

6. The gesture recognition method based on ultrasonic array and deep learning according to claim 1, characterized in that, The recognition model includes convolutional neural networks, residual networks, and lightweight convolutional networks. A transfer learning model based on ResNet-50 is used to classify ultrasound amplitude images. The model convergence is achieved by freezing some base layers, replacing classification layers, and small-sample augmentation training. During training, amplitude scaling, slight rotation, slight translation, noise perturbation, or time clipping are applied to the training samples to achieve data augmentation. The data augmentation is constrained by the array physical coordinates and the spatiotemporal semantics of the images.

7. A gesture recognition system based on ultrasonic array and deep learning, characterized in that, include: An ultrasonic array is used to acquire the user's pressure information and convert the pressure information into an echo signal; A receiving conditioning circuit is used to preprocess the echo signal to obtain a simulated echo signal; Amplitude imaging module, used to convert analog echo signals into two-dimensional grayscale images with spatiotemporal continuity; A microcontroller is used to input the spatiotemporally continuous two-dimensional grayscale image into the trained recognition model to obtain the probability of various dynamic gestures, and to take the dynamic gesture with the highest probability as the identification result. The microcontroller is connected to the ultrasonic array, which is connected to the microcontroller via a receiving conditioning circuit and an amplitude imaging module.

8. The gesture recognition system based on ultrasonic array and deep learning according to claim 7, characterized in that, A transmission boost drive circuit is provided between the microcontroller and the ultrasonic transmitting array.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the gesture recognition method based on ultrasonic array and deep learning as described in any one of claims 1-6.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the gesture recognition method based on ultrasonic array and deep learning as described in any one of claims 1-6.

Images