Ultrasonic host control method and device based on gesture recognition, equipment and medium

Abstract

The embodiment of the invention discloses an ultrasonic host control method and device based on gesture recognition, equipment and a medium, and relates to the technical field of intelligent control, and the method comprises the steps: collecting a reflection signal of a hand of a user through a preset millimeter wave radar sensor; processing the reflected signal, and extracting motion feature information of the hand of the user; based on the motion feature information, recognizing a gesture of a user through a pre-trained gesture recognition model; and controlling the ultrasonic host to execute corresponding operation according to the recognized gesture. The system reliability is guaranteed by means of the environmental adaptability of the millimeter wave radar, the gesture recognition precision is guaranteed through multi-dimensional motion feature analysis, comprehensive control is achieved by means of extensible gesture function mapping, the problems of cross infection risk and inconvenient operation existing in traditional contact operation are effectively solved, and the system reliability is improved. And the health safety and the man-machine interaction efficiency of the ultrasonic host in a medical scene are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of intelligent control technology, and in particular to an ultrasonic host control method, device, equipment and medium based on gesture recognition. Background Technology

[0002] Currently, operating ultrasound machines primarily relies on medical staff directly touching physical buttons, knobs, and other control components. This contact-based operation method has significant drawbacks in medical environments such as hospitals. First, frequent contact increases the risk of cross-infection between patients and medical staff, especially during periods of high infectious disease prevalence. Second, in special scenarios such as surgery, medical staff's hands are often occupied or need to be kept sterile. If they must touch the equipment to operate it, it not only causes inconvenience but may also affect surgical efficiency and overall safety.

[0003] To overcome the drawbacks of contact-based operation, some non-contact control technologies have been attempted for application in the medical device field. However, these existing technologies still have many shortcomings in practical applications. First, their gesture recognition accuracy is low, making them prone to misinterpretation and resulting in inaccurate execution of control commands. Furthermore, these technologies can only recognize a limited variety of gestures, making it difficult to meet the control needs of various complex functions in clinical settings. Moreover, they are easily affected by unconscious movements or the movement of other objects in the environment, leading to frequent false triggering, which seriously affects the normal use and stability of the equipment.

[0004] It is evident that existing technologies have not yet provided a non-contact control solution that can balance recognition accuracy, operational reliability, and comprehensive functionality. This has become a pressing technical problem to be solved in the human-machine interaction of current ultrasound mainframes. Summary of the Invention

[0005] This invention provides a method, apparatus, device, and medium for controlling an ultrasonic host based on gesture recognition, aiming to solve the problem that the prior art has not yet provided a non-contact control scheme that can balance recognition accuracy, operational reliability, and comprehensive functionality.

[0006] In a first aspect, embodiments of the present invention provide an ultrasound host control method based on gesture recognition, comprising: The reflected signals from the user's hand are collected by a pre-set millimeter-wave radar sensor; The reflected signal is processed to extract the motion feature information of the user's hand; Based on the motion feature information, the user's gestures are identified through a pre-trained gesture recognition model; The ultrasound host is controlled to perform corresponding operations based on the recognized gestures.

[0007] Optionally, processing the reflected signal to extract motion feature information of the user's hand includes: A two-dimensional fast Fourier transform is performed on the reflected signal to obtain a distance-Doppler heatmap; The position information of the user's hand in three-dimensional space and the average Doppler velocity are extracted from the distance-Doppler heat map; Based on the position information of continuous time frames, the motion velocity vector of the user's hand is obtained by differential calculation, and the motion velocity vector includes the velocity magnitude and direction; Based on the location information in continuous time frames, the movement trajectory of the user's hand is determined by connecting the spatial points corresponding to the location information, and the trajectory information of the movement trajectory is extracted.

[0008] Optionally, recognizing the user's gestures using a pre-trained gesture recognition model based on the motion feature information includes: The motion velocity vectors at different time points are synthesized to obtain the overall motion vector of the hand. The overall motion vector and the motion feature information are input into a pre-trained gesture recognition model for gesture classification, and the recognition result of the user's gesture is output. The motion feature information includes at least one of the motion velocity vector, the trajectory information, the position information, and the average Doppler velocity.

[0009] Optionally, controlling the ultrasound host to perform corresponding operations based on the recognized gestures includes: In response to the recognition of a preset unlock gesture, the ultrasound host is switched from the gesture lock state to the gesture unlock state; In the gesture unlock state, in response to recognizing a preset function control gesture, a corresponding function control command is generated and executed; In response to the recognition of a preset locking gesture, the ultrasound host is switched from the gesture unlock state to the gesture lock state.

[0010] Optionally, the method further includes: When preset conditions are met, the ultrasound host will automatically switch from the gesture unlock state back to the gesture lock state.

[0011] Optionally, at least one global gesture that is not restricted by the gesture lock state is predefined; the step of controlling the ultrasound host to perform the corresponding operation based on the recognized gesture further includes: In response to the recognition of the global gesture, regardless of whether the ultrasound host is currently in the gesture locked state or the gesture unlocked state, a corresponding control command is generated and executed.

[0012] Optionally, the control commands corresponding to the global gesture include commands to clear fault information and / or commands to enter the frequency sweep interface.

[0013] Secondly, embodiments of the present invention also provide an ultrasound host control device based on gesture recognition, which includes a unit for performing the above-described method.

[0014] Thirdly, embodiments of the present invention also provide a computer device, which includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the above-described method.

[0015] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the above-described method.

[0016] This invention provides a method, device, equipment, and medium for controlling an ultrasound host based on gesture recognition. The method includes: acquiring reflected signals from a user's hand using a preset millimeter-wave radar sensor; processing the reflected signals to extract motion feature information of the user's hand; recognizing the user's gesture based on the motion feature information using a pre-trained gesture recognition model; and controlling the ultrasound host to perform corresponding operations according to the recognized gesture. This invention ensures system reliability through the environmental adaptability of millimeter-wave radar, guarantees gesture recognition accuracy through multi-dimensional motion feature analysis, and achieves comprehensive control through scalable gesture function mapping. It effectively solves the cross-infection risks and operational inconveniences of traditional contact operations, significantly improving the hygiene and safety and human-computer interaction efficiency of the ultrasound host in medical scenarios. Attached Figure Description

[0017] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the following description of the embodiments will be briefly introduced. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 A flowchart illustrating an ultrasound host control method based on gesture recognition provided in an embodiment of the present invention; Figure 2 This is a schematic block diagram of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0021] It should also be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the invention. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0022] It should also be further understood that the term "and / or" as used in this specification and the appended claims refers to any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0023] As used in this specification and the appended claims, the term "if" may be interpreted, depending on the context, as "when," "once," "in response to determination," or "in response to detection." Similarly, the phrase "if determined" or "if [described condition or event] is detected" may be interpreted, depending on the context, as "once determined," "in response to determination," "once [described condition or event] is detected," or "in response to detection of [described condition or event]."

[0024] Please see Figure 1 This invention proposes a gesture recognition-based ultrasonic host control method, which includes the following steps: S1 collects reflected signals from the user's hand using a pre-set millimeter-wave radar sensor.

[0025] In practical implementation, the use of a pre-set millimeter-wave radar sensor to collect reflected signals from the user's hand lays the physical foundation for the high reliability of the entire solution. As a non-optical sensing method, millimeter-wave radar's detection is independent of ambient lighting conditions, allowing it to operate stably in both brightly lit and dimly lit operating rooms. Furthermore, it can penetrate non-metallic obstructions such as thin layers of smoke or sterile drapes, ensuring the continuity and robustness of signal acquisition in complex medical scenarios. This contrasts sharply with optical camera solutions, which are susceptible to changes in lighting and obstructions, fundamentally guaranteeing the system's always-available nature.

[0026] S2, process the reflected signal to extract the motion feature information of the user's hand.

[0027] In practice, processing the collected reflected signals and extracting the motion feature information of the user's hand is the core step in improving the accuracy of gesture recognition. The raw reflected signals are inherently chaotic and unusable directly; they must undergo specialized processing to extract key parameters that characterize the essence of hand movements. This processing typically involves complex signal processing algorithms, such as using a two-dimensional Fast Fourier Transform to convert the time-domain signal to the frequency domain, thereby resolving the distance and velocity information of the hand target. The extracted motion feature information is no longer simply image pixels. It includes deeper physical quantities such as the precise position of the hand in three-dimensional space, instantaneous velocity vectors, and continuous motion trajectories. These quantified features provide a rich, accurate, and low-noise data foundation for subsequent intelligent recognition, enabling the system to go beyond simple appearance recognition and understand the dynamic characteristics of hand movements. This is a crucial prerequisite for achieving high-precision gesture recognition.

[0028] In some preferred embodiments, the above step "processing the reflected signal to extract motion feature information of the user's hand" specifically includes the following steps: performing a two-dimensional fast Fourier transform on the reflected signal to obtain a distance-Doppler heatmap; extracting the position information of the user's hand in three-dimensional space and the average Doppler velocity from the distance-Doppler heatmap; calculating the motion velocity vector of the user's hand based on the position information of continuous time frames through difference calculation, the motion velocity vector including velocity magnitude and direction; determining the motion trajectory of the user's hand by connecting the spatial points corresponding to the position information based on the position information of continuous time frames, and extracting the trajectory information of the motion trajectory.

[0029] In practice, the extraction process of motion feature information includes generating a distance-Doppler heatmap and extracting hand position information, average Doppler velocity, motion velocity vector, and motion trajectory information from it. The synergistic effect of these features greatly improves the accuracy and richness of gesture recognition.

[0030] The key step in signal processing is to perform a two-dimensional fast Fourier transform on the reflected signal to obtain a range-Doppler heatmap. This transforms the original time-domain signal into range and velocity dimensions, thus clearly separating targets at different distances and radial velocities, providing a data foundation for accurate positioning and velocity measurement.

[0031] Furthermore, the positional information of the hand in three-dimensional space is extracted from the heatmap, enabling the system to accurately perceive the spatial coordinates of the hand. This is the foundation for recognizing the starting point of static and dynamic gestures. The calculated motion velocity vector, containing both magnitude and direction, quantitatively describes the instantaneous motion state of the hand in space, allowing the system to accurately distinguish between similar but different actions, such as rapid swiping and slow movement. The mean Doppler velocity provides a scalar representation of the hand's radial velocity relative to the radar, complementing the motion velocity vector and further enhancing the description of motion patterns.

[0032] Furthermore, the motion trajectory and its features (such as length and curvature) determined based on continuous time-point location information enable the system to understand the path and shape of the gesture from a macroscopic perspective. This is crucial for recognizing complex trajectory gestures such as drawing circles and rotating. In summary, by extracting this series of multi-level and multi-dimensional motion feature information, the system constructs a feature set that comprehensively describes the hand movement state, providing rich and high-quality input data for subsequent gesture recognition models, thus laying a solid foundation for high-precision and high-reliability gesture classification.

[0033] Specifically, in one embodiment, the above-mentioned motion feature information is extracted in the following way: First, the position information of the user's hand in three-dimensional space is extracted from the range-Doppler heatmap. By detecting the energy peaks in the range-Doppler heatmap, one or more energy concentration regions corresponding to the user's hand are identified, each region being jointly identified by specific range cells and Doppler cells. Based on the radial distance represented by the range cells and the angle of arrival information calculated from the phase difference of the received signals between each antenna channel of the millimeter-wave radar sensor receiving antenna array, the coordinate position of the user's hand in three-dimensional space is comprehensively calculated.

[0034] Further, the average Doppler velocity of the user's hand is extracted. In the distance-Doppler heatmap, multiple cells associated with the user's hand are identified, and the signal amplitude values ​​of these cells and the radial velocity values ​​mapped to their respective Doppler cells are obtained. The radial velocity values ​​of each cell are weighted and averaged using their signal amplitude values ​​as weights to calculate the average Doppler velocity of the user's hand.

[0035] Further, based on the position information at different time points, the motion velocity vector of the user's hand is calculated. The three-dimensional spatial position information of the hand is continuously acquired across multiple time frames, and the positional change between adjacent time frames is differentially calculated to obtain the displacement difference of the hand along each coordinate axis in three-dimensional space. Dividing the displacement difference by the corresponding time interval yields the instantaneous motion velocity vector of the hand within that time interval. This motion velocity vector includes the magnitude of the velocity and the direction indicated by the components along the three axes. Optionally, a smoothing filter is applied to the instantaneous velocity across multiple consecutive time frames to suppress measurement noise and obtain a more robust motion velocity vector estimate.

[0036] Furthermore, based on the location information at different time points, the motion trajectory of the user's hand is determined, and the trajectory information is extracted. A series of consecutive time frames are connected sequentially to form the hand's three-dimensional spatial location points, i.e., the motion trajectory. Curve fitting is performed on the motion trajectory, and feature information characterizing the trajectory morphology is extracted based on the fitted trajectory curve. This trajectory information includes, but is not limited to, the total length of the trajectory, the curvature variation characteristics of the trajectory, the main direction of the trajectory, and the geometric shape category presented by the trajectory.

[0037] S3, Based on the motion feature information, the user's gestures are recognized by a pre-trained gesture recognition model.

[0038] In practice, based on the extracted motion feature information, a pre-trained gesture recognition model identifies the user's gestures. This step transforms the data prepared in the previous steps into specific operational intentions. The pre-trained gesture recognition model, such as a deep neural network or support vector machine trained on a large amount of gesture data, possesses powerful nonlinear mapping and pattern recognition capabilities. It can learn and memorize the complex correspondence between various gestures and their feature information. When new feature data is input, the gesture recognition model can quickly compare, analyze, and classify it, outputting the specific gesture category. For example, the gesture recognition model can accurately distinguish between upward and downward swipes, even though their trajectories are similar but their directions are opposite; this is achieved by analyzing the directional features of the motion velocity vector. This AI-based recognition method not only recognizes various gestures, meeting the complex control requirements of the ultrasound host, but also possesses strong anti-interference capabilities and adaptability, filtering out unintentional gesture jitters or execution deviations caused by individual differences, thus ensuring the accuracy and stability of the recognition results.

[0039] In some preferred embodiments, the above step "based on the motion feature information, identify the user's gesture through a pre-trained gesture recognition model" specifically includes the following steps: synthesizing the motion velocity vectors at different time points to obtain the overall motion vector of the hand; inputting the overall motion vector and the motion feature information into the pre-trained gesture recognition model for gesture classification, and outputting the recognition result of the user's gesture, wherein the motion feature information includes at least one of the motion velocity vector, the trajectory information, the position information, and the average Doppler velocity.

[0040] In practice, the gesture recognition process involves synthesizing motion velocity vectors at different time points to obtain an overall motion vector. This overall motion vector, along with other motion feature information, is then input into the model for gesture classification, significantly improving the accuracy and robustness of gesture recognition, especially dynamic gesture recognition. Synthesizing motion velocity vectors at different time points essentially involves temporal integration and summarization of the hand movement process, enabling the capture of the overall trend and macroscopic intent of the hand movements.

[0041] For example, a "waving" gesture may consist of multiple tiny instantaneous velocity vectors, each with slightly fluctuating directions. However, the synthesized overall motion vector clearly shows its dominant direction of motion, effectively filtering out unavoidable minor jitters or unintentional tremors during the gesture's execution and reducing the interference of these noises on the recognition results. This recognition method based on overall motion vectors allows the system to focus more on the global semantics of the gesture rather than local details, thereby improving its tolerance for differences in the execution of the same gesture by different individuals.

[0042] Furthermore, the synthesized overall motion vector, along with motion feature information (such as position and trajectory), is input into the pre-trained gesture recognition model, achieving fusion judgment of motion information. The model can simultaneously utilize instantaneous detailed features and macroscopic trend features for comprehensive decision-making. For example, it can combine trajectory shape and overall motion direction to accurately distinguish between "swiping left" and "swiping right," or combine position changes and velocity synthesis results to identify specific gesture sequences. This multi-feature fusion recognition strategy greatly enhances the model's understanding and discrimination capabilities, making gesture recognition results more accurate and reliable, effectively avoiding misjudgments and omissions, and ensuring that the ultrasound host can accurately respond to the operator's intentions.

[0043] S4, based on the recognized gesture, control the ultrasound host to perform the corresponding operation.

[0044] In practice, the ultrasound host is controlled to perform corresponding operations based on the recognized gestures. This step realizes the final conversion from interactive intent to device behavior, demonstrating the comprehensiveness of the solution's functions and operational reliability. By pre-defining a set of intuitive mapping relationships between gestures and host functions, such as waving the palm to switch modes and rotating the wrist to adjust parameters, users can achieve precise control of the host through natural body language. This contactless operation mode completely eliminates the risk of cross-infection caused by touching physical buttons or touchscreens, which is of great significance for maintaining a sterile environment in the operating room and protecting the health and safety of both medical staff and patients. At the same time, when hands are occupied or need to be kept sterile, medical staff can remotely control the host without interrupting their current work, greatly improving the smoothness and efficiency of the operation process.

[0045] In some preferred embodiments, the above step "controlling the ultrasound host to perform corresponding operations according to the recognized gesture" specifically includes the following steps: in response to recognizing a preset unlock gesture, switching the ultrasound host from a gesture-locked state to a gesture-unlocked state; in the gesture-unlocked state, in response to recognizing a preset function control gesture, generating and executing a corresponding function control command; in response to recognizing a preset lock gesture, switching the ultrasound host from a gesture-unlocked state to a gesture-locked state.

[0046] In practical implementation, the gesture control state of the ultrasound host is managed by recognizing preset unlock and lock gestures, and the system responds to function control gestures in the unlocked state. This technical solution introduces a state machine management mechanism, which greatly reduces the system's false trigger rate and improves the stability and safety of equipment operation. In a medical environment, there may be unintentional movements or arm swings by other medical staff around the equipment. Without state isolation, these movements are easily misinterpreted by the system as control commands, causing the ultrasound host to perform unexpected operations, which may interfere with the diagnosis or treatment process. By setting a gesture lock state as the default or accessible state, it is ensured that irrelevant environmental movements will not trigger host actions most of the time, thus effectively filtering environmental interference. Only when the user actively issues a specific, preset unlock gesture does the system enter the gesture unlock state, ready to receive function control commands. This makes the system activation intentional; that is, the user must express their control intention through a clear action that is not usually unintentionally performed. This is equivalent to adding a "safety gate" to gesture control. Similarly, through preset lock gestures, users can actively restore the system to a safe lock state after completing the operation. This proactive status management gives users a clear sense of control over the system, preventing misoperation and avoiding unnecessary consumption of system resources during non-operational periods, thus ensuring that the ultrasound host can operate stably and reliably in complex medical environments.

[0047] In some preferred embodiments, at least one global gesture that is not restricted by the gesture lock state is predefined; the above step "controlling the ultrasound host to perform the corresponding operation according to the recognized gesture" further includes the following step: in response to the recognition of the global gesture, regardless of whether the ultrasound host is currently in the gesture lock state or the gesture unlock state, a corresponding control command is generated and executed.

[0048] In practical implementation, by pre-defining at least one global gesture that is not restricted by the gesture lock state, a clever balance is achieved between ensuring system security and maintaining operational convenience. This provides a quick access channel for specific urgent or frequently used functions, avoiding operational delays or risks that may arise due to state restrictions. While the gesture lock state effectively prevents accidental triggering, it can also present a problem: in emergency situations or specific scenarios, even if the system is locked, the user may need to immediately perform a critical operation. For example, if an ultrasound host malfunctions and an unlock gesture is required to clear the alarm, it increases the number of steps and delays processing time. Pre-defined global gestures resolve this contradiction. Global gestures are designed to have the highest priority; regardless of whether the system is currently locked or unlocked, as long as the gesture is recognized, the corresponding control command will be executed immediately. This means that for functions defined as global gestures, users do not need to be concerned with the current system state and can trigger them directly and quickly. This design greatly improves the response speed of critical operations and simplifies the operation process. It ensures that even when the system is locked to prevent accidental touches, critical functions (such as emergency stop, fault clearing, mode switching, etc.) remain readily available, thus avoiding secondary operational obstacles that may be introduced by state management. This makes the entire gesture control system highly safe, agile in dealing with emergencies, and provides an excellent user experience.

[0049] In some preferred embodiments, the control commands corresponding to the global gesture include commands to clear fault information and / or commands to enter the frequency sweep interface.

[0050] In practice, the control commands corresponding to the global gestures include instructions to clear fault information and / or instructions to enter the scanning interface. Clearing fault information and entering the scanning interface are both crucial functions of the ultrasound host in clinical use, and sometimes require rapid response. Assigning the fault-clearing instruction to a global gesture means that when the host displays a fault message, medical staff can immediately and unconditionally clear the fault state with a specific gesture, without first confirming whether the system is unlocked. This immediacy is essential for maintaining the smooth progress of surgical or examination procedures, quickly restoring the equipment to a usable state, avoiding treatment interruptions caused by cumbersome unlocking procedures, and ensuring medical efficiency and safety. Similarly, setting the function of entering the scanning interface as a global gesture allows medical staff to quickly initiate or switch to the core scanning diagnostic mode under any circumstances (including when the system is locked). This is of significant value in scenarios requiring rapid initiation of examinations or emergency switching of scanning parameters, ensuring the timeliness of diagnostic work.

[0051] In some preferred embodiments, the method further includes the following step: when a preset condition is met, automatically switching the ultrasound host from the gesture unlock state back to the gesture lock state.

[0052] In practical implementation, a feature has been added to automatically switch the ultrasound host from gesture unlock state back to gesture lock state when preset conditions are met, further improving the system's security and intelligence level, and effectively preventing potential risks caused by users forgetting to lock the device. The preset conditions are, for example, that no valid function control gestures are recognized within a preset time period after entering the gesture unlock state.

[0053] In clinical practice, after adjusting the ultrasound machine using gestures, medical staff may quickly shift their attention to the patient or surgical procedure, potentially forgetting to execute the lock gesture. If the system remains unlocked, any unintentional movement that conforms to the functional control gesture (e.g., arm movement while wiping the equipment, hand movements of other people passing by) may be captured and misexecuted by the system, altering the machine's parameter settings. This could affect examination results or, in severe cases, adversely impact the patient or the equipment.

[0054] By introducing an automatic timeout locking mechanism, the system automatically reverts to a secure locked state if it fails to detect any valid functional control gestures for a certain period while in the unlocked state, fundamentally preventing the aforementioned situation from occurring. This "foolproof" design does not rely on user memory or active operation; the system autonomously restores itself to a safe state, greatly enhancing the device's fault tolerance. These preset conditions (such as time thresholds) can be flexibly configured according to actual application scenarios, balancing convenience and safety. This allows the entire gesture control system to provide convenience while possessing higher safety and automation levels, ensuring the rigor of medical procedures and the reliability of equipment use.

[0055] Furthermore, in one embodiment, the correspondence between gestures and functions is shown in Table 1 below.

[0056]

[0057] Table 1. Gesture Function Comparison Table This invention proposes a gesture recognition-based ultrasound host control method, comprising: acquiring reflected signals from a user's hand using a preset millimeter-wave radar sensor; processing the reflected signals to extract motion feature information of the user's hand; recognizing the user's gesture based on the motion feature information using a pre-trained gesture recognition model; and controlling the ultrasound host to perform corresponding operations according to the recognized gesture. This invention ensures system reliability through the environmental adaptability of millimeter-wave radar, guarantees gesture recognition accuracy through multi-dimensional motion feature analysis, and achieves comprehensive control through scalable gesture function mapping. It effectively solves the cross-infection risk and operational inconvenience problems associated with traditional contact operations, significantly improving the hygiene and safety and human-computer interaction efficiency of the ultrasound host in medical settings.

[0058] Accordingly, this invention provides a schematic block diagram of an ultrasound host control device based on gesture recognition. Corresponding to the above-described ultrasound host control method based on gesture recognition, this invention also provides an ultrasound host control device based on gesture recognition. This gesture-recognition-based ultrasound host control device includes a unit for executing the above-described ultrasound host control method based on gesture recognition, and can be configured in a desktop computer, tablet computer, laptop computer, or other terminal. Specifically, the gesture-recognition-based ultrasound host control device includes: The acquisition unit is used to acquire reflected signals from the user's hand through a preset millimeter-wave radar sensor; An extraction unit is used to process the reflected signal and extract the motion feature information of the user's hand; The recognition unit is used to recognize the user's gestures based on the motion feature information using a pre-trained gesture recognition model. An execution unit is used to control the ultrasound host to perform corresponding operations based on the recognized gesture.

[0059] In some preferred embodiments, processing the reflected signal to extract motion feature information of the user's hand includes: A two-dimensional fast Fourier transform is performed on the reflected signal to obtain a distance-Doppler heatmap; The position information of the user's hand in three-dimensional space and the average Doppler velocity are extracted from the distance-Doppler heat map; Based on the position information of continuous time frames, the motion velocity vector of the user's hand is obtained by differential calculation, and the motion velocity vector includes the velocity magnitude and direction; Based on the location information in continuous time frames, the movement trajectory of the user's hand is determined by connecting the spatial points corresponding to the location information, and the trajectory information of the movement trajectory is extracted.

[0060] In some preferred embodiments, recognizing the user's gestures using a pre-trained gesture recognition model based on the motion feature information includes: The motion velocity vectors at different time points are synthesized to obtain the overall motion vector of the hand. The overall motion vector and the motion feature information are input into a pre-trained gesture recognition model for gesture classification, and the recognition result of the user's gesture is output. The motion feature information includes at least one of the motion velocity vector, the trajectory information, the position information, and the average Doppler velocity.

[0061] In some preferred embodiments, controlling the ultrasound host to perform corresponding operations based on the recognized gestures includes: In response to the recognition of a preset unlock gesture, the ultrasound host is switched from the gesture lock state to the gesture unlock state; In the gesture unlock state, in response to recognizing a preset function control gesture, a corresponding function control command is generated and executed; In response to the recognition of a preset locking gesture, the ultrasound host is switched from the gesture unlock state to the gesture lock state.

[0062] In some preferred embodiments, it further includes: The switching unit is used to automatically switch the ultrasound host from the gesture unlock state back to the gesture lock state when preset conditions are met.

[0063] In some preferred embodiments, at least one global gesture that is not restricted by the gesture lock state is predefined; the step of controlling the ultrasound host to perform corresponding operations based on the recognized gesture further includes: In response to the recognition of the global gesture, regardless of whether the ultrasound host is currently in the gesture locked state or the gesture unlocked state, a corresponding control command is generated and executed.

[0064] In some preferred embodiments, the control commands corresponding to the global gesture include commands to clear fault information and / or commands to enter the frequency sweep interface.

[0065] It should be noted that those skilled in the art can clearly understand that the specific implementation process of the above-mentioned gesture recognition-based ultrasound host control device and its various units can be referred to the corresponding descriptions in the foregoing method embodiments. For the sake of convenience and brevity, these details will not be repeated here.

[0066] The aforementioned gesture recognition-based ultrasound host control device can be implemented as a computer program, which can, for example... Figure 2 It runs on the computer device shown.

[0067] Please see Figure 2 , Figure 2 This is a schematic block diagram of a computer device provided in an embodiment of this application. The computer device 500 can be a terminal or a server. The terminal can be an electronic device with communication functions, such as a smartphone, tablet, laptop, desktop computer, personal digital assistant, or wearable device. The server can be a standalone server or a server cluster composed of multiple servers.

[0068] The computer device 500 includes a processor 502, a memory, and a network interface 505 connected via a system bus 501. The memory may include a non-volatile storage medium 503 and internal memory 504.

[0069] The non-volatile storage medium 503 can store an operating system 5031 and a computer program 5032. When the computer program 5032 is executed, it enables the processor 502 to execute an ultrasound host control method based on gesture recognition.

[0070] The processor 502 provides computing and control capabilities to support the operation of the entire computer device 500.

[0071] The internal memory 504 provides an environment for the operation of the computer program 5032 in the non-volatile storage medium 503. When the computer program 5032 is executed by the processor 502, the processor 502 can execute an ultrasound host control method based on gesture recognition.

[0072] The network interface 505 is used for network communication with other devices. Those skilled in the art will understand that the above structure is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device 500 to which the present application is applied. A specific computer device 500 may include more or fewer components than shown in the figures, or combine certain components, or have different component arrangements.

[0073] The processor 502 is used to run a computer program 5032 stored in the memory to perform the following steps: The reflected signals from the user's hand are collected by a pre-set millimeter-wave radar sensor; The reflected signal is processed to extract the motion feature information of the user's hand; Based on the motion feature information, the user's gestures are identified through a pre-trained gesture recognition model; The ultrasound host is controlled to perform corresponding operations based on the recognized gestures.

[0074] In some preferred embodiments, processing the reflected signal to extract motion feature information of the user's hand includes: A two-dimensional fast Fourier transform is performed on the reflected signal to obtain a distance-Doppler heatmap; The position information of the user's hand in three-dimensional space and the average Doppler velocity are extracted from the distance-Doppler heat map; Based on the position information of continuous time frames, the motion velocity vector of the user's hand is obtained by differential calculation, and the motion velocity vector includes the velocity magnitude and direction; Based on the location information in continuous time frames, the movement trajectory of the user's hand is determined by connecting the spatial points corresponding to the location information, and the trajectory information of the movement trajectory is extracted.

[0075] In some preferred embodiments, recognizing the user's gestures using a pre-trained gesture recognition model based on the motion feature information includes: The motion velocity vectors at different time points are synthesized to obtain the overall motion vector of the hand. The overall motion vector and the motion feature information are input into a pre-trained gesture recognition model for gesture classification, and the recognition result of the user's gesture is output. The motion feature information includes at least one of the motion velocity vector, the trajectory information, the position information, and the average Doppler velocity.

[0076] In some preferred embodiments, controlling the ultrasound host to perform corresponding operations based on the recognized gestures includes: In response to the recognition of a preset unlock gesture, the ultrasound host is switched from the gesture lock state to the gesture unlock state; In the gesture unlock state, in response to recognizing a preset function control gesture, a corresponding function control command is generated and executed; In response to the recognition of a preset locking gesture, the ultrasound host is switched from the gesture unlock state to the gesture lock state.

[0077] In some preferred embodiments, the method further includes: When preset conditions are met, the ultrasound host will automatically switch from the gesture unlock state back to the gesture lock state.

[0078] In some preferred embodiments, at least one global gesture that is not restricted by the gesture lock state is predefined; the step of controlling the ultrasound host to perform corresponding operations based on the recognized gesture further includes: In response to the recognition of the global gesture, regardless of whether the ultrasound host is currently in the gesture locked state or the gesture unlocked state, a corresponding control command is generated and executed.

[0079] In some preferred embodiments, the control commands corresponding to the global gesture include commands to clear fault information and / or commands to enter the frequency sweep interface.

[0080] It should be understood that in the embodiments of this application, the processor 502 may be a central processing unit (CPU), or it may be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0081] It will be understood by those skilled in the art that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program may be stored in a storage medium, which is a computer-readable storage medium. The computer program is executed by at least one processor in the computer system to implement the process steps of the embodiments of the above methods.

[0082] Therefore, the present invention also provides a storage medium. This storage medium can be a computer-readable storage medium. The storage medium stores a computer program. When executed by a processor, the computer program causes the processor to perform the following steps: The reflected signals from the user's hand are collected by a pre-set millimeter-wave radar sensor; The reflected signal is processed to extract the motion feature information of the user's hand; Based on the motion feature information, the user's gestures are identified through a pre-trained gesture recognition model; The ultrasound host is controlled to perform corresponding operations based on the recognized gestures.

[0083] In some preferred embodiments, processing the reflected signal to extract motion feature information of the user's hand includes: A two-dimensional fast Fourier transform is performed on the reflected signal to obtain a distance-Doppler heatmap; The position information of the user's hand in three-dimensional space and the average Doppler velocity are extracted from the distance-Doppler heat map; Based on the position information of continuous time frames, the motion velocity vector of the user's hand is obtained by differential calculation, and the motion velocity vector includes the velocity magnitude and direction; Based on the location information in continuous time frames, the movement trajectory of the user's hand is determined by connecting the spatial points corresponding to the location information, and the trajectory information of the movement trajectory is extracted.

[0084] In some preferred embodiments, recognizing the user's gestures using a pre-trained gesture recognition model based on the motion feature information includes: The motion velocity vectors at different time points are synthesized to obtain the overall motion vector of the hand. The overall motion vector and the motion feature information are input into a pre-trained gesture recognition model for gesture classification, and the recognition result of the user's gesture is output. The motion feature information includes at least one of the motion velocity vector, the trajectory information, the position information, and the average Doppler velocity.

[0085] In some preferred embodiments, controlling the ultrasound host to perform corresponding operations based on the recognized gestures includes: In response to the recognition of a preset unlock gesture, the ultrasound host is switched from the gesture lock state to the gesture unlock state; In the gesture unlock state, in response to recognizing a preset function control gesture, a corresponding function control command is generated and executed; In response to the recognition of a preset locking gesture, the ultrasound host is switched from the gesture unlock state to the gesture lock state.

[0086] In some preferred embodiments, the method further includes: When preset conditions are met, the ultrasound host will automatically switch from the gesture unlock state back to the gesture lock state.

[0087] In some preferred embodiments, at least one global gesture that is not restricted by the gesture lock state is predefined; the step of controlling the ultrasound host to perform corresponding operations based on the recognized gesture further includes: In response to the recognition of the global gesture, regardless of whether the ultrasound host is currently in the gesture locked state or the gesture unlocked state, a corresponding control command is generated and executed.

[0088] In some preferred embodiments, the control commands corresponding to the global gesture include commands to clear fault information and / or commands to enter the frequency sweep interface.

[0089] The storage medium is a physical, non-transient storage medium, such as a USB flash drive, external hard drive, read-only memory (ROM), magnetic disk, or optical disk, or any other physical storage medium capable of storing program code. The computer-readable storage medium can be non-volatile or volatile.

[0090] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0091] In the several embodiments provided by this invention, it should be understood that the disclosed apparatus and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For example, the division of each unit is merely a logical functional division, and there may be other division methods in actual implementation. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed.

[0092] The steps in the method of this invention can be adjusted, merged, or reduced in order according to actual needs. The units in the device of this invention can be merged, divided, or reduced according to actual needs. Furthermore, the functional units in the various embodiments of this invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0093] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, a terminal, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention.

[0094] In the above embodiments, the descriptions of each embodiment have different focuses. For parts that are not described in detail in a certain embodiment, please refer to the relevant descriptions in other embodiments.

[0095] Obviously, those skilled in the art can make various modifications and variations to this invention without departing from its spirit and scope. Since these modifications and variations fall within the scope of the claims and their equivalents, this invention also intends to include these modifications and variations.

[0096] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and these modifications or substitutions should all be covered within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A method for controlling an ultrasonic host based on gesture recognition, characterized in that, include: The reflected signals from the user's hand are collected by a pre-set millimeter-wave radar sensor; The reflected signal is processed to extract the motion feature information of the user's hand; Based on the motion feature information, the user's gestures are identified through a pre-trained gesture recognition model; The ultrasound host is controlled to perform corresponding operations based on the recognized gestures.

2. The ultrasound host control method based on gesture recognition according to claim 1, characterized in that, The process of processing the reflected signal to extract the motion feature information of the user's hand includes: A two-dimensional fast Fourier transform is performed on the reflected signal to obtain a distance-Doppler heatmap; The position information of the user's hand in three-dimensional space and the average Doppler velocity are extracted from the distance-Doppler heat map; Based on the position information of continuous time frames, the motion velocity vector of the user's hand is obtained by differential calculation, and the motion velocity vector includes the velocity magnitude and direction; Based on the location information in continuous time frames, the movement trajectory of the user's hand is determined by connecting the spatial points corresponding to the location information, and the trajectory information of the movement trajectory is extracted.

3. The ultrasound host control method based on gesture recognition according to claim 2, characterized in that, The step of recognizing the user's gestures using a pre-trained gesture recognition model based on the motion feature information includes: The motion velocity vectors at different time points are synthesized to obtain the overall motion vector of the hand. The overall motion vector and the motion feature information are input into a pre-trained gesture recognition model for gesture classification, and the recognition result of the user's gesture is output. The motion feature information includes at least one of the motion velocity vector, the trajectory information, the position information, and the average Doppler velocity.

4. The ultrasound host control method based on gesture recognition according to claim 1, characterized in that, The step of controlling the ultrasound host to perform corresponding operations based on the recognized gestures includes: In response to the recognition of a preset unlock gesture, the ultrasound host is switched from the gesture lock state to the gesture unlock state; In the gesture unlock state, in response to recognizing a preset function control gesture, a corresponding function control command is generated and executed; In response to the recognition of a preset locking gesture, the ultrasound host is switched from the gesture unlock state to the gesture lock state.

5. The ultrasound host control method based on gesture recognition according to claim 4, characterized in that, The method further includes: When preset conditions are met, the ultrasound host will automatically switch from the gesture unlock state back to the gesture lock state.

6. The ultrasound host control method based on gesture recognition according to claim 4, characterized in that, At least one global gesture is predefined and is not restricted by the gesture lock state; the step of controlling the ultrasound host to perform the corresponding operation based on the recognized gesture also includes: In response to the recognition of the global gesture, regardless of whether the ultrasound host is currently in the gesture locked state or the gesture unlocked state, a corresponding control command is generated and executed.

7. The ultrasound host control method based on gesture recognition according to claim 6, characterized in that, The control commands corresponding to the global gestures include commands to clear fault information and / or commands to enter the frequency sweep interface.

8. An ultrasound host control device based on gesture recognition, characterized in that, Includes a unit for performing the method as described in any one of claims 1-7.

9. A computer device, characterized in that, The computer device includes a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the method as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The storage medium stores a computer program that, when executed by a processor, can implement the method as described in any one of claims 1-7.

Images