Instrument operation method based on gestures

Abstract

The invention relates to the technical field of man-machine interaction, in particular to a gesture-based instrument operation method, which comprises the following steps of: capturing gesture information of an operator in real time through gesture acquisition equipment; the gesture recognition module receives gesture information captured by the gesture acquisition equipment and converts the gesture information into a preset instruction signal; the medical instrument control module receives the instruction signal and executes corresponding operation based on the instruction signal, the method is based on a medical instrument gesture model, the model is integrated to acquisition equipment to realize edge calculation, and the gesture of wearing gloves is acquired in real time; instruction conversion is carried out based on gesture acquisition equipment, and the gesture of wearing the glove is converted into an instrument instruction; the collection device is driven based on the instrument state, it is ensured that the instrument enters the working state and is driven, system resource waste is avoided, it is ensured that the instrument and the collection device are high in availability, the glove wearing gesture is efficiently and accurately recognized, traditional input equipment does not need to be used, and operation convenience and man-machine interaction efficiency are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of human-computer interaction, and in particular to an instrument operation method based on gestures. Background Art

[0002] In the existing interactive operation of medical instruments, the main input methods are keyboard input, mouse input, touch screen input and scanner input.

[0003] However, in medical laboratories, workers often need to wear protective clothing and protective gloves. In this environment, using traditional input methods to operate medical instruments often leads to input errors due to the inconvenience of workers wearing protective clothing and protective gloves, thus affecting work efficiency. Summary of the invention

[0004] The purpose of the present invention is to provide an instrument operation method based on gestures, aiming to solve the problem that in traditional input methods, due to the inconvenience of workers wearing protective clothing and protective gloves, input errors may occur, affecting work efficiency.

[0005] To achieve the above object, the present invention provides a gesture-based instrument operation method, comprising the following steps:

[0006] Capture the operator's gesture information in real time through gesture acquisition equipment;

[0007] The gesture recognition module receives the gesture information captured by the gesture acquisition device and converts it into a preset command signal;

[0008] The medical instrument control module receives the instruction signal and performs corresponding operations based on the instruction signal.

[0009] Wherein, the gesture acquisition device includes a camera, an infrared sensor and a depth sensor.

[0010] The gesture acquisition device can capture the operator's gesture information in real time and convert it into a data format recognizable by a computer.

[0011] The gesture recognition module stores preset gesture information and corresponding command signals, processes and matches the gesture information captured by the gesture acquisition device with the gesture information in the gesture feature value, and outputs a matching command signal.

[0012] Among them, the gesture information processing and matching includes data processing, feature extraction and matching algorithm. The data processing includes removing noise from the image or sensor data and unifying the gesture data to the same scale. The feature extraction includes extracting key features from the gesture image or sensor data, such as gesture shape, size, direction and motion trajectory. The matching algorithm includes comparing the extracted gesture features with the predefined gesture template, finding the most similar template as the recognition result, and using the deep learning algorithm to classify and recognize the gesture features.

[0013] The present invention discloses a gesture-based instrument operation method, which uses a gesture acquisition device to capture the operator's gesture information in real time; a gesture recognition module receives the gesture information captured by the gesture acquisition device and converts it into a preset command signal; a medical instrument control module receives the command signal and performs corresponding operations based on the command signal. The method is based on a medical instrument gesture model, integrates the model into the acquisition device to realize edge computing, and collects gloved gestures in real time; performs command conversion based on the gesture acquisition device to realize the conversion of gloved gestures into instrument commands; drives the acquisition device based on the instrument state to ensure that the instrument enters the working state and drives the acquisition device, avoids waste of system resources, ensures high availability of the instrument and the acquisition device, efficiently and accurately recognizes gloved gestures, does not need to use traditional input devices, improves operation convenience and human-computer interaction efficiency, can avoid erroneous operations caused by inconvenient operation, can add new gesture commands and corresponding operations as needed, is easy to expand and upgrade, and solves the problem that the traditional input method causes input erroneous operations and affects work efficiency due to the inconvenience of workers wearing protective clothing and protective gloves. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0015] Figure 1 It is a functional diagram.

[0016] Figure 2It is a business architecture diagram, in which Clk: clock signal, Rst_n: low-level effective reset signal, Po_flag: flag input port, Po_data: port input data, Threshold: threshold, Out_data: output data, Out_flag: output flag, sobel_model: Sobel operator detection module, flag: gesture feature value, order_model: instrument instruction conversion module, 0x1F0A, 0x401B, 0xC230, 0xE71B are examples of instrument instructions.

[0017] Figure 3 It is a data flow graph.

[0018] Figure 4 It is a flow chart of a gesture-based instrument operation method provided by the present invention. DETAILED DESCRIPTION

[0019] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present invention, and should not be construed as limiting the present invention.

[0020] See also Figures 1 to 4 The present invention provides a gesture-based instrument operation method, comprising the following steps:

[0021] S1 captures the operator's gesture information in real time through gesture acquisition equipment;

[0022] Specifically, the gesture acquisition device may be a device based on technologies such as a camera, an infrared sensor, a depth sensor, etc., which can capture the operator's gesture information in real time and convert it into a data format recognizable by a computer.

[0023] S2 gesture recognition module receives the gesture information captured by the gesture acquisition device and converts it into a preset command signal;

[0024] Specifically, the gesture recognition module includes gesture feature values ​​and gesture recognition algorithms. The gesture feature values ​​store preset gesture information and corresponding command signals. The feature value extraction information, storage information and corresponding command signals are described as follows:

[0025] Feature extraction process: First, common gestures and the specific environment of wearing gloves and protective clothing are sampled. In the feature extraction stage, key information in the image (such as color, shape, texture, etc.) is extracted as the basis for the subsequent recognition process. These features are usually expressed in the form of numerical values ​​or vectors, that is, eigenvalues. Feature extraction methods include edge detection and corner point detection to generate specific forms of eigenvalues.

[0026] Storage of eigenvalues: They are stored in computer memory in the form of numerical matrices and vectors. These data structures facilitate subsequent calculations and processing. In order to reduce the requirements for storage space and transmission bandwidth, the eigenvalues ​​are compressed and encoded before storage, which helps reduce data redundancy and improve storage efficiency.

[0027] Characteristic values ​​and command signals: Core commands include but are not limited to "start", "pause", "next page", "previous page", "confirm", "cancel" and "close pop-up window".

[0028] The gesture recognition algorithm is used to process and match the gesture information captured by the gesture acquisition device with the gesture information in the gesture feature value, and output a matching command signal. The description of the gesture information acquisition, matching method, and output signal is as follows:

[0029] Gesture information collection: Using a camera or depth camera to capture hand images or videos can capture two-dimensional or three-dimensional information of the hand.

[0030] Gesture information processing and matching includes data processing, feature extraction and matching algorithms.

[0031] Data preprocessing includes denoising, normalization and data enhancement.

[0032] Denoising: removing noise from images or sensor data to improve data quality;

[0033] Normalization: Unify the gesture data to the same scale for easy subsequent processing;

[0034] Data enhancement: Increase the diversity of gesture data through transformation, rotation, scaling, etc. to improve the generalization ability of the model.

[0035] Feature extraction

[0036] Extract key features from gesture images or sensor data, such as gesture shape, size, direction, motion trajectory, etc.

[0037] Static gestures usually extract features from a single frame image, while dynamic gestures need to consider temporal features.

[0038] Matching algorithms include template matching and deep learning.

[0039] Template matching: Compare the extracted gesture features with the predefined gesture templates to find the most similar template as the recognition result.

[0040] Deep learning: Use deep learning algorithms (such as convolutional neural networks (CNN) and recurrent neural networks (RNN)) to classify and recognize gesture features. These algorithms can automatically learn gesture features and have high recognition accuracy and robustness.

[0041] Output signal

[0042] Command signal: Convert the recognized gestures into specific instructions or commands, such as "start", "pause", "confirm", "cancel", "next page", "previous page", "close pop-up window", etc. These command signals can directly control the medical equipment to perform corresponding operations.

[0043] Feedback signal: In order to provide a better user experience, the gesture recognition system can also output feedback signals to the user, such as prompt sounds for successful recognition, visual feedback, etc. These feedback signals can help users understand the recognition results and current status of the system.

[0044] S3 The medical instrument control module receives the command signal and performs corresponding operations based on the command signal.

[0045] The working environment and related configuration installation of the gesture-based instrument operation method are described as follows:

[0046] 1. Gesture recognition device (a recognition and analysis device equipped with ARM MCU, integrated camera and edge computing module);

[0047] 2. Installation near medical instrument displays;

[0048] 3. The gesture algorithm model is integrated into the self-developed gesture acquisition device;

[0049] 4. The acquisition device is connected to the medical instrument through the USB interface to realize power supply and command issuance;

[0050] 5. Control process: the acquisition device collects data in real time, the algorithm module analyzes gesture commands, and the commands are converted and issued;

[0051] 6. The instrument is connected to a gesture recognition device, which detects in real time whether there is a gesture of a person that satisfies the trigger command.

[0052] The method is based on the medical instrument gesture model, and the model is integrated into the acquisition device to realize edge computing, and the gloved gestures are acquired in real time; the command conversion is performed based on the gesture acquisition device, and the gloved gestures are converted into instrument commands; the acquisition device is driven based on the instrument status to ensure that the instrument enters the working state and drives the acquisition device, avoids the waste of system resources, ensures the high availability of the instrument and the acquisition device, and efficiently and accurately recognizes the gloved gestures without the need to use traditional input devices, thereby improving the convenience of operation and the efficiency of human-computer interaction, avoiding misoperation caused by inconvenient operation, and adding new gesture commands and corresponding operations as needed, which is easy to expand and upgrade.

[0053] What is disclosed above is only a preferred embodiment of a gesture-based instrument operation method of the present invention. Of course, it cannot be used to limit the scope of rights of the present invention. Ordinary technicians in this field can understand that all or part of the processes of the above embodiment and equivalent changes made according to the claims of the present invention are still within the scope of the invention.

Claims

1. A gesture-based instrument operation method, characterized in that: The following steps are involved: Capture the operator's gesture information in real time through gesture acquisition equipment; The gesture recognition module receives the gesture information captured by the gesture acquisition device and converts it into a preset command signal; The medical instrument control module receives the instruction signal and performs corresponding operations based on the instruction signal.

2. The gesture-based instrument operation method according to claim 1, characterized in that: The gesture acquisition device includes a camera, an infrared sensor and a depth sensor.

3. The gesture-based instrument operation method according to claim 1, characterized in that: The gesture acquisition device can capture the operator's gesture information in real time and convert it into a data format recognizable by a computer.

4. The gesture-based instrument operation method according to claim 1, characterized in that: The gesture recognition module stores preset gesture information and corresponding command signals, processes and matches the gesture information captured by the gesture acquisition device with the gesture information in the gesture feature value, and outputs a matching command signal.

5. The gesture-based instrument operation method according to claim 1, characterized in that: The gesture information processing and matching includes data processing, feature extraction and matching algorithm. The data processing includes removing noise from the image or sensor data and unifying the gesture data to the same scale. The feature extraction includes extracting key features from the gesture image or sensor data, such as gesture shape, size, direction and motion trajectory. The matching algorithm includes comparing the extracted gesture features with the predefined gesture template, finding the most similar template as the recognition result, and using the deep learning algorithm to classify and recognize the gesture features.

Images