Voice control ultrasonic wave adjustment setting method based on CTC-Attention mixed architecture

Abstract

The invention relates to a voice control ultrasonic wave adjustment setting method based on a CTC-Attention mixed architecture, and the method comprises the following steps: S1, receiving a voice instruction of a doctor through a voice receiving device, and carrying out the preprocessing of an original voice signal; s2, constructing an acoustic model of a CTC-Attention hybrid architecture, and training the acoustic model to obtain a trained acoustic model; s3, inputting the preprocessed voice signal into the trained acoustic model, and performing voice recognition through CTC path probability alignment and Attention context dependence joint optimization; s4, outputting an identification result and decoding the identification result into a text instruction; and S5, analyzing the operation intention according to the text instruction, and controlling the ultrasonic instrument to execute a corresponding state adjustment operation. According to the method, the voice signal of the operator is received, the voice signal is recognized and analyzed, the obtained result is optimized and then converted into the character content with the meaning, then the ultrasonic energy with the intensity corresponding to the character content is generated by the ultrasonic instrument based on the character content, and the ultrasonic instrument is effectively used for medical operation.

Description

Technical Field

[0001] The present invention belongs to the field of speech recognition technology, and in particular relates to a voice-controlled ultrasonic adjustment setting of a CTC-Attention hybrid architecture. Background Art

[0002] Voice signals are the primary form of communication in real life. With the rapid development of science and technology, voice signals have gradually become a key form of human-computer interaction. Compared to manual operation, voice interaction frees our hands and greatly facilitates our daily lives. The 21st century is an era of vigorous development of information technology. As a key component of this human-computer interaction branch, speech recognition technology is a key interface for human-computer interaction, making human-machine interaction more automated and intelligent. It has become the primary way for machines to understand human language and has promoted the development of artificial intelligence.

[0003] When voice control is used, i.e., analyzing or recognizing user intent or voice commands expressed in natural language, artificial intelligence algorithms are preferably used. This is particularly suitable for mapping a high-dimensional input space consisting of multiple different speech sequences onto a target space consisting of multiple defined control commands based on natural speech input. In traditional medical procedures, doctors use their hands to fine-tune ultrasound imaging. As the number of routine ultrasound examinations continues to grow, ultrasound doctors repeat this manual labor day after day, year after year, which can become a burden. Many ultrasound doctors suffer from varying degrees of frozen shoulder and tenosynovitis, and urgently need to free their hands. In interventional procedures, because the surgeon holds the ultrasound probe in one hand and the needle in the other, ultrasound machine adjustments are often performed by an assistant. This presents two problems: first, the need for an assistant increases labor costs; second, varying levels of understanding of the ultrasound image or the surgeon's intended procedure can lead to coordination issues, delaying time and potentially compromising medical safety.

[0004] Chinese patent document CN 115424608A discloses a method for identity authentication based on machine learning. The method uses the built-in speaker of a smart device to generate ultrasonic waves. When the user speaks a preset authentication password, the built-in microphone of the smart device is used as a signal receiver to collect the voice signal during the user's pronunciation and the ultrasonic signal reflected from the user's lips. The voice signal and the ultrasonic signal are normalized, and the features of the voice signal and the envelope information features of the ultrasonic signal are extracted, wherein the features of the voice signal reflect the physiological characteristics of the user's pronunciation. The extracted features of the voice signal and the envelope information features of the ultrasonic signal are input into a trained classification model to identify the legitimacy of the user's identity. The present invention innovatively combines physiological behavior and voice features for identity authentication, and can achieve effective identity authentication even based on a single syllable.

[0005] Chinese patent document CN 113576527A discloses a method for ultrasonic input judgment using voice control, comprising: S1, an operator holding a plurality of microphone devices on an ultrasonic probe to receive sound input in real time, obtaining multiple voice signals, and then judging whether there is sound input. If it is judged that there is no sound input, the microphone is used to receive the sound again. If it is judged that there is sound input, the method proceeds to the next step; S2, performing ICA analysis on the multiple voice signals to extract useful signals; S3, locating the sound source of the extracted useful signals and judging the position of the sound source. If it is judged to be within a pre-set range, the method returns to S1; if it is judged not to be within a pre-set range, the method proceeds to the next step; S4, performing sound fingerprint judgment on the useful signals. If it is judged that the sound fingerprint is not the operator's sound fingerprint, the method returns to S1; if it is judged to be the operator's sound fingerprint, the method proceeds to the next step; S5, recognizing the voice content of the useful signals, and completing the ultrasonic judgment operation required for the voice content.

[0006] However, there is currently no report on a voice-controlled ultrasonic adjustment setting method based on a CTC-Attention hybrid architecture. Summary of the Invention

[0007] The purpose of the present invention is to address the deficiencies in the prior art and provide a method for voice-controlled ultrasonic adjustment settings based on a CTC-Attention hybrid architecture.

[0008] A method for setting up voice-controlled ultrasonic wave adjustment based on a CTC-Attention hybrid architecture includes the following steps:

[0009] S1: Receive the doctor's voice instructions through the sound receiving device and pre-process the original voice signal;

[0010] S2: Build and train an acoustic model of the CTC-Attention hybrid architecture to obtain a trained acoustic model.

[0011] S3: The preprocessed speech signal is input into the trained acoustic model, and speech recognition is performed through the joint optimization of CTC path probability alignment and Attention context dependency.

[0012] S4: Output recognition results and decode them into text instructions;

[0013] S5: Analyze the operation intention according to the text instruction and control the ultrasonic instrument to perform the corresponding state adjustment operation.

[0014] As a preferred example, the acoustic model in step S2 includes:

[0015] Encoder module: uses a multi-layer bidirectional LSTM to extract the temporal context representation of speech features;

[0016] CTC module: receives the encoder output and solves the problem of input and output sequence length alignment through the blank label mechanism;

[0017] Attention module: This module uses an additive attention mechanism enhanced with medical features to dynamically weight the encoder output and generate a context vector.

[0018] Joint decoder: Fusion of CTC path probability and Attention context vector to generate the final recognition result.

[0019] As a preferred example, the alignment rules of the blank tag mechanism are as follows: silent segment: mandatory insertion of "_" tag; numeric parameter interval: prohibition of insertion of "_" (to ensure numeric continuity); start of medical term: delay insertion of "_" by 2 frames;

[0020] The additive attention mechanism

[0021] The additive attention model is used to dynamically calculate the weight distribution of the encoder's hidden state and generate a context vector. Its mathematical expression is:

[0022]

[0023] Medical Signature Enhancement Unit

[0024] Construction of a benchmark vector library: Collect ultrasonic operation instructions and extract the acoustic feature vectors of keywords;

[0025] Keyword triggering: When the decoder recognizes a medical operation keyword, the enhancement mechanism is activated;

[0026] Acoustic matching: Compute the cosine similarity between the encoder state and the pre-trained medical keyword baseline vectors.

[0027] As a preferred example, the encoder module comprises 4 layers of bidirectional LSTM, with 512 hidden units in each layer;

[0028] The Key-Value dimension of the Attention module is compressed to 256 dimensions;

[0029] The output labels of the CTC module are delayed by 2-3 time steps.

[0030] As a preferred example, the joint optimization in S3 adopts a multi-task learning strategy, and the loss function is:

[0031] Loss=λL CTC +(1-λ)L Attention .

[0032] As a preferred example, during the multi-task training process:

[0033] The training samples are divided into training set, validation set and test set in the ratio of 8:1:1;

[0034] The blank label probability of LCTC is accumulated through dynamic programming path gradient;

[0035] The gradient of LAttention is back-propagated through the Teacher Forcing strategy.

[0036] As a preferred example, the ultrasonic instrument state adjustment operation in S5 includes:

[0037] S51: Calling and entering information into the patient database;

[0038] S52: Ultrasonic probe type switching and preset mode loading;

[0039] S53: Real-time adjustment of 2D image gain / depth / focus;

[0040] S54: Calling and adjusting ultrasonic non-two-dimensional functions.

[0041] As a preferred example, the front end of the encoder module is connected to a convolutional feature extraction layer, comprising:

[0042] The first convolutional layer uses a 3*5 asymmetric convolution kernel as the time-frequency filter, with a step size of (1, 2) and 64 output channels.

[0043] The second convolution layer uses a 3*3 symmetric convolution kernel as a time-frequency filter with a step size of (1,1) and 128 output channels.

[0044] ReLU activation and 2*2 maximum pooling are used after each convolutional layer.

[0045] In a second aspect, the present invention provides a computer-readable storage medium storing a computer program, which, when executed by hardware, is used to perform a method for voice-controlled ultrasonic adjustment settings based on a CTC-Attention hybrid architecture.

[0046] The advantages of the present invention are: the present invention receives the operator's voice signal, recognizes and analyzes the voice signal, optimizes the obtained result and converts it into text content with the meaning, and then based on the text content, the ultrasonic instrument generates ultrasonic energy of the corresponding intensity, thereby effectively utilizing the ultrasonic instrument for medical operations, reducing the fatigue and deviation of the operator caused by manual control of the ultrasonic instrument, helping to reduce the physical burden of front-line ultrasound doctors and ensure the medical safety and efficacy of ultrasonic interventional surgery. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] Figure 1 : Multi-layer bidirectional LSTM (BiLSTM) basic structure.

[0048] Figure 2 : Attention module infrastructure.

[0049] Figure 3 : Fusion of CTC path probability and Attention context vector generates the final recognition result path. DETAILED DESCRIPTION

[0050] Below in conjunction with specific embodiment, further set forth the present invention.Should be understood that these embodiments are only used to illustrate the present invention and are not used in limiting the scope of the present invention.In addition, should be understood that after reading the content of the present invention record, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall equally within the scope limited by the application's appended claims.

[0051] Example 1

[0052] A method for setting up voice-controlled ultrasonic wave adjustment based on a CTC-Attention hybrid architecture includes the following steps:

[0053] S1: Receive the doctor's voice instructions through the sound receiving device and pre-process the original voice signal;

[0054] S2: Build and train an acoustic model of the CTC-Attention hybrid architecture to obtain a trained acoustic model.

[0055] S3: The preprocessed speech signal is input into the trained acoustic model, and speech recognition is performed through the joint optimization of CTC path probability alignment and Attention context dependency.

[0056] S4: Output recognition results and decode them into text instructions;

[0057] S5: Analyze the operation intention according to the text instruction and control the ultrasonic instrument to perform the corresponding state adjustment operation. The acoustic model in S2 includes:

[0058] ● Encoder module: Uses multi-layer bidirectional LSTM to extract the temporal context representation of speech features; the basic structure of multi-layer bidirectional LSTM (BiLSTM) is shown in Figure 1 .

[0059] CTC module: Receives encoder output and uses a blank label mechanism to solve the problem of input and output sequence length alignment;

[0060] Blank label mechanism alignment rules: see Table 1

[0061] Table 1

[0062]

[0063] Based on the blank label mechanism, the gradient calculation optimization objective of the CTC module is designed as follows:

[0064]

[0065] Where μ is the blank tag penalty factor used to suppress over-insertion

[0066] ●Attention module: Dynamically weights the encoder output using an additive attention mechanism enhanced with medical features to generate a context vector; see the Attention module infrastructure for details. Figure 2 .

[0067] Additive Attention Mechanism

[0068] The additive attention model is used to dynamically calculate the weight distribution of the encoder's hidden state and generate a context vector. Its mathematical expression is: (standard model, no special protection points)

[0069]

[0070] Medical Signature Enhancement Unit

[0071] Benchmark vector library construction: 2,000 ultrasound operation instructions were collected and acoustic feature vectors of the following keywords were extracted: {abdominal probe, cardiac probe, color Doppler ultrasound mode, elastic mode, image storage, video recording...}, to improve the accuracy of medical keyword recognition.

[0072] Keyword triggering: When the decoder status recognizes a medical operation keyword (such as "color Doppler ultrasound mode"), the enhancement mechanism is activated;

[0073] Acoustic matching: Calculate the cosine similarity between the encoder state and the pre-trained medical keyword baseline vector, and increase the attention weight of the matching frame by a factor of 5.0.

[0074] ●Joint decoder: Fusion of CTC path probability and Attention context vector generates the final recognition result. Fusion of CTC path probability and Attention context vector generates the final recognition result path. Figure 3 .

[0075] Medical status perception and safety verification are based on the current status and usage rules of the ultrasound equipment, verifying the final recognition results to avoid high-risk actions.

[0076] The encoder module contains 4 layers of bidirectional LSTM, with 512 hidden units in each layer;

[0077] The Key-Value dimension of the Attention module is compressed to 256 dimensions to reduce the risk of overfitting;

[0078] The output labels of the CTC module are delayed by 2-3 time steps to avoid over-reliance on future information to interfere with real-time recognition.

[0079] CTC module delayed output mechanism

[0080] Delay window design:

[0081] ■2 frame delay (25ms): used for single-digit parameter commands (such as "Gain +5")

[0082] ■3-frame delay (37.5ms): used for compound commands (such as "depth 12cm")

[0083] Cases of medical error prevention: See Table 2

[0084] Table 2

[0085]

[0086] The joint optimization in S3 adopts a multi-task learning strategy, and the loss function is:

[0087] Loss= λL CTC +(1-λ) L Attention

[0088] in:

[0089] LCTC calculates the probabilities of all aligned paths through a forward-backward algorithm;

[0090] L CTC =- ln ∑n∈B -1 (Y)P(π|X)

[0091] π: path sequence (extension sequence containing blank label ″_″)

[0092] B -1 :fold function (remove duplicate characters and blank tags)

[0093] ·L Attention The cross entropy loss function is used to calculate the difference between the predicted text and the true label;

[0094] Medical directives are weighted by tiers:

[0095]

[0096] Using w t Improve the weight of action command keywords in speech recognition training (including: abdomen (probe), superficial (probe), heart (probe), thyroid (mode), abdomen (mode), carotid (mode), heart (mode), color ultrasound (mode), two-dimensional (mode), elasticity (mode), angiography (mode), focus position, image gain, image depth, deep, shallow, strong, weak, stored image, video).

[0097] ●λ is the balance coefficient, and its value range is [0.3, 0.7].

[0098] During the multi-task training process:

[0099] ● The training samples are divided into training set, validation set and test set in a ratio of 8:1:1;

[0100] The original training sample set is divided into three mutually exclusive subsets according to a preset ratio: 80% of the samples are used as the training set (Training Set) for direct learning and gradient updating of model weights; 10% of the samples are used as the validation set (Validation Set) to independently monitor model performance (such as loss function value, specific evaluation indicators) during training, so as to adjust hyperparameters (such as learning rate, regularization strength) in real time and implement the early stopping mechanism (Early Stopping) to prevent overfitting; the remaining 10% of the samples are used as the test set (Test Set), which is only enabled after the model training is completely completed, and is used to simulate unknown data in real scenarios and objectively evaluate the final generalization ability and robustness of the model.

[0101] ●LCTC’s blank label probability is accumulated through dynamic programming path gradient;

[0102] The gradient of attention is back-propagated through the Teacher Forcing strategy.

[0103] The ultrasonic instrument state adjustment operation in S5 includes:

[0104] 1. Calling and entering information into the patient database;

[0105] 2. Ultrasound probe type switching and preset mode loading (e.g. abdominal probe - liver / kidney mode; superficial probe - thyroid / breast mode, etc.);

[0106] 3. Real-time adjustment of 2D image gain / depth / focus (strong / weak image enhancement Gain, deep / shallow depth, deep / shallow focus position);

[0107] 4. Calling and adjusting ultrasonic non-two-dimensional functions (such as CDFI, PW, CW, Elasto, SWE, CEUS, etc.).

[0108] The front end of the encoder module is connected to the convolutional feature extraction layer, including:

[0109] ● First convolutional layer: uses a 3*5 asymmetric convolution kernel as a time-frequency filter, with a stride of (1, 2) and 64 output channels;

[0110] ● Second convolutional layer: uses a 3*3 symmetric convolution kernel as a time-frequency filter, with a stride of (1,1) and 128 output channels;

[0111] ●ReLU activation and 2*2 maximum pooling are used after each convolutional layer.

[0112] The above is only a preferred embodiment of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and supplements without departing from the method of the present invention. These improvements and supplements should also be regarded as the scope of protection of the present invention.

Claims

1. A voice-controlled ultrasonic adjustment setting method based on a CTC-Attention hybrid architecture, characterized in that: The following steps are involved: S1: Receive the doctor's voice instructions through the sound receiving device and pre-process the original voice signal; S2: Build and train an acoustic model of the CTC-Attention hybrid architecture to obtain a trained acoustic model. S3: The preprocessed speech signal is input into the trained acoustic model, and speech recognition is performed through the joint optimization of CTC path probability alignment and Attention context dependency. S4: Output recognition results and decode them into text instructions; S5: Analyze the operation intention according to the text instruction and control the ultrasonic instrument to perform the corresponding state adjustment operation.

2. The method for setting up voice-controlled ultrasonic wave adjustment according to claim 1, characterized in that: The acoustic model in step S2 includes: Encoder module: uses a multi-layer bidirectional LSTM to extract the temporal context representation of speech features; CTC module: receives the encoder output and solves the problem of input and output sequence length alignment through the blank label mechanism; Attention module: This module uses an additive attention mechanism enhanced with medical features to dynamically weight the encoder output and generate a context vector. Joint decoder: Fusion of CTC path probability and Attention context vector to generate the final recognition result.

3. The method for setting up voice-controlled ultrasonic wave adjustment according to claim 2, characterized in that: The alignment rules of the blank label mechanism are as follows: silent segment: mandatory insertion of "_" label, numeric parameter interval: prohibition of insertion of "_" (to ensure numeric continuity), start of medical term: delay insertion of "_" by 2 frames; the additive attention mechanism The additive attention model is used to dynamically calculate the weight distribution of the encoder hidden state and generate the context vector, which is mathematically expressed as: Medical Signature Enhancement Unit Construction of a benchmark vector library: Collect ultrasonic operation instructions and extract the acoustic feature vectors of keywords; Keyword triggering: When the decoder recognizes a medical operation keyword, the enhancement mechanism is activated; Acoustic matching: Compute the cosine similarity between the encoder state and the pre-trained medical keyword baseline vectors.

4. The voice-controlled ultrasonic wave adjustment setting method according to claim 2, The encoder module contains 4 layers of bidirectional LSTM, with 512 hidden units in each layer; The Key-Value dimension of the Attention module is compressed to 256 dimensions; The output labels of the CTC module are delayed by 2-3 time steps.

5. The voice-controlled ultrasonic wave adjustment setting method according to claim 1 or 2, characterized in that: The joint optimization in S3 adopts a multi-task learning strategy, and the loss function is: Loss=λL CTC +(1-λ)L Attention 。 6. The method for setting up voice-controlled ultrasonic wave adjustment according to claim 5, characterized in that: During the multi-task training process: The training samples are divided into training set, validation set and test set in the ratio of 8:1:1; The blank label probability of LCTC is accumulated through dynamic programming path gradient; The gradient of LAttention is back-propagated through the Teacher Forcing strategy.

7. The voice-controlled ultrasonic wave adjustment setting method according to claim 1 or 2, characterized in that: The ultrasonic instrument state adjustment operation in S5 includes: S51: Calling and entering information into the patient database; S52: Ultrasonic probe type switching and preset mode loading; S53: Real-time adjustment of 2D image gain / depth / focus; S54: Calling and adjusting ultrasonic non-two-dimensional functions.

8. The voice-controlled ultrasonic wave adjustment setting method according to claim 1 or 2, characterized in that: The front end of the encoder module is connected to the convolutional feature extraction layer, including: The first convolutional layer uses a 3*5 asymmetric convolution kernel as the time-frequency filter, with a step size of (1, 2) and 64 output channels. The second convolutional layer uses a 3*3 symmetric convolution kernel as a time-frequency filter with a stride of (1,1) and 128 output channels. ReLU activation and 2*2 maximum pooling are used after each convolutional layer.

9. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by hardware, it is used to perform the method according to any one of claims 1 to 9.

Images