Handheld ultrasonic multilingual voice control method

Abstract

The invention relates to the technical field of ultrasonic inspection equipment, in particular to a handheld ultrasonic multilingual voice control method, which comprises the following steps: acquiring an input voice sequence, and extracting vowel syllables in the input voice sequence; classifying according to the extracted vowel syllables to determine the language of the input voice sequence; and acquiring a voice recognition model of the corresponding language according to the language, and recognizing the input voice sequence by using the voice recognition model to obtain an operation instruction. Aiming at the problem of poor effect of a voice control system of handheld ultrasonic equipment in the prior art, various voice recognition models of different languages are preset to realize effective recognition of control instructions of different languages; in addition, in consideration of the problem that cross-language matching cannot be performed due to low equipment efficiency, a classification process for extracting vowel syllables and predicting languages is introduced in advance, so that feature quantities involved in the classification process are reduced, the processing efficiency is improved, and the recognition method can be applied to low-power-consumption equipment.

Description

Technical Field

[0001] The invention relates to the technical field of ultrasonic inspection equipment, and in particular to a multi-language voice control method for handheld ultrasound. Background Art

[0002] Palmtop ultrasound equipment, also known as handheld ultrasound, is a palm-sized, easy-to-control medical ultrasound device that combines a host and a probe into one. It is usually composed of a probe, a processor, a display, and a battery. It has the characteristics of small size, light weight, and low power consumption.

[0003] For example, Chinese patent CN201710560949.0 discloses a handheld three-dimensional ultrasound imaging system and method, including a handheld ultrasound probe for scanning and acquiring ultrasound images; a display, control and processing terminal connected to the handheld ultrasound probe by wired or wireless connection. It also includes: a handheld three-dimensional spatial positioning system, which is connected to the handheld ultrasound probe and moves with the movement of the handheld ultrasound probe, and is connected to the display, control and processing terminal by wired or wireless connection, and is used to independently locate the three-dimensional position of the handheld ultrasound probe. By applying the handheld three-dimensional ultrasound imaging system and method, the huge spatial positioning system in the existing three-dimensional ultrasound imaging system is transformed into a portable and ready-to-use spatial positioning system, so that handheld three-dimensional ultrasound imaging can be widely used.

[0004] However, in actual implementation, the inventors found that handheld ultrasound devices are often used for on-site scanning, such as for simple examinations of patients in emergency treatment, hospital rounds, etc. At this time, in order to improve operational efficiency, a voice control system is often added to the handheld ultrasound device. However, existing voice control systems usually only support single language control, which is not conducive to use. Summary of the invention

[0005] In view of the above problems existing in the prior art, a multi-language voice control method for handheld ultrasound is now provided.

[0006] The specific technical solutions are as follows:

[0007] A multi-language voice control method for handheld ultrasound, comprising:

[0008] Step S1: obtaining an input speech sequence, and extracting vowel syllables from the input speech sequence;

[0009] Step S2: classifying the extracted vowel syllables to determine the language of the input speech sequence;

[0010] Step S3: acquiring a speech recognition model of the corresponding language according to the language, and using the speech recognition model to recognize the input speech sequence to obtain an operation instruction.

[0011] On the other hand, the step S1 comprises:

[0012] Step S11: acquiring the input speech sequence, and performing Fourier transform on the input speech sequence to obtain time-frequency domain data;

[0013] Step S12: segmenting the time-frequency domain data in the time domain according to the signal amplitude to obtain syllable intervals;

[0014] Step S13: classify each of the syllable intervals to obtain the vowel syllables.

[0015] On the other hand, the step S12 comprises:

[0016] Step S121: extracting zero amplitude points in the time domain for the time-frequency domain data, and segmenting the time-frequency domain data according to the zero amplitude points to obtain multiple groups of word time-frequency domain data;

[0017] Step S122: using a sliding window to scan each group of the word time-frequency domain data according to a preset step length, and determining the sliding window amplitude mean at each position;

[0018] Step S123: segmenting according to the sliding window amplitude mean to obtain the syllable intervals.

[0019] On the other hand, the step S13 comprises:

[0020] Step S131: extracting frequency domain signals for the syllable intervals respectively to obtain a first type of frequency domain signal corresponding to the breathy voice frequency band and a second type of frequency domain signal not belonging to the breathy voice frequency band;

[0021] Step S132: determining the breath sound energy proportion corresponding to the syllable interval according to the amplitude of the first type of frequency domain signal and the amplitude of the second type of frequency domain signal;

[0022] Step S133: Compare the breathy voice energy proportion with a pre-determined breathy voice energy threshold to determine the vowel syllable.

[0023] On the other hand, the step S2 comprises:

[0024] Step S21: for each group of vowel syllables, respectively map them to the Mel scale to obtain nonlinear perceptual features;

[0025] Step S22: performing discrete cosine transform according to the nonlinear perceptual feature to obtain a feature vector;

[0026] Step S23: classifying the vowel syllables according to the feature vectors, and determining the language type according to the classification result.

[0027] On the other hand, in step S23, a language recognition model is used for classification, and the language recognition model includes:

[0028] A cascading module, wherein the cascading module cascades multiple groups of feature vectors to obtain a fusion vector;

[0029] A convolutional network, wherein the convolutional network is connected to the cascade module, and the convolutional network extracts the feature vectors in sequence to obtain extracted features;

[0030] A fully connected layer, wherein the fully connected layer is connected to the convolutional network, and the fully connected layer predicts the language according to the extracted features.

[0031] On the other hand, the step S3 comprises:

[0032] Step S31: acquiring a speech recognition model of the corresponding language according to the language, and using the speech recognition model to recognize the input speech sequence;

[0033] Step S32: determining whether the speech recognition model is capable of recognizing the input speech sequence;

[0034] If so, output the operation instruction;

[0035] If not, go to step S33;

[0036] Step S33: matching the pre-recognition result of the speech recognition model with the instruction template to output the operation instruction.

[0037] On the other hand, before executing step S1, the method further includes:

[0038] Step A1: Acquire a real-time audio signal and input it into a buffer area, and determine whether the input speech sequence exists according to the effective signal length in the buffer area.

[0039] The above technical solution has the following advantages or beneficial effects:

[0040] In view of the problem that the voice control system of the handheld ultrasound device in the prior art only supports a single language and has poor effect in actual use, in this embodiment, voice recognition models of multiple languages ​​are preset in the handheld ultrasound device to achieve effective recognition of control instructions in different languages; at the same time, considering the problem that the handheld ultrasound device has low computational efficiency and cannot use relatively complex models for cross-language matching, a classification process for vowel syllable extraction and language prediction is also introduced in advance, which reduces the number of features involved in the classification process, improves processing efficiency, and enables the recognition method to be used in low-power devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The embodiments of the present invention will be described more fully with reference to the attached drawings, which are provided for illustration and description only and are not intended to limit the scope of the present invention.

[0042] Figure 1 It is an overall schematic diagram of an embodiment of the present invention;

[0043] Figure 2 This is a schematic diagram of step S1 of an embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of step S12 of an embodiment of the present invention;

[0045] Figure 4 This is a schematic diagram of step S13 of an embodiment of the present invention;

[0046] Figure 5 This is a schematic diagram of step S2 of an embodiment of the present invention;

[0047] Figure 6 A schematic diagram of a language recognition model according to an embodiment of the present invention;

[0048] Figure 7 This is a schematic diagram of step S3 of an embodiment of the present invention;

[0049] Figure 8 Schematic diagram of step A1 of an embodiment of the present invention. DETAILED DESCRIPTION

[0050] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0051] It should be noted that, in the absence of conflict, the embodiments of the present invention and the features in the embodiments may be combined with each other.

[0052] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, but they are not intended to limit the present invention.

[0053] The present invention comprises:

[0054] A multi-language voice control method for handheld ultrasound, such as Figure 1 As shown, including:

[0055] Step S1: obtaining an input speech sequence, and extracting vowel syllables in the input speech sequence;

[0056] Step S2: classifying the extracted vowel syllables to determine the language of the input speech sequence;

[0057] Step S3: Obtain a speech recognition model of the corresponding language according to the language, and use the speech recognition model to recognize the input speech sequence to obtain an operation instruction.

[0058] Specifically, in view of the problem that the voice control system of the handheld ultrasonic device in the prior art only supports a single language and has poor effect during actual use, in this embodiment, voice recognition models of multiple different languages ​​are preset in the handheld ultrasonic device to achieve effective recognition of control instructions in different languages; at the same time, considering the problem that the handheld ultrasonic device has low computational efficiency and cannot use relatively complex models for cross-language matching, a classification process for extracting vowel syllables and predicting languages ​​is also pre-introduced. Since vowel syllables in different languages ​​often have different pronunciation habits, only distinguishing vowel syllables reduces the feature quantity involved in the classification process, improves processing efficiency, and enables the recognition method to be used in low-power devices.

[0059] In one embodiment, Figure 2 As shown, step S1 includes:

[0060] Step S11: obtaining an input speech sequence, and performing Fourier transform on the input speech sequence to obtain time-frequency domain data;

[0061] Step S12: segmenting the time-frequency domain data in the time domain according to the signal amplitude to obtain syllable intervals;

[0062] Step S13: Classify each syllable interval to obtain vowel syllables.

[0063] Specifically, in order to achieve a better recognition effect, in this embodiment, for the input speech sequence, the input speech sequence is first subjected to Fourier transform to obtain time-frequency domain data, and then the time domain signal is analyzed. First, the time-frequency domain data is segmented in the time domain according to the signal amplitude to obtain the syllable interval corresponding to each pronounced syllable, thereby achieving the trimming of the signal length to be classified. Then, each syllable interval is classified to obtain vowel syllables. The above process realizes the splitting of the input speech sequence with a long length, so that only a single vowel syllable needs to be classified in the subsequent model recognition process, reducing the number of model layers.

[0064] In one embodiment, Figure 3 As shown, step S12 includes:

[0065] Step S121: extracting zero amplitude points in the time domain for the time-frequency domain data, and segmenting the time-frequency domain data according to the zero amplitude points to obtain multiple groups of word time-frequency domain data;

[0066] Step S122: using a sliding window to scan each group of word time-frequency domain data according to a preset step size, and determining the sliding window amplitude mean at each position;

[0067] Step S123: segmenting according to the sliding window amplitude mean to obtain syllable intervals.

[0068] Specifically, in order to achieve a better segmentation effect, in this embodiment, for the time-frequency domain data of the complete sequence, the sequence is first traversed in the time domain direction, and the amplitude at each point is determined. When the zero amplitude point is extracted, it usually corresponds to the interval area between two words in the instruction input process, then the time-frequency domain data is segmented according to the zero amplitude point, and the zero amplitude point is removed to obtain multiple groups of word time-frequency domain data. Subsequently, a sliding window is used to scan each group of word time-frequency domain data according to a preset step size, and multiple sampling windows are generated. For each sampling window, the mean of the amplitude corresponding to the sampling point in the sampling window is calculated respectively, as the sliding window amplitude mean. According to the sliding window amplitude mean, variance statistics are performed to determine the minimum value among the multiple sliding window amplitude means, which corresponds to the weaker vocalization part in the word, that is, the interval between two syllables. On this basis, segmentation is performed to obtain syllable intervals in segments.

[0069] In one embodiment, Figure 4 As shown, step S13 includes:

[0070] Step S131: extracting frequency domain signals for each syllable interval to obtain a first type of frequency domain signal corresponding to the breathy voice frequency band and a second type of frequency domain signal not belonging to the breathy voice frequency band;

[0071] Step S132: determining the breath energy ratio corresponding to the syllable interval according to the amplitude of the first frequency domain signal and the amplitude of the second frequency domain signal;

[0072] Step S133: Compare the breathy voice energy ratio with a pre-determined breathy voice energy threshold to determine a vowel syllable.

[0073] Specifically, in order to achieve a better extraction effect for vowel syllables, in this embodiment, for each syllable interval, the signal components are bandpass filtered in the frequency domain to obtain a first-class frequency domain signal corresponding to the breathy frequency band and a second-class frequency domain signal that does not belong to the breathy frequency band. Subsequently, the amplitude at each frequency point is calculated along the time series direction to obtain the frequency point amplitude mean at each frequency point, and the sum of the amplitudes of the first-class frequency domain signals and the sum of the amplitudes of the second-class frequency domain signals are accumulated according to the classification of the frequency point amplitude mean, and the sum of the amplitudes of the first-class frequency domain signals and the sum of the amplitudes of the second-class frequency domain signals are divided to obtain the breathy energy ratio. The breathy energy ratio is compared with the pre-divided breathy energy threshold to determine the vowel syllable. Since the airflow of vowel syllables is relatively smooth when they are pronounced, their breathy components are usually less, and the identification of vowel syllables can be achieved based on this method.

[0074] In one embodiment, Figure 5 As shown, step S2 includes:

[0075] Step S21: for each group of vowel syllables, respectively map them to the Mel scale to obtain nonlinear perceptual features;

[0076] Step S22: performing discrete cosine transform according to the nonlinear perceptual features to obtain a feature vector;

[0077] Step S23: Classify the vowel syllables according to the feature vectors, and determine the language according to the classification results.

[0078] Specifically, in order to achieve a better classification effect, in this embodiment, each group of vowel syllables is first mapped to the Mel scale to extract nonlinear perceptual features, and then discrete cosine transform is performed according to the nonlinear perceptual features to obtain feature vectors, and classification is performed based on the feature vectors using a model to predict the language. Since the classification object is determined to be the selected vowel syllables before classification, there are certain differences in their pronunciation methods between different languages, so a better matching effect can be achieved.

[0079] In one embodiment, Figure 6 As shown, in step S23, a language recognition model is used for classification, and the language recognition model includes:

[0080] Cascade module 1, cascade module 1 cascades multiple groups of feature vectors to obtain a fusion vector;

[0081] Convolutional network 2, convolutional network 2 is connected to cascade module 1, and convolutional network 2 extracts feature vectors in sequence to obtain extracted features;

[0082] Fully connected layer 3, fully connected layer 3 is connected to convolutional network 2, and fully connected layer 3 predicts the language according to the extracted features.

[0083] Specifically, in order to achieve better recognition effect, in this embodiment, the above-mentioned language recognition model is designed, wherein the cascade module 1 cascades multiple groups of feature vectors corresponding to each recognized vowel syllable to obtain a fusion vector, and then, the feature vectors are extracted in turn through multiple convolutional layers in the convolutional network 2 to obtain deep information therein to obtain extracted features, and finally, the language is predicted according to the extracted features through the fully connected layer 3.

[0084] In one embodiment, Figure 7 As shown, step S3 includes:

[0085] Step S31: obtaining a speech recognition model of the corresponding language according to the language, and using the speech recognition model to recognize the input speech sequence;

[0086] Step S32: determining whether the speech recognition model is capable of recognizing the input speech sequence;

[0087] If so, output the operation instruction;

[0088] If not, go to step S33;

[0089] Step S33: Match the pre-recognition result of the speech recognition model with the instruction template to output the operation instruction.

[0090] Specifically, in order to improve the matching accuracy, in this embodiment, instruction templates of different languages ​​are also set. After the speech recognition model of the corresponding language is obtained according to the language, the speech recognition model is first used to recognize the input speech sequence, and then the pre-recognition result and confidence of the speech recognition model are obtained, and whether the speech recognition model can recognize the input speech sequence is judged according to the confidence and confidence threshold. If the confidence is too low, the pre-recognition result of the speech recognition model is matched with the instruction template, and the closest instruction is obtained as the instruction that the user may input, and the operation instruction is corrected before output.

[0091] In one embodiment, Figure 8 As shown, before executing step S1, the method further includes:

[0092] Step A1: Acquire a real-time audio signal and input it into a buffer, and determine whether there is an input speech sequence according to the effective signal length in the buffer.

[0093] Specifically, in order to achieve a better recording effect, in this embodiment, the real-time audio signal is obtained and cached through the buffer area. At the same time, a shorter sampling window is used to sample the current input real-time audio signal, and it is determined whether the average amplitude of the signal in the sampling window reaches the preset speech volume threshold. If it reaches, it is counted. When the count value reaches the preset length, it is determined that there is valid data, and the input voice sequence is intercepted from the position of the first sampling window that triggers the speech volume threshold.

[0094] The above are only preferred embodiments of the present invention, and are not intended to limit the implementation methods and protection scope of the present invention. Those skilled in the art should be aware that all solutions obtained by equivalent substitutions and obvious changes made using the description and illustrations of the present invention should be included in the protection scope of the present invention.

Claims

1. A multi-language voice control method for handheld ultrasound, characterized in that: include: Step S1: obtaining an input speech sequence, and extracting vowel syllables from the input speech sequence; Step S2: classifying the extracted vowel syllables to determine the language of the input speech sequence; Step S3: acquiring a speech recognition model of the corresponding language according to the language, and using the speech recognition model to recognize the input speech sequence to obtain an operation instruction.

2. The multilingual voice control method according to claim 1, characterized in that: The step S1 comprises: Step S11: acquiring the input speech sequence, and performing Fourier transform on the input speech sequence to obtain time-frequency domain data; Step S12: segmenting the time-frequency domain data in the time domain according to the signal amplitude to obtain syllable intervals; Step S13: classify each of the syllable intervals to obtain the vowel syllables.

3. The multilingual voice control method according to claim 2, characterized in that: The step S12 comprises: Step S121: extracting zero amplitude points in the time domain for the time-frequency domain data, and segmenting the time-frequency domain data according to the zero amplitude points to obtain multiple groups of word time-frequency domain data; Step S122: using a sliding window to scan each group of the word time-frequency domain data according to a preset step length, and determining the sliding window amplitude mean at each position; Step S123: segmenting according to the sliding window amplitude mean to obtain the syllable intervals.

4. The multilingual voice control method according to claim 2, characterized in that: The step S13 comprises: Step S131: extracting frequency domain signals for the syllable intervals respectively to obtain a first type of frequency domain signal corresponding to the breathy voice frequency band and a second type of frequency domain signal not belonging to the breathy voice frequency band; Step S132: determining the breath sound energy proportion corresponding to the syllable interval according to the amplitude of the first type of frequency domain signal and the amplitude of the second type of frequency domain signal; Step S133: Compare the breathy voice energy proportion with a pre-determined breathy voice energy threshold to determine the vowel syllable.

5. The multilingual voice control method according to claim 1, characterized in that: The step S2 comprises: Step S21: for each group of vowel syllables, respectively map them to the Mel scale to obtain nonlinear perceptual features; Step S22: performing discrete cosine transform according to the nonlinear perceptual feature to obtain a feature vector; Step S23: classifying the vowel syllables according to the feature vectors, and determining the language type according to the classification result.

6. The multilingual voice control method according to claim 5, characterized in that: In step S23, a language recognition model is used for classification, and the language recognition model includes: A cascading module, wherein the cascading module cascades multiple groups of feature vectors to obtain a fusion vector; A convolutional network, wherein the convolutional network is connected to the cascade module, and the convolutional network extracts the feature vectors in sequence to obtain extracted features; A fully connected layer, wherein the fully connected layer is connected to the convolutional network, and the fully connected layer predicts the language according to the extracted features.

7. The multilingual voice control method according to claim 1, characterized in that: The step S3 comprises: Step S31: acquiring a speech recognition model of the corresponding language according to the language, and using the speech recognition model to recognize the input speech sequence; Step S32: determining whether the speech recognition model is capable of recognizing the input speech sequence; If so, output the operation instruction; If not, go to step S33; Step S33: matching the pre-recognition result of the speech recognition model with the instruction template to output the operation instruction.

8. The multilingual voice control method according to claim 1, characterized in that: Before executing step S1, the method further includes: Step A1: Acquire a real-time audio signal and input it into a buffer area, and determine whether the input speech sequence exists according to the effective signal length in the buffer area.

Images