Intelligent voice device operation method based on multilingual phoneme sequence

Abstract

The application discloses a kind of intelligent voice device operation methods based on multilingual phoneme sequence, comprising the following working steps: step S1, the voice instruction input by user is acquired, and the phoneme sequence is converted to phoneme sequence;Step S2, using multilingual phoneme recognition model is analyzed to phoneme sequence, preferentially match Chinese phoneme model, when matching degree is lower than preset threshold value, other language phoneme model is automatically introduced to supplement matching;Step S3, according to the phoneme sequence parsed, in the mapping database of preestablished, find the standard mandarin instruction corresponding thereto.This application breaks through the mandarin standardization limit by multilingual phoneme recognition, adapts to different pronunciation user, improves voice interaction ability;User does not need to have standard mandarin pronunciation ability, can use voice control intelligent device;User can define voice instruction according to self language habit, support multiple expression modes, realize more natural voice interaction.

Description

Technical Field

[0001] This invention relates to the field of intelligent voice device technology based on phoneme sequences, and in particular to an operation method for an intelligent voice device based on multilingual phoneme sequences. Background Technology

[0002] With the rapid development of artificial intelligence technology, smart devices such as smart speakers have been widely used in daily life. These devices typically execute user commands through voice recognition technology, greatly improving the interactive experience between users and devices. However, existing smart voice devices still face some challenges. They are mainly designed for users with standard Mandarin pronunciation, and there is still a high barrier to entry for users with non-standard Mandarin pronunciation or those who speak a less common dialect.

[0003] For most smart devices, voice recognition systems are typically designed based on the assumption that users can pronounce Mandarin accurately, with little consideration for differences in region, culture, and language background. For individuals with non-standard Mandarin pronunciation or who speak a less common dialect, existing devices often struggle to accurately recognize their commands. This not only reduces the accuracy of voice recognition but also severely impacts user experience and device adoption.

[0004] Although some smart device manufacturers have attempted to improve this issue in recent years by adding dialect recognition capabilities, current solutions still have significant limitations. While these technologies optimize dialect recognition to some extent, they still cannot effectively solve the recognition problem for users with particularly non-standard pronunciations or significant dialect differences. Especially in scenarios involving some niche dialects, existing systems cannot accurately perform speech recognition, still leading to recognition errors or failures, thus affecting the user's operating experience.

[0005] Therefore, how to improve the accuracy of speech recognition for users with different language backgrounds by smart devices, especially for users with non-standard Mandarin pronunciation or those speaking minority dialects, remains a pressing technical challenge that needs to be addressed.

[0006] Therefore, we have designed an intelligent voice device operation method based on multilingual phoneme sequences to solve the above problems. Summary of the Invention

[0007] The purpose of this invention is to address the shortcomings of existing technologies by proposing an intelligent voice device operation method based on multilingual phoneme sequences.

[0008] To achieve the above objectives, the present invention adopts the following technical solution: A method for operating an intelligent voice device based on multilingual phoneme sequences includes the following steps: Step S1: Obtain the user's voice input and convert the voice signal into a phoneme sequence; Step S2: Use a multilingual phoneme recognition model to parse the phoneme sequence, prioritize matching the Chinese phoneme model, and automatically introduce other language phoneme models for supplementary matching when the matching degree is lower than the preset threshold. Step S3: Based on the parsed phoneme sequence, search for the corresponding standard Mandarin command in the preset mapping database and obtain the corresponding smart device operation; Step S4: Execute the smart device operation corresponding to the standard Mandarin command; Step S5: Optimize phoneme sequence matching rules through user interaction feedback mechanism to improve recognition accuracy.

[0009] Preferably, the phoneme recognition model uses Wav2Vec 2.0 or other self-supervised learning models to extract phoneme-level features from the input speech, thereby enhancing multilingual adaptability.

[0010] Preferably, in step S3, a mapping database between phoneme sequences and standard Mandarin instructions is established, and users are allowed to customize the mapping relationship between input phoneme sequences and standard instructions.

[0011] Preferably, the user interaction feedback mechanism in step S5 includes: (1) Record the actual execution results of the user's voice commands; (2) The user confirms whether the operation is correct. If it is incorrect, the user manually corrects it and stores the corrected phoneme sequence matching relationship to optimize the database.

[0012] Preferably, the voice commands may include Mandarin, non-standard Mandarin, dialects, or foreign languages, and through multilingual phoneme sequence matching, the smart device can recognize and execute the corresponding commands.

[0013] Preferably, the smart device includes one or more of the following: smart speaker, smartphone, smart home control device, and in-vehicle voice assistant.

[0014] Preferred, including: (1) Voice receiving module, used to receive user voice commands; (2) Speech processing module, used to convert speech signals into phoneme sequences and match them using a multilingual phoneme recognition model; (3) Instruction mapping module, used to convert the matched phoneme sequence into standard Mandarin instructions and query the corresponding smart device operation; (4) Equipment control module, used to perform corresponding smart device operations; (5) User feedback module, used to receive user feedback information and optimize phoneme matching rules.

[0015] Preferably, the speech processing module uses deep learning algorithms to extract phoneme-level features in order to improve the accuracy of speech recognition.

[0016] Preferably, the user feedback module allows users to manually adjust the mapping relationship between voice commands and device operations, and dynamically optimizes it through a cloud database.

[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. Improve the device's voice interaction capabilities By using multilingual phoneme recognition, this invention breaks through the limitations of standardization of Mandarin, adapts to users with different pronunciations, and improves voice interaction capabilities.

[0018] 2. Lower the barrier to speech recognition Users do not need to have standard Mandarin pronunciation skills to use voice control for smart devices.

[0019] 3. Enhance user customization capabilities Users can define voice commands according to their own language habits, supporting multiple expression methods to achieve a more natural voice interaction. Attached Figure Description

[0020] Figure 1 This is a flowchart of a method for operating an intelligent voice device based on multilingual phoneme sequences, as proposed in this invention. Detailed Implementation

[0021] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0022] This invention aims to address the problem of low voice recognition accuracy encountered by users with non-standard Mandarin or speaking minority dialects when operating smart voice devices. Existing smart voice devices generally require users to have a certain level of standard Mandarin pronunciation. When a user's pronunciation is not standard or their dialect differs significantly from standard Mandarin, the system cannot correctly recognize the voice, leading to operation failures or misrecognition, thus affecting the user experience. This invention overcomes this problem by combining phoneme recognition technology with a multilingual phoneme model. Even if the user's pronunciation is non-standard or they use a dialect, it can still achieve high-accuracy voice command recognition, improving the voice interaction experience.

[0023] Reference Figure 1 A method for operating an intelligent voice device based on multilingual phoneme sequences includes the following steps: Step S1: Obtain the user's voice input and convert the voice signal into a phoneme sequence; Step S2: The phoneme sequence is parsed using a multilingual phoneme recognition model. The Chinese phoneme model is matched first. When the matching degree is lower than a preset threshold, other language phoneme models are automatically introduced for supplementary matching. The phoneme recognition model uses Wav2Vec 2.0 or other self-supervised learning models to extract phoneme-level features from the input speech to enhance multilingual adaptability. Step S3: Based on the parsed phoneme sequence, search for the corresponding standard Mandarin instruction in the preset mapping database and obtain the corresponding smart device operation. In step S3, a mapping database of phoneme sequence and standard Mandarin instruction is established, and users are allowed to customize the mapping relationship between input phoneme sequence and standard instruction. Step S4: Execute the smart device operation corresponding to the standard Mandarin command; Step S5: Optimize phoneme sequence matching rules through user interaction feedback mechanism to improve recognition accuracy.

[0024] The user interaction feedback mechanism in step S5 includes: (1) Record the actual execution results of the user's voice commands; (2) The user confirms whether the operation is correct. If it is incorrect, the user manually corrects it and stores the corrected phoneme sequence matching relationship to optimize the database.

[0025] More specifically, the voice commands may include Mandarin, non-standard Mandarin, dialects, or foreign languages, and through multilingual phoneme sequence matching, the smart device can recognize and execute the corresponding commands. The smart device includes one or more of the following: smart speaker, smartphone, smart home control device, and in-vehicle voice assistant.

[0026] Preferred, including: (1) Voice receiving module, used to receive user voice commands; (2) Speech processing module, used to convert speech signals into phoneme sequences and match them using a multilingual phoneme recognition model; (3) Instruction mapping module, used to convert the matched phoneme sequence into standard Mandarin instructions and query the corresponding smart device operation; (4) Equipment control module, used to perform corresponding smart device operations; (5) User feedback module, used to receive user feedback information and optimize phoneme matching rules.

[0027] Preferably, the speech processing module uses deep learning algorithms to extract phoneme-level features in order to improve the accuracy of speech recognition.

[0028] Preferably, the user feedback module allows users to manually adjust the mapping relationship between voice commands and device operations, and dynamically optimizes it through a cloud database.

[0029] The premise of this invention is that existing smart devices already possess traditional voice recognition capabilities (such as the technical solution in prior art 1), enabling them to perform corresponding operations after recognizing Mandarin voice commands. The content of the Mandarin voice commands can be commands such as "play music" or "turn on the air conditioner," and these commands correspond to the operations that the smart device can perform, such as starting music playback or starting the air conditioner.

[0030] Example 1. Present users with executable standard Mandarin commands built into the smart device through system dialogue interfaces, voice prompts, or instruction manuals. ,in , Given a set containing n Mandarin instructions. .For example, The content is the instruction to "play music".

[0031] 2. Users follow standardized Mandarin instructions. The user can customize their voice input. Users can provide standard Mandarin commands based on their own language habits or pronunciation characteristics. The audible expression includes, but is not limited to, dialects, non-standard Mandarin, foreign languages, and multilingual expressions, to express the user's understanding of standard Mandarin instructions. The user's intention to interact with the content. For example, a user could use a dialect pronunciation of "play music" or "play music" instead of the standard Mandarin pronunciation of "play music," as a way to interact with the content. Custom sound input for commands.

[0032] 3. Acquisition of smart devices After the custom sound input is given, phoneme recognition processing is performed, converting the audio input into a phoneme sequence (existing mature solutions such as the Wav2Vec 2.0 self-supervised model can all generate phoneme sequences from audio; however, since this is not the focus of this technical solution, it will not be elaborated here). During the recognition process, the audio is prioritized for recognition as Chinese phonemes. If some segments in the audio have a low matching degree with the Chinese phoneme model, other language phoneme models (such as English, French, etc.) are incorporated as supplements for multilingual phoneme recognition. After recognition is completed, the multilingual phoneme sequence corresponding to the custom sound is output, and the output result is a set of phoneme sequences. ,in For Chinese phonemes / phonemes of other languages.

[0033] 4. Phoneme sequence and A mapping relationship is established between the instructions and the corresponding smart device operations, and this mapping relationship is incorporated into the corresponding library of smart device operations for phoneme sequences. Through this mapping relationship, when a smart device subsequently collects external sounds and identifies the acquired phoneme sequences, it can match the corresponding phoneme sequences. At that time, the smart device executes the original standard Mandarin instructions. Corresponding smart device operations. Different phoneme sequences can correspond to the same device operation, making it convenient for users to drive the same operation on smart devices using multiple commonly used expressions. For example, the four corresponding phoneme recognition sequences generated after inputting "play music", "play music", "sing a song", and "sing" in dialect can all correspond to the same device operation action of "start music playback" in the corresponding library.

[0034] 5. After recognizing the external sound phoneme sequence, the output is the recognized phoneme sequence. ,in For Chinese phonemes / phonemes of other languages.

[0035] 6. Obtain the phoneme sequence The phoneme sequence is compared with the phoneme sequence stored in the corresponding library. The phoneme sequence comparison process is as follows: ① Extract the phoneme sequence. The first phoneme in Filter the first phoneme in the corresponding library that is the same The phoneme sequence is included in the phoneme sequence set. ② Extract phoneme sequences The second phoneme Filter the set of phoneme sequences The second phoneme in the middle is the same The phoneme sequence is included in the phoneme sequence set. ③ Repeated extraction process Next, among them Until the phoneme sequence set There is one and only one phoneme sequence.

[0036] 7. Phoneme Sequence Set If there is exactly one phoneme sequence, the comparison is considered successful, and the phoneme sequence is determined. If the phoneme sequence matches the one in the corresponding library, execute the smart device operation corresponding to that phoneme sequence.

[0037] The foregoing has shown and described the basic principles, main features, and advantages of the present invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention, all of which fall within the scope of protection claimed by the present invention. The scope of protection of the present invention is defined by the appended claims and their equivalents.

Claims

1. A method for operating an intelligent voice device based on multilingual phoneme sequences, characterized in that, The work includes the following steps: Step S1: Obtain the user's voice input and convert the voice signal into a phoneme sequence; Step S2: Use a multilingual phoneme recognition model to parse the phoneme sequence, prioritize matching the Chinese phoneme model, and automatically introduce other language phoneme models for supplementary matching when the matching degree is lower than the preset threshold. Step S3: Based on the parsed phoneme sequence, search for the corresponding standard Mandarin command in the preset mapping database and obtain the corresponding smart device operation; Step S4: Execute the smart device operation corresponding to the standard Mandarin command; Step S5: Optimize phoneme sequence matching rules through user interaction feedback mechanism to improve recognition accuracy.

2. The method for operating an intelligent voice device based on multilingual phoneme sequences according to claim 1, characterized in that, The phoneme recognition model uses Wav2Vec 2.0 or other self-supervised learning models to extract phoneme-level features from the input speech, thereby enhancing multilingual adaptability.

3. The method for operating an intelligent voice device based on multilingual phoneme sequences according to claim 1, characterized in that, In step S3, a mapping database between phoneme sequences and standard Mandarin instructions is established, and users are allowed to customize the mapping relationship between input phoneme sequences and standard instructions.

4. The method for operating an intelligent voice device based on multilingual phoneme sequences according to claim 1, characterized in that, The user interaction feedback mechanism in step S5 includes: (1) Record the actual execution results of the user's voice commands; (2) The user confirms whether the operation is correct. If it is incorrect, the user manually corrects it and stores the corrected phoneme sequence matching relationship to optimize the database.

5. The method for operating an intelligent voice device based on multilingual phoneme sequences according to claim 1, characterized in that, The voice commands can include Mandarin, non-standard Mandarin, dialects, or foreign languages, and through multilingual phoneme sequence matching, the smart device can recognize and execute the corresponding commands.

6. The method for operating an intelligent voice device based on multilingual phoneme sequences according to claim 1, characterized in that, The smart devices include one or more of the following: smart speakers, smartphones, smart home control devices, and in-vehicle voice assistants.

7. The method for operating an intelligent voice device based on multilingual phoneme sequences according to claim 1, characterized in that, include: (1) Voice receiving module, used to receive user voice commands; (2) Speech processing module, used to convert speech signals into phoneme sequences and match them using a multilingual phoneme recognition model; (3) Instruction mapping module, used to convert the matched phoneme sequence into standard Mandarin instructions and query the corresponding smart device operation; (4) Equipment control module, used to perform corresponding smart device operations; (5) User feedback module, used to receive user feedback information and optimize phoneme matching rules.

8. The method for operating an intelligent voice device based on multilingual phoneme sequences according to claim 1, characterized in that, The speech processing module uses deep learning algorithms to extract phoneme-level features in order to improve the accuracy of speech recognition.

9. The method for operating an intelligent voice device based on multilingual phoneme sequences according to claim 1, characterized in that, The user feedback module allows users to manually adjust the mapping relationship between voice commands and device operations, and dynamically optimizes it through a cloud database.

Images