Device self-learning method, device, and medium

Abstract

The application relates to a device self-learning method, a device and a medium, and belongs to the technical field of voice recognition. The method comprises the following steps: acquiring n same voice instructions for triggering the same device function; determining the posterior probability of each phoneme of a preset phoneme sequence in each voice frame in each voice instruction; determining phonemes with posterior probability ranking in the front m to obtain candidate phonemes of the voice frame; m is a positive integer; for voice frames with the same frame timing in different voice instructions, x phonemes are determined according to n*m candidate phonemes corresponding to the voice frames with the same frame timing to obtain a target phoneme set; and the optimal phoneme combination corresponding to the voice instruction is determined from the target phoneme sets corresponding to each frame timing. The method can solve the problem that the response speed of an electronic device is slow and the efficiency is low when adding a custom recognition word to the electronic device because a large amount of training resources are consumed each time training is performed, and the model training consumes a relatively long time.

Description

[0001] The application relates to a device self-learning method, a device and a medium, and belongs to the technical field of voice recognition.

[0002] More and more electronic devices have added voice control functions, such as smart bracelets, televisions, air conditioners and smart phones. Therefore, the electronic devices usually have audio acquisition components to obtain voice data of a user, and recognize the voice data according to a pre-trained recognition model to obtain corresponding recognition words. In the case that the recognition words match a recognition list pre-stored in the electronic device, the electronic device executes a device function corresponding to the recognition words.

[0003] In a traditional device self-learning method, when a user adds a custom recognition word to the recognition list, an audio training set corresponding to the recognition word needs to be obtained, and the recognition model is retrained through the audio training set. Only the trained recognition model can recognize the custom recognition word of the user and execute the device function corresponding to the custom recognition word.

[0004] Since the recognition model needs to be retrained every time a custom recognition word is added, and each training needs to consume a large amount of training resources, and the model training consumes a long time, when the custom recognition word is added to the electronic device, the response speed of the electronic device is slow, and the efficiency is too low, which affects the user experience.

[0005] The application provides a device self-learning method, a device and a medium, which can solve the problem that the response speed of the electronic device is slow and the efficiency is too low when a custom recognition word is added to the electronic device due to the fact that the recognition model needs to be retrained every time a custom recognition word is added, and each training needs to consume a large amount of training resources, and the model training consumes a long time. The application provides the following technical solutions:

[0006] In a first aspect, a device self-learning method is improved, and the method comprises:

[0007] n same voice instructions for triggering the same device function are obtained, each voice instruction comprising at least one voice frame; n is a positive integer;

[0008] For each voice frame in each voice instruction, the posterior probability of each phoneme of a preset phoneme sequence in the voice frame is determined; the preset phoneme sequence is a phoneme sequence of a predefined common word;

[0009] The phonemes with a posterior probability ranking in the front m are determined to obtain the candidate phonemes of the voice frame; m is a positive integer;

[0010] ​​​For the speech frames with the same frame timing in different speech instructions, x phonemes are determined according to n×m candidate phonemes corresponding to the speech frames with the same frame timing, so as to obtain a target phoneme set; x is a positive integer;

[0011] An optimal phoneme combination corresponding to the speech instruction is determined from the target phoneme sets corresponding to the respective frame timings, so as to trigger the device function based on the optimal phoneme combination.

[0012] Optionally, the step of determining the optimal phoneme combination corresponding to the speech instruction from the target phoneme sets corresponding to the respective frame timings comprises:

[0013] One phoneme is extracted from each of the target phoneme sets corresponding to the respective frame timings, so as to obtain a phoneme combination;

[0014] If at least two phonemes adjacent in the frame timing are the same in the phoneme combination, the at least two phonemes are merged into the same phoneme;

[0015] Each of the phoneme combinations after the merging operation is scored, and the optimal phoneme combination is determined as the value with the highest score.

[0016] Optionally, the step of scoring each of the phoneme combinations after the merging operation and determining the optimal phoneme combination as the value with the highest score further comprises:

[0017] A preset initial consonant-vowel sound pronunciation rule is obtained;

[0018] Each phoneme in the phoneme combination is deleted using the initial consonant-vowel sound pronunciation rule.

[0019] Optionally, the step of determining x phonemes from n×m candidate phonemes corresponding to the speech frames with the same frame timing in different speech instructions to obtain a target phoneme set comprises:

[0020] In a case where there are the same candidate phonemes in the n×m candidate phonemes corresponding to the speech frames with the same frame timing, the posterior probabilities of the same candidate phonemes are accumulated, so as to obtain y candidate phonemes after the posterior probability accumulation; y is a positive integer;

[0021] x phonemes with the highest posterior probabilities are determined from the y candidate phonemes.

[0022] Optionally, after the step of determining the optimal phoneme combination corresponding to the speech instruction from the target phoneme sets corresponding to the respective frame timings, the method further comprises:

[0023] A matching degree between the optimal phoneme combination and each speech instruction is determined, so as to obtain n matching degrees;

[0024] In a case that the z matching degrees are greater than or equal to the preset matching degree respectively, the device function is triggered based on the optimal phoneme combination, and the z is a positive integer less than or equal to n.

[0025] Optionally, the method comprises:

[0026] In a case that the z confidence degrees are less than the preset confidence degree, updated n same voice instructions are acquired to trigger the step of determining a posterior probability of each phoneme of a preset phoneme sequence in each voice frame in each voice instruction; determining phonemes with a posterior probability ranking in front of m as candidate phonemes of the voice frame; determining x phonemes from n×m candidate phonemes corresponding to the voice frames with the same frame timing in different voice instructions to obtain a target phoneme set; and determining an optimal phoneme combination corresponding to the voice instruction from the target phoneme sets corresponding to each frame timing.

[0027] Optionally, before the step of acquiring n same voice instructions for triggering the same device function, the method further comprises:

[0028] The n same initial voice instructions for triggering the same device function are acquired.

[0029] Each initial voice instruction is framed to obtain the voice instruction.

[0030] Optionally, the preset phoneme sequence is a tonal pinyin phoneme.

[0031] In a second aspect, an electronic device is provided, which comprises a processor and a memory connected to the processor, and the memory stores a program. When the processor executes the program, the device self-learning method provided in the first aspect is implemented.

[0032] In a third aspect, a computer readable storage medium is provided, and the storage medium stores a program. When the program is executed by a processor, the device self-learning method provided in the first aspect is implemented.

[0033] The beneficial effects of the present application at least include: by acquiring n same voice instructions for triggering the same device function; for each voice frame in each voice instruction, determining the posterior probability of each phoneme of a preset phoneme sequence in the voice frame; determining the phonemes with posterior probability ranking in the front m to obtain the candidate phonemes of the voice frame; m is a positive integer; for the voice frames with the same frame timing in different voice instructions, determining x phonemes according to n×m candidate phonemes corresponding to the voice frames with the same frame timing to obtain a target phoneme set; determining the optimal phoneme combination corresponding to the voice instruction from the target phoneme sets corresponding to each frame timing to trigger the device function based on the optimal phoneme combination; can solve the problem that the response speed and efficiency of the electronic device are low when adding a custom recognition word to the electronic device because the recognition model needs to be retrained once for each addition of a custom recognition word, each training consumes a lot of training resources, and the model training consumes a lot of time; from the target phoneme set corresponding to each frame, the optimal phoneme combination is determined, and the optimal phoneme combination is used to trigger the device function, so when the user wants to add a custom recognition word, the electronic device does not need to train the recognition model using the audio training set, but only needs to add the optimal phoneme combination into the recognition list corresponding to the device function, so that the device function can be triggered by the optimal phoneme combination, thereby improving the response speed and efficiency of the electronic device.

[0034] At the same time, when the user adds a custom recognition word, the electronic device can customize exclusive voice instructions for users with different accents and different language habits by acquiring the same voice instruction multiple times, without using official standard languages such as Mandarin, which can improve the intelligent level of the electronic device and improve the comfort of use.

[0035] In addition, the same phonemes with adjacent frame timing in the phoneme combination are merged into the same phoneme, because the voice frames with adjacent frame timing may correspond to the same phoneme when the frame length is short, so merging the same phonemes with adjacent frame timing can make the merged phoneme combination more in line with the language habits of the user, and improve the accuracy of the obtained phoneme combination.

[0036] In addition, deleting each phoneme in the phoneme combination using the initial-final pronunciation rule can ensure that the obtained phoneme combination conforms to the initial-final pronunciation rule and improves the standardization of the obtained phoneme combination.

[0037] In addition, the matching degree between the optimal phoneme combination and each voice instruction is determined to obtain n matching degrees; in the case that z matching degrees are greater than or equal to a preset matching degree, the device function is triggered based on the optimal phoneme combination, which can ensure that the optimal phoneme combination and the voice instruction have small differences, i.e., the voice instruction of the user can trigger the device function, thereby improving the intelligent level of the electronic device.

[0038] The above description is only an overview of the technical solution of this application. In order to better understand the technical means of this application and to implement it in accordance with the contents of the specification, the preferred embodiments of this application are described in detail below with reference to the accompanying drawings. [Attached Image Description]

[0039] Figure 1 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application;

[0040] Figure 2 This is a flowchart of a device self-learning method provided in one embodiment of this application;

[0041] Figure 3 This is a flowchart illustrating a specific example of a device self-learning method provided in one embodiment of this application;

[0042] Figure 4 This is a block diagram of a device self-learning apparatus provided in one embodiment of this application;

[0043] Figure 5 This is a block diagram of an electronic device provided in one embodiment of this application.

Detailed Implementation Methods

[0044] The specific embodiments of this application will be described in further detail below with reference to the accompanying drawings and examples. The following examples are used to illustrate this application, but are not intended to limit the scope of this application.

[0045] Figure 1 This is a schematic diagram of the structure of an electronic device 10 provided in one embodiment of this application. According to... Figure 1 It is known that the electronic device includes at least: a device function component 110 and an audio acquisition component 120.

[0046] Device functional component 110 refers to the component on electronic device 10 used to realize device functions. For example, the refrigeration component and heating component of air conditioning equipment, the lighting component of lighting equipment, etc. This embodiment does not limit the device functional component 110 of electronic device 10.

[0047] The functions provided by the device functional components 110 may be the same or different.

[0048] In this embodiment, the voice commands for triggering different functions of the device functional components 110 are different. The voice command refers to the voice used to trigger the device functional components 110.

[0049] Alternatively, the same device function can be triggered by different voice commands.

[0050] For example, the cooling function of an air conditioner can be triggered by voice commands such as "It's hot," "Turn on cooling," or "Cool down."

[0051] The electronic device 10 also has an audio acquisition component 120 to obtain voice instructions input by a user. The audio acquisition component 120 can be a microphone, a microphone array, a sound card, etc., and the implementation of the audio acquisition component 120 is not limited in the embodiment.

[0052] The number of the audio acquisition component 120 can be one or two, and the number of the audio acquisition component 120 is not limited in the embodiment.

[0053] In the embodiment, the audio acquisition component 120 and the functional device component are integrated in the same device, and in other embodiments, the audio acquisition component 120 and the functional device component can also be located in different devices in communication, and the implementation of the audio acquisition component 120 and the functional device component is not limited in the embodiment.

[0054] The electronic device 10 also has a storage function to store voice instructions obtained by the audio acquisition device.

[0055] In the conventional electronic device 10, there is usually an identification model, and the identification list of the identification model stores trigger words for triggering device functions. After the audio acquisition component 120 obtains the voice instructions of the user, the identification model is used to identify the voice instructions to obtain an identified word. If the trigger word in the identification list corresponding to the identified word matches the device function, the electronic device 10 executes the device function corresponding to the identified word.

[0056] Therefore, when the user expects to add a custom identified word to the identification list, the electronic device 10 needs to obtain an audio training set corresponding to the identified word, retrain the identification model through the audio training set, and only the trained identification model can identify the custom identified word and execute the device function corresponding to the custom identified word.

[0057] However, since the identification model needs to be retrained every time a custom identified word is added, and each training needs to consume a large amount of training resources, and the model training consumes a long time, when the custom identified word is added to the electronic device 10, it will cause the problem of slow response speed and low efficiency of the electronic device 10.

[0058] To address the aforementioned issues, in this embodiment, the electronic device 10 is configured to: acquire n identical voice commands for triggering the same device function; for each voice frame in each voice command, determine the posterior probability of each phoneme in the preset phoneme sequence within the voice frame; determine the phonemes ranked in the top m by posterior probability to obtain candidate phonemes for the voice frame; for voice frames with the same frame timing in different voice commands, determine x phonemes based on the n×m candidate phonemes corresponding to the voice frames with the same frame timing to obtain a target phoneme set; and determine the optimal phoneme combination corresponding to the voice command from the target phoneme set corresponding to each frame timing, so as to trigger the device function based on the optimal phoneme combination.

[0059] Each voice instruction includes at least one voice frame. The preset phoneme sequence is a predefined sequence of phonemes of commonly used words, and n, m, and x are all positive integers.

[0060] Posterior probability refers to the probability that an event has already occurred and the cause of that event was a certain factor.

[0061] In this embodiment, by determining the optimal phoneme combination from the target phoneme set corresponding to each frame and using the optimal phoneme combination to trigger the device function, when the user wants to add a custom recognition word, the electronic device does not need to use the audio training set to train the recognition model. Instead, it only needs to add the optimal phoneme combination to the recognition list corresponding to the device function, and the device function can be triggered through the optimal phoneme combination. Therefore, the response speed and response efficiency of the electronic device can be improved.

[0062] Meanwhile, when users add custom recognition words, electronic devices can acquire the same voice command multiple times, allowing them to customize exclusive voice commands for users with different accents and language habits, without needing to use official standard languages ​​such as Mandarin. This can improve the intelligence of electronic devices and enhance user comfort.

[0063] The device self-learning method proposed in this application will be described below. The following embodiments illustrate the application of the device self-learning method in an electronic device, specifically in a processor within the electronic device. In actual implementation, this method can also be used in other devices that are communicatively connected to the electronic device, such as user terminals or servers. User terminals include, but are not limited to, mobile phones, computers, tablets, wearable devices, etc. This embodiment does not limit the implementation methods of other devices or user terminals.

[0064] The communication connection can be wired or wireless. The wireless communication method can be short-range communication or wireless communication, etc. This embodiment does not limit the communication method between the self-moving device and other devices.Figure 2 is a flowchart of a device self-learning method provided by an embodiment of the present application. The method comprises at least the following steps:

[0065] In step 201, n same voice instructions for triggering the same device function are acquired.

[0066] The device function refers to a function provided by an electronic device.

[0067] For example, the air conditioning device provides the refrigeration function and the heating function, and the lighting device provides the lighting function, etc. The embodiment does not limit the implementation manner of the device function.

[0068] Different users have different accents and different language habits. For the same device function, the pronunciation of the recognition word customized by different users is different. Therefore, acquiring the voice instruction of the user can help different users customize the exclusive voice instruction.

[0069] In the case where n is greater than or equal to 2, multiple inputs of the same voice instruction can also improve the accuracy. Optionally, before acquiring the n same voice instructions for triggering the same device function, the method further comprises: acquiring n same initial voice instructions for triggering the same device function; and performing frame division on each initial voice instruction to obtain the voice instruction.

[0070] The frame division refers to dividing the initial voice instruction into multiple same small segments, and each small segment becomes a frame.

[0071] Specifically, each frame can be 10 ms or 25 ms. The embodiment does not limit the length of each frame.

[0072] Illustratively, the frame division can be implemented by using a moving window function. Commonly used window functions include but are not limited to a rectangular window, a Hanning window, a Hamming window, etc. The embodiment does not limit the implementation manner of the frame division.

[0073] In step 202, for each voice frame in each voice instruction, the posterior probability of each phoneme in a preset phoneme sequence in the voice frame is determined.

[0074] Each voice frame corresponds to a phoneme. The greater the posterior probability, the higher the phoneme probability corresponding to the current voice frame. By determining the posterior probability of each phoneme in the preset phoneme sequence in the voice frame, the phoneme most suitable for the current voice frame can be determined.

[0075] The preset phoneme sequence can be acquired from a corpus collected in a network, acquired from other devices, or pre-stored in the electronic device. The embodiment does not limit the implementation manner of the preset phoneme sequence.

[0076] Optionally, the preset phoneme sequence is a tonal pinyin phoneme.

[0077] For example, a3 in the preset phoneme sequence is a three-tone a, and o1 is a one-tone o.

[0078] Optionally, the preset phoneme sequence includes 121 phoneme sequences. In other embodiments, the preset phoneme sequence can also be 48 English phonemes or 32 Mandarin phonemes, and the implementation of the preset phoneme sequence is not limited in the embodiment.

[0079] Optionally, for each speech frame in each voice instruction, determining the posterior probability of each phoneme in the preset phoneme sequence in the speech frame includes: determining the posterior probability using a neural network model.

[0080] The neural network model can be a feedforward sequential memory network (FSMN) or a recurrent neural network (RNN), and the implementation of the neural network is not limited in the embodiment.

[0081] In step 203, the phonemes with the top m posterior probabilities are determined to obtain candidate phonemes of the speech frame.

[0082] In the case where m is greater than or equal to 2, the top m phonemes of each frame are used as the candidate phonemes of the speech frame. Since the posterior probability of each phoneme in the current frame is only the probability of each phoneme in the current frame, not the probability in the entire voice instruction, retaining the m candidate phonemes can expand the selection range for subsequent use, thereby indirectly improving the accuracy.

[0083] In step 204, for the speech frames with the same frame timing in different voice instructions, x phonemes are determined according to the n×m candidate phonemes corresponding to the speech frames with the same frame timing, to obtain a target phoneme set.

[0084] In the case where n is greater than or equal to 2, the speech frames with the same frame timing in multiple identical voice instructions can have the same candidate phonemes.

[0085] Optionally, for the speech frames with the same frame timing in different voice instructions, x phonemes are determined according to the n×m candidate phonemes corresponding to the speech frames with the same frame timing, to obtain a target phoneme set, including: in the case where there are the same candidate phonemes in the n×m candidate phonemes corresponding to the speech frames with the same frame timing, the posterior probabilities of the same candidate phonemes are accumulated to obtain y candidate phonemes after the posterior probabilities are accumulated; and x phonemes with the highest posterior probabilities are determined from the y candidate phonemes.

[0086] Wherein, y is a positive integer.

[0087] The same candidate phoneme existing in the speech frames with the same frame timing in the multiple same speech instructions indicates that the same candidate phoneme has a greater probability of being the phoneme corresponding to the speech frame, and therefore the probability can be accumulated to increase the posterior probability of the candidate phoneme.

[0088] Optionally, for the speech frames with the same frame timing in different speech instructions, x phonemes are determined according to the n x m candidate phonemes corresponding to the speech frames with the same frame timing to obtain a target phoneme set, including: scoring the m candidate phonemes in each speech frame according to the posterior probability ranking; in the case that the same candidate phonemes exist in the n x m candidate phonemes corresponding to the speech frames with the same frame timing, the scores of the same candidate phonemes are accumulated to obtain y candidate phonemes after score accumulation; and x phonemes with the highest scores are determined from the y candidate phonemes.

[0089] Specifically, in each speech frame, the greater the posterior probability, the higher the ranking, and the higher the score.

[0090] Step 205, determining the optimal phoneme combination corresponding to the speech instruction from the target phoneme sets corresponding to the respective frame timings.

[0091] Specifically, the optimal phoneme combination is stored in the recognition list corresponding to the device function, so as to trigger the device function in the case that the speech instruction input by the user matches the optimal phoneme combination.

[0092] Since the electronic device does not need to convert the speech instruction into a recognized word in text form when the user inputs a speech instruction corresponding to a custom recognized word, but only needs to store the optimal phoneme combination obtained from the speech instruction in the recognition list, the step of converting the phoneme combination into a recognized word in text form can be reduced, and the response speed and efficiency can be improved.

[0093] Optionally, in order to obtain the phoneme combination with the greatest probability in the entire speech instruction, the optimal phoneme combination corresponding to the speech instruction is determined from the target phoneme sets corresponding to the respective frame timings, including: extracting one phoneme from each target phoneme set corresponding to the respective frame timings to obtain a phoneme combination; in the phoneme combination, if at least two phonemes adjacent in frame timing are the same, the at least two phonemes are merged into the same phoneme; and scoring each phoneme combination after the merging operation, and determining the value with the highest score as the optimal phoneme combination.

[0094] Since in the case that the frame duration of the speech frame is short, the speech frames adjacent in frame timing can correspond to the same phoneme, merging the same phonemes adjacent in frame timing into the same phoneme makes the phoneme combination more in line with the user's speech habits, and improves the accuracy of the obtained phoneme combination.

[0095] It should be noted that in the case of the preset phoneme sequence being 121 adjusted pinyin phonemes, the preset phoneme sequence includes a special phoneme "blk", blk does not participate in pronunciation in the phoneme combination, but in the case that one of the frames in the phoneme combination is blk, and the phonemes of the two frames adjacent to the blk frame in time sequence are the same, and the same phoneme is not blk, the same phonemes of the two frames adjacent to the blk frame in time sequence are not merged.

[0096] Optionally, scoring each phoneme combination after performing the merging operation, and determining the optimal phoneme combination as the value with the highest score, includes: scoring each phoneme combination after performing the merging operation using a speech recognition model, and determining the optimal phoneme combination as the value with the highest score.

[0097] The speech recognition model can be a Connectionist Temporal Classification (CTC) model based on a neural network, or a sequence-to-sequence (seq2seq) model, and the like, and the implementation of the speech recognition model is not limited in the embodiment.

[0098] Optionally, the phoneme combination should follow the initial-final pronunciation rule, and scoring each phoneme combination after performing the merging operation, and determining the optimal phoneme combination as the value with the highest score, further includes: obtaining a preset initial-final pronunciation rule; and deleting each phoneme in the phoneme combination using the initial-final pronunciation rule.

[0099] Optionally, after obtaining the optimal phoneme combination, the optimal phoneme combination needs to be verified, and the optimal phoneme combination corresponding to the voice command is determined from the target phoneme set corresponding to each frame time sequence, and the optimal phoneme combination after obtaining the optimal phoneme combination further includes: determining the matching degree between the optimal phoneme combination and each voice command to obtain n matching degrees; and in the case that the z matching degrees are greater than or equal to a preset matching degree, triggering a device function based on the optimal phoneme combination.

[0100] Wherein, z is a positive integer less than or equal to n.

[0101] In the case that the z matching degrees are greater than or equal to the preset matching degree, it is determined that the optimal phoneme combination passes the verification, in other words, the matching degrees of the z voice commands are greater than or equal to the preset matching degree, which means that the voice command input by the user has a small difference with the optimal phoneme combination, i.e., the voice command input by the user subsequently can trigger the device function, i.e., the user successfully adds a custom recognition word to the electronic device.

[0102] In a case where the z matching degrees are less than the preset matching degree, the n same updated voice instructions are acquired to trigger the execution of the following steps: for each voice frame in each voice instruction, determining a posterior probability of each phoneme in the preset phoneme sequence in the voice frame; determining phonemes with top m posterior probabilities to obtain candidate phonemes of the voice frame; for voice frames with same frame timing in different voice instructions, determining x phonemes from n x m candidate phonemes corresponding to the voice frames with the same frame timing to obtain a target phoneme set; and determining an optimal phoneme combination corresponding to the voice instruction from the target phoneme sets corresponding to respective frame timings.

[0103] In a case where the z matching degrees are less than the preset matching degree, it is determined that the optimal phoneme combination fails to pass the verification, that is, the matching degrees of the z voice instructions are less than the preset matching degree, which indicates that the voice instruction input by the user is greatly different from the optimal phoneme combination, that is, the voice instruction of the user cannot trigger the device function, and thus the n same voice instructions need to be re-acquired to re-execute the step of obtaining the optimal phoneme combination.

[0104] In the following, the device self-learning method provided in the embodiment is described with a specific example. In the example, n is 3, the preset phoneme sequence is 121 tonal pinyin phonemes, m is 10, and x is 3, and the method is described with reference to FIG. 8. Figure 3

[0105] In step 301, three same voice instructions for triggering the same device function are acquired, and each voice instruction includes at least one voice frame.

[0106] In step 302, the three same voice instructions are sequentially input into the FSMN, and for each voice frame in each voice instruction, a posterior probability of each phoneme in the preset phoneme sequence in the voice frame is determined.

[0107] In step 303, phonemes with top 10 posterior probabilities are determined to obtain candidate phonemes of the voice frame.

[0108] In step 304, the candidate phonemes are scored according to the posterior probability ranking in each voice frame, and the higher the posterior probability ranking is, the higher the score is.

[0109] In step 305, for voice frames with the same frame timing in different voice instructions, in a case where there are same candidate phonemes in 30 candidate phonemes corresponding to the voice frames with the same frame timing, scores of the same candidate phonemes are added to obtain y candidate phonemes after score addition.

[0110] In step 306, three phonemes with the highest scores are determined from the y candidate phonemes to obtain a target phoneme set.

[0111] ​Step 307, determining the optimal phoneme combination corresponding to the voice instruction from the target phoneme set corresponding to each frame time sequence;

[0112] Step 308, storing the optimal phoneme combination into the recognition list corresponding to the device function, so as to trigger the device function based on the optimal phoneme combination.

[0113] In the embodiment, by obtaining n same voice instructions for triggering the same device function; for each voice frame in each voice instruction, determining the posterior probability of each phoneme of the preset phoneme sequence in the voice frame; determining the phonemes with posterior probability ranking in the front m, to obtain the candidate phonemes of the voice frame; m is a positive integer; for the voice frames with the same frame time sequence in different voice instructions, determining x phonemes according to n*m candidate phonemes corresponding to the voice frames with the same frame time sequence, to obtain the target phoneme set; determining the optimal phoneme combination corresponding to the voice instruction from the target phoneme set corresponding to each frame time sequence, so as to trigger the device function based on the optimal phoneme combination; the problem that the response speed and efficiency of the electronic device are low when adding the custom recognition word can be solved, because the recognition model needs to be retrained once for each addition of the custom recognition word, and each training needs to consume a large amount of training resources, and the model training consumes a long time; the optimal phoneme combination is determined from the target phoneme set corresponding to each frame, and the device function is triggered by using the optimal phoneme combination, so when the user wants to add the custom recognition word, the electronic device does not need to train the recognition model using the audio training set, but only needs to add the optimal phoneme combination into the recognition list corresponding to the device function, so that the device function can be triggered by the optimal phoneme combination, thereby the response speed and efficiency of the electronic device can be improved.

[0114] Meanwhile, when the user adds the custom recognition word, the electronic device can customize the exclusive voice instruction for the user with different accents and different language habits by obtaining the same voice instruction multiple times, without using the official standard language such as Mandarin, so that the intelligent degree of the electronic device can be improved, and the comfort degree of use can be improved.

[0115] In addition, the same phonemes with adjacent frame time sequences are merged into the same phoneme in the phoneme combination, because when the frame length is short, the voice frames with adjacent frame time sequences may correspond to the same phoneme, so the merging of the same phonemes with adjacent frame time sequences can make the merged phoneme combination more consistent with the language habits of the user, and improve the accuracy of the obtained phoneme combination.

[0116] In addition, the deletion of each phoneme in the phoneme combination using the initial-final pronunciation rule can ensure that the obtained phoneme combination conforms to the initial-final pronunciation rule, and improve the standardization of the obtained phoneme combination.

[0117] In addition, the matching degrees between the optimal phoneme combination and each voice instruction are determined, n matching degrees are obtained, and in a case where z matching degrees are greater than or equal to a preset matching degree, the device function is triggered based on the optimal phoneme combination, which can ensure that the optimal phoneme combination is less different from the voice instruction, that is, the voice instruction of the user can trigger the device function, and therefore the intelligent degree of the electronic device can be improved.

[0118] Figure 4 is a block diagram of a device self-learning apparatus provided in an embodiment of the present application. The apparatus includes at least the following modules: a voice acquisition module 410, a probability determination module 420, a candidate determination module 430, a target determination module 440, and an optimal determination module 450.

[0119] The voice acquisition module 410 is configured to acquire n same voice instructions for triggering a same device function.

[0120] The probability determination module 420 is configured to determine, for each voice frame in each voice instruction, a posterior probability of each phoneme of a preset phoneme sequence in the voice frame.

[0121] The candidate determination module 430 is configured to determine phonemes with a posterior probability ranked in a front m, to obtain a candidate phoneme of the voice frame.

[0122] The target determination module 440 is configured to determine, for voice frames with a same frame timing in different voice instructions, x phonemes from n x m candidate phonemes corresponding to the voice frames with the same frame timing, to obtain a target phoneme set.

[0123] The optimal determination module 450 is configured to determine, from the target phoneme sets corresponding to respective frame timings, an optimal phoneme combination corresponding to the voice instruction.

[0124] For related details, refer to the method embodiments described above.

[0125] It should be noted that the device self-learning apparatus provided in the above embodiments is only taken as an example for the division of the functional modules, and in actual applications, the above functions can be completed by different functional modules according to needs, that is, the internal structure of the device self-learning apparatus is divided into different functional modules to complete all or part of the functions described above. In addition, the device self-learning apparatus and the device self-learning method provided in the above embodiments belong to the same concept, and the specific implementation process is described in the method embodiments, which will not be repeated here.

[0126] Figure 5 is a block diagram of an electronic device provided in an embodiment of the present application. The device includes at least a processor 501 and a memory 502.

[0127] The processor 501 can include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 501 can be implemented in at least one of a hardware form of a DSP (Digital Signal Processing), an FPGA (Field-Programmable Gate Array), a PLA (Programmable Logic Array). The processor 501 can also include a main processor and a coprocessor, the main processor being a processor for processing data in an awake state, also known as a CPU (Central Processing Unit), and the coprocessor being a low-power processor for processing data in a standby state. In some embodiments, the processor 501 can be integrated with a GPU (Graphics Processing Unit) that is responsible for rendering and drawing the content required to be displayed by the display screen. In some embodiments, the processor 501 can also include an AI (Artificial Intelligence) processor for processing machine learning-related computing operations.

[0128] The memory 502 can include one or more computer-readable storage media, which can be non-transitory. The memory 502 can also include a high-speed random access memory, and a non-volatile memory such as one or more disk storage devices, flash storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 502 is used to store at least one instruction for being executed by the processor 501 to implement the device self-learning method provided by the method embodiments in the present application.

[0129] In some embodiments, the electronic device can also optionally include a peripheral device interface and at least one peripheral device. The processor 501, the memory 502, and the peripheral device interface can be connected through a bus or a signal line. Each peripheral device can be connected to the peripheral device interface through a bus, a signal line, or a circuit board. Illustratively, the peripheral devices include, but are not limited to, radio frequency circuitry, a touch display screen, audio circuitry, and a power supply, etc.

[0130] Of course, the electronic device can also include fewer or more components, which are not limited in the present embodiment.

[0131] Optionally, the present application also provides a computer-readable storage medium, which stores a program, the program being loaded and executed by a processor to implement the device self-learning method of the above-mentioned method embodiments.

[0132] Optionally, the application further provides a computer product, comprising a computer readable storage medium, wherein a program is stored in the computer readable storage medium, and the program is loaded and executed by a processor to implement the device self-learning method of the above method embodiment.

[0133] Any combination of the technical features in the above-described embodiments can be made, and to make the description concise, all possible combinations of the technical features in the above-described embodiments are not described, however, as long as the combination of the technical features does not exist, it should be considered as the scope of the description.

[0134] The above-described embodiments only express several implementation manners of the application, the description is more specific and detailed, but it should not be understood as a limitation on the patent scope of the application. It should be pointed out that, for ordinary skilled in the art, without departing from the concept of the application, a number of modifications and improvements can be made, which are all within the protection scope of the application. Therefore, the patent protection scope of the application should be subject to the appended claims.

Claims

1. A device self-learning method characterized by, The method comprises: acquiring n same voice instructions for triggering the same device function, each voice instruction comprising at least one voice frame; n is a positive integer; for each voice frame in each voice instruction, determining the posterior probability of each phoneme in a preset phoneme sequence in the voice frame; the preset phoneme sequence is a phoneme sequence of a predefined commonly used word; determining phonemes with posterior probabilities ranked in the top m to obtain candidate phonemes of the voice frame; m is a positive integer; for voice frames with the same frame timing in different voice instructions, determining x phonemes from n×m candidate phonemes corresponding to the voice frames with the same frame timing to obtain a target phoneme set; x is a positive integer; in the case that there are same candidate phonemes in the n×m candidate phonemes corresponding to the voice frames with the same frame timing, the posterior probabilities of the same candidate phonemes are accumulated to obtain y candidate phonemes after the posterior probabilities are accumulated; y is a positive integer; x phonemes with the highest posterior probabilities are determined from the y candidate phonemes; from the target phoneme sets corresponding to each frame timing, determining an optimal phoneme combination corresponding to the voice instructions to trigger the device function based on the optimal phoneme combination; the determination of the optimal phoneme combination corresponding to the voice instructions from the target phoneme sets corresponding to each frame timing comprises: extracting one phoneme from each target phoneme set corresponding to each frame timing to obtain a phoneme combination; in the phoneme combination, if at least two phonemes with adjacent frame timings are same, the at least two phonemes are merged into the same phoneme; each phoneme combination after the merging operation is scored, and the optimal phoneme combination is determined as the value with the highest score.

2. The method of claim 1, wherein, The scoring of each phoneme combination after the merging operation and the determination of the optimal phoneme combination as the value with the highest score further comprise: acquiring a preset initial consonant-vowel sound pronunciation rule; using the initial consonant-vowel sound pronunciation rule to delete each phoneme in the phoneme combination.

3. The method of claim 1, wherein, After the determination of the optimal phoneme combination corresponding to the voice instructions from the target phoneme sets corresponding to each frame timing, the method further comprises: determining the matching degree between the optimal phoneme combination and each voice instruction to obtain n matching degrees; in the case that z matching degrees are greater than or equal to the preset matching degree, triggering the device function based on the optimal phoneme combination, z is a positive integer less than or equal to n.

4. The method of claim 3, wherein, The method comprises: in the case that the z matching degrees are less than the preset matching degree, acquiring updated n same voice instructions to trigger the steps of determining, for each voice frame in each voice instruction, the posterior probability of each phoneme in a preset phoneme sequence in the voice frame; determining phonemes with posterior probabilities ranked in the top m to obtain candidate phonemes of the voice frame; for voice frames with the same frame timing in different voice instructions, determining x phonemes from n×m candidate phonemes corresponding to the voice frames with the same frame timing to obtain a target phoneme set; and determining an optimal phoneme combination corresponding to the voice instructions from the target phoneme sets corresponding to each frame timing.

5. The method of claim 1, wherein, Before the acquisition of n same voice instructions for triggering the same device function, the method further comprises: acquire n same initial voice instructions for triggering the same device function; frame each initial voice instruction to obtain the voice instruction.

6. The method according to any one of claims 1 to 5, characterized in that, The preset phoneme sequence is a tonal pinyin phoneme.

7. An electronic device, comprising: The electronic device includes a processor and a memory connected to the processor, the memory stores a program, and the processor executes the program to implement the device self-learning method of any one of claims 1-6.

8. A computer-readable storage medium, characterized in that, The storage medium stores a program, and the program is executed by the processor to implement the device self-learning method of any one of claims 1-6.

Images