Methods and systems for constructing speech recognition models and speech processing

Abstract

Embodiments of the present disclosure provide methods and systems for constructing a speech recognition model and speech processing. In the method for constructing a speech recognition model of the present disclosure, a target keyword is obtained, and a synonym group semantically associated with the target keyword is determined. Subsequently, a language model is trained using the target keyword and the synonym group to obtain a target language model. A first decoding graph is generated according to the target language model, wherein the first decoding graph indicates a plurality of decoding paths satisfying a grammar constraint rule determined based on the target keyword and the synonym group, and a speech recognition model is determined based on the first decoding graph. In this way, embodiments of the present disclosure can construct a speech recognition model with keyword generalization recognition capability.

Description

Technical Field

[0001] The embodiments of this disclosure are primarily related to the field of computers, and more specifically, to methods and systems for constructing speech recognition models and speech processing. Background Technology

[0002] With the development of computer technology, speech recognition technology has been widely applied to various aspects of people's lives. Semantic understanding is a crucial technology for achieving voice interaction during speech recognition. This is especially true for lightweight devices with limited computing power (such as smart home devices), where semantic understanding is key to enabling voice control.

[0003] However, currently available lightweight devices that support voice recognition typically cannot support custom semantics, and are completely unable to handle voice input with the same or similar semantics, which greatly affects the user's voice interaction experience. Summary of the Invention

[0004] The embodiments of this disclosure provide a scheme for constructing a speech recognition model and speech processing.

[0005] In a first aspect of this disclosure, a method for constructing a speech recognition model is provided. The method includes: obtaining target keywords; obtaining synonyms semantically associated with the target keywords; training a language model using the target keywords and synonyms to obtain a target language model; generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths satisfying grammatical constraints determined based on the target keywords and synonyms; and determining a speech recognition model based on the first decoding graph.

[0006] In some embodiments of the first aspect, the method may be performed, for example, by a first computing device with strong computing capabilities. The first computing device may include, for example, a cloud-based or embedded heavy-duty device with strong computing capabilities for constructing a speech recognition model. The first computing device may also include, for example, a user terminal device. In some embodiments of the first aspect, one or more steps in the method may also be performed collaboratively by the user terminal and the cloud.

[0007] In some embodiments of the first aspect, the obtained speech recognition model may, for example, be deployed to a second computing device. The second computing device may include an embedded lightweight device, for example, having limited computing power, for performing speech processing using the deployed speech recognition model. Examples of the second computing device may include, but are not limited to: smart home devices (e.g., air conditioners, refrigerators, washing machines, televisions, speakers, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.), or in-vehicle devices, etc.

[0008] Based on this approach, embodiments of this disclosure can construct a speech recognition model with keyword generalization recognition capability, thereby enabling a second computing device with relatively low computing power to have keyword generalization recognition capability, thereby improving the user's interactive experience.

[0009] In some embodiments of the first aspect, the target keywords include keywords from voice input by an audio collector located at a user terminal. In other embodiments of the first aspect, the target keywords include keywords from text input by a text collector located at a user terminal.

[0010] For example, if the first computing device is a user terminal, the user can directly provide voice input or text input to the user terminal so that the user terminal can extract keywords from the voice input or text input to build a speech recognition model.

[0011] For example, if the first computing device is a different computing device from the user terminal, such as a cloud device or an edge computing device, the user can, for example, use the interface provided by the user terminal to input voice or text. Such voice or text input can be sent to the first computing device so that the first computing device can obtain keywords to build a speech recognition model.

[0012] This approach allows users to customize the keywords supported by the speech recognition model, thereby improving the personalization of the speech recognition model.

[0013] In some embodiments of the first aspect, determining a group of synonyms that are semantically associated with a target keyword includes: determining the semantics of the target keyword; and determining a group of synonyms based at least on the semantics of the target keyword, wherein the semantic difference between each synonym in the group of synonyms and the semantics of the target keyword is less than a difference threshold.

[0014] In this way, the first computing device can automatically generate related synonyms based on semantics, without relying on user input, thereby improving and reducing user interaction overhead.

[0015] In some embodiments of the first aspect, determining synonym groups based at least on the semantics of the target keyword includes: determining synonym groups based on the semantics and length of the target keyword, wherein the difference between the length of each synonym in the synonym group and the length of the target keyword is less than a length threshold. The length of the keyword may, for example, represent the number of characters or words included in the keyword. In this way, synonyms in the synonym group can have similar lengths, thereby reducing the complexity of decoding searches using a decoding graph.

[0016] In some embodiments of the first aspect, determining a group of synonyms based at least on the semantics of the target keyword includes: obtaining a plurality of candidate synonyms based on the semantics of the target keyword; providing the plurality of candidate synonyms to a user; and determining a group of synonyms from the plurality of candidate synonyms based on user input received from the user, wherein the user input indicates that at least one of the plurality of candidate synonyms is excluded or confirmed.

[0017] Based on this approach, the synonym pairs used to train the speech recognition model can be further adjusted based on user feedback, which can make the obtained speech recognition model more in line with users' usage habits.

[0018] In some embodiments of the first aspect, the target keyword includes at least a first keyword and a second keyword, wherein determining the speech recognition model based on the first decoding graph includes: obtaining a first set of decoding paths and a second set of decoding paths from the first decoding graph, wherein the first set of decoding paths includes decoding paths corresponding to the first keyword and a first set of synonyms semantically associated with the first keyword, and the second set of decoding paths includes decoding paths corresponding to the second keyword and a second set of synonyms semantically associated with the second keyword; generating a first subgraph based on the first set of decoding paths; generating a second subgraph based on the second set of decoding paths; and determining the speech recognition model based at least on the first subgraph and the second subgraph. In this manner, the generated decoding graph has low complexity, supports faster decoding search, thereby reducing computational and storage overhead.

[0019] In some embodiments of the first aspect, a first subgraph indicates a first decoding path and a second decoding path, wherein the first decoding path is the decoding path corresponding to a first keyword, and the second decoding path is the decoding path corresponding to a synonym in a first synonym group, and the first decoding path and each second decoding path have the same weight in the first subgraph. Based on this approach, faster decoding search for expanded synonyms can be achieved, thereby reducing computational and storage overhead.

[0020] In some embodiments of the first aspect, obtaining the target keyword includes: obtaining a first keyword group based on pre-stored historical keywords and received keywords; and in response to determining that the number of keywords in the first keyword group exceeds a predetermined threshold, obtaining the target keyword from the first keyword group based on the predetermined threshold. For example, the first computing device may retain only the keywords of a predetermined threshold number in the first keyword group as the target keyword.

[0021] This approach avoids using too many keywords to train the language model, thus ensuring that the decoding graph is lightweight and suitable for devices with limited computing resources.

[0022] In some embodiments of the first aspect, obtaining target keywords from a first keyword group based on a predetermined threshold includes: obtaining target keywords from the first keyword group according to the attributes of keywords in the target keywords, wherein the number of target keywords is a predetermined threshold. For example, one or more historical keywords that were created earliest can be deleted from the first keyword group to obtain the predetermined threshold number of keywords.

[0023] In some embodiments of the first aspect, obtaining target keywords from a first keyword group based on a predetermined threshold includes: obtaining target keywords from the first keyword group according to user instructions, wherein the number of target keywords is a predetermined threshold. For example, user input can be used to select which keywords in the first keyword group to retain as target keywords.

[0024] In some embodiments of the first aspect, the first computing device may also instruct a target computing device (e.g., a second computing device) to provide a speech recognition model for deployment on the target computing device. This approach enables the automatic deployment of the speech recognition model.

[0025] In a second aspect of this disclosure, a method for speech processing is provided. The method includes: receiving speech input; and using a speech recognition model to determine a text representation associated with the speech input, wherein the speech recognition model is obtained based on the following process: acquiring target keywords; acquiring synonyms semantically associated with the target keywords; training a language model using the target keywords and synonyms to obtain a target language model; generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths satisfying grammatical constraint rules determined based on the target keywords and synonyms; and determining the speech recognition model based on the first decoding graph.

[0026] In some embodiments of the second aspect, the speech recognition model may be obtained by a first computing device. The first computing device may, for example, include a cloud-based or embedded device with strong computing capabilities for constructing the speech recognition model. The first computing device may also include, for example, a user terminal device.

[0027] In some embodiments of the second aspect, the step of obtaining the speech recognition model may also be performed collaboratively by the user terminal and the cloud.

[0028] In some embodiments of the second aspect, the speech processing method may be performed, for example, by a second computing device. The second computing device may include an embedded lightweight device, for example, having limited computing power, to perform speech processing using a deployed speech recognition model. Examples of the second computing device may include, but are not limited to: smart home devices (e.g., air conditioners, refrigerators, washing machines, televisions, speakers, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.), or in-vehicle devices, etc.

[0029] Based on this approach, embodiments of this disclosure enable, for example, a second computing device with lower computing power to have keyword generalization recognition capabilities, thereby improving the user's voice interaction experience.

[0030] In some embodiments of the second aspect, the target keywords include keywords from voice input by an audio collector located at a user terminal. In other embodiments of the second aspect, the target keywords include keywords from text input by a text collector located at a user terminal.

[0031] For example, if the first computing device is a user terminal, the user can directly provide voice input or text input to the user terminal so that the user terminal can extract keywords from the voice input or text input to build a speech recognition model.

[0032] For example, if the first computing device is a different computing device from the user terminal, such as a cloud device or an edge computing device, the user can, for example, use the interface provided by the user terminal to input voice or text. Such voice or text input can be sent to the first computing device so that the first computing device can obtain keywords to build a speech recognition model.

[0033] In this way, users can customize the keywords that the speech recognition model can support, thereby increasing the personalization of the speech recognition model.

[0034] In some embodiments of the second aspect, determining a group of synonyms associated with the semantics of the target keyword includes: determining the semantics of the target keyword; and determining a group of synonyms based at least on the semantics of the target keyword, wherein the semantics of each synonym in the group of synonyms differs from the semantics of the target keyword by less than a difference threshold.

[0035] In this way, the first computing device can automatically generate related synonyms based on semantics, without relying on user input, thereby improving and reducing user interaction overhead.

[0036] In some embodiments of the second aspect, determining synonym groups based at least on the semantics of the target keyword includes: determining synonym groups based on the semantics and length of the target keyword, wherein the difference between the length of each synonym in the synonym group and the length of the target keyword is less than a length threshold. The length of the keyword can, for example, represent the number of characters or words included in the keyword. In this way, synonyms in the synonym group can have similar lengths, thereby reducing the complexity of decoding searches using a decoding graph.

[0037] In some embodiments of the second aspect, determining a group of synonyms based at least on the semantics of the target keyword includes: obtaining a plurality of candidate synonyms based on the semantics of the target keyword; providing the plurality of candidate synonyms to a user; and determining a group of synonyms from the plurality of candidate synonyms based on user input received from the user, wherein the user input indicates that at least one of the plurality of candidate synonyms is excluded or confirmed.

[0038] Based on this approach, the synonym pairs used to train the speech recognition model can be further adjusted based on user feedback, which can make the obtained speech recognition model more in line with users' usage habits.

[0039] In some embodiments of the second aspect, the target keywords include at least a first keyword and a second keyword. Determining the speech recognition model based on the first decoding graph includes: obtaining a first set of decoding paths and a second set of decoding paths from the first decoding graph. The first set of decoding paths includes decoding paths corresponding to the first keyword and a first set of synonyms semantically associated with the first keyword. The second set of decoding paths includes decoding paths corresponding to the second keyword and a second set of synonyms semantically associated with the second keyword. A first subgraph is generated based on the first set of decoding paths, and a second subgraph is generated based on the second set of decoding paths. The speech recognition model is then determined based at least on the first and second subgraphs. In this manner, the generated decoding graph has lower complexity, enabling faster decoding searches and thus reducing computational and storage overhead.

[0040] In some embodiments of the second aspect, the first subgraph indicates a first decoding path and a second decoding path. The first decoding path is the decoding path corresponding to the first keyword, and the second decoding path is the decoding path corresponding to a synonym in the first synonym group. The first decoding path and each second decoding path have the same weight in the first subgraph. Based on this approach, faster decoding search for the expanded synonyms can be achieved, thereby reducing computational and storage overhead.

[0041] In some embodiments of the second aspect, obtaining the target keyword includes: obtaining a first keyword group based on pre-stored historical keywords and received keywords; and in response to determining that the number of keywords in the first keyword group exceeds a predetermined threshold, obtaining the target keyword from the first keyword group based on the predetermined threshold. For example, only keywords of a predetermined threshold number in the first keyword group may be retained as the target keyword.

[0042] This approach avoids using too many keywords to train the language model, thus ensuring that the decoding graph is lightweight and suitable for devices with limited computing resources.

[0043] In some embodiments of the second aspect, obtaining target keywords from a first keyword group based on a predetermined threshold includes: obtaining target keywords from the first keyword group according to the attributes of the keywords in the target keywords, wherein the number of target keywords is a predetermined threshold. For example, one or more historical keywords that were created earliest can be deleted from the first keyword group to obtain the predetermined threshold number of keywords.

[0044] In some embodiments of the second aspect, obtaining target keywords from the first keyword group based on a predetermined threshold includes: obtaining target keywords from the first keyword group according to user instructions, wherein the number of target keywords is the predetermined threshold. For example, user input can be used to select which keywords in the first keyword group to retain as target keywords.

[0045] In some embodiments of the second aspect, the second computing device may also perform an action corresponding to the text representation. Alternatively, the second computing device may also generate a corresponding control command based on the text representation and send it to the third computing device, causing the third computing device to perform the corresponding action.

[0046] In some embodiments of the second aspect, the text represents a synonym corresponding to a target keyword or a group of synonyms.

[0047] In a third aspect of this disclosure, a speech model construction system is provided. The system includes a keyword acquisition unit for acquiring target keywords; a synonym acquisition unit for acquiring synonym phrases semantically related to the target keywords; a model training unit for training a language model using the target keywords and synonym phrases to obtain a target language model; a decoding graph generation unit for generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths satisfying grammatical constraints determined based on the target keywords and synonym phrases; and a model determination unit for determining a speech recognition model based on the first decoding graph.

[0048] In some embodiments of the third aspect, the speech model building system may, for example, include a first computing device with strong computing capabilities. The first computing device may, for example, include a cloud-based or embedded heavy-duty device with strong computing capabilities for constructing a speech recognition model. The first computing device may also include, for example, a user terminal device. In some embodiments of the third aspect, the method may also be performed collaboratively by the user terminal and the cloud.

[0049] In some embodiments of the third aspect, the obtained speech recognition model may, for example, be deployed to a second computing device. The second computing device may include an embedded lightweight device, for example, having limited computing power, to perform speech processing using the deployed speech recognition model. Examples of the second computing device may include, but are not limited to: smart home devices (e.g., air conditioners, refrigerators, washing machines, televisions, speakers, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.), or in-vehicle devices, etc.

[0050] Based on this approach, embodiments of this disclosure can construct a speech recognition model with keyword generalization recognition capability, thereby enabling a second computing device with relatively low computing power to have keyword generalization recognition capability, thereby improving the user's interactive experience.

[0051] In some embodiments of the third aspect, the target keywords include keywords from voice input by an audio collector located at a user terminal. In other embodiments of the third aspect, the target keywords include keywords from text input by a text collector located at a user terminal.

[0052] For example, if the first computing device is a user terminal, the user can directly provide voice input or text input to the user terminal so that the user terminal can extract keywords from the voice input or text input to build a speech recognition model.

[0053] For example, if the first computing device is a different computing device from the user terminal, such as a cloud device or an edge computing device, the user can, for example, use the interface provided by the user terminal to input voice or text. Such voice or text input can be sent to the first computing device so that the first computing device can obtain keywords to build a speech recognition model.

[0054] In this way, users can customize the keywords that the speech recognition model can support, thereby increasing the personalization of the speech recognition model.

[0055] In some embodiments of the third aspect, the synonym acquisition unit is further configured to: determine the semantics of the target keyword; and determine a group of synonyms based at least on the semantics of the target keyword, wherein the difference between the semantics of each synonym in the group of synonyms and the semantics of the target keyword is less than a difference threshold.

[0056] In this way, the first computing device can automatically generate related synonyms based on semantics, without relying on user input, thereby improving and reducing user interaction overhead.

[0057] In some embodiments of the third aspect, the synonym acquisition unit is further configured to: determine a group of synonyms based on the semantics and length of the target keyword, wherein the difference between the length of each synonym in the group and the length of the target keyword is less than a length threshold. The length of the keyword can, for example, represent the number of characters or words included in the keyword. In this way, synonyms in the group can have similar lengths, thereby reducing the complexity of decoding searches using a decoding graph.

[0058] In some embodiments of the third aspect, the synonym acquisition unit is further configured to: acquire multiple candidate synonyms based on the semantics of the target keyword; provide multiple candidate synonyms to the user; and determine a group of synonyms from the multiple candidate synonyms based on user input received from the user, wherein the user input indicates that at least one candidate synonym among the multiple candidate synonyms is excluded or confirmed.

[0059] Based on this approach, the synonym pairs used to train the speech recognition model can be further adjusted based on user feedback, which can make the obtained speech recognition model more in line with users' usage habits.

[0060] In some embodiments of the third aspect, the target keywords include at least a first keyword and a second keyword, wherein the model merging unit is further configured to: obtain a first set of decoding paths and a second set of decoding paths from a first decoding graph, the first set of decoding paths including decoding paths corresponding to the first keyword and a first synonym group semantically associated with the first keyword, and the second set of decoding paths including decoding paths corresponding to the second keyword and a second synonym group semantically associated with the second keyword; generate a first subgraph based on the first set of decoding paths and generate a second subgraph based on the second set of decoding paths; and determine a speech recognition model based at least on the first subgraph and the second subgraph. In this manner, the generated decoding graph has lower complexity, enabling faster decoding search, thereby reducing computational and storage overhead.

[0061] In some embodiments of the third aspect, the first subgraph indicates a first decoding path and a second decoding path. The first decoding path is the decoding path corresponding to the first keyword, and the second decoding path is the decoding path corresponding to the synonyms in the first synonym group. The first decoding path and each second decoding path have the same weight in the first subgraph. Based on this approach, faster decoding search for the expanded synonyms can be achieved, thereby reducing computational and storage overhead.

[0062] In some embodiments of the third aspect, the keyword acquisition unit is further configured to: acquire a first keyword group based on pre-stored historical keywords and received keywords; and in response to determining that the number of keywords in the first keyword group exceeds a predetermined threshold, acquire a target keyword from the first keyword group based on the predetermined threshold.

[0063] This approach avoids using too many keywords to train the language model, thus ensuring that the decoding graph is lightweight and suitable for devices with limited computing resources.

[0064] In some embodiments of the third aspect, the keyword acquisition unit is further configured to: acquire target keywords from a first keyword group based on the attributes of keywords in the target keywords, wherein the number of target keywords is a predetermined threshold. For example, one or more historical keywords that were created earliest can be deleted from the first keyword group to obtain the predetermined threshold number of keywords.

[0065] In some embodiments of the third aspect, the keyword acquisition unit is further configured to: acquire target keywords from a first keyword group according to user instructions, wherein the number of target keywords is a predetermined threshold. For example, the user may select which keywords in the first keyword group to retain as target keywords based on user input.

[0066] In some embodiments of the third aspect, the speech model building system may also instruct the provision of a speech recognition model to a target computing device (e.g., a second computing device) for deployment on the target computing device. Based on this approach, automated deployment of the speech recognition model can be supported.

[0067] In a fourth aspect of this disclosure, a speech processing system is provided. The system includes: a speech input unit for receiving speech input; and a speech processing unit for determining a text representation associated with the speech input using a speech recognition model, wherein the speech recognition model is obtained based on the following process: acquiring target keywords; acquiring synonyms semantically associated with the target keywords; training a language model using the target keywords and synonyms to obtain a target language model; generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths satisfying grammatical constraint rules determined based on the target keywords and synonyms; and determining the speech recognition model based on the first decoding graph.

[0068] In some embodiments of the fourth aspect, the speech recognition model may be obtained by a first computing device. The first computing device may, for example, include a cloud-based or embedded device with strong computing capabilities to perform the construction of the speech recognition model. The first computing device may also include, for example, a user terminal device.

[0069] In some embodiments of the fourth aspect, the step of obtaining the speech recognition model may also be performed collaboratively by the user terminal and the cloud.

[0070] In some embodiments of the fourth aspect, the voice processing system may include, for example, a second computing device. The second computing device may include an embedded lightweight device, for example, having limited computing power, for performing voice processing using a deployed speech recognition model. Examples of the second computing device may include, but are not limited to: smart home devices (e.g., air conditioners, refrigerators, washing machines, televisions, speakers, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.), or in-vehicle devices, etc.

[0071] Based on this approach, embodiments of this disclosure enable, for example, a second computing device with lower computing power to have keyword generalization recognition capabilities, thereby improving the user's voice interaction experience.

[0072] In some embodiments of the fourth aspect, the target keywords include keywords from voice input by an audio collector located at a user terminal. In other embodiments of the fourth aspect, the target keywords include keywords from text input by a text collector located at a user terminal.

[0073] For example, if the first computing device is a user terminal, the user can directly provide voice input or text input to the user terminal so that the user terminal can extract keywords from the voice input or text input to build a speech recognition model.

[0074] For example, if the first computing device is a different computing device from the user terminal, such as a cloud device or an edge computing device, the user can, for example, use the interface provided by the user terminal to input voice or text. Such voice or text input can be sent to the first computing device so that the first computing device can obtain keywords to build a speech recognition model.

[0075] In this way, users can customize the keywords that the speech recognition model can support, thereby increasing the personalization of the speech recognition model.

[0076] In some embodiments of the fourth aspect, determining a group of synonyms that are semantically associated with the target keyword includes: determining the semantics of the target keyword; and determining a group of synonyms based at least on the semantics of the target keyword, wherein the semantic difference between each synonym in the group of synonyms and the semantics of the target keyword is less than a difference threshold.

[0077] In this way, the first computing device can automatically generate related synonyms based on semantics, without relying on user input, thereby improving and reducing user interaction overhead.

[0078] In some embodiments of the fourth aspect, determining synonym groups based at least on the semantics of the target keyword includes: determining synonym groups based on the semantics and length of the target keyword, wherein the difference between the length of each synonym in the synonym group and the length of the target keyword is less than a length threshold. The length of the keyword can, for example, represent the number of characters or words included in the keyword. In this way, synonyms in the synonym group can have similar lengths, thereby reducing the complexity of decoding searches using a decoding graph.

[0079] In some embodiments of the fourth aspect, determining a group of synonyms based at least on the semantics of the target keyword includes: obtaining a plurality of candidate synonyms based on the semantics of the target keyword; providing the plurality of candidate synonyms to a user; and determining a group of synonyms from the plurality of candidate synonyms based on user input received from the user, wherein the user input indicates that at least one of the plurality of candidate synonyms is excluded or confirmed.

[0080] Based on this approach, the synonym pairs used to train the speech recognition model can be further adjusted based on user feedback, which can make the obtained speech recognition model more in line with users' usage habits.

[0081] In some embodiments of the fourth aspect, the target keywords include at least a first keyword and a second keyword, wherein determining the speech recognition model based on the first decoding graph includes: obtaining a first set of decoding paths and a second set of decoding paths from the first decoding graph, wherein the first set of decoding paths includes decoding paths corresponding to the first keyword and a first set of synonyms semantically associated with the first keyword, and the second set of decoding paths includes decoding paths corresponding to the second keyword and a second set of synonyms semantically associated with the second keyword; generating a first subgraph based on the first set of decoding paths and generating a second subgraph based on the second set of decoding paths; and determining the speech recognition model based at least on the first subgraph and the second subgraph. Based on this approach, the generated decoding graph has lower complexity, enabling faster decoding search, thereby reducing computational and storage overhead.

[0082] In some embodiments of the fourth aspect, the first subgraph indicates a first decoding path and a second decoding path, where the first decoding path corresponds to the first keyword, and the second decoding path corresponds to the synonyms in the first synonym group. The first decoding path and each second decoding path have the same weight in the first subgraph. Based on this approach, faster decoding search for the expanded synonyms can be achieved, thereby reducing computational and storage overhead.

[0083] In some embodiments of the fourth aspect, obtaining the target keyword includes: obtaining a first keyword group based on pre-stored historical keywords and received keywords; and in response to determining that the number of keywords in the first keyword group exceeds a predetermined threshold, obtaining the target keyword from the first keyword group based on the predetermined threshold.

[0084] This approach avoids using too many keywords to train the language model, thus ensuring that the decoding graph is lightweight and suitable for devices with limited computing resources.

[0085] In some embodiments of the fourth aspect, obtaining target keywords from the first keyword group based on a predetermined threshold includes: obtaining target keywords from the first keyword group according to the attributes of the keywords in the target keywords, wherein the number of target keywords is a predetermined threshold. For example, one or more historical keywords that were created earliest can be deleted from the first keyword group to obtain the predetermined threshold number of keywords.

[0086] In some embodiments of the fourth aspect, obtaining target keywords from the first keyword group based on a predetermined threshold includes: obtaining target keywords from the first keyword group according to user instructions, wherein the number of target keywords is a predetermined threshold. For example, user input can be used to select which keywords in the first keyword group to retain as target keywords.

[0087] In some embodiments of the fourth aspect, the speech processing system may also perform actions corresponding to the text representation. Alternatively, the second computing device may also generate corresponding control commands based on the text representation and send them to the third computing device, causing the third computing device to perform the corresponding actions.

[0088] In some embodiments of the fourth aspect, the text representation corresponds to a synonym in the target keyword or a group of synonyms.

[0089] In a fifth aspect of this disclosure, a method for constructing a speech recognition model is provided. The method includes: acquiring target language information; acquiring a group of synonymous phoneme sequences associated with the target language information, the group of synonymous phoneme sequences including at least one of the synonymous phoneme sequences, the at least one synonymous phoneme sequence being a phoneme sequence corresponding to a word or phrase semantically similar to the target language information; training a language model using the group of synonymous phoneme sequences to obtain a target language model; generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths satisfying grammatical constraint rules determined based on the group of synonymous phoneme sequences; and determining a speech recognition model based on the first decoding graph.

[0090] In some embodiments of the fifth aspect, the method may be performed, for example, by a first computing device with strong computing capabilities. The first computing device may include, for example, a cloud-side or embedded heavy-duty device with strong computing capabilities for constructing a speech recognition model. The first computing device may also include, for example, a user terminal device. In some embodiments of the fifth aspect, one or more steps in the method may also be performed collaboratively by the user terminal and the cloud.

[0091] In some embodiments of the fifth aspect, the obtained speech recognition model may, for example, be deployed to a second computing device. The second computing device may include an embedded lightweight device, for example, having limited computing power, for performing speech processing using the deployed speech recognition model. Examples of the second computing device may include, but are not limited to: smart home devices (e.g., air conditioners, refrigerators, washing machines, televisions, speakers, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.), or in-vehicle devices, etc.

[0092] Based on this approach, embodiments of this disclosure can construct a speech recognition model with generalized recognition capabilities for phoneme sequences associated with target language information, thereby enabling, for example, a second computing device with lower computing power to have phoneme sequence generalized recognition capabilities, thereby improving the user's interactive experience.

[0093] In some embodiments of the fifth aspect, the target language information may include speech or text. In some embodiments, the target language information includes speech input from an audio acquisition device located at the user terminal. In other embodiments, keywords of text input are obtained from a text acquisition device at the user terminal. In some embodiments, the target language information may be short instruction words or instruction statements, such as "turn off," "stop," "pause," "increase volume," "increase volume," etc.

[0094] For example, if the first computing device is a user terminal, the user can directly provide voice input or text input to the user terminal so that the user terminal can extract target language information from the voice input or text input in order to construct a speech recognition model.

[0095] For example, if the first computing device is a different computing device from the user terminal, such as a cloud device or edge computing device, the user can, for example, use the interface provided by the user terminal to input voice or text. Such voice or text input can be sent to the first computing device, enabling it to acquire target language information for constructing a speech recognition model. In this way, the user can customize the phoneme sequences supported by the speech recognition model, thereby improving the personalization of the speech recognition model.

[0096] In some embodiments of the fifth aspect, obtaining a set of synonymous phoneme sequences associated with target language information includes: determining the semantics of the target language information; and determining the set of synonymous phoneme sequences based at least on the semantics of the target language information, wherein the difference between the semantics of each synonymous phoneme sequence in the set of synonymous phoneme sequences and the semantics of the target language information is less than a difference threshold.

[0097] In this way, the first computing device can automatically generate associated synonymous phoneme sequences based on semantics, without relying on user input, thereby improving and reducing user interaction overhead.

[0098] In some embodiments of the fifth aspect, determining a group of synonymous phoneme sequences based at least on the semantics of the target language information includes: determining a target phoneme sequence corresponding to the target language information; and determining a group of synonymous phoneme sequences based on the semantics and length of the target phoneme sequence, wherein the difference between the length of each synonymous phoneme sequence in the group and the length of the target phoneme sequence is less than a length threshold. The length of the phoneme sequence may, for example, represent the number of phonemes (e.g., initials and finals) included in the phoneme sequence. In some embodiments, if the target language information is text, the phoneme sequence corresponding to the text can be obtained as the target phoneme sequence through a pronunciation dictionary. If the target language information is speech, the phoneme sequence of the speech can be obtained as the target phoneme sequence through an acoustic model.

[0099] This approach allows synonymous phoneme sequences within a group to have similar lengths, thereby reducing the complexity of decoding searches using decoding graphs.

[0100] In some embodiments of the fifth aspect, obtaining a group of synonymous phoneme sequences associated with the semantics of target language information includes: obtaining multiple candidate synonyms based on the semantics of target keywords corresponding to the target language information; providing multiple candidate synonyms to a user; determining a group of synonyms from the multiple candidate synonyms based on user input received from the user, wherein the user input indicates that at least one candidate synonym among the multiple candidate synonyms is excluded or confirmed; and obtaining the group of synonymous phoneme sequences based on a pronunciation dictionary and the group of synonyms.

[0101] Based on this approach, the synonymous phoneme sequence groups used to train the speech recognition model can be further adjusted based on user feedback, which enables the obtained speech recognition model to better suit users' usage habits.

[0102] In some embodiments of the fifth aspect, obtaining a group of synonymous phoneme sequences associated with the semantics of the target phoneme sequence includes: receiving voice input from a user; and generating the group of synonymous phoneme sequences based on the voice input. For example, the semantics of the voice input are obtained based on keywords corresponding to the voice input, thereby generating the group of synonymous phoneme sequences.

[0103] Based on this approach, the synonymous phoneme sequence groups used to train the speech recognition model can be further supplemented by user feedback in the form of voice input, which enables the obtained speech recognition model to be more in line with the user's usage habits.

[0104] In some embodiments of the fifth aspect, the target language information includes at least first language information and second language information, wherein determining the speech recognition model based on the first decoding graph includes: obtaining a first set of decoding paths and a second set of decoding paths from the first decoding graph, wherein the first set of decoding paths includes decoding paths for a first group of synonymous phoneme sequences associated with the first language information, and the second set of decoding paths includes decoding paths for a second group of synonymous phoneme sequences associated with the second language information; generating a first subgraph based on the first set of decoding paths; generating a second subgraph based on the second set of decoding paths; and determining the speech recognition model based at least on the first and second subgraphs. In this manner, the generated decoding graph has lower complexity, supports faster decoding search, and thus reduces computational and storage overhead.

[0105] In some embodiments of the fifth aspect, the first subgraph indicates a first decoding path and a second decoding path, which are decoding paths in the first group of synonymous phoneme sequences, and the first and second decoding paths have the same weight in the first subgraph. Based on this approach, a faster decoding search for the extended synonymous phoneme sequences can be achieved, thereby reducing computational and storage overhead.

[0106] In some embodiments of the fifth aspect, obtaining target language information includes: obtaining a first language information group based on pre-stored historical language information and received language information; and in response to determining that the number of language information in the first language information group exceeds a predetermined threshold, obtaining the target language information from the first language information group based on the predetermined threshold.

[0107] This approach avoids using too much linguistic information to train the language model, thus ensuring that the decoding graph is lightweight and suitable for devices with limited computing resources.

[0108] In some embodiments of the fifth aspect, obtaining target language information from a first language information group based on a predetermined threshold includes: obtaining target language information from the first language information group according to the attributes of language information in the target language information, wherein the number of target language information is a predetermined threshold. For example, one or more historical language information items that were created earliest can be deleted from the first language information group to obtain the predetermined threshold number of language information items.

[0109] In some embodiments of the fifth aspect, obtaining target language information from the first language information group based on a predetermined threshold includes: obtaining target language information from the first language information group according to user instructions, wherein the number of target language information is the predetermined threshold. For example, user input can be used to select which language information from the first language information group to retain as target language information.

[0110] In some embodiments of the fifth aspect, the first computing device may also instruct a target computing device (e.g., a second computing device) to provide a speech recognition model for deployment on the target computing device. This approach enables the automatic deployment of the speech recognition model.

[0111] In a sixth aspect of this disclosure, a speech processing method is provided, comprising: receiving a speech command input; using a speech recognition model to obtain a phoneme sequence representation of the speech input, the speech recognition model being configured to recognize the speech command based on a group of phoneme sequences that are semantically synonymous with the command; and executing a command corresponding to the phoneme sequence representation if the phoneme sequence representation corresponds to a phoneme sequence in the phoneme sequence group.

[0112] In some embodiments of the sixth aspect, the speech recognition model may be obtained by a first computing device. The first computing device may, for example, include a cloud-based or embedded device with strong computing capabilities for constructing the speech recognition model. The first computing device may also include, for example, a user terminal device.

[0113] In some embodiments of the sixth aspect, the step of obtaining the speech recognition model may, for example, be performed collaboratively by the user terminal and the cloud.

[0114] In some embodiments of the sixth aspect, the speech processing method may be performed, for example, by a second computing device. The second computing device may include an embedded lightweight device, for example, having limited computing power, for performing speech processing using a deployed speech recognition model. Examples of the second computing device may include, but are not limited to: smart home devices (e.g., air conditioners, refrigerators, washing machines, televisions, speakers, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.), or in-vehicle devices, etc.

[0115] Based on this approach, embodiments of this disclosure enable, for example, a second computing device with lower computing power to have phoneme sequence generalization recognition capabilities without needing to recognize keywords for natural language understanding, thereby reducing the performance requirements of the device and improving the user's voice interaction experience.

[0116] In some embodiments of the sixth aspect, the speech recognition model is obtained based on the following process: acquiring target language information; acquiring a group of synonymous phoneme sequences associated with the target language information, the group of synonymous phoneme sequences including at least one synonymous phoneme sequence, the synonymous phoneme sequence being a phoneme sequence corresponding to a word or phrase semantically similar to the target language information; training a language model using the group of synonymous phoneme sequences to obtain a target language model; generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths that satisfy grammatical constraint rules determined based on the group of synonymous phoneme sequences; and determining a speech recognition model based on the first decoding graph.

[0117] Based on this approach, the constructed speech recognition model can achieve the ability to generalize and recognize custom target language information.

[0118] In some embodiments of the sixth aspect, if the phoneme sequence representation does not match any phoneme sequence in the phoneme sequence group, a notification of no recognition result is provided. Based on this approach, user speech can be recognized in real time and efficiently, improving the user's voice interaction experience.

[0119] In some embodiments of the sixth aspect, the speech recognition model is configured to recognize a first group of phoneme sequences having a first semantic synonym and a second group of phoneme sequences having a second semantic synonym. The method may further include executing a first instruction if the phoneme sequence represents a first phoneme sequence corresponding to a first phoneme sequence in the first group of phoneme sequences, and executing a second instruction different from the first instruction if the phoneme sequence represents a second phoneme sequence corresponding to a second group of phoneme sequences. In this manner, the speech recognition model can be used to recognize phoneme sequences in groups of phoneme sequences with different semantics, thereby enabling the execution of instructions corresponding to user intent.

[0120] In some embodiments of the sixth aspect, using a speech recognition model to obtain a phoneme sequence representation of a speech command may include: using an acoustic model to generate emission probabilities from speech features of the speech command input to phonemes; recognizing the speech command input by inputting the emission probabilities to a speech recognition model; and causing the speech recognition model to output a phoneme sequence representation. Based on this approach, a corresponding phoneme sequence can be obtained from a speech-form command for matching phoneme sequences within a phoneme sequence set that the speech recognition model can recognize. In a seventh aspect of this disclosure, a speech model construction system is provided, comprising: a target language information acquisition unit for acquiring target language information; a synonymous phoneme sequence group acquisition unit for acquiring a synonymous phoneme sequence group associated with the target language information, the synonymous phoneme sequence group including at least one synonymous phoneme sequence, the at least one synonymous phoneme sequence being a phoneme sequence corresponding to a word or phrase semantically similar to the target language information; a model training unit for training a language model using the synonymous phoneme sequence group to obtain a target language model; a decoding graph generation unit for generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths satisfying grammatical constraint rules determined based on the synonymous phoneme sequence group; and a model determination unit for determining the speech recognition model based on the first decoding graph.

[0121] In some embodiments of the seventh aspect, the speech model building system according to the seventh aspect may be implemented, for example, by a first computing device with strong computing power. The first computing device may, for example, include a cloud-based or embedded heavy-duty device, which may have strong computing power for performing the construction of the speech recognition model. The first computing device may also include, for example, a user terminal device. In some embodiments of the seventh aspect, the system may also be implemented by collaboration between the user terminal and the cloud.

[0122] In some embodiments of the seventh aspect, the obtained speech recognition model may, for example, be deployed to a second computing device. The second computing device may include an embedded lightweight device, for example, having limited computing power, for performing speech processing using the deployed speech recognition model. Examples of the second computing device may include, but are not limited to: smart home devices (e.g., air conditioners, refrigerators, washing machines, televisions, speakers, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.), or in-vehicle devices, etc.

[0123] Based on this approach, embodiments of this disclosure can construct a speech recognition model with generalized recognition capabilities for phoneme sequences associated with target language information, thereby enabling, for example, a second computing device with lower computing power to have phoneme sequence generalized recognition capabilities, thereby improving the user's interactive experience.

[0124] In some embodiments of the seventh aspect, the target language information may include speech or text. In some embodiments, the target language information includes speech input from an audio acquisition device located at a user terminal. In other embodiments, keywords of text input are obtained from a text acquisition device at the user terminal. In some embodiments, the target language information may be short instruction words or instruction statements, such as "turn off," "stop," "pause," "increase volume," "increase volume," etc.

[0125] For example, if the first computing device is a user terminal, the user can directly provide voice input or text input to the user terminal so that the user terminal can extract target language information from the voice input or text input in order to construct a speech recognition model.

[0126] For example, if the first computing device is a different computing device from the user terminal, such as a cloud device or edge computing device, the user can, for example, use the interface provided by the user terminal to input voice or text. Such voice or text input can be sent to the first computing device, enabling it to acquire target language information for constructing a speech recognition model. In this way, the user can customize the phoneme sequences supported by the speech recognition model, thereby improving the personalization of the speech recognition model.

[0127] In some embodiments of the seventh aspect, the synonymous phoneme sequence group acquisition unit may also be used to: determine the semantics of the target language information; and determine the synonymous phoneme sequence group based at least on the semantics of the target language information, wherein the difference between the semantics of each synonymous phoneme sequence in the synonymous phoneme sequence group and the semantics of the target language information is less than a difference threshold.

[0128] In this way, the first computing device can automatically generate associated synonymous phoneme sequences based on semantics, without relying on user input, thereby improving and reducing user interaction overhead.

[0129] In some embodiments of the seventh aspect, the synonym phoneme sequence group acquisition unit can also be used to: determine a target phoneme sequence corresponding to the target language information; and determine a synonym phoneme sequence group based on the semantics and length of the target phoneme sequence, wherein the difference between the length of each synonym phoneme sequence in the synonym phoneme sequence group and the length of the target phoneme sequence is less than a length threshold. The length of the phoneme sequence may, for example, represent the number of phonemes (e.g., initials and finals) included in the phoneme sequence. In some embodiments, if the target language information is text, the phoneme sequence corresponding to the text can be obtained as the target phoneme sequence through a pronunciation dictionary. If the target language information is speech, the phoneme sequence of the speech can be obtained as the target phoneme sequence through an acoustic model.

[0130] This approach allows synonymous phoneme sequences within a group to have similar lengths, thereby reducing the complexity of decoding searches using decoding graphs.

[0131] In some embodiments of the seventh aspect, the synonym phoneme sequence group acquisition unit may also be used to: acquire multiple candidate synonyms based on the semantics of target keywords corresponding to target language information; provide multiple candidate synonyms to a user; determine a synonym group from multiple candidate synonyms based on user input received from the user, wherein the user input indicates that at least one candidate synonym among the multiple candidate synonyms is excluded or confirmed; and acquire a synonym phoneme sequence group based on a pronunciation dictionary and a synonym group.

[0132] Based on this approach, the synonymous phoneme sequence groups used to train the speech recognition model can be further adjusted based on user feedback, which enables the obtained speech recognition model to better suit users' usage habits.

[0133] In some embodiments of the seventh aspect, the synonym phoneme sequence group acquisition unit can also be used to: receive voice input from a user; and generate a synonym phoneme sequence group based on the voice input. For example, the semantics of the voice input can be obtained based on keywords corresponding to the voice input, thereby generating the synonym phoneme sequence group.

[0134] Based on this approach, the synonymous phoneme sequence groups used to train the speech recognition model can be further supplemented by user feedback in the form of voice input, which enables the obtained speech recognition model to be more in line with the user's usage habits.

[0135] In some embodiments of the seventh aspect, the target language information includes at least first language information and second language information, wherein the model determination unit can further be used to: obtain a first set of decoding paths and a second set of decoding paths from a first decoding graph, wherein the first set of decoding paths includes decoding paths for a first group of synonymous phoneme sequences associated with the first language information, and the second set of decoding paths includes decoding paths for a second group of synonymous phoneme sequences associated with the second language information; generate a first subgraph based on the first set of decoding paths; generate a second subgraph based on the second set of decoding paths; and determine a speech recognition model based at least on the first and second subgraphs. In this manner, the generated decoding graph has lower complexity, enabling faster decoding search, thereby reducing computational and storage overhead.

[0136] In some embodiments of the seventh aspect, the first subgraph indicates a first decoding path and a second decoding path, which are decoding paths in the first group of synonymous phoneme sequences, and the first and second decoding paths have the same weight in the first subgraph. Based on this approach, a faster decoding search for the extended synonymous phoneme sequences can be achieved, thereby reducing computational and storage overhead.

[0137] In some embodiments of the seventh aspect, the target language information acquisition unit may also be used to: acquire a first language information group based on pre-stored historical language information and received language information; and in response to determining that the number of language information in the first language information group exceeds a predetermined threshold, acquire target language information from the first language information group based on the predetermined threshold.

[0138] This approach avoids using too much linguistic information to train the language model, thus ensuring that the decoding graph is lightweight and suitable for devices with limited computing resources.

[0139] In some embodiments of the seventh aspect, the target language information acquisition unit may further be used to: acquire target language information from a first language information group based on the attributes of the language information in the target language information, wherein the number of target language information is a predetermined threshold. For example, one or more historical language information items that were created earliest can be deleted from the first language information group to obtain the predetermined threshold number of language information items.

[0140] In some embodiments of the seventh aspect, the target language information acquisition unit may further be used to: acquire target language information from a first language information group according to user instructions, wherein the number of target language information is a predetermined threshold. For example, user input may be used to select which language information from the first language information group to retain as target language information.

[0141] In some embodiments of the seventh aspect, the first computing device may also instruct a target computing device (e.g., a second computing device) to provide a speech recognition model for deployment on the target computing device. This approach enables the automatic deployment of the speech recognition model.

[0142] In an eighth aspect of this disclosure, a speech processing system is provided, comprising: a speech command input unit for receiving a speech command input; a speech processing unit for using a speech recognition model to obtain a phoneme sequence representation of the speech command input, the speech recognition model being configured to recognize the speech command input based on a group of phoneme sequences that are semantically synonymous with the command; and for executing an instruction corresponding to the phoneme sequence representation when the phoneme sequence representation corresponds to a phoneme sequence in the phoneme sequence group.

[0143] In some embodiments of the eighth aspect, the speech recognition model may be obtained by a first computing device. The first computing device may, for example, include a cloud-based or embedded device with strong computing capabilities for constructing the speech recognition model. The first computing device may also include, for example, a user terminal device.

[0144] In some embodiments of the eighth aspect, the voice processing system may be executed by a second computing device, for example. The second computing device may include an embedded lightweight device, for example, having limited computing power, for performing voice processing using a deployed speech recognition model. Examples of the second computing device may include, but are not limited to: smart home devices (e.g., air conditioners, refrigerators, washing machines, televisions, speakers, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.), or in-vehicle devices, etc.

[0145] Based on this approach, embodiments of this disclosure enable, for example, a second computing device with lower computing power to have phoneme sequence generalization recognition capabilities without needing to recognize keywords for natural language understanding, thereby reducing the performance requirements of the device and improving the user's voice interaction experience.

[0146] In some embodiments of the eighth aspect, the speech recognition model is obtained based on the following process: acquiring target language information; acquiring a set of synonymous phoneme sequences associated with the target language information, the set of synonymous phoneme sequences including at least one synonymous phoneme sequence, the synonymous phoneme sequence being a phoneme sequence corresponding to a word or phrase semantically similar to the target language information; training a language model using the set of synonymous phoneme sequences to obtain a target language model; generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths that satisfy grammatical constraint rules determined based on the set of synonymous phoneme sequences; and determining a speech recognition model based on the first decoding graph.

[0147] Based on this approach, the constructed speech recognition model can achieve the ability to generalize and recognize custom target language information.

[0148] In some embodiments of the eighth aspect, the speech processing unit can also be used to: provide a notification of no recognition result if the phoneme sequence representation does not match any phoneme sequence in the phoneme sequence group. Based on this approach, user speech can be recognized in real time and efficiently, improving the user's voice interaction experience.

[0149] In some embodiments of the eighth aspect, the speech recognition model is configured to recognize a first group of phoneme sequences having a first semantic synonym and a second group of phoneme sequences having a second semantic synonym. The speech processing unit can also be configured to: execute a first instruction if the phoneme sequence represents a first phoneme sequence corresponding to a first phoneme sequence in the first phoneme sequence group, and execute a second instruction different from the first action if the phoneme sequence represents a second phoneme sequence corresponding to a second phoneme sequence in the second phoneme sequence group. Based on this, the speech recognition model can be used to recognize phoneme sequences in phoneme sequence groups with different semantics, thereby enabling the execution of instructions corresponding to user intent.

[0150] In some embodiments of the eighth aspect, the speech processing unit may further be used to: generate emission probabilities of speech features to phonemes from a speech command input using an acoustic model; recognize the speech command input by inputting the emission probabilities to a speech recognition model; and cause the speech recognition model to output a phoneme sequence representation. In this manner, a corresponding phoneme sequence can be obtained from a speech command to match a phoneme sequence in a group of phoneme sequences that the speech recognition model can recognize.

[0151] In a ninth aspect of this disclosure, a first computing device is provided. The first computing device includes: at least one computing unit; at least one memory coupled to the at least one computing unit and storing instructions for execution by the at least one computing unit, wherein, when executed by the at least one computing unit, the instructions cause the first computing device to perform some or all of the steps of the method in the first aspect or any implementation thereof, or to perform some or all of the steps of the method in the fifth aspect or any implementation thereof.

[0152] In a tenth aspect of this disclosure, a second computing device is provided. The second computing device includes: at least one computing unit; at least one memory coupled to the at least one computing unit and storing instructions for execution by the at least one computing unit, wherein, when executed by the at least one computing unit, the instructions cause the second computing device to perform some or all of the steps of the method in the second aspect or any implementation thereof, or to perform some or all of the steps of the method in the sixth aspect or any implementation thereof.

[0153] In the eleventh aspect of this disclosure, a computer-readable storage medium is provided, having stored thereon one or more computer instructions, wherein one or more computer instructions are executed by a processor to implement the method of the first aspect or any implementation thereof, or to perform part or all of the steps of the method of the fifth aspect or any implementation thereof.

[0154] In the twelfth aspect of this disclosure, a computer-readable storage medium is provided that stores one or more computer instructions, wherein the one or more computer instructions are executed by a processor to implement the method of the second aspect or any implementation thereof, or to perform part or all of the steps of the method of the sixth aspect or any implementation thereof.

[0155] In the thirteenth aspect of this disclosure, a computer program product is provided that, when the computer program product is run on a computer, causes the computer to execute instructions for some or all of the steps of the method in the first aspect or any implementation thereof, or to execute some or all of the steps of the method in the fifth aspect or any implementation thereof.

[0156] In the fourteenth aspect of this disclosure, a computer program product is provided that, when run on a computer, causes the computer to execute instructions for some or all of the steps of the method in the second aspect or any implementation thereof, or to execute some or all of the steps of the method in the sixth aspect or any implementation thereof.

[0157] Understandably, the first computing device of the ninth aspect, the computer storage medium of the eleventh aspect, or the computer program product of the thirteenth aspect described above are all used to execute the method provided in the first aspect. Therefore, the explanations or descriptions regarding the first aspect also apply to the ninth, eleventh, and thirteenth aspects. Furthermore, the beneficial effects achievable by the ninth, eleventh, and thirteenth aspects can be referred to in the beneficial effects described in the corresponding methods, and will not be repeated here.

[0158] Understandably, the second computing device of aspect ten, the computer storage medium of aspect twelfth, or the computer program product of aspect fourteenth provided above are all used to execute the method provided in aspect two. Therefore, the explanations or descriptions regarding aspect two also apply to aspects ten, twelfth, and fourteenth. Furthermore, the beneficial effects achievable by aspects ten, twelfth, and fourteenth can be referred to in the beneficial effects of the corresponding methods, and will not be repeated here. Attached Figure Description

[0159] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. In the drawings, the same or similar reference numerals denote the same or similar elements, wherein: Figure 1 A schematic block diagram of an example environment in which embodiments of the present disclosure may be implemented is shown; Figure 2 A to Figure 2 D illustrates an example user interface according to some embodiments of this disclosure; Figure 3 A schematic block diagram of another example environment in which embodiments of the present disclosure may be implemented is shown; Figure 4 A schematic block diagram is shown of yet another example environment in which embodiments of the present disclosure may be implemented; Figure 5 A flowchart illustrating the process of constructing a speech recognition model according to some embodiments of the present disclosure is shown; Figure 6 A flowchart illustrating an example process for obtaining target keywords according to some embodiments of this disclosure is shown; Figure 7 A flowchart illustrating an example process for determining synonym pairs according to some embodiments of this disclosure is shown; Figure 8 A schematic diagram illustrating an example process for training a language model according to some embodiments of the present disclosure is shown; Figure 9 A schematic diagram of the generation of a decoding map according to some embodiments of the present disclosure is shown; Figure 10 A schematic diagram illustrating subgraph clustering of a decoded graph according to some embodiments of the present disclosure is shown; Figure 11 Schematic diagrams of example sub-figures according to some embodiments of the present disclosure are shown; Figure 12 A flowchart illustrating an example process of speech processing according to some embodiments of the present disclosure is shown; Figure 13 A flowchart illustrating an example process for determining a group of synonymous phoneme sequences according to an embodiment of this disclosure is shown; Figure 14 A schematic diagram illustrating an example process for generating a decoded graph according to an embodiment of the present disclosure is shown; Figure 15 A schematic diagram illustrating an example process for clustering synonymous phoneme sequences according to an embodiment of the present disclosure is shown; Figure 16 Schematic diagrams of example sub-diagrams according to some embodiments of the present disclosure are shown; Figure 17 A flowchart illustrating an example process of speech processing according to some embodiments of the present disclosure is shown; Figure 18 A flowchart illustrating an example process for determining speech recognition results according to embodiments of the present disclosure is shown; Figure 19 A flowchart illustrating an example process of a speech processing method according to an embodiment of the present disclosure is shown; Figure 20 A schematic block diagram of an example speech recognition system according to some specific embodiments of the present disclosure is shown; Figure 21 A schematic block diagram of a speech model construction system according to an embodiment of the present disclosure is shown; Figure 22 A block diagram of a speech processing system according to an embodiment of the present disclosure is shown; Figure 23 A block diagram of a speech model construction system according to an embodiment of the present disclosure is shown; Figure 24 A block diagram of a speech processing system according to an embodiment of the present disclosure is shown; Figure 25 A schematic block diagram of an example device that can be used to implement embodiments of the present disclosure is shown.

[0160] In the various figures, the same or similar reference numerals represent the same or similar elements. Detailed Implementation

[0161] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0162] In the description of embodiments of this disclosure, the term "comprising" and similar terms should be understood as open-ended inclusion, i.e., "including but not limited to". The term "based on" should be understood as "at least partially based on". The term "one embodiment" or "the embodiment" should be understood as "at least one embodiment". The terms "first", "second", etc., may refer to different or the same objects. Other explicit and implicit definitions may also be included below.

[0163] As discussed above, for lightweight devices with limited computing power (e.g., smart home devices), some traditional speech recognition solutions typically rely on other devices with stronger computing power (e.g., mobile phones or cloud servers) to achieve semantic understanding of voice input. Alternatively, some lightweight devices can deploy lightweight speech recognition models to achieve local semantic understanding; however, such speech recognition models cannot support user-defined semantics, let alone handle voice input with the same semantics. This will have a significant impact on the user's voice interaction experience.

[0164] Embodiments of this disclosure provide keyword-based speech recognition models and phoneme sequence-based speech recognition models. Here, keywords can be textual representations of language, and phoneme sequences can be sequences of speech units divided according to the natural properties of speech. Depending on the language type (e.g., Chinese or English), phonemes can be, for example, initials and finals, phonetic symbols, or any other form.

[0165] Example Environment 1 Figure 1 A schematic diagram of an example environment 100 in which various embodiments of the present disclosure can be implemented is shown. For example... Figure 1 As shown, environment 100 includes a first computing device 130. The first computing device 130 can be, for example, a device with strong computing capabilities, including but not limited to: cloud servers, smartphones, laptops, tablets, desktop computers, or edge computing devices. In this disclosure, such a first computing device 130 can also be referred to as a heavy-duty device. Figure 1 In the example, such a first computing device 130 may include, for example, a cloud-side device.

[0166] According to embodiments of this disclosure, the first computing device 130 can acquire target keywords, train a language model based on synonyms semantically associated with the target keywords, and further obtain a speech recognition model. In some embodiments, the target keywords may include, for example, keywords input by the user 105. Figure 1 As shown, user 105 can, for example, use user terminal 110 to configure keywords. User 105 may, for example, want to configure a second computing device 150 to enable speech recognition of the keyword "enhance voice".

[0167] According to embodiments of this disclosure, the first computing device 130 can also acquire target language information, train a language model based on a group of synonymous phoneme sequences associated with the target language information, and further obtain a speech recognition model. In some embodiments, the target phoneme sequence may, for example, include speech 115 provided by user 105. Figure 1 As shown, user 105 may, for example, use user terminal 110 to configure a group of synonymous phoneme sequences. User 105 may, for example, wish to configure a second computing device 150 to enable speech recognition of the phoneme sequence corresponding to "raise sound".

[0168] exist Figure 1 In the example, the second computing device 150 is shown as a smart speaker. It should be understood that the smart speaker is merely one example of the second computing device 150, and the second computing device 150 may also include other suitable devices, such as smart home devices (e.g., air conditioners, refrigerators, washing machines, televisions, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.), or in-vehicle devices, etc. The second computing device 150 typically has relatively weak computing power and is also referred to as a lightweight device in this disclosure. This disclosure is not intended to limit the specific form of the second computing device 150.

[0169] In some embodiments, such as Figure 2 As shown in Figure A, user 105 can access the smart home configuration user interface 200A through user terminal 110, and can, for example, select "living room speaker" to configure the smart speaker. It should be understood that user 105 can also select other suitable smart home devices for configuration. This is for illustrative purposes only. Figure 1 and Figure 2 The scenario uses "living room speakers" as an example.

[0170] like Figure 2 As shown in Figure B, user terminal 110 can, for example, provide user 105 with a user interface 200B for configuring custom keywords, enabling user 105 to modify, add, or delete keywords that the smart speaker can recognize via voice. Figure 2 As shown in B, the smart speaker, for example, added three custom keywords previously created by user 105: "lower volume", "mute", and "switch signal source".

[0171] For each custom keyword, user 105 can, for example, view the synonyms supported by the smart speaker. Taking "lower the volume" as an example, user 105 can, for example, click "view synonyms" to view the synonym phrases supported by the smart speaker for "lower the volume," such as "reduce the volume" and "turn down the volume," etc. Figure 2 As shown in C.

[0172] In some embodiments, user 105 may also edit the synonym groups supported by the smart speaker to delete or modify synonyms.

[0173] like Figure 2 As shown in Figure D, user 105 can also configure new keywords, for example. In some embodiments, user 105 can input the text "raise your voice" via the touchscreen provided by user terminal 110. Alternatively, user 105 can also directly record the voice corresponding to "raise your voice" by clicking the microphone button, for example.

[0174] like Figure 2 As shown in D, user 105 can, for example, customize the action corresponding to the keyword, that is, what operation the smart speaker should perform when it recognizes the keyword. Figure 2 In example D, based on the rules of this configuration, the smart speaker can, for example, perform the action of "increasing the speaker volume" after recognizing the keyword "increase the sound".

[0175] In some embodiments, a pronunciation dictionary can be used to convert the keywords obtained in the above manner and other keywords with the same semantics into corresponding phoneme sequences for obtaining synonymous phoneme sequence groups.

[0176] Continue to refer to Figure 1 User terminal 110 may, for example, provide the received voice input 115 or text input 120 to a first computing device 130, such as a cloud server, via wireless communication.

[0177] like Figure 1 As shown, the first computing device 130 can obtain the user-input keyword "raise your voice" or the corresponding phoneme sequence (e.g., through a pronunciation dictionary) from the voice input 115 or text input 120, and thereby determine the target keyword or the corresponding target phoneme sequence for training the language model. In some embodiments, the target keyword may include, for example, default system keywords, user-defined keywords previously used, and keywords that the user wishes to add.

[0178] Alternatively, the target keywords may include only keywords associated with the user's personalization, and the default system keywords may not be repeatedly retrieved in response to the user's custom operations, so that the generated speech recognition model 140 is only used to support speech recognition of user-defined keywords. Similarly, the target phoneme sequence may include only phoneme sequences associated with the user's personalization.

[0179] After acquiring the target keywords, the first computing device 130 can determine synonym groups semantically associated with the target keywords, and use the target keywords and synonym groups to train a language model, further obtaining a speech recognition model 140. Similarly, after acquiring target language information, the first computing device 130 can determine synonymous phoneme sequence groups associated with the target language information, and use the synonymous phoneme sequence groups to train a language model, further obtaining a speech recognition model 140. The process of obtaining the speech recognition model 140 will be described in detail below. Figure 2 Detailed descriptions will not be provided here.

[0180] In some embodiments, the speech recognition model 140 obtained using target keywords and synonyms can be deployed to a smart speaker specified by user 105. After the speech recognition model 140 is deployed on the smart speaker, it will be able to support speech recognition of target keywords and associated synonyms. For example, user 155 (who may be the same or different from user 105) can provide voice input 160 to the smart speaker, such as "increase volume". Accordingly, the smart speaker can use the speech recognition model 140 to process the voice input 160 and determine the text representation 170 corresponding to the voice input 160, namely "increase volume".

[0181] It should be understood that text representation 170 can be the text corresponding to all speech segments of voice input 160, or text representation 170 can be the text corresponding to a portion of speech segments of voice input 160. For example, user 155 can input "please increase the volume" by voice, and the smart speaker can accordingly use speech recognition model 140 to recognize the keyword "increase the volume".

[0182] In some embodiments, the voice input 160 received by the smart speaker may correspond to a custom keyword configured by the user 105, such as "raise the voice", or it may correspond to a synonym automatically determined by the first computing device 130, such as "increase the volume".

[0183] In some embodiments, the smart speaker can perform an action corresponding to the text representation 170 based on pre-configured rules. For example, user 105 has previously configured the corresponding action to be "increase speaker volume". This allows the smart speaker to perform the action of "increase speaker volume" after recognizing the keyword "increase volume".

[0184] In some embodiments, text representation 170 may also trigger another device, different from the smart speaker, to perform a corresponding action. For example, the user may configure the corresponding action to "increase the volume of the living room TV". Accordingly, when the smart speaker recognizes "increase volume", it can send a command to the "living room TV" to increase the TV's volume.

[0185] In some embodiments, the speech recognition model 140 obtained using a group of synonymous phoneme sequences can also be deployed to a smart speaker specified by user 105. After the speech recognition model 140 is deployed on the smart speaker, it will be able to support speech recognition for groups of synonymous phoneme sequences. For example, user 155 (who may be the same or different from user 105) can provide voice input 160 to the smart speaker, such as "increase the volume". Accordingly, the smart speaker can process the voice input 160 using the speech recognition model 140 and determine the phoneme sequence representation 180 of the user's voice input 160, namely, "tisheng yin liang".

[0186] It should be understood that when recognizing speech input based on phoneme sequences, the phoneme sequence representation 180 can be the phoneme sequence corresponding to all speech segments of the speech input 160, or it can be the phoneme sequence corresponding to a portion of the speech segments of the speech input 160. For example, user 155 can input "qing ti sheng yinliang" (please increase volume), and the smart speaker can accordingly use the speech recognition model 140 to recognize the included phoneme sequence "ti sheng yinliang".

[0187] In some embodiments, the voice input 160 received by the smart speaker may correspond to a custom phoneme sequence configured by the user 105, such as "ti gao sheng yin" (raise the voice), or it may correspond to a synonymous phoneme sequence automatically determined by the first computing device 130, such as "ti sheng yin liang" (raise the volume).

[0188] In some embodiments, the smart speaker can perform an action corresponding to the phoneme sequence representation 180 based on pre-configured rules. For example, user 105 has previously configured the corresponding action to be "increase speaker volume". This allows the smart speaker to perform the action of "increase speaker volume" after recognizing the phoneme sequence representation "ti sheng yin liang".

[0189] In some embodiments, the phoneme sequence representation 180 can also trigger another device, different from the smart speaker, to perform a corresponding action. For example, the user can configure the corresponding action to "increase the volume of the living room TV". Accordingly, when the smart speaker recognizes "ti sheng yin liang", it can send a command to the "living room TV" to increase the TV's volume.

[0190] Example Environment 2 Figure 3 A schematic diagram of an example environment 300 in which various embodiments of the present disclosure can be implemented is shown. For example... Figure 3 As shown, environment 300 includes a first computing device 330. Figure 3 In the example, such a first computing device 330 may include, for example, a cloud-side device.

[0191] and Figure 1 Unlike the environment 100 shown, user 305 can, for example, configure custom keywords that the second computing device 310 is expected to recognize by speech by directly providing voice input 320 to the second computing device 310 to be configured. Figure 3 In the example, the second computing device 310 is shown as a smart speaker. It should be understood that the smart speaker is only one example of the second computing device 310. The second computing device 310 may also include other suitable devices, such as smart home devices (e.g., air conditioners, refrigerators, washing machines, televisions, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.) or in-vehicle devices, etc.

[0192] Accordingly, the voice input 320 can be sent to the first computing device 330, for example, via a wired or wireless network. The first computing device 330 can, for example, use appropriate speech recognition technology and pre-specified grammar rules to extract the keyword "raise your voice" or the phoneme sequence "ti gao sheng yin" from the voice input 320.

[0193] Reference Figure 1 The process discussed is similar; the first computing device 330 can acquire target keywords including "enhance the voice," and can further acquire synonym phrases 335 that are phonetically associated with the target keywords. The first computing device 330 can further utilize the target keywords and synonym phrases 335 to obtain a speech recognition model 340.

[0194] The first computing device 330 can also acquire target language input including “ti gao sheng yin”, and can further acquire a group of synonymous phoneme sequences (not shown in the figure) associated with the target language input. The first computing device 330 can further utilize the group of synonymous phoneme sequences to obtain a speech recognition model 340.

[0195] In some embodiments, the speech recognition model 340 can be further deployed to a smart speaker to enable the smart speaker to recognize the keyword "increase volume" and its corresponding synonyms as defined by user 305. For example, user 355 (who may be the same as or different from user 305) can provide speech input 360 to the smart speaker, such as "increase volume". Accordingly, the smart speaker can utilize the speech recognition model 340 to process the speech input 360 and determine either the text representation 370 corresponding to the speech input 360, i.e., "increase volume", or the phoneme sequence representation 380 corresponding to the speech input 360, i.e., "ti sheng yin liang".

[0196] It should be understood that text representation 370 may be the text corresponding to all speech segments of voice input 360, or it may be the text corresponding to a portion of the speech segments of voice input 360. For example, user 355 may input "please increase the volume" via voice, and the smart speaker may accordingly use speech recognition model 340 to recognize the keyword "increase the volume". In some embodiments, the voice input 360 received by the smart speaker may correspond to a custom keyword configured by user 305, such as "raise the voice", or it may correspond to a synonym automatically determined by the first computing device 330, such as "increase the volume". In some embodiments, the smart speaker may also perform an action corresponding to the text representation 370 based on pre-configured rules.

[0197] Similarly, the phoneme sequence representation 380 can be the phoneme sequence corresponding to all speech segments of the voice input 360, or it can be the text corresponding to a portion of the speech segments of the voice input 360. For example, user 355 can input "qing ti sheng yin liang" (please increase volume) via voice, and the smart speaker can accordingly use the speech recognition model 340 to recognize the phoneme sequence "ti sheng yin liang" (increase volume).

[0198] In some embodiments, the voice input 360 received by the smart speaker may correspond to a custom phoneme sequence configured by the user 305, such as "ti gao sheng yin" (raise the voice), or it may correspond to a synonymous phoneme sequence automatically determined by the first computing device 330, such as "ti sheng yin liang" (increase the volume). In some embodiments, the smart speaker may also perform an action corresponding to the phoneme sequence representation 380 based on pre-configured rules.

[0199] Example Environment 3 Figure 4 A schematic diagram of yet another example environment 400 in which various embodiments of this disclosure can be implemented is shown. (See diagram for reference.) Figure 4 As shown, environment 400 includes a first computing device 430. In Figure 4 In the example, the first computing device 430 may be, for example, a user terminal, such as a smartphone or tablet.

[0200] and Figure 1 Unlike the environment 100 shown, user 405 can configure the second computing device 450, for example, through an interface provided by a user terminal. Figure 4 In this context, the second computing device 450 is shown as a smart TV; however, it should be understood that the smart TV is merely one example of the second computing device 450, which may also include other suitable devices, such as smart home devices (e.g., air conditioners, refrigerators, washing machines, speakers, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.) or in-vehicle devices, etc.

[0201] Reference Figure 1 The processes discussed are 100 different, in Figure 4 In environment 400, the user terminal can directly determine the keyword "raise the voice" or the corresponding phoneme sequence from the voice input 410 or text input 420 without sending the voice input 410 or text input 420 to the cloud device.

[0202] Furthermore, the user terminal can also acquire target keywords, which may include, for example, keywords determined based on voice input 410 or text input 420, or keywords previously defined by the user. The user terminal can further acquire synonym phrases 435 semantically related to the target keywords, and based on... Figure 1 A similar process is used to obtain the speech recognition model 440.

[0203] Similarly, the user terminal can also acquire target language information (e.g., text or speech). Such target language information may include, for example, a phoneme sequence determined based on speech input 410 or text input 420, and may also include phoneme sequences previously defined by the user. The user terminal can further acquire a group of synonymous phoneme sequences (not shown in the figure) associated with the target speech language information, and based on... Figure 1 A similar process is used to obtain the speech recognition model 440.

[0204] In some embodiments, the speech recognition model 440 obtained using the target keywords and synonyms 435 can be deployed on the smart TV specified by the user 405.

[0205] After the speech recognition model 440 is deployed on the smart TV, it will be able to support semantic recognition of target keywords and associated synonym phrases 435. For example, user 455 (which may be the same as or different from user 405) can provide voice input 460 to the smart TV, such as "raise the volume". Accordingly, the smart TV can utilize the speech recognition model 440 to process the voice input 460 and determine the text representation 470 corresponding to the voice input 460, namely "raise the volume".

[0206] It should be understood that text representation 470 can be the text corresponding to all speech segments of voice input 460, or text representation 470 can be the text corresponding to a portion of speech segments of voice input 460. For example, user 455 can input "please increase the volume" by voice, and the smart TV can accordingly use the voice recognition model 440 to recognize the keyword "increase the volume" included in the input.

[0207] In some embodiments, the voice input 460 received by the smart TV may correspond to a custom keyword configured by the user 405, such as "raise the voice", or it may correspond to a synonym automatically determined by the first computing device 430, such as "increase the volume".

[0208] In some embodiments, the smart TV can perform the action corresponding to the text representation 470 based on pre-configured rules. For example, user 405 has previously configured the corresponding action to be "increase TV volume". This allows the smart TV to perform the action of "increase TV volume" after recognizing the keyword "increase volume".

[0209] In some embodiments, text representation 470 may also trigger another device, different from the smart TV, to perform a corresponding action. For example, the user may configure the corresponding action to "increase the volume of the living room speaker." Accordingly, upon recognizing "increase volume," the smart TV may send a command to the "living room speaker" to increase the speaker's volume.

[0210] In other embodiments, the speech recognition model 440 obtained using a group of synonymous phoneme sequences can be deployed on a smart TV specified by user 405.

[0211] After the speech recognition model 440 is deployed on the smart TV, it will be able to support semantic recognition of synonymous phoneme sequence groups. For example, user 455 (which may be the same as or different from user 405) can provide speech input 460 to the smart TV, such as "ti sheng yin liang" (raising the volume). Accordingly, the smart TV can use the speech recognition model 440 to process the speech input 460 and determine the phoneme sequence representation 480 corresponding to the speech input 460, namely "tisheng yin liang".

[0212] It should be understood that the phoneme sequence representation 480 can be the text corresponding to all speech segments of the voice input 460, or it can be the text corresponding to a portion of the speech segments of the voice input 460. For example, user 455 can input "qing ti sheng yin liang" (please increase volume) by voice, and the smart TV can accordingly use the voice recognition model 440 to recognize the phoneme sequence "ti sheng yin liang" included in the input.

[0213] In some embodiments, the voice input 460 received by the smart TV may correspond to a custom phoneme sequence configured by the user 405, such as “ti gao sheng yin” (raise the voice), or it may correspond to a synonymous phoneme sequence automatically determined by the first computing device 430, such as “ti sheng yin liang” (raise the volume).

[0214] In some embodiments, the smart TV can perform an action corresponding to the phoneme sequence representation 480 based on pre-configured rules. For example, user 405 has previously configured the corresponding action to be "increase TV volume". This allows the smart TV to perform the "increase TV volume" action after recognizing the phoneme sequence "ti sheng yin liang" (increase volume).

[0215] In some embodiments, the phoneme sequence representation 480 can also trigger another device, different from the smart TV, to perform a corresponding action. For example, the user can configure the corresponding action to "increase the volume of the living room speaker." Accordingly, after recognizing "ti sheng yin liang," the smart TV can send a command to the "living room speaker" to increase the speaker's volume.

[0216] Construction of speech recognition model According to various embodiments of this disclosure, a scheme for constructing a speech recognition model is provided. This scheme constructs the speech recognition model based on keywords. In embodiments of this disclosure, target keywords are obtained, and synonyms semantically associated with the target keywords are determined. Subsequently, a language model is trained using the target keywords and synonyms to obtain a target language model. The target language model, an acoustic model, and a pronunciation dictionary are merged to obtain a speech recognition model, which is a decoding graph. Thus, embodiments of this disclosure can construct a speech recognition model with keyword generalization recognition capabilities.

[0217] The following will be referenced Figures 5 to 11 This describes the speech processing process according to some embodiments of the present disclosure. Figure 5 A flowchart of an example process 500 for speech processing according to some embodiments of the present disclosure is shown. Process 500 may be executed, for example, by the first computing device discussed above, such as... Figure 1 The first computing device 130 in Figure 3 The first computing device 330 or Figure 3 The first computing device 430 is used in the process. Process 500 can also be executed collaboratively by a terminal device and a cloud-side device, for example. For ease of description, process 500 is described below using the first computing device as an example.

[0218] like Figure 5 As shown in box 502, the first computing device acquires the target keyword.

[0219] As discussed above, target keywords may include keywords entered by the user. In some embodiments, the first computing device may determine the user-entered keywords from different types of data (e.g., text data or audio data). References will follow... Figure 6 This describes the detailed process of determining the keywords entered by the user. Figure 6 A flowchart of an example process 600 for determining keywords according to an embodiment of this disclosure is shown.

[0220] like Figure 6 As shown, in box 602, the first computing device can acquire target keyword data. In some embodiments, the first computing device can acquire voice input from an audio acquisition device. Figure 1 In the example, the first computing device 130 can obtain voice input 115 from the user terminal 110. Figure 3 In the example, the first computing device 130 may, for instance, obtain voice input 320 from a second computing device 310 where the speech recognition model is to be deployed. Figure 4 In the example, the first computing device 430 may be a terminal device that can directly acquire voice input 410 using a voice acquisition device (e.g., a microphone).

[0221] Alternatively or additionally, the first computing device may also acquire text input via a text acquisition device. Figure 1 In the example, the first computing device 130 can obtain text input 120 from the user terminal 110. Figure 4 In the example, the first computing device 430 may be a terminal device that can directly acquire text input 420 using a text collector (e.g., a touchscreen).

[0222] In box 604, the first computing device can determine whether the keyword data is audio or text. If the type is text, process 600 can proceed to box 608, where the first computing device can, for example, directly determine the keywords from the text input.

[0223] If it is determined in box 604 that the type of the target keyword data is audio, then process 600 can proceed to box 606, whereby the first computing device can, for example, use ASR (Automatic Speech Recognition) to recognize the voice input. Accordingly, the voice input can be converted into corresponding text. Further, in box 608, the first computing device can determine the keywords entered by the user from the text based on the voice input 110.

[0224] Alternatively or additionally, target keywords may also include pre-stored historical keywords. Such historical keywords may, for example, include default system keywords. These system keywords may, for example, be specified by the manufacturer of the second computing device. Alternatively, pre-stored historical keywords may also include user-defined historical keywords, such as… Figure 2 The keywords shown in B include "lowering the volume" and so on.

[0225] In some embodiments, to avoid the generated speech recognition model from becoming too complex, the first computing device may also limit the number of keywords in the keyword group used to train the language model.

[0226] After determining the keywords input by the user, the first computing device can determine a first keyword group based on the user-input keywords and pre-stored historical keywords. If the number of keywords in the first keyword group exceeds a predetermined threshold, the first computing device retrieves the target keywords from the first keyword group based on the predetermined threshold. For example, the first computing device can retain only the keywords within the first keyword group that meet the predetermined threshold as the target keywords. This approach avoids using too many keywords for training the language model, ensuring the decoding graph is lightweight and suitable for devices with limited computing resources.

[0227] In some embodiments, the first computing device may obtain the target keywords from the first keyword group based on the attributes of the keywords in the target keywords, wherein the number of target keywords is the predetermined threshold. Such attributes may include, for example, keyword type (e.g., system-created or user-defined) or keyword creation time. For example, a predetermined threshold number of keywords may be retained from the first keyword group based on their creation time, such that the keyword with the earliest creation time is deleted.

[0228] Alternatively, if the target keywords include default system keywords, these system keywords can always be used as target keywords. Correspondingly, the first computing device can also select one or more keywords from user-defined keywords as target keywords based on the difference between a predetermined threshold and the number of system keywords.

[0229] Alternatively, the pre-stored historical keywords may include only user-defined keywords. Accordingly, the predetermined threshold can be used to limit the number of user-defined keywords supported by the speech recognition model. Thus, if the first keyword group already includes a predetermined number of user-defined keywords, the first computing device can, for example, select a predetermined threshold number of user-defined keywords from the first keyword group as target keywords.

[0230] In some embodiments, the first computing device may also obtain target keywords from a first keyword group based on user input. Figure 2 As an example, the first computing device, for instance, allows a user to configure up to three custom keywords. Once three custom keywords have been configured, if the user wishes to add new custom keywords, the user terminal can prompt the user to select which keywords should be retained / deleted from the three already configured custom keywords, ensuring that the number of target keywords used for training remains at a predetermined threshold.

[0231] In this way, embodiments of the present disclosure can support personalized customization of the speech recognition model deployed at the second computing device.

[0232] Continue to refer to Figure 5 In box 504, the first computing device acquires synonyms semantically associated with the target keyword. For a newly received keyword, the first computing device can determine synonyms from a thesaurus based on the keyword's semantics. In some embodiments, the thesaurus may be maintained locally on the first computing device, or it may be maintained on a remote device different from the first computing device. For pre-stored historical keywords, the first computing device can, for example, directly acquire previously stored synonyms without needing to redetermine synonyms from the thesaurus.

[0233] The following will be referenced Figure 7 To describe the specific process of identifying synonym phrases. Figure 7 A flowchart of an example process 700 for determining synonym groups according to an embodiment of the present disclosure is shown.

[0234] like Figure 7 As shown, in box 702, the first computing device can acquire keywords. In box 704, the first computing device can, for example, use natural language understanding technology to determine the first semantics of the target keywords. In box 706, the first computing device can search a lexicon for multiple candidate synonyms that are close to the first semantics based on the first semantics. Specifically, the semantic difference between each determined candidate synonym and the first semantics is less than a predetermined difference threshold.

[0235] In some embodiments, the plurality of candidate synonyms can be directly identified as a group of synonyms for training a language model.

[0236] In some embodiments, process 700 may further include block 708, in which the first computing device can filter multiple candidate synonyms. In some embodiments, the first computing device may filter based on the length difference between candidate synonyms and target keywords, such that the length difference between each synonym in the determined synonym group and the keyword is less than a length threshold.

[0237] For example, the first computing device can select only candidate synonyms with the same length as the target keyword from among multiple candidate synonyms as the synonym group to be used for training. Based on this approach, the generated decoding graph can have a simpler structure, making it more suitable for deployment on a second computing device with lower computing power.

[0238] In some embodiments, the first computing device may also provide a plurality of candidate synonyms to the user and determine a group of synonyms from the plurality of candidate synonyms based on user input received from the user, wherein the user input indicates that at least one of the plurality of candidate synonyms is excluded or confirmed.

[0239] For example, the first computing device may provide the plurality of candidate synonyms to the user through an appropriate means (e.g., voice broadcast or display on a screen) and receive feedback from the user regarding the candidate synonyms. Such feedback may, for example, indicate that at least one candidate synonym is confirmed or at least one candidate synonym is excluded. For instance, the user may determine which of the candidate synonyms should be retained or excluded by clicking on the displayed candidate synonyms on the screen. Alternatively, the user may also indicate which candidate synonyms should be retained or excluded by voice input.

[0240] Based on this approach, embodiments of this disclosure can adjust the synonyms used to train the speech recognition model based on user feedback. This enables the obtained speech recognition model to better suit user habits and avoids automatically expanding some synonyms that users do not want.

[0241] In some embodiments, to avoid the constructed speech recognition model being too complex, the first computing device may also ensure that the number of synonyms included in the determined synonym group 135 does not exceed a predetermined number. Accordingly, when there are more than a predetermined number of candidate synonyms, the first computing device may, for example, select the predetermined number of candidate synonyms that are semantically closest as the synonym group 135.

[0242] Continue to refer to Figure 5 In box 506, the first computing device trains a training model based on target keywords and synonyms to obtain a target language model.

[0243] Specifically, the first computing device can construct a training dataset for training a language model based on target keywords and synonyms, and obtain the target language model based on the training dataset. Figure 8 A schematic diagram 800 illustrates an example process for training a language model according to some embodiments of the present disclosure.

[0244] like Figure 8 As shown, the training dataset 805 constructed based on the target keywords and synonyms can be provided to the language model training module 810. Specifically, the language model training module 810 may include a feature extraction module 815 for generating input features based on the training dataset 805 and providing the input features to the model training module 820 to obtain a target language model 825, which can indicate the grammatical constraint rules determined based on the target keywords and synonyms.

[0245] Examples of target language models 825 include, but are not limited to: N-gram models based on N-grams, RNN-LM models based on neural networks, JSGF models based on regular syntax, etc. This disclosure is not intended to limit the specific type of language model.

[0246] Continue to refer to Figure 5 In box 508, the first computing device generates a first decoding graph based on the target language model. The first decoding graph indicates multiple decoding paths that satisfy the grammatical constraint rules determined based on the target keywords and synonyms.

[0247] The target language model, acoustic model, and pronunciation dictionary are merged to obtain a speech recognition model, which is a decoding graph.

[0248] In some embodiments, after generating the target language model 525, the first computing device can generate a decoding map based on the target language model 525 and an existing pronunciation dictionary and acoustic model. In some embodiments, the acoustic model can be trained offline or online. The acoustic model can also adopt various model structures such as DNN-HMM, LSTM-HMM, TDNN-HMM, etc., and this disclosure is not intended to limit the type of acoustic model or the training process.

[0249] In some embodiments, the first computing device generates the decoding graph, for example, based on the HCLG (HMM + Context + Lexicon + Grammar) decoding graph construction process. Figure 9 A schematic diagram 900 illustrates an example process for generating a decoded graph according to an embodiment of the present disclosure.

[0250] like Figure 9 As shown, the first computing device first uses the model merging unit 915 to merge the target language model 905 (e.g., Figure 8 The target language model (825) and pronunciation dictionary (910) are used to generate the merged model 1 (920).

[0251] Optionally, if the acoustic model 940 is trained using monophones, the first computing device can directly merge the merging model 1 90920 with the acoustic model 940 without considering context-dependent phonemes. Conversely, if the acoustic model 940 takes into account the context of phonemes during training, for example, by training with multiple phonemes, the first computing device can first use the model merging unit 930 to merge the merging model 1 920 and the context-dependent phonemes 925 to generate the merging model 2 935, and then use the model merging unit 945 to merge the merging model 2 935 and the acoustic model 940 to generate the decoding graph 950. The decoding graph 950, also known as the HCLG decoding model, is used to indicate multiple decoding paths based on the grammatical constraint rules determined by the target keyword and synonyms.

[0252] Continue to refer to Figure 5 In box 510, the first computing device determines a speech recognition model based on the first decoded graph.

[0253] In some embodiments, the first computing device may directly use the decoded graph 950 as the final speech recognition model.

[0254] In some embodiments, the target keywords may include at least a first keyword and a second keyword. To improve the efficiency of decoding graph search, the first computing device may also perform synonym clustering on the obtained decoding graph. Figure 10A schematic diagram 1000 illustrates an example process of synonym clustering according to an embodiment of the present disclosure.

[0255] like Figure 10 As shown, the first computing device can utilize the synonym subgraph clustering module 1020 to process the first decoded graph 1010 (e.g., Figure 9 The first computing device performs synonym clustering on the decoding graph 950. Specifically, the first computing device can obtain a first set of decoding paths and a second set of decoding paths from the first decoding graph. The first set of decoding paths includes decoding paths corresponding to a first keyword and a first set of synonyms semantically associated with the first keyword, and the second set of decoding paths includes decoding paths corresponding to a second keyword and a second set of synonyms semantically associated with the second keyword. Further, the first computing device can generate a first subgraph based on the first set of decoding paths and a second subgraph based on the second set of decoding paths. Figure 11 A schematic diagram of example sub-Figure 1100 according to some embodiments of the present disclosure is shown. For example... Figure 11 As shown, subgraph 1100 includes the decoding path corresponding to the keyword "enhance the sound" and synonyms.

[0256] Furthermore, the first computing device can determine the speech recognition model based at least on the first subgraph and the second subgraph. Specifically, the first computing device can, for example, generate a second decoded graph based on the first and second subgraphs as a speech recognition model. Figure 10 As shown, in some embodiments, the decoded graph after subgraph clustering can be directly used as the second decoded graph 1040 and used as the final speech recognition model. When the target keyword includes multiple keywords, the generated second decoded graph 1040 can include multiple independent subgraphs corresponding to the multiple keywords.

[0257] Alternatively, the first computing device can also utilize the subgraph weight adjustment module 1030 to perform subgraph weight adjustment on the decoded graph after subgraph clustering. Specifically, the first computing device assigns the same weight to the first decoding path corresponding to the target keyword as to the second decoding path corresponding to a synonym in a synonym group, thereby obtaining the final decoded graph 1040. Figure 11 As an example, the decoding path corresponding to the target keyword "enhance the sound" has the same weight as the decoding path corresponding to the synonym "increase volume". Based on this approach, faster decoding searches for the expanded synonyms can be achieved, thereby reducing computational and storage overhead.

[0258] Based on the methods discussed above, the first computing device can automatically expand related synonym groups based on the target keywords and construct a decoding graph for the second computing device. This allows the generated decoding graph to meet both the requirements of lightweight design and the ability of the second computing device to perform keyword generalization recognition.

[0259] In some embodiments, the first computing device may also instruct a target computing device (e.g., a second computing device) to provide a speech recognition model for deployment on the target computing device. Exemplarily, the first computing device may send the speech recognition model to the second computing device via wired or wireless communication for deployment. Alternatively, the first computing device may store the model in a predetermined storage device, enabling the second computing device to automatically retrieve the speech recognition model from the storage device for deployment.

[0260] According to various embodiments of this disclosure, another scheme for constructing a speech recognition model is provided. This scheme constructs the speech recognition model based on phoneme sequences. In embodiments of this disclosure, target language information is obtained, and a group of synonymous phoneme sequences associated with the target language information is determined. Subsequently, the language model is trained using the group of synonymous phoneme sequences to obtain a target language model. The target language model can be merged with an acoustic model to obtain a speech recognition model, which is a decoding graph. Thus, embodiments of this disclosure can construct a speech recognition model with phoneme sequence generalization recognition capability.

[0261] The following will be referenced Figures 12 to 17 This describes the speech processing process according to some embodiments of the present disclosure. Figure 12 A flowchart of an example process 1200 for speech processing according to some embodiments of the present disclosure is shown. Process 1200 may be executed, for example, by the first computing device discussed above, such as... Figure 1 The first computing device 130 in Figure 3 The first computing device 330 or Figure 3 The first computing device 430 is used in the process. Process 1200 can also be executed collaboratively by a terminal device and a cloud-side device, for example. For ease of description, process 1200 is described below using the first computing device as an example.

[0262] like Figure 12 As shown, in box 1202, the first computing device acquires target language information. In some embodiments, the first computing device can determine the target language information from different types of data (e.g., text data or audio data). The first computing device can acquire speech input from an audio acquisition device. Figure 1 In the example, the first computing device 130 can obtain voice input 115 from the user terminal 110. Figure 3In the example, the first computing device 130 may, for instance, obtain voice input 320 from a second computing device 310 where the speech recognition model is to be deployed. Figure 4 In the example, the first computing device 430 may be a terminal device that can directly acquire voice input 410 using a voice acquisition device (e.g., a microphone).

[0263] Alternatively or additionally, the first computing device may also acquire text input via a text acquisition device. Figure 1 In the example, the first computing device 130 can obtain text input 120 from the user terminal 110. Figure 4 In the example, the first computing device 430 may be a terminal device that can directly acquire text input 420 using a text collector (e.g., a touchscreen).

[0264] Alternatively or additionally, the target language information may also include pre-stored historical language information. Historical language information may include speech or text. Such historical language information may, for example, include default system language information. This system language information may, for example, be specified by the manufacturer of the second computing device. Alternatively, the pre-stored historical language information may also include user-defined historical language information, for example, with... Figure 2 The “lower volume” option shown in B corresponds to the audio or text.

[0265] In some embodiments, to avoid the generated speech recognition model being too complex, the first computing device may also limit the number of phoneme sequences in the group of synonymous phoneme sequences used to train the language model.

[0266] After determining the target language information input by the user, the first computing device can determine a first language information group based on the user-input target language information and pre-stored historical language information. If the number of language information in the first language information group exceeds a predetermined threshold, the first computing device retrieves the target language information from the first language information group based on the predetermined threshold. For example, the first computing device can retain only the predetermined threshold number of language information in the first language information group as the target phoneme sequence. In this way, the number of target language information used to train the language model can be avoided, thus ensuring that the decoding graph is lightweight and suitable for devices with limited computing resources.

[0267] In some embodiments, the first computing device may obtain target language information from a first language information group based on attributes of the language information, wherein the number of target language information items is a predetermined threshold. Such attributes may include, for example, the type of language information (e.g., system-created or user-defined) or the creation time of the language information. For example, a predetermined threshold number of language information items may be retained from the first language information group based on their creation time, such that the earliest created language information item is deleted.

[0268] Alternatively, if the target language information includes default system language information, this system language information can always be used as the target language information. Accordingly, the first computing device can also select one or more language information from user-defined language information as the target language information based on the difference between a predetermined threshold and the number of system language information.

[0269] Alternatively, the pre-stored historical language information may include only user-defined language information. Accordingly, the predetermined threshold can be used to limit the number of user-defined language information pieces supported by the speech recognition model. Thus, if the first language information group already includes a predetermined number of user-defined language pieces, the first computing device can, for example, select a predetermined threshold number of user-defined language pieces from the first language information group as the target language information.

[0270] In some embodiments, the first computing device may also obtain target language information from a first language information group based on user input. Figure 2 As an example, the first computing device, for instance, allows a user to configure up to three custom keywords. Once these three custom keywords have been configured, if the user wishes to add new ones, the user terminal can prompt the user to select which keywords should be retained / deleted from the three already configured keywords. The retained keywords and their synonym phoneme sequences can then be used to train the target language model. In this way, the amount of language information used for training can be guaranteed to be within a predetermined threshold.

[0271] In this way, embodiments of the present disclosure can support personalized customization of the speech recognition model deployed at the second computing device.

[0272] Continue to refer to Figure 12 In box 1204, the first computing device acquires a group of synonymous phoneme sequences associated with the target language information. The group of synonymous phoneme sequences includes at least one synonymous phoneme sequence. The at least one synonymous phoneme sequence contains phoneme sequences corresponding to words or phrases semantically similar to the target language information.

[0273] The first computing device can determine synonymous phoneme sequence groups based on the semantics of the target language information. For example, for target language information in speech form, it can map the information to keywords using a pronunciation dictionary, then retrieve synonymous keywords from a word database, and then map them to phoneme sequences using the pronunciation dictionary. For target language information in text form, the corresponding keywords can be obtained directly through word segmentation.

[0274] In some embodiments, the dictionary may be maintained locally on the first computing device, or it may be maintained on a remote device different from the first computing device. For pre-stored historical language information, the first computing device may, for example, directly retrieve previously stored synonymous phoneme sequence groups without needing to redetermine the synonymous phoneme sequence groups.

[0275] The following will be referenced Figure 13 This describes the specific process of determining the sequence of synonymous phonemes. Figure 13 A flowchart illustrating an example process 1300 for determining a group of synonymous phoneme sequences according to an embodiment of the present disclosure is shown.

[0276] like Figure 13 As shown, in box 1302, the first computing device can acquire target language information. In some embodiments, the target language information can be converted into corresponding keywords. In box 1304, the first computing device can, for example, utilize natural language understanding technology to determine a first semantic of the target language information. In box 1306, the first computing device can search for multiple candidate keywords that are close to the first semantic, for example, through a thesaurus. Specifically, the semantic difference between each determined candidate synonym and the first semantic is less than a predetermined difference threshold.

[0277] Then, in box 1308, the first computing device can obtain phoneme sequences of multiple candidate keywords, for example, through a pronunciation dictionary. In some embodiments, the phoneme sequences of multiple candidate keywords can be directly determined as a group of synonymous phoneme sequences for training a language model.

[0278] In some embodiments, process 1300 may further include block 1310, in which the first computing device may filter phoneme sequences of multiple candidate synonyms. In some embodiments, the first computing device may filter based on the length difference between the phoneme sequences of the candidate synonyms and the phoneme sequences of the target language information, such that the length difference between each synonym phoneme sequence in the determined group of synonym phoneme sequences and the phoneme sequences of the target language information is less than a length threshold.

[0279] For example, the first computing device can select only candidate synonymous phoneme sequences with the same length as the target phoneme sequence from among multiple candidate synonymous phoneme sequences as the group of synonymous phoneme sequences to be used for training. Based on this approach, the generated decoding graph can have a simpler structure, making it more suitable for deployment on a second computing device with lower computing power.

[0280] In some embodiments, the first computing device may also provide the user with a plurality of candidate synonymous phoneme sequences or corresponding text, and determine a group of synonymous phoneme sequences from the plurality of candidate synonymous phoneme sequences based on user input received from the user, wherein the user input indicates that at least one candidate synonymous phoneme sequence among the plurality of candidate synonymous phoneme sequences is excluded or confirmed.

[0281] For example, the first computing device may provide the plurality of candidate synonymous phoneme sequences to the user through an appropriate means (e.g., voice broadcast or display on a screen), and receive feedback information from the user regarding the plurality of candidate synonymous phoneme sequences. Such feedback information may, for example, indicate that at least one candidate synonymous phoneme sequence is confirmed or at least one candidate synonymous phoneme sequence is excluded. For example, the user may determine which of the plurality of candidate synonymous phoneme sequences should be retained or excluded by clicking on the displayed plurality of candidate synonymous phoneme sequences or their corresponding text. Alternatively, the user may also indicate which candidate synonymous phoneme sequences should be retained or excluded by voice input.

[0282] Based on this approach, embodiments of this disclosure can adjust the synonymous phoneme sequences used to train the speech recognition model based on user feedback. This enables the obtained speech recognition model to better suit the user's usage habits and avoids automatically expanding some synonymous phoneme sequences that the user does not want.

[0283] In some embodiments, to avoid the constructed speech recognition model being too complex, the first computing device may also ensure that the number of synonymous phoneme sequences included in the determined synonymous phoneme sequence group does not exceed a predetermined number. Accordingly, when there are more than a predetermined number of candidate synonymous phoneme sequences, the first computing device may, for example, select a predetermined number of candidate synonymous phoneme sequences that are semantically closest as the synonymous phoneme sequence group.

[0284] Referring back to the figure, in box 1206, the first computing device uses a group of synonymous phoneme sequences to train a training model to obtain a target language model.

[0285] Specifically, the first computing device can construct a training dataset for training a language model based on synonymous phoneme sequence groups, and obtain the target language model based on this training dataset. Example processes and references for training language models are provided. Figure 8The process described is similar, so I won't repeat it here.

[0286] The target language model can indicate grammatical constraint rules determined based on target keywords and synonyms. Examples of target language models include, but are not limited to, N-gram models based on N-grams, RNN-LM models based on neural networks, and JSGF models based on regular syntax. This disclosure is not intended to limit the specific type of language model.

[0287] Continue to refer to Figure 12 In box 1208, the first computing device generates a first decoding graph based on the target language model. The first decoding graph indicates multiple decoding paths that satisfy the grammatical constraint rules determined based on the synonymous phoneme sequence group.

[0288] After generating the target language model, the first computing device can generate a decoding map based on the target language model and an existing acoustic model. In some embodiments, the acoustic model can be trained offline or online. The acoustic model can also adopt various model structures such as DNN-HMM, LSTM-HMM, TDNN-HMM, etc., and this disclosure is not intended to limit the type of acoustic model or the training process.

[0289] Figure 14 A schematic diagram illustrating an example process for generating a decoded map according to an embodiment of this disclosure is shown. Compared to Figure 9 The process shown, process 1400 does not require the use of a pronunciation dictionary, and language model 1520 is obtained based on phoneme sequence training.

[0290] If the acoustic model 1440 is trained using monophones, the first computing device can directly merge the merging model 1435 with the acoustic model 1440, without considering context-dependent phonemes. Conversely, if the acoustic model 1440 takes into account the context of phonemes during training, for example, by training with multiple phonemes, the first computing device can first use the model merging unit 1430 to merge the language model 1420 and the context-dependent phonemes 1425 to generate the merging model 1435, and then use the model merging unit 1445 to merge the merging model 1435 and the acoustic model 1440 to generate the decoding graph 1450. The decoding graph 1450 is used to indicate multiple decoding paths based on the grammatical constraint rules determined by the target phoneme sequence and the synonymous phoneme sequence group.

[0291] Continue to refer to Figure 12 In box 1210, the first computing device determines a speech recognition model based on the first decoded graph.

[0292] In some embodiments, the first computing device may directly use the decoded figure 1450 as the final speech recognition model.

[0293] In some embodiments, the target phoneme sequence may include, for example, at least a first phoneme sequence and a second phoneme sequence. To reduce the efficiency of decoding graph search, the first computing device may also perform synonymous phoneme sequence clustering on the obtained decoding graph. Figure 15 A schematic diagram 1500 illustrates an example process for clustering synonymous phoneme sequences according to an embodiment of the present disclosure.

[0294] like Figure 15 As shown, the first computing device can utilize the synonymous phoneme sequence subgraph clustering module 1520 to perform clustering on the first decoded graph 1510 (e.g., Figure 14 The first computing device performs synonymous phoneme sequence clustering on the first decoding graph (1450). Specifically, the first computing device can obtain a first set of decoding paths and a second set of decoding paths from the first decoding graph. The first set of decoding paths includes decoding paths corresponding to the first phoneme sequence and a first group of synonymous phoneme sequences semantically associated with the first phoneme sequence, and the second set of decoding paths includes decoding paths corresponding to the second phoneme sequence and a second group of synonymous phoneme sequences semantically associated with the second phoneme sequence. Further, the first computing device can generate a first subgraph based on the first set of decoding paths and a second subgraph based on the second set of decoding paths. Figure 16 A schematic diagram of example sub-figure 1600 according to some embodiments of the present disclosure is shown. For example... Figure 16 As shown, subgraph 1600 includes the decoding path corresponding to the phoneme sequence "ti gao sheng yin" (raising the voice) and the group of synonymous phoneme sequences.

[0295] Furthermore, the first computing device can determine the speech recognition model based at least on the first subgraph and the second subgraph. Specifically, the first computing device can, for example, generate a second decoded graph based on the first and second subgraphs as a speech recognition model. Figure 15 As shown, in some embodiments, the decoded graph after subgraph clustering can be directly used as the second decoded graph 1540 and used as the final speech recognition model. When the target phoneme sequence includes multiple phoneme sequences, the generated second decoded graph 1540 may include multiple independent subgraphs corresponding to the multiple phoneme sequences.

[0296] Alternatively, the first computing device can also utilize the subgraph weight adjustment module 1530 to perform subgraph weight adjustment on the decoded graph after subgraph clustering. Specifically, the first computing device assigns the same weight to the first decoding path corresponding to the target phoneme sequence as to the second decoding path corresponding to the synonymous phoneme sequence in the synonymous phoneme sequence group, thereby obtaining the final decoded graph 1540. Figure 16As an example, the decoding path corresponding to the target phoneme sequence "ti gao sheng yin" has the same weight as the decoding path corresponding to the synonym "ti sheng yin liang" (increase volume). Based on this approach, a faster decoding search can be achieved for the extended synonymous phoneme sequences, thereby reducing computational and storage overhead.

[0297] Based on the methods discussed above, the first computing device can automatically expand the target phoneme sequence into associated synonymous phoneme sequence groups and construct a decoding graph for the second computing device. This allows the generated decoding graph to meet both the requirements of lightweight design and the ability of the second computing device to generalize phoneme sequence recognition.

[0298] In some embodiments, the first computing device may also instruct a target computing device (e.g., a second computing device) to provide a speech recognition model for deployment on the target computing device. Exemplarily, the first computing device may send the speech recognition model to the second computing device via wired or wireless communication for deployment. Alternatively, the first computing device may store the model in a predetermined storage device, enabling the second computing device to automatically retrieve the speech recognition model from the storage device for deployment.

[0299] Use of speech recognition models According to various embodiments of this disclosure, a scheme for speech processing is also provided. In embodiments of this disclosure, speech input is received, and a speech recognition model is used to determine the text representation associated with the speech input. The speech recognition model is obtained based on the following process: acquiring target keywords; acquiring synonyms semantically associated with the target keywords; training a language model using the target keywords and synonyms to obtain a target language model; and merging the target language model, an acoustic model, and a pronunciation dictionary to obtain a speech recognition model, where the speech recognition model is a decoding graph. Based on this approach, embodiments of this disclosure enable, for example, computing devices with relatively low computing power to have keyword generalization recognition capabilities, thereby improving the user's voice interaction experience.

[0300] The following will be referenced Figures 17 to 19 This describes the speech processing process according to some embodiments of the present disclosure. Figure 17 A flowchart of an example process 1700 for speech processing according to some embodiments of the present disclosure is shown. Process 1700 may be executed, for example, by a second computing device discussed above, such as... Figure 1 The second computing device 150 in Figure 3 The second computing device 310 or Figure 3 The second computing device 450 in the system.

[0301] like Figure 17 As shown, in block 1702, the second computing device receives voice input. In some embodiments, the second computing device may receive voice input via an audio acquisition device (e.g., a microphone) located locally on the second computing device or via an audio acquisition device communicatively coupled to the second computing device.

[0302] For example, as discussed above, Figure 1 The second computing device 150 can receive voice input 160 from user 155. Figure 3 The second computing device 310 can receive voice input 360 from the user 355. Figure 4 The second computing device 450 can receive voice input 460 from the user 455.

[0303] In box 1704, the second computing device uses a speech recognition model to determine the text representation associated with the speech input. This speech recognition model is acquired by the first computing device using keyword training data based on the process discussed above. For details on the specific construction process of the speech recognition model, please refer to the above section on... Figures 5 to 11 The details described will not be elaborated upon here. Figure 18 A flowchart of an example process for determining speech recognition results according to embodiments of the present disclosure is further illustrated. Figure 18 As shown, in box 1802, the second computing device can acquire the speech signal, and in box 1804, it preprocesses the signal. Subsequently, in box 1806, the second computing device can perform frame-by-frame windowing processing on the preprocessed signal; in box 1808, the second computing device can extract features; in box 1810, the second computing device can perform decoding search based on the extracted features and using the deployed decoding graph; in box 18012, the second computing device can use the decoding graph to obtain the recognition result, i.e., the text representation or phoneme sequence representation associated with the speech input.

[0304] In some embodiments, for a speech recognition model obtained from keyword training data, the text representation may correspond to a synonym of the target keyword or a group of synonyms.

[0305] In some embodiments, the second computing device can also perform an action corresponding to the text representation. For example, the second computing device can query predetermined action rules based on the determined text representation to determine the corresponding action that the second computing device should perform. For instance, based on the text representation "increase volume," the second computing device (e.g., a smart speaker) can perform the action of increasing the speaker volume.

[0306] In some embodiments, the second computing device may also generate corresponding control commands based on the text representation and send them to the third computing device, so that the third computing device performs the corresponding action. For example, the second computing device 150 may be a smart speaker. When the text representation is "turn on the TV", the smart speaker may send a power-on command to the corresponding smart TV, so that the smart TV automatically turns on.

[0307] Based on this approach, the second computing device can utilize a speech recognition model with keyword generalization capabilities to process speech input in order to detect the keywords it contains.

[0308] In some embodiments, the speech recognition model can also be obtained using phoneme sequence training data. For details on the specific construction process of this speech recognition model, please refer to the above section. Figures 12 to 16 The details described will not be elaborated here. Instead of process 1800, the second computing device, after receiving semantic input, can use a speech recognition model to determine the phoneme sequence representation associated with the speech input.

[0309] In some embodiments, for a speech recognition model obtained from phoneme sequence training data, the phoneme sequence representation may correspond to a target phoneme sequence or a synonymous phoneme sequence from a group of synonymous phoneme sequences.

[0310] In some embodiments, the second computing device can also perform actions corresponding to a phoneme sequence representation. For example, the second computing device can query predetermined action rules based on the determined phoneme sequence representation to determine the corresponding action that the second computing device should perform. For instance, based on the phoneme sequence representation "ti sheng yin liang" (increase volume), the second computing device (e.g., a smart speaker) can perform the action of increasing the speaker volume.

[0311] In some embodiments, the second computing device may also generate corresponding control commands based on the phoneme sequence representation and send them to the third computing device, so that the third computing device performs the corresponding action. For example, the second computing device 150 may be a smart speaker. When the phoneme sequence representation is "da kai dian shi" (turn on the TV), the smart speaker may send a power-on command to the corresponding smart TV, so that the smart TV automatically turns on.

[0312] Based on this approach, the second computing device can use a speech recognition model with phoneme sequence generalization recognition capabilities to process speech input in order to detect the phoneme sequences it contains.

[0313] Figure 19A flowchart of an example process 1900 of a speech processing method according to an embodiment of the present disclosure is further shown. Process 1900 may be executed, for example, by a second computing device discussed above, such as... Figure 1 The second computing device 150 in Figure 3 The second computing device 310 or Figure 3 The second computing device 450 in the system.

[0314] In block 1902, the second computing device receives voice command input. In some embodiments, the second computing device may receive voice input via an audio acquisition device (e.g., a microphone) located locally on the second computing device or via an audio acquisition device communicatively coupled to the second computing device.

[0315] In box 1904, the second computing device utilizes a speech recognition model to obtain a phoneme sequence representation of the speech input. Here, the speech recognition model is configured to recognize semantically related groups of phoneme sequences. For details on the specific construction process of this speech recognition model, please refer to the section above. Figures 12 to 16 The content described.

[0316] In box 1906, if the phoneme sequence representation corresponds to a phoneme sequence in the phoneme sequence group, execute the instruction corresponding to the phoneme sequence representation.

[0317] In some embodiments, the speech recognition model is obtained based on the following process: acquiring target language information; acquiring a group of synonymous phoneme sequences associated with the target language information, the group of synonymous phoneme sequences including at least one synonymous phoneme sequence, the at least one synonymous phoneme sequence being a phoneme sequence corresponding to a word or phrase semantically similar to the target language information; training a language model using the group of synonymous phoneme sequences to obtain a target language model; generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths that satisfy grammatical constraint rules determined based on the group of synonymous phoneme sequences; and determining a speech recognition model based on the first decoding graph.

[0318] In some embodiments, if the phoneme sequence representation does not match any phoneme sequence in the phoneme sequence group, the second computing device can provide a notification of no recognition result. A no-recognition result can be represented as a garbage characterization such as "SIL" and discarded.

[0319] In some embodiments, the speech recognition model is configured to recognize a first phoneme sequence group having a first associated semantic and a second phoneme sequence group having a second associated semantic. Process 2000 may further include: performing a first action if the phoneme sequence represents a first phoneme sequence corresponding to the first phoneme sequence group; and performing a second action different from the first action if the phoneme sequence represents a second phoneme sequence corresponding to the second phoneme sequence group.

[0320] In some embodiments, using a speech recognition model to obtain a phoneme sequence representation of a speech input may include: using an acoustic model to generate speech features of the speech input into phoneme emission probabilities; recognizing the speech input by inputting the emission probabilities into a speech recognition model; and causing the speech recognition model to output a phoneme sequence representation.

[0321] Example systems and devices Figure 20 A schematic block diagram of an example speech recognition system 2000 according to some specific embodiments of the present disclosure is shown. Figure 20 As shown, the speech recognition system 2000 may include cloud-side or embedded heavy-duty devices 2020, examples of which include, but are not limited to, cloud-side servers, smartphones, laptops, tablets, desktops, or edge computing devices.

[0322] like Figure 20 As shown, the cloud-side or embedded heavy-duty device 2020 can acquire keyword input data. In some embodiments, the keyword input data may be acquired, for example, by a custom keyword input module 2015 deployed in the keyword receiving device 2010. In some embodiments, the keyword receiving device 2010 may be a device different from the cloud-side or embedded heavy-duty device 2020, and transmits the keyword input data to the transmission communication module 2050 in the cloud-side or embedded heavy-duty device 2020 via wired or wireless communication. In some embodiments, the keyword receiving device 2010 may also be, for example, an embedded lightweight device 2055 for deploying a speech recognition model.

[0323] Alternatively, the keyword receiving device 2010 can also be the same device as the cloud-side or embedded heavy device 2020. In this case, the output of the custom keyword input unit 2015 can be directly provided to the data preprocessing unit 2025.

[0324] like Figure 20 As shown, the data preprocessing module 2025 can determine custom keywords based on the received keyword input data. For example, when the keyword input data is text data, the data preprocessing module 2025 can directly determine custom keywords based on the text input. Furthermore, when the keyword input data is audio data, the data preprocessing module 2025 can first use automatic speech recognition technology to convert the audio data into text data, and then further determine the custom keywords from the text data.

[0325] In some embodiments, the data preprocessing module 2025 can determine the target keyword based on custom keywords and pre-stored historical keywords. In some embodiments, the synonym augmentation module 2030 can determine synonym phrases associated with the target keyword from a thesaurus based on the semantics of the target keyword.

[0326] In some embodiments, the model training module 2035 can train a language model based on target keywords and synonyms, and store the language model in a model library 2040. The model library 2040 can, for example, maintain pre-trained acoustic models, language models, and pronunciation dictionaries. In some embodiments, the model library 2040 can also be maintained on a cloud server.

[0327] Accordingly, the trained language model can also be provided to the decoding graph construction module 2045 to generate a decoding graph for the embedded light device 2055 based on the language model, the acoustic model stored in the model library 2040, and the pronunciation dictionary.

[0328] It should be understood that the above references Figures 5 to 16 The various embodiments discussed regarding the construction of speech recognition models are also applicable to the process of generating decoding maps in cloud-based or embedded heavy-duty devices in 2020, and will not be described in detail here.

[0329] In some embodiments, such as Figure 20 As shown, the generated decoded image can be sent to the keyword recognition and detection unit 2060 in the embedded lightweight device 2055 via the transmission communication module 2050, so that the keyword recognition and detection unit 2060 can use the decoded image to process the received voice input in order to determine the text representation corresponding to the voice input.

[0330] It should be understood that the above references Figures 17 to 19 The various embodiments of speech recognition discussed are also applicable to the process of speech recognition in the embedded light device 2055, and will not be described in detail here.

[0331] The above describes an example speech recognition system 2100 based on keywords for speech recognition. This disclosure also provides an exemplary speech recognition system based on phoneme sequences, which uses phoneme sequences instead of keywords in text form to construct a speech recognition model. The constructed speech recognition model recognizes speech input as a phoneme sequence and recognizes speech based on the phoneme sequence rather than keywords. The overall structure of this speech recognition model is similar to... Figure 20 The speech recognition model is similar to 2000, so I won't go into details here.

[0332] Figure 21A schematic block diagram of a speech model building system 2100 according to an embodiment of the present disclosure is further shown. The speech model building system 2100 may include multiple units for performing tasks such as... Figure 5 The corresponding steps in process 500 discussed herein. For example... Figure 21 As shown, the speech model construction system 2100 includes a keyword acquisition unit 2110 for acquiring target keywords; a synonym acquisition unit 2120 for acquiring synonym phrases semantically related to the target keywords; a model training unit 2130 for training a language model using the target keywords and synonym phrases to obtain a target language model; a decoding graph generation unit 2140 for generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths that satisfy the grammatical constraint rules determined based on the target keywords and synonym phrases; and a model determination unit 2150 for determining a speech recognition model based on the first decoding graph.

[0333] In some embodiments, target keywords include keywords from voice input from an audio acquisition device located at a user terminal.

[0334] In other embodiments, target keywords include keywords from text input from a text collector located at a user terminal.

[0335] In some embodiments, the synonym acquisition unit 2120 is further configured to: determine the semantics of the target keyword; and determine a group of synonyms based at least on the semantics of the target keyword, wherein the difference between the semantics of each synonym in the group of synonyms and the semantics of the target keyword is less than a difference threshold.

[0336] In some embodiments, the synonym acquisition unit 2120 is further configured to: determine a group of synonyms based on the semantics and length of the target keyword, wherein the difference between the length of each synonym in the group and the length of the target keyword is less than a length threshold. The length of the keyword may, for example, represent the number of characters or words included in the keyword.

[0337] In some embodiments, the synonym acquisition unit 2120 is further configured to: acquire multiple candidate synonyms based on the semantics of the target keyword; provide multiple candidate synonyms to the user; and determine a group of synonyms from the multiple candidate synonyms based on user input received from the user, wherein the user input indicates that at least one candidate synonym among the multiple candidate synonyms is excluded or confirmed.

[0338] In some embodiments, the target keywords include at least a first keyword and a second keyword, wherein the model determination unit 1550 is further configured to: obtain a first set of decoding paths and a second set of decoding paths from a first decoding graph, wherein the first set of decoding paths includes decoding paths corresponding to the first keyword and a first synonym group semantically associated with the first keyword, and the second set of decoding paths includes decoding paths corresponding to the second keyword and a second synonym group semantically associated with the second keyword; generate a first subgraph based on the first set of decoding paths and generate a second subgraph based on the second set of decoding paths; and determine a speech recognition model based at least on the first subgraph and the second subgraph.

[0339] In some embodiments, the first subgraph indicates a first decoding path and a second decoding path. The first decoding path is the decoding path corresponding to the first keyword, and the second decoding path is the decoding path corresponding to the synonyms in the first synonym group. The first decoding path and each second decoding path have the same weight in the first subgraph.

[0340] In some embodiments, the keyword acquisition unit 2110 is further configured to: acquire a first keyword group based on pre-stored historical keywords and received keywords; and in response to determining that the number of keywords in the first keyword group exceeds a predetermined threshold, delete at least one keyword in the first keyword group, and the remaining keywords in the first keyword group after deleting at least one keyword are the target keywords.

[0341] In some embodiments, the keyword acquisition unit 2110 is further configured to: delete at least one keyword from the target keywords based on the attributes of the keywords in the target keywords.

[0342] In some embodiments, the keyword acquisition unit 2110 is further configured to: delete at least one keyword in the target first keyword group according to user instructions.

[0343] In some embodiments, the speech model building system 2100 may further include a communication unit for providing a speech recognition model to a second computing device for deploying the speech recognition model on the second computing device.

[0344] It should be understood that the units in the speech model building system 2100 can be implemented using hardware units, software units, or a combination of hardware and software units.

[0345] Figure 22 A block diagram of a speech processing system 2200 according to an embodiment of the present disclosure is further shown. The speech processing system 2200 may include multiple units for performing corresponding steps in process 1700 discussed in FIG. 1700. Figure 22As shown, the speech processing system 2200 includes a speech input unit 2210 for receiving speech input; and a speech processing unit 2220 for determining a text representation associated with the speech input using a speech recognition model. The speech recognition model is obtained based on the following process: acquiring target keywords; acquiring synonyms semantically associated with the target keywords; generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths that satisfy grammatical constraints determined based on the target keywords and synonyms; and determining the speech recognition model based on the first decoding graph.

[0346] In some embodiments, target keywords include keywords from voice input from an audio acquisition device located at a user terminal.

[0347] In other embodiments, target keywords include keywords from text input from a text collector located at a user terminal.

[0348] In some embodiments, determining a group of synonyms associated with the semantics of a target keyword includes: determining the semantics of the target keyword; and determining a group of synonyms based at least on the semantics of the target keyword, wherein the semantic difference between each synonym in the group of synonyms and the semantics of the target keyword is less than a difference threshold.

[0349] In some embodiments, determining synonym groups based at least on the semantics of the target keyword includes: determining synonym groups based on the semantics of the target keyword and the length of the target keyword, wherein the difference between the length of each synonym in the synonym group and the length of the target keyword is less than a length threshold. The length of the keyword may, for example, represent the number of characters or words included in the keyword.

[0350] In some embodiments, determining a group of synonyms based at least on the semantics of the target keyword includes: obtaining a plurality of candidate synonyms based on the semantics of the target keyword; providing the plurality of candidate synonyms to a user; and determining a group of synonyms from the plurality of candidate synonyms based on user input received from the user, wherein the user input indicates that at least one of the plurality of candidate synonyms is excluded or confirmed.

[0351] In some embodiments, the target keywords include at least a first keyword and a second keyword, wherein determining the speech recognition model based on the first decoding graph includes: obtaining a first set of decoding paths and a second set of decoding paths from the first decoding graph, wherein the first set of decoding paths includes decoding paths corresponding to the first keyword and a first synonym group semantically associated with the first keyword, and the second set of decoding paths includes decoding paths corresponding to the second keyword and a second synonym group semantically associated with the second keyword; generating a first subgraph based on the first set of decoding paths; generating a second subgraph based on the second set of decoding paths; and determining the speech recognition model based at least on the first subgraph and the second subgraph.

[0352] In some embodiments, the first subgraph indicates a first decoding path and a second decoding path. The first decoding path is the decoding path corresponding to the first keyword, and the second decoding path is the decoding path corresponding to the synonyms in the first synonym group. The first decoding path and each second decoding path have the same weight in the first subgraph.

[0353] In some embodiments, obtaining target keywords includes: obtaining a first keyword group based on pre-stored historical keywords and received keywords; and in response to determining that the number of keywords in the first keyword group exceeds a predetermined threshold, deleting at least one keyword in the first keyword group, and the remaining keywords in the first keyword group after deleting at least one keyword are the target keywords.

[0354] In some embodiments, deleting at least one keyword from the target keywords includes deleting at least one keyword from the target keywords based on the attributes of the keywords in the target keywords. For example, the earliest created historical keyword can be deleted based on its creation time.

[0355] In some embodiments, deleting at least one keyword in the target first keyword group includes deleting at least one keyword in the target first keyword group according to user instructions.

[0356] In some embodiments, the speech processing system 2200 may further include an action execution unit for performing actions corresponding to the text representation.

[0357] In some embodiments, the voice processing system 2200 may further include a device control unit for generating corresponding control commands based on text representations and sending them to a third computing device so that the third computing device performs the corresponding actions.

[0358] In some embodiments, the text represents a synonym corresponding to a target keyword or a group of synonyms.

[0359] It should be understood that the units in the voice processing system 2200 can be implemented using hardware units, software units, or a combination of hardware and software units. An example of the voice input unit 2210 may include a microphone for receiving voice input, and an example of the voice processing unit 2220 may include a processing device for performing voice recognition operations.

[0360] It should be understood that the speech model building system 2100 and / or the speech processing system 2200 can be implemented using application-specific integrated circuits, one or more FPGAs (Field Programmable Gate Arrays), PLDs (Programmable Logic Devices), controllers, state machines, gate logic, discrete hardware components, any other suitable circuits, or any combination of circuits capable of performing the various processes of this disclosure, chips, boards, or communication devices, etc.

[0361] Figure 23 A block diagram of a speech model building system 2300 according to an embodiment of the present disclosure is further shown. The speech model building system 2300 may include a plurality of units for performing, for example, Figure 12 The corresponding steps in process 1200 discussed herein. For example... Figure 23 As shown, the speech model construction system 2300 includes: a target language information acquisition unit 2310, used to acquire target language information; a synonymous phoneme sequence group acquisition unit 2320, used to acquire synonymous phoneme sequence groups associated with the target language information, wherein the synonymous phoneme sequence group includes at least one synonymous phoneme sequence, and the at least one synonymous phoneme sequence is a phoneme sequence corresponding to a word or phrase semantically similar to the target language information; a model training unit 2330, used to train a language model using the synonymous phoneme sequence groups to obtain a target language model; a decoding graph generation unit 2340, used to generate a first decoding graph based on the target language model, wherein the first decoding graph indicates multiple decoding paths that satisfy the grammatical constraint rules determined based on the synonymous phoneme sequence groups; and a model determination unit 2350, used to determine the speech recognition model based on the first decoding graph.

[0362] In some embodiments, target language information may include speech or text. In some embodiments, target language information includes speech input from an audio acquisition device located at the user terminal. In other embodiments, keywords of text input are obtained from a text acquisition device at the user terminal. In some embodiments, target language information may be short instruction words or instruction statements, such as "close," "stop," "pause," "increase volume," "increase volume," etc.

[0363] For example, if the first computing device is a user terminal, the user can directly provide voice input or text input to the user terminal so that the user terminal can extract target language information from the voice input or text input in order to construct a speech recognition model.

[0364] For example, if the first computing device is a different computing device from the user terminal, such as a cloud device or edge computing device, the user can, for example, use the interface provided by the user terminal to input voice or text. Such voice or text input can be sent to the first computing device, enabling it to acquire target language information for constructing a speech recognition model. In this way, the user can customize the phoneme sequences supported by the speech recognition model, thereby improving the personalization of the speech recognition model.

[0365] In some embodiments, the synonymous phoneme sequence group acquisition unit 2320 can also be used to: determine the semantics of the target language information; and determine the synonymous phoneme sequence group based at least on the semantics of the target language information, wherein the difference between the semantics of each synonymous phoneme sequence in the synonymous phoneme sequence group and the semantics of the target language information is less than a difference threshold.

[0366] In some embodiments, the synonym phoneme sequence group acquisition unit 2320 can also be used to: determine a target phoneme sequence corresponding to the target language information; and determine a synonym phoneme sequence group based on the semantics and length of the target phoneme sequence, wherein the difference between the length of each synonym phoneme sequence in the synonym phoneme sequence group and the length of the target phoneme sequence is less than a length threshold. The length of the phoneme sequence can, for example, represent the number of phonemes (e.g., initials and finals) included in the phoneme sequence. In some embodiments, if the target language information is text, the phoneme sequence corresponding to the text can be obtained as the target phoneme sequence through a pronunciation dictionary. If the target language information is speech, the phoneme sequence of the speech can be obtained as the target phoneme sequence through an acoustic model.

[0367] In some embodiments, the synonym phoneme sequence group acquisition unit 2320 may also be used to: acquire multiple candidate synonyms based on the semantics of target keywords corresponding to target language information; provide multiple candidate synonyms to the user; determine a synonym group from multiple candidate synonyms based on user input received from the user, wherein the user input indicates that at least one candidate synonym among the multiple candidate synonyms is excluded or confirmed; and acquire a synonym phoneme sequence group based on a pronunciation dictionary and a synonym group.

[0368] In some embodiments, the synonym phoneme sequence group acquisition unit 2320 can also be used to: receive voice input from a user; and generate a synonym phoneme sequence group based on the voice input. For example, the semantics of the voice input can be obtained based on the keywords corresponding to the voice input, thereby generating the synonym phoneme sequence group.

[0369] In some embodiments, the target language information includes at least first language information and second language information, wherein the model determination unit 2330 may further be used to: obtain a first set of decoding paths and a second set of decoding paths from a first decoding graph, wherein the first set of decoding paths includes decoding paths for a first group of synonymous phoneme sequences associated with the first language information, and the second set of decoding paths includes decoding paths for a second group of synonymous phoneme sequences associated with the second language information; generate a first subgraph based on the first set of decoding paths; generate a second subgraph based on the second set of decoding paths; and determine a speech recognition model based at least on the first subgraph and the second subgraph.

[0370] In some embodiments, the first subgraph indicates a first decoding path and a second decoding path, which are decoding paths in the first group of synonymous phoneme sequences, and the first and second decoding paths have the same weight in the first subgraph. Based on this approach, a faster decoding search for the extended synonymous phoneme sequences can be achieved, thereby reducing computational and storage overhead.

[0371] In some embodiments, the target language information acquisition unit 2310 may also be used to: acquire a first language information group based on pre-stored historical language information and received language information; and in response to determining that the number of language information in the first language information group exceeds a predetermined threshold, acquire target language information from the first language information group based on the predetermined threshold.

[0372] In some embodiments, the target language information acquisition unit 2310 can also be used to: acquire target language information from a first language information group based on the attributes of the language information in the target language information, wherein the number of target language information is a predetermined threshold. For example, one or more historical language information items that were created earliest can be deleted from the first language information group to obtain the predetermined threshold number of language information items.

[0373] In some embodiments, the target language information acquisition unit 2310 can also be used to: acquire target language information from a first language information group according to user instructions, wherein the number of target language information is a predetermined threshold. For example, user input can be used to select which language information in the first language information group to retain as target language information.

[0374] In some embodiments, the first computing device may also instruct the provision of a speech recognition model to a target computing device (e.g., a second computing device) for deployment of the speech recognition model on the target computing device.

[0375] Figure 24 A block diagram of a speech processing system 2400 according to an embodiment of the present disclosure is further shown. The speech model building system 2300 may include multiple units for performing tasks such as... Figure 19 The corresponding steps in the process discussed in 1900. For example... Figure 24 As shown, the speech processing system 2400 includes: a speech command input unit 2410 for receiving speech command input; a speech processing unit 2420 for using a speech recognition model to obtain a phoneme sequence representation of the speech command input, wherein the speech recognition model is configured to recognize the speech command input based on a group of phoneme sequences that are semantically synonymous with the command; and for executing a command corresponding to the phoneme sequence representation when the phoneme sequence representation corresponds to a phoneme sequence in the phoneme sequence group.

[0376] In some embodiments, the speech recognition model may be obtained by a first computing device. The first computing device may, for example, include a cloud-based or embedded device with strong computing power to construct the speech recognition model. The first computing device may also include, for example, a user terminal device.

[0377] In some embodiments, the voice processing system 2400 may be executed by a second computing device, for example. The second computing device may include an embedded lightweight device, for example, having limited computing power, to perform voice processing using a deployed speech recognition model. Examples of the second computing device may include, but are not limited to: smart home devices (e.g., air conditioners, refrigerators, washing machines, televisions, speakers, etc.), smart wearable devices (e.g., wristbands, watches, glasses, etc.), or in-vehicle devices, etc.

[0378] In some embodiments, the speech recognition model is obtained based on the following process: acquiring target language information; acquiring a group of synonymous phoneme sequences associated with the target language information, the group of synonymous phoneme sequences including at least one synonymous phoneme sequence, the synonymous phoneme sequence being a phoneme sequence corresponding to a word or phrase semantically similar to the target language information; training a language model using the group of synonymous phoneme sequences to obtain a target language model; generating a first decoding graph based on the target language model, the first decoding graph indicating multiple decoding paths that satisfy grammatical constraint rules determined based on the group of synonymous phoneme sequences; and determining a speech recognition model based on the first decoding graph.

[0379] In some embodiments, the speech processing unit 2420 can also be used to: provide a notification of no recognition result if the phoneme sequence representation does not match any phoneme sequence in the phoneme sequence group. Based on this approach, user speech can be recognized in real time and efficiently, improving the user's voice interaction experience.

[0380] In some embodiments, the speech recognition model is configured to recognize a first phoneme sequence group having a first semantic synonym and a second phoneme sequence group having a second semantic synonym. The speech processing unit 2420 may also be configured to: execute a first instruction if the phoneme sequence represents a first phoneme sequence corresponding to a first phoneme sequence in the first phoneme sequence group, and execute a second instruction different from the first instruction if the phoneme sequence represents a second phoneme sequence corresponding to a second phoneme sequence in the second phoneme sequence group.

[0381] In some embodiments, the speech processing unit may also be used to: generate the emission probabilities of speech features to phonemes from the speech command input using an acoustic model; recognize the speech command input by inputting the emission probabilities to a speech recognition model; and cause the speech recognition model to output a phoneme sequence representation.

[0382] Figure 25 A schematic block diagram of an example device 2500 that can be used to implement embodiments of the present disclosure is shown. For example, a first computing device according to an embodiment of the present disclosure (e.g., Figure 1 The first computing device 130 in Figure 3 The first computing device 330 or Figure 3 The first computing device 430) and / or the second computing device (e.g., Figure 1 The second computing device 150 in Figure 3 The second computing device 310 or Figure 3 The second computing device (450) can be implemented by device 2500. As shown, device 2500 includes a central processing unit (CPU) 2501, which can perform various appropriate actions and processes according to computer program instructions stored in read-only memory (ROM) 2502 or loaded from storage unit 2508 into random access memory (RAM) 2503. Various programs and data required for the operation of device 2500 can also be stored in RAM 2503. CPU 2501, ROM 2502, and RAM 2503 are interconnected via bus 2504. Input / output (I / O) interface 2505 is also connected to bus 2404.

[0383] Multiple components in device 2500 are connected to I / O interface 2505, including: input unit 2506, such as keyboard, mouse, etc.; output unit 2507, such as various types of monitors, speakers, etc.; storage unit 2508, such as disk, optical disk, etc.; and communication unit 2509, such as network card, modem, wireless transceiver, etc. Communication unit 2509 allows device 2500 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0384] The various processes and procedures described above, such as processes 500, 600, 700, 800, 900, 1000, 1200, 1300, 1400, 1500, 1700, 1800, and 1900, can be executed by processing unit 2501. For example, in some embodiments, the above processes can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 2508. In some embodiments, part or all of the computer program can be loaded and / or installed on device 2500 via ROM 2502 and / or communication unit 2509. When the computer program is loaded into RAM 2503 and executed by CPU 2501, one or more actions of the processes described above can be performed.

[0385] This disclosure can be a method, apparatus, system, and / or computer program product. A computer program product may include a computer-readable storage medium having computer-readable program instructions loaded thereon for performing various aspects of this disclosure.

[0386] Computer-readable storage media can be tangible devices capable of holding and storing instructions for use by an instruction execution device. Computer-readable storage media can be, for example—but not limited to—electrical storage devices, magnetic storage devices, optical storage devices, electromagnetic storage devices, semiconductor storage devices, or any suitable combination of the foregoing. More specific examples (a non-exhaustive list) of computer-readable storage media include: portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), static random access memory (SRAM), portable compact disc read-only memory (CD-ROM), digital multifunction disc (DVD), memory sticks, floppy disks, mechanical encoding devices, such as punch cards or recessed protrusions storing instructions thereon, and any suitable combination of the foregoing. The computer-readable storage media used herein are not to be construed as transient signals themselves, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through waveguides or other transmission media (e.g., light pulses through fiber optic cables), or electrical signals transmitted through wires.

[0387] The computer-readable program instructions described herein can be downloaded from computer-readable storage media to various computing / processing devices, or downloaded via a network, such as the Internet, local area network, wide area network, and / or wireless network, to an external computer or external storage device. The network may include copper transmission cables, fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards them to the computer-readable storage media in the respective computing / processing device.

[0388] Computer program instructions used to perform the operations of this disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-dependent instructions, microcode, firmware instructions, status setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Smalltalk, C++, etc., and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The computer-readable program instructions may execute entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider). In some embodiments, electronic circuitry, such as programmable logic circuitry, field-programmable gate arrays (FPGAs), or programmable logic arrays (PLAs), is personalized by utilizing the status information of the computer-readable program instructions to implement various aspects of this disclosure.

[0389] Various aspects of this disclosure are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this disclosure. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0390] These computer-readable program instructions can be provided to a processing unit of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to produce a machine such that, when executed by the processing unit of the computer or other programmable data processing apparatus, they create means for implementing the functions / actions specified in one or more blocks of the flowchart and / or block diagram. These computer-readable program instructions can also be stored in a computer-readable storage medium that causes a computer, programmable data processing apparatus, and / or other device to operate in a particular manner. Thus, the computer-readable medium storing the instructions comprises an article of manufacture that includes instructions for implementing aspects of the functions / actions specified in one or more blocks of the flowchart and / or block diagram.

[0391] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to perform the functions / actions specified in one or more boxes of a flowchart and / or block diagram.

[0392] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of an instruction, which contains one or more executable instructions for implementing a specified logical function. In some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0393] Various embodiments of this disclosure have been described above. These descriptions are exemplary and not exhaustive, nor are they limited to the disclosed embodiments. Many modifications and variations will be apparent to those skilled in the art without departing from the scope and spirit of the disclosed embodiments. The terminology used herein is chosen to best explain the principles, practical applications, or improvements to the technology in the market, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for constructing a speech recognition model, comprising: Obtain target keywords; Obtain synonyms that are semantically related to the target keyword; A language model is trained using the target keywords and the synonyms to obtain the target language model; A first decoding graph is generated based on the target language model, and the first decoding graph indicates multiple decoding paths that satisfy the grammatical constraint rules determined based on the target keywords and the synonyms. as well as Based on the first decoded image, the speech recognition model is determined. The target keywords include at least a first keyword and a second keyword, and determining the speech recognition model based on the first decoding graph includes: A first set of decoding paths and a second set of decoding paths are obtained from the first decoding graph. The first set of decoding paths includes decoding paths corresponding to the first keyword and a first synonym group semantically associated with the first keyword. The second set of decoding paths includes decoding paths corresponding to the second keyword and a second synonym group semantically associated with the second keyword. A first sub-graph is generated based on the first set of decoding paths; A second subgraph is generated based on the second set of decoding paths; and The speech recognition model is determined based at least on the first subgraph and the second subgraph.

2. The method of claim 1, wherein determining the synonym group semantically associated with the target keyword comprises: Determine the semantics of the target keywords; as well as The synonym group is determined at least based on the semantics of the target keyword, wherein the semantic difference between each synonym in the synonym group and the semantics of the target keyword is less than a difference threshold.

3. The method according to claim 2, wherein determining the synonym group based at least on the semantics of the target keyword includes: Based on the semantics and length of the target keyword, the synonym group is determined, wherein the difference between the length of each synonym in the synonym group and the length of the target keyword is less than a length threshold.

4. The method according to claim 2, wherein determining the synonym group based at least on the semantics of the target keyword comprises: Based on the semantics of the target keywords, multiple candidate synonyms are obtained; Provide the user with the aforementioned multiple candidate synonyms; as well as Based on user input received from the user, the synonym group is determined from the plurality of candidate synonyms, wherein the user input indicates that at least one of the plurality of candidate synonyms is excluded or confirmed.

5. The method according to claim 1, wherein the first subgraph indicates a first decoding path and a second decoding path, the first decoding path being a decoding path corresponding to the first keyword, the second decoding path being a decoding path corresponding to a synonym in the first synonym group, and the first decoding path and each of the second decoding paths having the same weight in the first subgraph.

6. The method according to claim 1, wherein obtaining the target keyword includes: Obtain the first keyword group based on the pre-stored historical keywords and the received keywords; as well as In response to determining that the number of keywords in the first keyword group exceeds a predetermined threshold, the target keyword is obtained from the first keyword group based on the predetermined threshold.

7. The method according to claim 6, wherein obtaining the target keyword from the first keyword group based on the predetermined threshold comprises: The target keyword is obtained from the first keyword group based on the attributes of the keywords in the target keyword, and the number of target keywords is the predetermined threshold.

8. The method according to claim 1, further comprising: The instruction is to provide the speech recognition model to the target computing device for deployment on the target computing device.

9. A speech processing method, comprising: Receive voice input; as well as A speech recognition model is used to determine the text representation associated with the speech input, wherein the speech recognition model is obtained by the method according to any one of claims 1 to 8.

10. The method of claim 9, further comprising: Perform the action corresponding to the text representation.

11. The method of claim 9, wherein the text represents a synonym corresponding to the target keyword or the group of synonyms.

12. A speech model construction system, comprising: The keyword acquisition unit is used to acquire target keywords; The synonym acquisition unit is used to acquire synonym groups that are semantically related to the target keyword; The model training unit is used to train a language model using the target keywords and the synonyms to obtain the target language model. A decoding graph generation unit is used to generate a first decoding graph based on the target language model, wherein the first decoding graph indicates multiple decoding paths that satisfy the grammatical constraint rules determined based on the target keywords and the synonyms; as well as The model determination unit is used to determine the speech recognition model based on the first decoded image. The target keywords include at least a first keyword and a second keyword, and determining the speech recognition model based on the first decoding graph includes: A first set of decoding paths and a second set of decoding paths are obtained from the first decoding graph. The first set of decoding paths includes decoding paths corresponding to the first keyword and a first synonym group semantically associated with the first keyword. The second set of decoding paths includes decoding paths corresponding to the second keyword and a second synonym group semantically associated with the second keyword. A first sub-graph is generated based on the first set of decoding paths; A second subgraph is generated based on the second set of decoding paths; and The speech recognition model is determined based at least on the first subgraph and the second subgraph.

13. A speech processing system, comprising: A voice input unit is used to receive voice input; as well as A speech processing unit is used to determine the text representation associated with the speech input using a speech recognition model. The speech recognition model mentioned above is obtained based on the following process: Obtain target keywords; Obtain synonyms that are semantically related to the target keyword; A language model is trained using the target keywords and the synonyms to obtain the target language model; A first decoding graph is generated based on the target language model, and the first decoding graph indicates multiple decoding paths that satisfy the grammatical constraint rules determined based on the target keywords and the synonyms. as well as Based on the first decoded image, the speech recognition model is determined. The target keywords include at least a first keyword and a second keyword, and determining the speech recognition model based on the first decoding graph includes: A first set of decoding paths and a second set of decoding paths are obtained from the first decoding graph. The first set of decoding paths includes decoding paths corresponding to the first keyword and a first synonym group semantically associated with the first keyword. The second set of decoding paths includes decoding paths corresponding to the second keyword and a second synonym group semantically associated with the second keyword. A first sub-graph is generated based on the first set of decoding paths; A second subgraph is generated based on the second set of decoding paths; and The speech recognition model is determined based at least on the first subgraph and the second subgraph.

14. A method for constructing a speech recognition model, comprising: Obtain target language information; Obtain a group of synonymous phoneme sequences associated with the target language information, wherein the group of synonymous phoneme sequences includes at least one synonymous phoneme sequence, and the at least one synonymous phoneme sequence is a phoneme sequence corresponding to a word or phrase that is semantically similar to the target language information; A language model is trained using the synonymous phoneme sequence group to obtain a target language model; A first decoding graph is generated based on the target language model, and the first decoding graph indicates multiple decoding paths that satisfy the grammatical constraint rules determined based on the synonymous phoneme sequence group; as well as Based on the first decoded image, the speech recognition model is determined. The target language information includes at least first language information and second language information, wherein determining the speech recognition model based on the first decoding map includes: A first set of decoding paths and a second set of decoding paths are obtained from the first decoding diagram. The first set of decoding paths includes the decoding path of a first synonymous phoneme sequence group associated with the first language information, and the second set of decoding paths includes the decoding path of a second synonymous phoneme sequence group associated with the second language information. A first sub-graph is generated based on the first set of decoding paths; A second subgraph is generated based on the second set of decoding paths; and The speech recognition model is determined based at least on the first subgraph and the second subgraph.

15. The method of claim 14, wherein obtaining the group of synonymous phoneme sequences associated with the target language information comprises: Determine the semantics of the target language information; as well as The synonymous phoneme sequence group is determined at least based on the semantics of the target language information, wherein the difference between the semantics of each synonymous phoneme sequence in the synonymous phoneme sequence group and the semantics of the target language information is less than a difference threshold.

16. The method of claim 15, wherein determining the group of synonymous phoneme sequences based at least on the semantics of the target language information comprises: Determine the target phoneme sequence corresponding to the target language information; as well as Based on the semantics of the target language information and the length of the target phoneme sequence, a group of synonymous phoneme sequences is determined, wherein the difference between the length of each synonymous phoneme sequence in the group and the length of the target phoneme sequence is less than a length threshold.

17. The method of claim 16, wherein obtaining the group of synonymous phoneme sequences associated with the target language information comprises: Based on the semantics of the target keywords corresponding to the target language information, multiple candidate synonyms are obtained; Provide the user with the aforementioned multiple candidate synonyms; Based on user input received from the user, a group of synonyms is determined from the plurality of candidate synonyms, wherein the user input indicates that at least one of the plurality of candidate synonyms is excluded or confirmed. as well as Based on the pronunciation dictionary and the synonym group, the synonym phoneme sequence group is obtained.

18. The method of claim 14, wherein obtaining a group of synonymous phoneme sequences semantically associated with the target phoneme sequence comprises: Receive voice input from the user; as well as The synonymous phoneme sequence group is generated based on the speech input.

19. The method of claim 14, wherein the first subgraph indicates a first decoding path and a second decoding path, the first decoding path and the second decoding path being decoding paths in the first group of decoding paths, and the first decoding path and the second decoding path having the same weight in the first subgraph.

20. The method of claim 14, wherein obtaining the target language information comprises: The first language information group is obtained based on the pre-stored historical language information and the received language information; as well as In response to determining that the number of language information in the first language information group exceeds a predetermined threshold, the target language information is obtained from the first language information group based on the predetermined threshold.

21. The method according to claim 20, wherein obtaining the target language information from the first language information group based on the predetermined threshold comprises: The target language information is obtained from the first phoneme sequence group based on the attributes of the language information in the first language information group, and the number of the target language information is the predetermined threshold.

22. The method according to claim 14, wherein, The target language information includes speech or text.

23. The method according to claim 14, further comprising: The instruction is to provide the speech recognition model to the target computing device for deployment on the target computing device.

24. A speech processing method, comprising: Receive voice command input; A speech recognition model is used to obtain the phoneme sequence representation of the speech command input. The speech recognition model is configured to recognize the speech command input based on a group of phoneme sequences that are semantically synonymous with the command. as well as When the phoneme sequence representation corresponds to a phoneme sequence in the group of synonymous phoneme sequences, an instruction corresponding to the phoneme sequence representation is executed, wherein the speech recognition model is obtained by the method according to any one of claims 14 to 23.

25. A speech model construction system, comprising: The target language information acquisition unit is used to acquire target language information. The synonym phoneme sequence group acquisition unit is used to acquire a synonym phoneme sequence group associated with the target language information. The synonym phoneme sequence group includes at least one synonym phoneme sequence, and the at least one synonym phoneme sequence is a phoneme sequence corresponding to a word or phrase that is semantically similar to the target language information. The model training unit is used to train a language model using the synonymous phoneme sequence group to obtain the target language model. A decoding graph generation unit is used to generate a first decoding graph based on the target language model, wherein the first decoding graph indicates multiple decoding paths that satisfy the grammatical constraint rules determined based on the synonymous phoneme sequence group; as well as The model determination unit is used to determine the speech recognition model based on the first decoded image. The target language information includes at least first language information and second language information, and Determining the speech recognition model based on the first decoding map includes: A first set of decoding paths and a second set of decoding paths are obtained from the first decoding diagram. The first set of decoding paths includes the decoding path of a first synonymous phoneme sequence group associated with the first language information, and the second set of decoding paths includes the decoding path of a second synonymous phoneme sequence group associated with the second language information. A first sub-graph is generated based on the first set of decoding paths; A second subgraph is generated based on the second set of decoding paths; and The speech recognition model is determined based at least on the first subgraph and the second subgraph.

26. A speech processing system, comprising: A voice command input unit is used to receive voice command input; Voice processing unit, used for A speech recognition model is used to obtain the phoneme sequence representation of the speech command input. The speech recognition model is configured to recognize the speech command input based on a group of phoneme sequences that are semantically synonymous with the command. as well as When the phoneme sequence representation corresponds to a phoneme sequence in the phoneme sequence group, the instruction corresponding to the phoneme sequence representation is executed. The speech recognition model is obtained by the method according to any one of claims 14 to 23.

27. An electronic device comprising: At least one computing unit; At least one memory coupled to the at least one computing unit and storing instructions for execution by the at least one computing unit, the instructions, when executed by the at least one computing unit, causing the device to perform the method according to any one of claims 1 to 11 or claims 14 to 23.

28. A computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the method according to any one of claims 1 to 11 or claims 14 to 23.

29. A computer program product comprising computer-executable instructions, wherein the computer-executable instructions, when executed by a processor, implement the method according to any one of claims 1 to 11 or claims 14 to 23.

Images