Speech recognition method and device, electronic equipment and storage medium
By identifying the dialect type of audio and using the corresponding speech recognition model, the problem of low accuracy in dialect recognition of the voice interaction system is solved, and effective recognition and interaction functions of dialect audio are realized.
Patent Information
- Application Number
- CN202310627148.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-30
- Publication Date
- 2024-12-03
AI Technical Summary
The existing voice interaction system cannot effectively recognize and process local languages (dialects), resulting in low accuracy of speech recognition and inability to use the voice interaction function normally.
By determining the dialect type of audio to be recognized, using the corresponding speech recognition model for speech recognition, and generating dialect-specific response statements based on the recognition results, the speech recognition and interaction of dialect audio are realized.
It improves the accuracy of dialect vocabulary and speech recognition, enhances the accuracy of semantic analysis, and realizes the function of human-computer interaction through dialects.
Smart Images

Figure CN119068865A_ABST
Abstract
Description
Technical Field
[0001] The present disclosure relates to the field of vehicle technology, and in particular to a speech recognition method and device, an electronic device, and a storage medium. Background Art
[0002] With the development of science and technology, voice interaction systems are applied to all aspects of our lives. Voice interaction functions are constantly improving, and can realize functions such as chatting with users and controlling smart devices.
[0003] At present, the voice interaction system only supports voice interaction using standard voice. If the local language (dialect) is used for interaction, the voice recognition accuracy will be low and the voice interaction function cannot be used normally. Therefore, how to use dialects to complete the interaction with users has become an urgent problem to be solved. Summary of the invention
[0004] The present disclosure provides a speech recognition method, device, electronic device and storage medium, the main purpose of which is to solve the problem of being unable to perform speech interaction through dialects.
[0005] According to a first aspect of the present disclosure, a speech recognition method is provided, comprising:
[0006] Determining a first dialect type of the audio to be recognized;
[0007] Performing speech recognition according to the audio to be recognized and the first dialect type to obtain a speech recognition result;
[0008] Obtaining a second dialect type for generating a dialect reply sentence corresponding to the audio to be recognized;
[0009] Generate a dialect reply sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result.
[0010] Optionally, performing speech recognition according to the audio to be recognized and the first dialect type to obtain a speech recognition result includes:
[0011] Determine a speech recognition model corresponding to the first dialect type; wherein different dialect types correspond to different speech recognition models;
[0012] Based on the determined speech recognition model, speech recognition is performed on the audio to be recognized to obtain a speech recognition result.
[0013] Optionally, determining the first dialect type of the audio to be recognized includes:
[0014] Determining the dialect type input by the user as the first dialect type of the audio to be recognized;
[0015] Alternatively, the first dialect type of the audio to be recognized is determined by using speech features in the audio to be recognized.
[0016] Optionally, generating a dialect reply sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result includes:
[0017] Performing natural language understanding according to the second dialect type of the audio to be recognized and the speech recognition result to obtain a natural language understanding result;
[0018] According to the natural speech understanding result and the second dialect type of the audio to be recognized, a reply sentence based on the dialect type is generated.
[0019] Optionally, the obtaining and generating of the second dialect type of the dialect reply sentence corresponding to the audio to be recognized includes:
[0020] determining the first dialect type as a second dialect type;
[0021] Alternatively, a dialect type specified by the user other than the first dialect type is determined as the second dialect type.
[0022] Optionally, before determining the first dialect type of the audio to be recognized, the method further includes:
[0023] Acquire audio to be recognized, and determine a first position of a sound source of the audio to be recognized;
[0024] Optionally, the obtaining and generating a second dialect type of a dialect reply sentence corresponding to the speech to be recognized includes:
[0025] Collecting a second audio to be recognized;
[0026] If it is determined that the position corresponding to the second audio to be recognized is not the first position, determining the dialect type of the second audio to be recognized, and determining a second position of the sound source of the second audio to be recognized;
[0027] Binding the second position to the corresponding dialect type;
[0028] The dialect type corresponding to the second position is determined as the second dialect type.
[0029] Optionally, the step of obtaining and generating a second dialect type of a dialect reply sentence corresponding to the speech to be recognized further includes:
[0030] Determine the identity information of each passenger, and determine the corresponding dialect type according to each identity information; wherein the correspondence between the passenger identity information and the dialect type is obtained by pre-configuration;
[0031] The dialect type corresponding to the identity information is determined as the second dialect type.
[0032] Optionally, generating a dialect reply sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result includes:
[0033] Generate a first dialect reply sentence according to the first dialect type and the speech recognition result;
[0034] A second dialect reply sentence is generated according to the second dialect type and the speech recognition result.
[0035] Optionally, after generating a dialect reply sentence corresponding to the audio to be recognized according to the dialect type and the speech recognition result, the method further includes:
[0036] Playing the first dialect reply statement through the speaker associated with the first position; and,
[0037] The speaker associated with the second position plays the second dialect reply sentence corresponding to the dialect type bound to the second position.
[0038] According to a second aspect of the present disclosure, there is provided a speech recognition device, comprising:
[0039] A determination unit, configured to determine a first dialect type of the audio to be recognized;
[0040] A recognition unit, configured to perform speech recognition according to the audio to be recognized and the first dialect type to obtain a speech recognition result;
[0041] A first acquisition unit is used to acquire a second dialect type for generating a dialect reply sentence corresponding to the audio to be recognized;
[0042] A generating unit is used to generate a dialect reply sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result.
[0043] Optionally, the identification unit includes:
[0044] A first determination module, used to determine a speech recognition model corresponding to the first dialect type; wherein different dialect types correspond to different speech recognition models;
[0045] The recognition module is used to perform speech recognition on the audio to be recognized based on the determined speech recognition model to obtain a speech recognition result.
[0046] Optionally, the determining unit is further used to: determine the dialect type input by the user as the first dialect type of the audio to be recognized;
[0047] Alternatively, the first dialect type of the audio to be recognized is determined by using speech features in the audio to be recognized.
[0048] Optionally, the generating unit includes:
[0049] A comprehension module, configured to perform natural language comprehension according to the second dialect type of the audio to be recognized and the speech recognition result, and obtain a natural language comprehension result;
[0050] A generation module is used to generate a reply sentence based on the dialect type according to the natural speech understanding result and the second dialect type of the audio to be recognized.
[0051] Optionally, the first acquiring unit is further used for:
[0052] determining the first dialect type as a second dialect type;
[0053] Alternatively, a dialect type specified by the user other than the first dialect type is determined as the second dialect type.
[0054] Optionally, the device further comprises:
[0055] A second acquisition unit, configured to acquire the audio to be recognized and determine a first position of a sound source of the audio to be recognized before the determination unit determines the first dialect type of the audio to be recognized;
[0056] Optionally, the first acquiring unit includes:
[0057] A collection module, used for collecting the second audio to be recognized;
[0058] A second determination module is used to determine the dialect type of the second audio to be recognized and the second position of the sound source of the second audio to be recognized when it is determined that the position corresponding to the second audio to be recognized is not the first position;
[0059] A binding module, used to bind the second position to a corresponding dialect type;
[0060] The third determining module is used to determine the dialect type corresponding to the second position and the first dialect type as the second dialect type.
[0061] Optionally, the first acquiring unit further includes:
[0062] A fourth determination module is used to determine the identity information of each passenger, and determine the corresponding dialect type according to each identity information; wherein the correspondence between the passenger's identity information and the dialect type is obtained by pre-configuration;
[0063] A fifth determining unit is configured to determine the dialect type corresponding to the identity information and the first dialect type as a second dialect type.
[0064] Optionally, the generating unit is further used for:
[0065] Generate a first dialect reply sentence according to the first dialect type and the speech recognition result;
[0066] A second dialect reply sentence is generated according to the second dialect type and the speech recognition result.
[0067] Optionally, the device further comprises:
[0068] a playing unit, configured to play the first dialect reply sentence through the speaker associated with the first position after the generating unit generates the dialect reply sentence corresponding to the audio to be recognized according to the dialect type and the speech recognition result; and
[0069] The speaker associated with the second position plays the second dialect reply sentence corresponding to the dialect type bound to the second position.
[0070] According to a third aspect of the present disclosure, there is provided an electronic device, including:
[0071] at least one processor; and
[0072] a memory communicatively connected to the at least one processor; wherein,
[0073] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.
[0074] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute the method described in the first aspect.
[0075] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method as described in the first aspect above.
[0076] The speech recognition method, device, electronic device and storage medium provided by the present disclosure have the following main technical solutions: first, determining the first dialect type of the audio to be recognized; performing speech recognition according to the audio to be recognized and the first dialect type to obtain a speech recognition result; second, obtaining the second dialect type for generating a dialect response sentence corresponding to the audio to be recognized; finally, generating a dialect response sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result. Compared with the related art, the embodiment of the present application first determines the dialect type of the audio data before speech recognition, performs speech recognition in combination with the determined dialect type, and generates a dialect response sentence corresponding to the dialect type according to the dialect type; speech recognition of dialect audio is realized, and specific dialect response sentences can be generated according to the dialect type to interact with the user, thereby realizing human-computer interaction through dialect.
[0077] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present application, nor is it intended to limit the scope of the present application. Other features of the present application will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS
[0078] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.
[0079] Figure 1 A flowchart of a speech recognition method provided by an embodiment of the present disclosure;
[0080] Figure 2 A flowchart of another speech recognition method provided by an embodiment of the present disclosure;
[0081] Figure 3 A flowchart of another speech recognition method provided by an embodiment of the present disclosure;
[0082] Figure 4 A schematic diagram of the structure of a speech recognition device provided by an embodiment of the present disclosure;
[0083] Figure 5 A schematic diagram of the structure of another speech recognition device provided by an embodiment of the present disclosure;
[0084] Figure 6 A schematic block diagram of an exemplary electronic device provided for an embodiment of the present disclosure. DETAILED DESCRIPTION
[0085] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.
[0086] The following describes the speech recognition method, device, electronic device and storage medium according to the embodiments of the present disclosure with reference to the accompanying drawings.
[0087] Figure 1 A flowchart of a speech recognition method provided by an embodiment of the present disclosure.
[0088] like Figure 1 As shown, the method comprises the following steps:
[0089] Step 101, determining a first dialect type of audio to be recognized.
[0090] The audio to be recognized is collected by using a microphone, such as that collected by a preset microphone in a vehicle. When determining the dialect type of the audio to be recognized, it can be determined based on the dialect features in the audio to be recognized, such as pronunciation, special words, etc. Specifically, the embodiments of the present application do not limit this.
[0091] Step 102: Perform speech recognition based on the audio to be recognized and the first dialect type to obtain a speech recognition result.
[0092] When performing speech recognition on the audio to be recognized, processing the dialect type described in the audio to be recognized together can improve the recognition accuracy of dialect vocabulary, thereby improving the accuracy of speech recognition. When performing semantic recognition on the speech recognition results, semantic recognition is performed in combination with the dialect type to make the semantic analysis results more accurate.
[0093] Step 103, obtaining a second dialect type for generating a dialect response sentence corresponding to the audio to be recognized.
[0094] In one implementable method of an embodiment of the present application, when generating a dialect reply sentence based on the audio to be recognized, the reply sentence is generated based on the second dialect type and the speech recognition result of the audio to be recognized, and a reply sentence with the characteristics of the second dialect type can be generated, such as specific dialect words, different accents, etc. Specifically, the embodiment of the present application does not limit the second dialect type.
[0095] It should be noted that the second dialect type can be configured in advance by the user. The user can configure the second dialect type according to his or her own preferences, or set the second dialect type to be the same as the first dialect type; the embodiment of the present application does not limit the second dialect type.
[0096] Those skilled in the art can understand that the various numerical numbers such as first and second involved in the present disclosure are only used for the convenience of description and are not used to limit the scope of the embodiments of the present disclosure, nor to limit the order of precedence.
[0097] Step 104: Generate a dialect reply sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result.
[0098] According to the speech recognition result, a dialect reply sentence having dialect characteristics of the second dialect type is generated, such as a dialect characteristic sentence, dialect words, dialect voice, etc. Specifically, the embodiment of the present application does not limit this.
[0099] The speech recognition method provided by the present disclosure has a main technical solution including: first, determining a first dialect type of the audio to be recognized; performing speech recognition according to the audio to be recognized and the first dialect type to obtain a speech recognition result; second, obtaining a second dialect type for generating a dialect response sentence corresponding to the audio to be recognized; finally, generating a dialect response sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result. Compared with the related art, the embodiment of the present application first determines the dialect type of the audio data before speech recognition, performs speech recognition in combination with the determined dialect type, and generates a dialect response sentence corresponding to the dialect type according to the dialect type; speech recognition of dialect audio is realized, and a specific dialect response sentence can be generated according to the dialect type to interact with the user, thereby realizing human-computer interaction through dialect.
[0100] In one possible implementation of the embodiment of the present application, when determining the first dialect type of the dialect type to be recognized in step 101, there are two determination methods. The first is to determine the dialect type input by the user as the first dialect type of the audio to be recognized; the user can set the first dialect type in advance, and in subsequent speech recognition, directly perform speech recognition on the first dialect type and the audio to be recognized, without the need to determine the first dialect type of the audio to be recognized; the second is to determine the first dialect type of the audio to be recognized by the speech features in the audio to be recognized. Before performing speech recognition on the audio to be recognized, the dialect type is identified using the speech features of the audio to be recognized according to the acoustic model; for detailed recognition methods, please refer to any implementation method in the prior art, and the embodiment of the present application will not be described one by one here.
[0101] In one implementable manner of the embodiment of the present application, in step 103, when obtaining the second dialect type for generating the dialect reply sentence corresponding to the audio to be recognized, there are two determination methods. The first method is to determine the first dialect type as the second dialect type; that is, to interact with the user in the same dialect as the user. The second method is to determine a dialect type other than the first dialect type specified by the user as the second dialect type. The user can set a dialect as the second dialect type in advance, and when generating a reply sentence, interact with the user according to the second dialect type specified by the user; in one implementable manner of the embodiment of the present application, the user can also set the first dialect type and the second dialect type to Mandarin. Specifically, the embodiment of the present application does not limit the first dialect type and the second dialect type.
[0102] In one possible implementation of the embodiment of the present application, when performing speech recognition according to the audio to be recognized and the first dialect type in step 102, a corresponding speech recognition model can be determined according to the first dialect type, and then the first dialect type and the audio to be recognized are used to enhance the recognition of the dialect, thereby improving the recognition accuracy of the dialect; please refer to Figure 2 , Figure 2 A flow chart of a speech recognition method provided by an embodiment of the present disclosure includes:
[0103] Step 201, determine a speech recognition model corresponding to the first dialect type; wherein different dialect types correspond to different speech recognition models.
[0104] Before executing this step, first train the corresponding speech models of different dialect types: obtain a certain amount of dialect audio data of a dialect type, and use the dialect audio data of the same dialect type to train the speech model corresponding to the dialect type to obtain a trained speech model; in one implementable method of the embodiment of the present application, the dialect type can be divided into dialect lines in units of prefecture-level cities. It should be noted that the more dialect audio data there is, the more accurate the recognition result of the trained speech model when recognizing the dialect, and the embodiment of the present application does not limit the amount of dialect audio data used for training.
[0105] Each dialect type has a corresponding speech recognition model. For example, the speech recognition model corresponding to the Shanghai dialect is trained based on the Shanghai dialect. Therefore, the speech recognition results corresponding to the Shanghai dialect are more accurate. It should be noted that this narrative method is only an exemplary description and is not a limitation on specific dialect types.
[0106] In one implementable method of the embodiment of the present application, the above embodiment is to train corresponding speech models for different dialect types respectively; during training, the following method can also be used: all dialect types use the same speech model, and are trained respectively through the audio data of each dialect, and the dialect recognition parameters corresponding to each dialect type are obtained through training; in actual application, after determining the first dialect type, the dialect recognition parameters corresponding to the first dialect type are determined, and the determined dialect recognition parameters are used in the dialect model; in the embodiment of the present application, different dialect types correspond to different speech recognition models as an example for explanation, but this narrative method is only an exemplary explanation, and is not a specific limitation on the specific implementation method, and the embodiment of the present application does not limit this.
[0107] Step 202: Perform speech recognition on the audio to be recognized based on the determined speech recognition model to obtain a speech recognition result.
[0108] In one possible implementation of the embodiment of the present application, the speech recognition model obtains at least two speech recognition results for the audio to be recognized, scores each speech recognition, and outputs the speech recognition result with the highest score.
[0109] See also Figure 3 , Figure 3 A flow chart of a speech recognition method provided in an embodiment of the present disclosure, wherein generating a dialect reply sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result comprises:
[0110] Step 301, performing natural language understanding according to the second dialect type of the audio to be recognized and the speech recognition result to obtain a natural language understanding result.
[0111] In one possible implementation method of the embodiment of the present application, the same word has different meanings in different dialect environments. Therefore, when performing natural language understanding on the speech recognition results, natural language understanding is performed together with the speech results according to the second dialect type to improve the accuracy of natural language understanding. For the detailed process of natural language understanding, please refer to any implementation method in the prior art, and the embodiment of the present application will not be described one by one here.
[0112] Step 302: Generate a reply sentence based on the dialect type according to the natural speech understanding result and the second dialect type of the audio to be recognized.
[0113] In one possible implementation of the embodiment of the present application, after generating a dialect response sentence corresponding to the audio to be recognized according to the dialect type and the speech recognition result, interaction is performed according to the dialect response sentence.
[0114] In one possible implementation of the embodiment of the present application, the user can set the interactive sound in advance, such as timbre, male or female voice, voice age and speaking speed, etc. Specifically, the embodiment of the present application does not limit the interactive sound.
[0115] In one possible implementation of the embodiment of the present application, before determining the first dialect type of the audio to be recognized, the method further includes:
[0116] Acquire the audio to be identified and determine the first position of the sound source of the audio to be identified; in one implementable method of the embodiment of the present application, the audio to be identified is collected by a microphone in the vehicle. In actual applications, multiple microphones can be arranged at different positions of the vehicle, and the sound source of the audio to be identified is determined by the time difference in collecting the same audio by different microphones; in the embodiment of the present application, the sound source is distinguished by seat position, such as the main driver's seat, the co-driver's seat, etc.).
[0117] In one possible implementation method of the embodiment of the present application, when arranging microphones, exclusive microphones can be arranged for different seats respectively. When determining the sound source of the audio to be identified, the seat position corresponding to the sound source can be determined based on the collected microphones; the embodiment of the present application does not limit the arrangement position and number of microphones.
[0118] In one possible implementation of the embodiment of the present application, when obtaining the second dialect type for generating the dialect reply sentence corresponding to the audio to be recognized in step 103, the following steps may also be referred to:
[0119] Collect the second audio to be recognized; when it is confirmed that the position corresponding to the second audio to be recognized is not the first position, determine the dialect type of the second audio to be recognized, and determine the second position of the sound source of the second audio to be recognized; in actual applications, people in the car may use one dialect when communicating with each other, and use another dialect when interacting with the car computer, such as passenger 1 uses Mandarin and passenger 2 uses Cantonese; when a person in the car interacts with the car computer interaction system, he needs to use a preset sentence to wake up the car computer and send the first audio to be recognized; the second audio to be recognized here is audio data other than the first audio to be recognized, such as chat audio data between passengers, etc. When determining the dialect type of the second audio to be recognized, first group it according to the sound source of the second audio data to be recognized, determine the seat position corresponding to each audio data, and determine the dialect type of the second audio to be recognized corresponding to each sound source; the position information of the second position (such as the co-pilot seat, the left side of the second row, etc.) is determined according to the second audio to be recognized, and the embodiment of the present application does not limit the number of second positions. It should be noted that the user's chat audio data is only used to determine the dialect type locally in the vehicle.
[0120] Bind the second position to the corresponding dialect type; the second position is the second position corresponding to the sound source of the second audio to be identified. The second position may include only one seat position, or may include all seats except the seat corresponding to the first position. Specifically, in the embodiment of the present application, the number of seat positions included in the second position is not limited.
[0121] The dialect type corresponding to the second position is determined as the second dialect type.
[0122] Following the description of the above application embodiment, the second dialect is rationally used to refer to a type of dialect (i.e., a dialect type used to guide the generation of a reply sentence). The second dialect type may include multiple dialect types, and it is not limited to a certain dialect type as the second dialect type.
[0123] In one possible implementation of the embodiment of the present application, when determining the second dialect type, the following method may also be used: it also includes: determining the identity information of each passenger, and determining the corresponding dialect type according to each identity information; wherein the correspondence between the identity information of the passenger and the dialect type is obtained by pre-configuration; the dialect type corresponding to the identity information and the first dialect type are determined as the second dialect type. In the vehicle system, the facial information and dialect types of different passengers are configured in advance and saved in the vehicle system; in actual application, the facial information of the user is collected by the camera in the vehicle, and compared with the facial information saved in the vehicle system, the identity information of the passenger is determined, and then the pre-configured dialect type is determined.
[0124] In one possible implementation of the embodiment of the present application, when generating a dialect reply sentence, a first dialect reply sentence is generated according to the first dialect type and the speech recognition result; a second dialect reply sentence is generated according to the second dialect type and the speech recognition result. And according to the binding information of the dialect type and the position information, the first dialect reply sentence is played by the speaker associated with the first position; and the second dialect reply sentence corresponding to the dialect type bound to the second position is played by the speaker associated with the second position.
[0125] In one possible implementation of the embodiment of the present application, speakers associated with different seat configurations may be provided in the vehicle, for example, at the headrest of each seat or on the adjacent vehicle body. Specifically, the embodiment of the present application does not limit the arrangement position and number of the speakers.
[0126] Corresponding to the above-mentioned speech recognition method, the present invention also provides a speech recognition device. Since the device embodiment of the present invention corresponds to the above-mentioned method embodiment, details not disclosed in the device embodiment can be referred to the above-mentioned method embodiment, and will not be described in detail in the present invention.
[0127] Figure 4 A schematic diagram of the structure of a speech recognition device provided by an embodiment of the present disclosure is shown in FIG. Figure 4 As shown, including:
[0128] A determination unit 41, configured to determine a first dialect type of the audio to be recognized;
[0129] A recognition unit 42, configured to perform speech recognition according to the audio to be recognized and the first dialect type to obtain a speech recognition result;
[0130] A first acquisition unit 43 is used to acquire a second dialect type for generating a dialect reply sentence corresponding to the audio to be recognized;
[0131] The generating unit 44 is used to generate a dialect reply sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result.
[0132] The speech recognition device provided by the present disclosure has a main technical solution including: first, determining a first dialect type of audio to be recognized; performing speech recognition according to the audio to be recognized and the first dialect type to obtain a speech recognition result; second, obtaining a second dialect type for generating a dialect response sentence corresponding to the audio to be recognized; finally, generating a dialect response sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result. Compared with the related art, the embodiment of the present application first determines the dialect type of audio data before speech recognition, performs speech recognition in combination with the determined dialect type, and generates a dialect response sentence corresponding to the dialect type according to the dialect type; speech recognition of dialect audio is realized, and specific dialect response sentences can be generated according to the dialect type to interact with the user, thereby realizing human-computer interaction through dialect.
[0133] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the identification unit 42 includes:
[0134] A first determination module 421 is used to determine a speech recognition model corresponding to the first dialect type; wherein different dialect types correspond to different speech recognition models;
[0135] The recognition module 422 is used to perform speech recognition on the audio to be recognized based on the determined speech recognition model to obtain a speech recognition result.
[0136] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the determining unit 41 is further used to: determine the dialect type input by the user as the first dialect type of the audio to be recognized;
[0137] Alternatively, the first dialect type of the audio to be recognized is determined by using speech features in the audio to be recognized.
[0138] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the generating unit 44 includes:
[0139] The understanding module 441 is used to perform natural language understanding according to the second dialect type of the audio to be recognized and the speech recognition result to obtain a natural language understanding result;
[0140] The generation module 442 is used to generate a reply sentence based on the dialect type according to the natural speech understanding result and the second dialect type of the audio to be recognized.
[0141] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the first acquisition unit 43 is further used for:
[0142] determining the first dialect type as a second dialect type;
[0143] Alternatively, a dialect type specified by the user other than the first dialect type is determined as the second dialect type.
[0144] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the device also includes:
[0145] A second acquisition unit 45 is used to acquire the audio to be recognized and determine a first position of a sound source of the audio to be recognized before the determination unit 41 determines the first dialect type of the audio to be recognized;
[0146] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the first acquisition unit 43 includes:
[0147] The collection module 431 is used to collect the second audio to be recognized;
[0148] A second determination module 432 is used to determine the dialect type of the second audio to be recognized and the second position of the sound source of the second audio to be recognized when it is determined that the position corresponding to the second audio to be recognized is not the first position;
[0149] A binding module 433, used to bind the second position to a corresponding dialect type;
[0150] The third determining module 434 is configured to determine the dialect type corresponding to the second position as the second dialect type.
[0151] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the first acquisition unit 43 also includes:
[0152] The fourth determination module 435 is used to determine the identity information of each passenger, and determine the corresponding dialect type according to each identity information; wherein the correspondence between the passenger's identity information and the dialect type is obtained by pre-configuration;
[0153] The fifth determining unit 436 is configured to determine the dialect type corresponding to the identity information as the second dialect type.
[0154] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the generating unit 44 is further used for:
[0155] Generate a first dialect reply sentence according to the first dialect type and the speech recognition result;
[0156] A second dialect reply sentence is generated according to the second dialect type and the speech recognition result.
[0157] Furthermore, in a possible implementation of this embodiment, as Figure 5 As shown, the device also includes:
[0158] a playing unit 46, configured to play the first dialect reply sentence through a preset speaker associated with the first position after the generating unit 44 generates the dialect reply sentence corresponding to the audio to be recognized according to the dialect type and the speech recognition result; and
[0159] According to the preset speaker associated with the second position, the second dialect reply sentence corresponding to the dialect type bound to the second position is played.
[0160] It should be noted that the above explanation of the method embodiment is also applicable to the device of this embodiment, and the principle is the same, which is not limited in this embodiment.
[0161] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.
[0162] Figure 6A schematic block diagram of an example electronic device 500 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.
[0163] like Figure 6 As shown, the device 500 includes a computing unit 501, which can perform various appropriate actions and processes according to a computer program stored in a ROM (Read-Only Memory) 502 or a computer program loaded from a storage unit 508 to a RAM (Random Access Memory) 503. In the RAM 503, various programs and data required for the operation of the device 500 can also be stored. The computing unit 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An I / O (Input / Output) interface 505 is also connected to the bus 504.
[0164] A number of components in the device 500 are connected to the I / O interface 505, including: an input unit 506, such as a keyboard, a mouse, etc.; an output unit 507, such as various types of displays, speakers, etc.; a storage unit 508, such as a disk, an optical disk, etc.; and a communication unit 509, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 509 allows the device 500 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.
[0165] The computing unit 501 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 501 include, but are not limited to, CPU (Central Processing Unit), GPU (Graphic Processing Units), various dedicated AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSP (Digital Signal Processor), and any appropriate processor, controller, microcontroller, etc. The computing unit 501 performs the various methods and processes described above, such as a speech recognition method. For example, in some embodiments, the speech recognition method may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 508. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 500 via ROM 502 and / or communication unit 509. When the computer program is loaded into RAM 503 and executed by the computing unit 501, one or more steps of the method described above may be performed. Alternatively, in other embodiments, the computing unit 501 may be configured to execute the aforementioned speech recognition method in any other appropriate manner (for example, by means of firmware).
[0166] Various embodiments of the systems and techniques described above herein may be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application Specific Standard Products), SOCs (System On Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include: being implemented in one or more computer programs that may be executed and / or interpreted on a programmable system including at least one programmable processor that may be a dedicated or general-purpose programmable processor that may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.
[0167] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.
[0168] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any suitable combination of the foregoing. More specific examples of machine-readable storage media may include electrical connections based on one or more lines, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only-Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.
[0169] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball), through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).
[0170] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: LAN (Local Area Network), WAN (Wide Area Network), the Internet, and blockchain networks.
[0171] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship between the client and the server is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services ("Virtual Private Server", or "VPS" for short). The server may also be a server of a distributed system, or a server combined with a blockchain.
[0172] It should be noted that artificial intelligence is a discipline that studies how computers can simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, planning, etc.), and includes both hardware-level and software-level technologies. Artificial intelligence hardware technologies generally include technologies such as sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, and big data processing; artificial intelligence software technologies mainly include computer vision technology, speech recognition technology, natural language processing technology, as well as machine learning / deep learning, big data processing technology, knowledge graph technology, and other major directions.
[0173] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.
[0174] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.
Claims
1. A speech recognition method, It is characterized in that include: Determining a first dialect type of the audio to be recognized; Performing speech recognition according to the audio to be recognized and the first dialect type to obtain a speech recognition result; Obtaining a second dialect type for generating a dialect reply sentence corresponding to the audio to be recognized; Generate a dialect reply sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result.
2. The method according to claim 1, It is characterized in that The performing speech recognition according to the audio to be recognized and the first dialect type to obtain a speech recognition result includes: Determine a speech recognition model corresponding to the first dialect type; wherein different dialect types correspond to different speech recognition models; Based on the determined speech recognition model, speech recognition is performed on the audio to be recognized to obtain a speech recognition result.
3. The method according to claim 2, It is characterized in that Determining the first dialect type of the audio to be recognized includes: Determining the dialect type input by the user as the first dialect type of the audio to be recognized; Alternatively, the first dialect type of the audio to be recognized is determined by using speech features in the audio to be recognized.
4. The method according to claim 1, It is characterized in that The step of obtaining and generating the second dialect type of the dialect reply sentence corresponding to the audio to be recognized includes: determining the first dialect type as a second dialect type; Alternatively, a dialect type specified by the user other than the first dialect type is determined as the second dialect type.
5. The method according to any one of claims 1 to 3, It is characterized in that Before determining the first dialect type of the audio to be recognized, the method further includes: Acquire audio to be recognized, and determine a first position of a sound source of the audio to be recognized.
6. The method according to claim 5, It is characterized in that The step of obtaining and generating the second dialect type of the dialect reply sentence corresponding to the speech to be recognized includes: Collecting a second audio to be recognized; If it is determined that the position corresponding to the second audio to be recognized is not the first position, determining the dialect type of the second audio to be recognized, and determining a second position of the sound source of the second audio to be recognized; Binding the second position to the corresponding dialect type; The dialect type corresponding to the second position is determined as the second dialect type.
7. The method according to claim 5, It is characterized in that The step of obtaining and generating a second dialect type of a dialect reply sentence corresponding to the speech to be recognized further comprises: Determine the identity information of each passenger, and determine the corresponding dialect type according to each identity information; wherein the correspondence between the passenger identity information and the dialect type is obtained by pre-configuration; The dialect type corresponding to the identity information is determined as the second dialect type.
8. The method according to claim 6 or 7, It is characterized in that Generating a dialect reply sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result includes: Generate a first dialect reply sentence according to the first dialect type and the speech recognition result; A second dialect reply sentence is generated according to the second dialect type and the speech recognition result.
9. The method according to claim 8, It is characterized in that After generating a dialect reply sentence corresponding to the audio to be recognized according to the dialect type and the speech recognition result, the method further includes: Playing the first dialect reply statement through the speaker associated with the first position; and, The speaker associated with the second position plays the second dialect reply sentence corresponding to the dialect type bound to the second position.
10. A speech recognition device, It is characterized in that include: A determination unit, configured to determine a first dialect type of the audio to be recognized; A recognition unit, configured to perform speech recognition according to the audio to be recognized and the first dialect type to obtain a speech recognition result; A first acquisition unit is used to acquire a second dialect type for generating a dialect reply sentence corresponding to the audio to be recognized; A generating unit is used to generate a dialect reply sentence corresponding to the audio to be recognized according to the second dialect type and the speech recognition result.
11. An electronic device, It is characterized in that include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 9.
12. A non-transitory computer-readable storage medium storing computer instructions, It is characterized in that The computer instructions are used to cause the computer to execute the method according to any one of claims 1-9.
13. A computer program product, It is characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 9.
Citation Information
Patent Citations
Voice interaction method, voice interaction device and computer readable storage medium
CN115424606A
Voice processing method and device and electronic equipment
CN115798457A
Intelligent outbound method and device, equipment and storage medium
CN115811573A
Voice interaction apparatus and program
JP2004109323A
System and method for data intensive local inference
US20110313767A1
Cited By
Speech assistance system
WO2026223859A1
