Speech processing method, apparatus, and electronic device

Abstract

The present disclosure provides a speech processing method and device, electronic equipment and storage medium, and relates to the technical fields of artificial intelligence, Internet of Vehicles, intelligent cockpit and automatic driving. The method comprises the following steps: obtaining to-be-processed speech data; performing combined speech processing on the to-be-processed speech data based on offline speech processing operations and online speech processing operations to obtain candidate processing results corresponding to different combined modes; and determining a target processing result corresponding to the to-be-processed speech data according to the candidate processing results. By combining online speech processing operations and offline speech processing operations to construct multiple speech processing channels, the response time of speech processing is ensured, and after obtaining candidate processing results corresponding to multiple speech processing modes, the candidate processing results are screened to select candidate processing results with higher accuracy. In this way, the accuracy of the speech processing result is enhanced while the response time is ensured, and the user experience is enhanced.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, Internet of Vehicles, intelligent cockpit and automatic driving, and in particular to a voice processing method and device and electronic equipment. BACKGROUND

[0002] With the continuous development of artificial intelligence, voice interaction is applied in various human-computer interaction scenarios, such as implementing a control process of an electronic device based on a voice instruction. Among them, the user voice needs to be subjected to voice recognition and semantic analysis to obtain the content of the user voice and understand the command information in the user voice. In related technologies, the voice processing process including voice recognition and semantic analysis can be implemented in two ways, offline and online. Online voice recognition and online semantic analysis are easily affected by network status, and the response time of voice interaction function is greatly related to network fluctuations, while the accuracy of offline voice recognition and offline semantic analysis is unstable. Therefore, how to balance the response time and processing accuracy of voice processing and enhance user experience has become a problem to be solved. SUMMARY

[0003] A voice processing method, device and electronic equipment are provided.

[0004] According to a first aspect, a voice processing method is provided, including: obtaining to-be-processed voice data; performing combined voice processing on the to-be-processed voice data based on offline voice processing operations and online voice processing operations to obtain candidate processing results corresponding to different combination modes; and determining a target processing result corresponding to the to-be-processed voice data according to the candidate processing results.

[0005] According to a second aspect, a voice processing device is provided, including: an obtaining module configured to obtain to-be-processed voice data; a voice processing module configured to perform combined voice processing on the to-be-processed voice data based on offline voice processing operations and online voice processing operations to obtain candidate processing results corresponding to different combination modes; and a determining module configured to determine a target processing result corresponding to the to-be-processed voice data according to the candidate processing results.

[0006] According to a third aspect, an electronic device is provided, including: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the voice processing method of the first aspect of the present disclosure.

[0007] According to a fourth aspect, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the speech processing method according to the first aspect of the present disclosure.

[0008] According to a fifth aspect, there is provided a computer program product comprising a computer program which, when executed by a processor, implements the steps of the speech processing method according to the first aspect of the present disclosure.

[0009] It should be understood that the description in this section is not intended to identify key or essential features of embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become apparent from the following description. BRIEF DESCRIPTION OF DRAWINGS

[0010] The accompanying drawings are used to better understand the present scheme, and do not constitute limitations on the present disclosure. Among them:

[0011] Figure 1 is a flowchart of a speech processing method according to the first embodiment of the present disclosure;

[0012] Figure 2 is a flowchart of a speech processing method according to the second embodiment of the present disclosure;

[0013] Figure 3 is a flowchart of a speech processing method according to the second embodiment of the present disclosure;

[0014] Figure 4 is a flowchart of a speech processing method according to the third embodiment of the present disclosure;

[0015] Figure 5 is a flowchart of a speech processing method according to the third embodiment of the present disclosure;

[0016] Figure 6 is a flowchart of a speech processing method according to the third embodiment of the present disclosure;

[0017] Figure 7 is a flowchart of a speech processing method according to the third embodiment of the present disclosure;

[0018] Figure 8 is a flowchart of a speech processing method according to the third embodiment of the present disclosure; DETAILED DESCRIPTION

[0019] Exemplary embodiments of the present disclosure are described herein with reference to the accompanying drawings, which are included to provide a thorough understanding of embodiments of the present disclosure, and are taken along with various details in order to further assist others skilled in the art to better understand the technology, and should not be construed as limiting the present disclosure. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, in the following description, descriptions of well-known functions and constructions are omitted for clarity and conciseness.

[0020] Artificial Intelligence (AI) is a technical science that studies and develops a theory, method, technology and application system for simulating, extending and expanding human intelligence. At present, AI technology has the advantages of high automation, high accuracy and low cost, and has been widely applied.

[0021] Internet of Vehicles (IoV) is a technology that uses wireless communication technology to effectively utilize all vehicle dynamic information in the information network platform, and provides different functional services in vehicle operation. Internet of Vehicles realizes "three network integration" by using new generation information communication technology, that is, the integration of in-vehicle network, inter-vehicle network and vehicle mobile Internet. Internet of Vehicles is to use sensing technology to sense the state information of vehicles, and realize intelligent management of traffic, intelligent decision of traffic information service and intelligent control of vehicles by means of wireless communication network and modern intelligent information processing technology.

[0022] Intelligent cockpit can be understood as updating and transforming the vehicle cockpit into a digital platform. The traditional car cockpit can only be used to mark various driving conditions, while the intelligent cockpit realizes intelligentization in driving, leisure and entertainment, and vehicle control.

[0023] Autonomous driving is based on video cameras, radar sensors and laser range finders installed on vehicles to understand the surrounding traffic conditions, and based on route maps to navigate the road ahead to realize autonomous driving of vehicles.

[0024] The speech processing method, device and electronic equipment of the embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0025] Figure 1 is a flowchart of the speech processing method according to the first embodiment of the present disclosure.

[0026] As shown in Figure 1 , the speech processing method of the embodiments of the present disclosure can specifically include the following steps:

[0027] S101, obtaining the to-be-processed speech data.

[0028] Specifically, the execution subject of the voice processing method can be a voice processing apparatus provided by the embodiments of the present disclosure. The voice processing apparatus can be a hardware device with data information processing capability and / or necessary software for driving the hardware device to work. Alternatively, the execution subject can include a workstation, a server, a computer, a user terminal and other devices. The user terminal includes, but is not limited to, a mobile phone, a computer, a smart voice interaction device, a smart home appliance, a vehicle-mounted terminal and the like.

[0029] The embodiments of the present disclosure can collect a voice instruction such as "confirm restart" from a user through an audio collection device, and can take the voice of the user or audio such as cached recording data as the voice data to be processed in the embodiments of the present disclosure.

[0030] In S102, the voice data to be processed is subjected to combined voice processing based on the offline voice processing operation and the online voice processing operation, so as to obtain candidate processing results corresponding to different combined modes.

[0031] In the embodiments of the present disclosure,

[0032] The combined voice processing of the voice data to be processed can be understood as the offline and online combined voice processing of the voice data to be processed. Each combined mode corresponds to a candidate processing result. These candidate processing results can be used to implement specific functions of subsequent voice interaction. For example, in the application scenario of voice query, the candidate processing results can include the text determined after the semantic analysis of the voice of the user, which can be used to form an executable instruction for querying the target query content.

[0033] In S103, a target processing result corresponding to the voice data to be processed is determined according to the candidate processing result.

[0034] In the embodiments of the present disclosure, in order to ensure the accuracy of the voice processing result, the candidate processing results corresponding to multiple combined modes are screened to determine the target processing result corresponding to the voice data to be processed. In some embodiments, the target processing result can be distributed to a specific function module to perform a specific function.

[0035] In summary, the speech processing method of the embodiments of the present disclosure acquires the to-be-processed speech data, performs combined speech processing on the to-be-processed speech data based on the offline speech processing operation and the online speech processing operation to obtain candidate processing results corresponding to different combined manners, and determines a target processing result corresponding to the to-be-processed speech data according to the candidate processing results. The embodiments of the present disclosure construct multiple speech processing channels by combining the offline speech processing operation and the online speech processing operation to guarantee the response time of speech processing, and after obtaining the candidate processing results corresponding to multiple speech processing manners, the embodiments of the present disclosure select the candidate processing result with higher accuracy from the candidate processing results, thereby guaranteeing the response time and enhancing the accuracy of the speech processing result and the user experience.

[0036] In some embodiments, the candidate processing result obtained by the embodiments of the present disclosure includes at least one of the following: an online candidate processing result obtained based on the online speech processing operation, an offline candidate processing result obtained based on the offline speech processing operation, and an offline-online fusion candidate processing result obtained based on the offline speech processing operation and the online speech processing operation.

[0037] In the embodiments of the present disclosure, the speech processing operation includes at least one of speech recognition and semantic analysis, for example, the offline speech processing operation can include offline speech recognition and offline semantic analysis, such as speech recognition by using an offline speech recognition engine and semantic analysis by using an offline semantic analysis engine; the online speech processing operation can include online speech recognition and online semantic analysis, such as online speech recognition by using an online speech recognition engine and semantic analysis by using an online semantic analysis engine or a remote semantic processing server.

[0038] The combined speech processing process described above can include a processing process in which online speech recognition and online semantic analysis are combined, or a processing process in which online speech recognition and offline semantic analysis are combined, or a processing process in which offline speech recognition and offline semantic analysis are combined, or a processing process in which offline speech recognition and online semantic analysis are combined.

[0039] Figure 2 FIG. 2 is a flowchart of a speech processing method according to a second embodiment of the present disclosure.

[0040] As shown in FIG. 2, the speech processing method according to the second embodiment of the present disclosure can include the following steps: Figure 2 As shown in FIG. 2, the speech processing method according to the second embodiment of the present disclosure can include the following steps: Figure 1 The speech processing method according to the embodiments of the present disclosure can specifically include the following steps:

[0041] S201, acquiring to-be-processed speech data.

[0042] S202, performing online speech recognition and offline speech recognition on the to-be-processed speech data respectively to obtain online speech recognition results and offline speech recognition results corresponding to the to-be-processed speech data.

[0043] In the embodiments of the present disclosure, the obtained to-be-processed speech data is subjected to online speech recognition to obtain an online speech recognition result and offline speech recognition to obtain an offline speech recognition result.

[0044] In the embodiments of the present disclosure, the obtained online speech recognition result is subjected to online semantic analysis to generate an online semantic analysis result. The online semantic analysis result is taken as an online candidate processing result corresponding to the first combination mode (online speech recognition and online semantic analysis combination).

[0045] In the embodiments of the present disclosure, the obtained online speech recognition result is subjected to online semantic analysis to generate an online semantic analysis result. The online semantic analysis result is taken as an online candidate processing result corresponding to the first combination mode (online speech recognition and online semantic analysis combination).

[0046] For example, after obtaining the online speech recognition result, the online speech recognition result can be provided to a semantic processing server, and a semantic analysis request is sent through a network to obtain an online semantic analysis result.

[0047] For another example, the process of subjecting the online speech recognition result to online semantic analysis is implemented by calling an online semantic analysis engine.

[0048] In the embodiments of the present disclosure, the obtained offline speech recognition result is subjected to offline semantic analysis to generate an offline semantic analysis result. The offline semantic analysis result is taken as an offline candidate processing result corresponding to the second combination mode (offline speech recognition and offline semantic analysis combination).

[0049] In the embodiments of the present disclosure, the obtained offline speech recognition result is subjected to offline semantic analysis to generate an offline semantic analysis result. The offline semantic analysis result is taken as an offline candidate processing result corresponding to the second combination mode (offline speech recognition and offline semantic analysis combination).

[0050] For example, the process of subjecting the offline speech recognition result to offline semantic analysis is implemented by calling an offline semantic analysis engine.

[0051] In the embodiments of the present disclosure, the obtained online speech recognition result is subjected to offline semantic analysis to generate an offline semantic analysis result. The offline semantic analysis result is taken as an online candidate processing result corresponding to the first combination mode (online speech recognition and online semantic analysis combination).

[0052] In the embodiments of the present disclosure, after obtaining the online speech recognition result, the online speech recognition result is subjected to online semantic analysis and offline semantic analysis at the same time to generate an offline semantic analysis result of subjecting the online speech recognition result to offline semantic analysis. The offline semantic analysis result of subjecting the online speech recognition result to offline semantic analysis is taken as an offline candidate processing result corresponding to the third combination mode (online speech recognition and offline semantic analysis combination).

[0053] For example, after obtaining the online speech recognition result, the online speech recognition result is provided to a local semantic analysis engine for semantic analysis, thereby realizing the process of performing offline semantic analysis on the online speech recognition result.

[0054] In S206, the target processing result corresponding to the to-be-processed speech data is determined according to the candidate processing result.

[0055] Specifically, S201 is the same as S101, and S206 is the same as S103, which will not be described herein.

[0056] Based on the above embodiment, the process of determining the target processing result corresponding to the to-be-processed speech data according to the candidate processing result in S206 can include the following steps:

[0057] The target processing result is determined from the candidate processing result based on the priority level of the candidate processing result, wherein the priority level of the online candidate processing result is the highest priority level, and the priority level of the offline candidate processing result is the lowest priority level.

[0058] The process of determining the target processing result from the candidate processing result based on the priority level of the candidate processing result can include the following process:

[0059] Since the process of online speech recognition or online semantic analysis is limited by the network state, the response time fluctuates greatly, and the offline semantic analysis result can be implemented locally and is not affected by the network state, therefore, the time of obtaining the candidate processing result corresponding to the above various combination modes will be different.

[0060] In the embodiments of the present disclosure, whether there is an offline candidate processing result in one or more candidate processing results obtained by timing detection can be detected by setting a detection time; if there is an offline candidate processing result, whether there is an online candidate processing result in the candidate processing result newly returned in the next preset time interval can be detected in the time interval; if there is an online candidate processing result, the online candidate processing result is determined as the target processing result; if there is no online candidate processing result, whether there is an offline-online fusion candidate processing result is detected; if there is an offline-online fusion candidate processing result, the offline-online fusion candidate processing result is determined as the target processing result; if there is no offline-online fusion candidate processing result, the offline candidate processing result is determined as the target processing result.

[0061] Furthermore, when checking for offline candidate processing results at a set detection time, if no offline candidate processing results are found, the system continues to check for offline-online fusion candidate processing results. If offline-online fusion candidate processing results are found, the system continues to check for online candidate processing results among the newly returned candidate processing results within the next preset time interval. If online candidate processing results are found, they are determined as the target processing result. If no online candidate processing results are found, offline-online fusion candidate processing results are determined as the target processing result. If no offline-online fusion candidate processing results are found, an extended waiting mechanism is initiated to wait for online candidate processing results and determine them as the target processing result.

[0062] For example, such as Figure 3 As shown, the decision center is used to receive the offline semantic parsing results and / or the offline-online fusion semantic parsing results returned by the offline semantic parsing component, as well as the online semantic parsing results returned by the online semantic parsing component, and to detect and judge these candidate processing results obtained by the decision center: here, the offline semantic parsing results will be represented by r1, the online semantic parsing results by r2, and the offline-online fusion semantic parsing results by r3. If r1 exists, a 500ms delay is initiated, and during the delay, r2 is checked for. If r2 exists, it is determined as the target processing result. If r2 does not exist, r3 is checked for. If r3 exists, it is determined as the target processing result. If r3 does not exist, r1 is determined as the target processing result. If r1 does not exist in the initial check, r3 is checked for. If r3 exists, a 500ms delay is initiated, and during the delay, r2 is checked for. If r2 exists, it is determined as the target processing result. If r2 does not exist, r3 is determined as the target processing result. If r3 does not exist, an ultra-long waiting mechanism is initiated to wait for r2, and r2 is determined as the target processing result.

[0063] Therefore, within the set response time range (such as the sum of the preset time interval and the detection time of the semantic parsing result), all candidate processing results are screened, and the candidate processing results with higher accuracy or priority are selected.

[0064] Furthermore, offline speech recognition is easily affected by speech generalization, resulting in low recognition accuracy. If the offline speech recognition results are directly used for semantic parsing, parsing errors are likely to occur, and the accuracy of the obtained semantic parsing results is not high. Therefore, the speech processing method disclosed in this paper also provides a process for determining whether the offline speech recognition results are usable.

[0065] like Figure 4 As shown, the following steps may be included:

[0066] S401, calculate the confidence of the offline speech recognition result.

[0067] In the embodiments of the present disclosure, the confidence of the offline speech recognition result can be determined according to the matching degree of the characters and / or syllables between the offline speech recognition result and the expected recognition result or the pre-stored speech recognition result. For example, the offline speech recognition result corresponding to the user voice is matched with the recognition result corresponding to the executable voice instruction stored locally in terms of characters and / or syllables. For example, the user voice "sunshade" can be recognized as "such year" or "this sunshade" in the offline speech recognition result, and the matching degree of the offline speech recognition result and "sunshade" in the local speech recognition library can be determined. In some embodiments, the recognition result participating in the matching stored locally can be pre-processed by fuzzy matching and generalization.

[0068] S402, obtain the character length of the offline speech recognition result.

[0069] In the embodiments of the present disclosure, the character length of the text of the offline speech recognition result is obtained, and the speech recognition result which is difficult to obtain useful information through semantic analysis is filtered out according to the character length.

[0070] S403, in response to the confidence being greater than or equal to the confidence threshold and the character length being greater than or equal to the length threshold, the offline speech recognition result is retained to perform the step of performing offline semantic analysis on the offline speech recognition result.

[0071] In the embodiments of the present disclosure, it is determined whether the above-mentioned confidence is greater than or equal to the preset confidence threshold, and it is determined whether the above-mentioned character length is greater than or equal to the preset length threshold. If the confidence is greater than or equal to the confidence threshold and the character length is greater than or equal to the length threshold, it is considered that the offline speech recognition result is available, and the offline speech recognition result is retained to perform offline semantic analysis on the offline speech recognition result.

[0072] The confidence threshold and the length threshold of the character length can be set as needed, and the present disclosure does not make any limitation. In some embodiments, the length threshold can be set to 2, so as to filter out the single-character speech recognition result which is difficult to analyze useful information.

[0073] S404, in response to the confidence being less than the confidence threshold or the character length being less than the length threshold, the offline speech recognition result is discarded.

[0074] Therefore, after obtaining the offline speech recognition result, the confidence of the offline speech recognition result is calculated and the character length of the offline speech recognition result is calculated, and it is determined whether the offline speech recognition result is available. If the offline speech recognition result is not available, the semantic analysis process is not performed, so as to guarantee the accuracy of the final candidate processing result.

[0075] In some embodiments, the judgment on whether the online speech resolution result is available based on the confidence and the character length can further guarantee the accuracy of the candidate processing result.

[0076] In some embodiments, the confidence of the candidate processing result, i.e., the semantic resolution result corresponding to different combination manners, can be calculated to detect whether the semantic resolution result is available, where the confidence of the semantic resolution result can be calculated according to the support degree of each vertical category to the instruction corresponding to the semantic resolution result.

[0077] In summary, the speech processing method of the embodiments of the present disclosure includes: obtaining to-be-processed speech data; performing combined speech processing on the to-be-processed speech data based on offline speech processing operations and online speech processing operations to obtain candidate processing results corresponding to different combination manners; and determining a target processing result corresponding to the to-be-processed speech data according to the candidate processing results. The embodiments of the present disclosure construct multiple speech processing channels by combining online speech processing operations and offline speech processing operations to guarantee the response time of speech processing, and after obtaining candidate processing results corresponding to multiple speech processing manners, the candidate processing results are screened to select a candidate processing result with higher accuracy within the response time, thereby guaranteeing the response time and enhancing the accuracy of the speech processing result to provide smoother and faster speech interaction experience and enhance user experience.

[0078] To describe the speech processing method of the embodiments of the present disclosure in detail, the following will be described in detail in combination with Figure 5 Figure 5 is an exemplary overall flowchart of the speech processing method according to the embodiments of the present disclosure.

[0079] S501, obtaining to-be-processed speech data. Steps S502 and S508 are executed.

[0080] S502, performing offline speech recognition on the to-be-processed speech data to obtain an offline speech recognition result.

[0081] S503, calculating the confidence of the offline speech recognition result.

[0082] S504, obtaining the character length of the offline speech recognition result.

[0083] S505, judging whether the confidence and the character length satisfy the following condition: the confidence is greater than or equal to a confidence threshold and the character length is greater than or equal to a length threshold; if yes, step S506 is executed; if no, step S507 is executed.

[0084] ​S506, the offline speech recognition result is retained, and offline semantic analysis is performed on the offline speech recognition result to obtain an offline semantic analysis result as an offline candidate processing result corresponding to the second combination mode. Step S511 is executed.

[0085] S507, the offline speech recognition result is discarded.

[0086] S508, online speech recognition is performed on the to-be-processed speech data to obtain an online speech recognition result. Steps S509 and S510 are executed.

[0087] S509, online semantic analysis is performed on the online speech recognition result to obtain an online semantic analysis result as an online candidate processing result corresponding to the first combination mode. Step S511 is executed.

[0088] S510, offline semantic analysis is performed on the online speech recognition result to obtain an offline-online fusion semantic analysis result as an offline-online fusion candidate processing result corresponding to the third combination mode. Step S511 is executed.

[0089] S511, based on the priority levels of the candidate processing results, a target processing result is determined from the candidate processing results, wherein the priority level of the online candidate processing result is the highest priority level, and the priority level of the offline candidate processing result is the lowest priority level.

[0090] Figure 6 is a block diagram of a speech processing apparatus according to the first embodiment of the present disclosure.

[0091] As shown in Figure 6 , the speech processing apparatus 600 of the embodiment of the present disclosure comprises an acquisition module 601, a speech processing module 602 and a determination module 603.

[0092] The acquisition module 601 is configured to acquire to-be-processed speech data.

[0093] The speech processing module 602 is configured to perform combined speech processing on the to-be-processed speech data based on offline speech processing operations and online speech processing operations to obtain candidate processing results corresponding to different combination modes.

[0094] The determination module 603 is configured to determine a target processing result corresponding to the to-be-processed speech data according to the candidate processing results.

[0095] It should be noted that the above explanation and description of the speech processing method embodiment also applies to the speech processing apparatus of the embodiment of the present disclosure, and the specific process will not be described here.

[0096] In summary, the speech processing apparatus of the embodiments of the present disclosure acquires speech data to be processed; performs combined speech processing on the speech data to be processed based on offline speech processing operations and online speech processing operations to obtain candidate processing results corresponding to different combined manners; and determines a target processing result corresponding to the speech data to be processed according to the candidate processing results. The embodiments of the present disclosure construct multiple speech processing channels by combining offline speech processing operations and online speech processing operations to ensure the response time of speech processing, and after obtaining candidate processing results corresponding to multiple speech processing manners, the embodiments of the present disclosure screen the candidate processing results without exceeding the response time, and select a candidate processing result with higher accuracy, thereby ensuring the response time and enhancing the accuracy of the speech processing result and the user experience.

[0097] Figure 7 is a block diagram of a speech processing apparatus according to a second embodiment of the present disclosure.

[0098] As shown in Figure 7 the speech processing apparatus 700 of the embodiments of the present disclosure includes an acquisition module 701, a speech processing module 702, and a determination module 703.

[0099] The acquisition module 701 has the same structure and function as the acquisition module 601 in the previous embodiment, the speech processing module 702 has the same structure and function as the speech processing module 602 in the previous embodiment, and the determination module 703 has the same structure and function as the determination module 603 in the previous embodiment.

[0100] Further, the candidate processing result includes at least one of an online candidate processing result obtained based on the online speech processing operation, an offline candidate processing result obtained based on the offline speech processing operation, and an offline-online fusion candidate processing result obtained based on the offline speech processing operation and the online speech processing operation.

[0101] Further, the voice processing operation comprises at least one of voice recognition and semantic analysis, and the voice processing module 702 comprises: a voice recognition unit 7021, configured to perform online voice recognition and offline voice recognition on the to-be-processed voice data respectively to obtain online voice recognition results and offline voice recognition results corresponding to the to-be-processed voice data; a first semantic analysis unit 7022, configured to perform online semantic analysis on the online voice recognition results to obtain online semantic analysis results as the online candidate processing results corresponding to a first combination manner of the different combination manners; a second semantic analysis unit 7023, configured to perform offline semantic analysis on the offline voice recognition results to obtain offline semantic analysis results as the offline candidate processing results corresponding to a second combination manner of the different combination manners; and a third semantic analysis unit 7024, configured to perform offline semantic analysis on the online voice recognition results to obtain offline semantic analysis results as the offline-online fusion candidate processing results corresponding to a third combination manner of the different combination manners.

[0102] Further, the determination module 703 comprises a determination unit configured to determine the target processing result from the candidate processing results based on priority levels of the candidate processing results, wherein the priority level of the online candidate processing result is the highest priority level, and the priority level of the offline candidate processing result is the lowest priority level.

[0103] Further, the determination unit comprises: a first detection sub-unit configured to detect whether the offline candidate processing result exists; a second detection sub-unit configured to, in response to the existence of the offline candidate processing result, continue to detect whether the online candidate processing result exists within a preset time interval; a first determination sub-unit configured to, in response to the existence of the online candidate processing result, determine the online candidate processing result as the target processing result; a third detection sub-unit configured to, in response to the non-existence of the online candidate processing result, detect whether the offline-online fusion candidate processing result exists; a second determination sub-unit configured to, in response to the existence of the offline-online fusion candidate processing result, determine the offline-online fusion candidate processing result as the target processing result; and a third determination sub-unit configured to, in response to the non-existence of the offline-online fusion candidate processing result, determine the offline candidate processing result as the target processing result.

[0104] Further, the determining unit further comprises: a fourth detecting subunit, configured to, in response to the absence of the offline candidate processing result, detect whether the online-infusion candidate processing result exists; a fifth detecting subunit, configured to, in response to the existence of the online-infusion candidate processing result, continue to detect whether the online candidate processing result exists within the preset time interval; a fourth determining subunit, configured to, in response to the existence of the online candidate processing result, determine the online candidate processing result as the target processing result; a fifth determining subunit, configured to, in response to the absence of the online candidate processing result, determine the online-infusion candidate processing result as the target processing result; and a sixth determining subunit, configured to, in response to the absence of the online-infusion candidate processing result, wait for the online candidate processing result and determine the online candidate processing result as the target processing result.

[0105] Further, the speech processing module 702 further comprises: a calculating unit configured to calculate a confidence of the offline speech recognition result; an obtaining unit configured to obtain a character length of the offline speech recognition result; a first comparing unit configured to, in response to the confidence being greater than or equal to a confidence threshold and the character length being greater than or equal to a length threshold, retain the offline speech recognition result to perform the step of performing offline semantic analysis on the offline speech recognition result; and a second comparing unit configured to, in response to the confidence being less than the confidence threshold or the character length being less than the length threshold, discard the offline speech recognition result.

[0106] To sum up, the speech processing apparatus provided in the embodiments of the present disclosure acquires the to-be-processed speech data, combines the to-be-processed speech data based on the offline speech processing operation and the online speech processing operation to obtain candidate processing results corresponding to different combination manners, and determines a target processing result corresponding to the to-be-processed speech data according to the candidate processing results. The embodiments of the present disclosure construct multiple speech processing channels by combining the online speech processing operation and the offline speech processing operation to guarantee the response time of speech processing, select the candidate processing result with higher accuracy from the candidate processing results obtained by multiple speech processing manners in the case of not exceeding the response time, and enhance the accuracy of the speech processing result while guaranteeing the response time, so as to provide smoother and faster speech interaction experience and enhance user experience.

[0107] In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision and disclosure of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0108] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0109] According to the embodiments of the present disclosure, the present disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0109] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0110] like Figure 8 As shown, the electronic device 800 includes a computing unit 801, which can perform various appropriate actions and processes based on a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. The RAM 803 may also store various programs and data required for the operation of the electronic device 800. The computing unit 801, ROM 802, and RAM 803 are interconnected via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0111] Multiple components in electronic device 800 are connected to I / O interface 805, including: input unit 806, such as keyboard, mouse, etc.; output unit 807, such as various types of displays, speakers, etc.; storage unit 808, such as disk, optical disk, etc.; and communication unit 809, such as network card, modem, wireless transceiver, etc. Communication unit 809 allows electronic device 800 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0112] The computing unit 801 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 801 performs the various methods and processes described above, such as... Figures 1 to 5The illustrated speech processing method. For example, in some embodiments, the speech processing method can be implemented as a computer software program tangibly embodied in a machine readable medium, such as storage unit 808. In some embodiments, portions or all of the computer program can be loaded and / or installed onto electronic device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded onto RAM 803 and executed by the computing unit 801, one or more steps of the semantic parsing method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the speech processing method by any other suitable means, such as by means of firmware.

[0113] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0114] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0115] In the context of this disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0116] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0117] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0118] The computer system can include clients and servers. This relationship can be. The servers are generally remote from the users and can be accessed via the Internet using a communication network. The relationship can be a client-server relationship over a communications network, and as such, the servers can be accessed by the clients using computer programs. The servers can be cloud servers, also known as cloud computing servers or cloud hosts, which are mainframe products in the cloud computing service system, and solve the defects of large management difficulty and weak business scalability in traditional physical hosts and VPS services (Virtual Private Server, or VPS for short). The servers can also be servers of a distributed system, or servers combined with a blockchain.

[0119] According to an embodiment of the present disclosure, the present disclosure also provides a computer program product comprising a computer program which, when executed by a processor, implements the steps of the voice processing method according to the above-mentioned embodiments of the present disclosure.

[0120] It should be understood that the steps shown in the above forms can be reordered, added or deleted. For example, the steps described in the present disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, and the present disclosure is not limited herein.

[0121] The above detailed description does not constitute a limitation on the protection scope of the present disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements and improvements within the spirit and principles of the present disclosure should be included in the protection scope of the present disclosure.

Claims

1. A speech processing method, comprising: obtaining to-be-processed speech data; performing combined speech processing on the to-be-processed speech data based on an offline speech processing operation and an online speech processing operation to obtain candidate processing results corresponding to different combined manners, wherein the candidate processing results include at least two of an online candidate processing result obtained based on the online speech processing operation, an offline candidate processing result obtained based on the offline speech processing operation, and an offline-online fusion candidate processing result obtained based on the offline speech processing operation and the online speech processing operation; and determining a target processing result corresponding to the to-be-processed speech data according to the candidate processing results; wherein the speech processing operation includes at least one of speech recognition and semantic analysis, and the performing combined speech processing on the to-be-processed speech data based on the offline speech processing operation and the online speech processing operation to obtain candidate processing results corresponding to different combined manners includes: performing online speech recognition and offline speech recognition on the to-be-processed speech data to obtain online speech recognition results and offline speech recognition results corresponding to the to-be-processed speech data; and performing offline semantic analysis on the online speech recognition results to obtain an offline-online fusion semantic analysis result as the offline-online fusion candidate processing result corresponding to a third combined manner of the different combined manners, wherein after the online speech recognition results are obtained, the offline semantic analysis is performed on the online speech recognition results while the online semantic analysis is performed on the online speech recognition results to generate the offline-online fusion semantic analysis result of the offline semantic analysis on the online speech recognition results, and the offline-online fusion semantic analysis result is taken as the third combined manner; and the determining a target processing result corresponding to the to-be-processed speech data according to the candidate processing results includes: determining the target processing result from the candidate processing results based on a priority level of the candidate processing results; wherein the determining the target processing result from the candidate processing results based on the priority level of the candidate processing results includes: detecting whether the offline candidate processing result exists; in response to the offline candidate processing result not existing, detecting whether the offline-online fusion candidate processing result exists; in response to the offline-online fusion candidate processing result existing, continuing to detect whether the online candidate processing result exists within a preset time interval; in response to the online candidate processing result existing, determining the online candidate processing result as the target processing result; in response to the online candidate processing result not existing, determining the offline-online fusion candidate processing result as the target processing result; in response to the offline-online fusion candidate processing result not existing, waiting for the online candidate processing result and determining the online candidate processing result as the target processing result.

2. The speech processing method of claim 1, wherein the combining speech processing of the to-be-processed speech data based on the offline speech processing operation and the online speech processing operation to obtain candidate processing results corresponding to different combination manners comprises: performing online semantic analysis on the online speech recognition result to obtain an online semantic analysis result as the online candidate processing result corresponding to a first combination manner of the different combination manners; and performing offline semantic analysis on the offline speech recognition result to obtain an offline semantic analysis result as the offline candidate processing result corresponding to a second combination manner of the different combination manners. The priority level of the online candidate processing result is the highest priority level, and the priority level of the offline candidate processing result is the lowest priority level. The determining of the target processing result from the candidate processing results based on the priority levels of the candidate processing results further comprises:

3. The voice processing method of claim 1, wherein, in response to the existence of the online candidate processing result, continuing to detect whether the online candidate processing result exists within a preset time interval; 4. The voice processing method of claim 3, wherein, in response to the existence of the online candidate processing result, determining the online candidate processing result as the target processing result; in response to the non-existence of the online candidate processing result, detecting whether the offline candidate processing result exists; in response to the existence of the offline candidate processing result, determining the offline candidate processing result as the target processing result; in response to the non-existence of the offline candidate processing result, determining the offline candidate processing result as the target processing result.

5. The speech processing method of claim 2, further comprising: calculating a confidence degree of the offline speech recognition result; obtaining a character length of the offline speech recognition result; in response to the confidence degree being greater than or equal to a confidence degree threshold and the character length being greater than or equal to a length threshold, retaining the offline speech recognition result to perform the step of performing offline semantic analysis on the offline speech recognition result; in response to the confidence degree being less than the confidence degree threshold or the character length being less than the length threshold, discarding the offline speech recognition result.

6. A speech processing apparatus, comprising: an obtaining module configured to obtain to-be-processed speech data; a speech processing module configured to combine speech processing of the to-be-processed speech data based on an offline speech processing operation and an online speech processing operation to obtain candidate processing results corresponding to different combination manners, the candidate processing results comprising at least two of an online candidate processing result based on the online speech processing operation, an offline candidate processing result based on the offline speech processing operation, and an offline-online fusion candidate processing result based on the offline speech processing operation and the online speech processing operation; a determining module configured to determine a target processing result corresponding to the to-be-processed speech data according to the candidate processing results; wherein the speech processing operation comprises at least one of speech recognition and semantic analysis, and the speech processing module comprises: ​ ​ The voice recognition unit is configured to perform online voice recognition and offline voice recognition on the voice data to be processed respectively to obtain online voice recognition results and offline voice recognition results corresponding to the voice data to be processed. The third semantic analysis unit is configured to perform offline semantic analysis on the online voice recognition results to obtain offline-online fusion semantic analysis results as the candidate processing results corresponding to the third combination manner in the different combination manners. The determination module comprises: The determination unit is configured to determine the target processing result from the candidate processing results based on the priority levels of the candidate processing results. The determination unit comprises: The first detection subunit is configured to detect whether the offline candidate processing result exists. The fourth detection subunit is configured to, in response to the offline candidate processing result not existing, detect whether the offline-online fusion candidate processing result exists. The fifth detection subunit is configured to, in response to the offline-online fusion candidate processing result existing, continue to detect whether the online candidate processing result exists within a preset time interval. The fourth determination subunit is configured to, in response to the online candidate processing result existing, determine the online candidate processing result as the target processing result. The fifth determination subunit is configured to, in response to the online candidate processing result not existing, determine the offline-online fusion candidate processing result as the target processing result. The sixth determination subunit is configured to, in response to the offline-online fusion candidate processing result not existing, wait for the online candidate processing result and determine the online candidate processing result as the target processing result.

7. The voice processing apparatus according to claim 6, wherein the voice processing module comprises: The first semantic analysis unit is configured to perform online semantic analysis on the online voice recognition results to obtain online semantic analysis results as the online candidate processing results corresponding to the first combination manner in the different combination manners. The second semantic analysis unit is configured to perform offline semantic analysis on the offline voice recognition results to obtain offline semantic analysis results as the offline candidate processing results corresponding to the second combination manner in the different combination manners.

8. The speech processing device of claim 6, wherein, The priority level of the online candidate processing result is the highest priority level, and the priority level of the offline candidate processing result is the lowest priority level.

9. The speech processing device of claim 8, wherein, The determination unit further comprises: The second detection subunit is configured to, in response to the offline candidate processing result existing, continue to detect whether the online candidate processing result exists within a preset time interval. The first determination subunit is configured to, in response to the online candidate processing result existing, determine the online candidate processing result as the target processing result. a third detecting subunit, configured to, in response to the absence of the online candidate processing result, detect whether the offline candidate processing result exists; a second determining subunit, configured to, in response to the existence of the offline-online fusion candidate processing result, determine the offline-online fusion candidate processing result as the target processing result; a third determining subunit, configured to, in response to the absence of the offline-online fusion candidate processing result, determine the offline candidate processing result as the target processing result.

10. The speech processing apparatus according to claim 7, wherein the speech processing module further comprises: a calculating unit, configured to calculate a confidence degree of the offline speech recognition result; an obtaining unit, configured to obtain a character length of the offline speech recognition result; a first comparing unit, configured to, in response to the confidence degree being greater than or equal to a confidence degree threshold and the character length being greater than or equal to a length threshold, retain the offline speech recognition result to perform the step of performing offline semantic analysis on the offline speech recognition result; a second comparing unit, configured to, in response to the confidence degree being less than the confidence degree threshold or the character length being less than the length threshold, discard the offline speech recognition result.

11. An electronic device, comprising: at least one processor; and a memory connected with the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-5.

12. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, The computer instructions are used to enable the computer to perform the method of any one of claims 1-5.

13. A computer program product comprising a computer program which, when executed by a processor, implements the steps of the method of any one of claims 1-5.

Images