A speech recognition method and related device

Abstract

The present application discloses a speech recognition method and related devices, which relate to the field of speech recognition technology. The method includes: obtaining a target speech; inputting the target speech into a speech recognition model on the end side for speech recognition, and transmitting the target speech to the cloud side so that the speech recognition model on the cloud side performs speech recognition on it; after obtaining the speech recognition result on the end side, using a large language model to determine whether the speech recognition result on the end side is reliable; if the speech recognition result on the end side is reliable, the speech recognition result on the end side is determined as the target speech recognition result; otherwise, the speech recognition result on the cloud side is determined as the target speech recognition result. The speech recognition method disclosed in the present application is an end-cloud collaborative speech recognition method. The method can accurately decide whether to ultimately adopt the speech recognition result on the end side or the cloud side based on the reliability judgment result of the speech recognition result on the end side by the large language model. The speech recognition method disclosed in the present application has a good speech recognition effect.

Description

Technical Field

[0001] The present application relates to the field of speech recognition technology, and in particular to a speech recognition method and related devices. Background Art

[0002] Speech recognition technology converts speech into text. It has a wide range of applications and has become a part of our daily lives and work.

[0003] In some scenarios (such as in-vehicle systems), voice recognition is often performed on the device side. This offers advantages such as low latency and data privacy and security.

[0004] However, since the speech recognition model on the end is usually small, poor recognition results may occur at times, and poor speech recognition results on the end will affect the user experience. Summary of the Invention

[0005] In view of this, the present application provides a speech recognition method and related apparatus to solve the problem that the speech recognition solution on the terminal side has poor recognition effect and affects the user experience. The technical solution is as follows:

[0006] The first aspect of the present application provides a speech recognition method, comprising:

[0007] Get the target voice;

[0008] Performing speech recognition on the target speech input terminal by the speech recognition model, and transmitting the target speech to the cloud side so that the speech recognition model on the cloud side performs speech recognition on the target speech;

[0009] After obtaining the speech recognition result on the device side, the reliability of the speech recognition result on the device side is determined using a large language model. The large language model is obtained by fine-tuning a pre-trained large language model on the device-cloud collaborative speech recognition task. The large language model has the ability to determine the reliability of the speech recognition result.

[0010] If the speech recognition result on the terminal side is reliable, determining the speech recognition result on the terminal side as the target speech recognition result;

[0011] If the speech recognition result on the terminal side is unreliable, after obtaining the speech recognition result on the cloud side, the speech recognition result on the cloud side is determined as the target speech recognition result.

[0012] In a possible implementation, the speech recognition method further includes:

[0013] If the speech recognition result on the terminal side is unreliable, extracting entity words from the speech recognition result on the cloud side, and determining the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side;

[0014] Determining whether the hot word library on the terminal side needs to be updated immediately based on the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side;

[0015] If it is determined that the hot word library on the end side does not need to be updated immediately, the extracted entity words are stored in the entity word database; if it is determined that the hot word library on the end side needs to be updated immediately, the extracted entity words and the entity words in the entity word database are added to the hot word library on the end side.

[0016] In one possible implementation, the large language model further has the ability to identify entity words from speech recognition results and determine the similarity between two speech recognition results;

[0017] The extracting entity words from the speech recognition result on the cloud side and determining the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side includes:

[0018] Utilizing the large language model, entity words are extracted from the speech recognition results on the cloud side, and similarity between the speech recognition results on the terminal side and the speech recognition results on the cloud side is determined.

[0019] In one possible implementation, determining whether the hot word library on the terminal side needs to be updated immediately based on the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side includes:

[0020] If the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side is greater than or equal to a preset similarity threshold, it is determined that there is no need to immediately update the hot word library on the terminal side;

[0021] If the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side is less than a preset similarity threshold, it is determined that the hot word library on the terminal side needs to be updated immediately.

[0022] In a possible implementation, the speech recognition method further includes:

[0023] After storing the extracted entity words in the entity word database, counting the number of entity words in the entity word database;

[0024] If the number of entity words in the entity word database is greater than or equal to a preset entity word number threshold, the entity words in the entity word database are added to the hot word library on the terminal side, and the entity words in the entity word database are deleted.

[0025] In one possible implementation, extracting entity words from the cloud-side speech recognition result using the large language model and determining the similarity between the terminal-side speech recognition result and the cloud-side speech recognition result includes:

[0026] Using the feature acquisition module of the large language model, extract features from the speech recognition result on the terminal side and the speech recognition result on the cloud side, respectively, to obtain features of the speech recognition result on the terminal side and features of the speech recognition result on the cloud side;

[0027] Utilizing the data distribution module of the large language model, the features of the cloud-side speech recognition result are distributed to the entity word recognition module of the large language model, and the features of the client-side speech recognition result and the features of the cloud-side speech recognition result are distributed to the similarity determination module of the large language model;

[0028] Using the entity word recognition module of the large language model, based on the characteristics of the speech recognition results on the cloud side, identify entity words in the speech recognition results on the cloud side;

[0029] The similarity determination module of the large language model is used to determine the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side based on the characteristics of the speech recognition result fed back by the cloud side and the characteristics of the speech recognition result on the terminal side.

[0030] A second aspect of the present application provides a speech recognition device, comprising: a speech acquisition module, a speech processing module, a terminal-side speech recognition result reliability determination module, and a target speech recognition result determination module;

[0031] The speech acquisition module is used to acquire the target speech;

[0032] The speech processing module is used to perform speech recognition on the speech recognition model on the target speech input side, and transmit the target speech to the cloud side so that the speech recognition model on the cloud side performs speech recognition on the target speech;

[0033] The device-side speech recognition result reliability determination module is configured to, after obtaining the device-side speech recognition result, use a large language model to determine whether the device-side speech recognition result is reliable. The large language model is obtained by fine-tuning a pre-trained large language model on a device-cloud collaborative speech recognition task, and the large language model is capable of determining the reliability of the speech recognition result.

[0034] The target speech recognition result determination module is used to determine the speech recognition result on the terminal side as the target speech recognition result if the speech recognition result on the terminal side is reliable; and to determine the speech recognition result on the cloud side as the target speech recognition result after obtaining the speech recognition result on the cloud side if the speech recognition result on the terminal side is unreliable.

[0035] A third aspect of the present application provides an electronic device, comprising at least one processor and a memory connected to the processor, wherein:

[0036] The memory is used to store computer programs;

[0037] The processor is used to execute the computer program so that the electronic device can implement the steps of any one of the above-mentioned speech recognition methods.

[0038] In a fourth aspect of the present application, a computer storage medium carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of any one of the above-mentioned speech recognition methods.

[0039] In a fifth aspect, the present application provides a computer program product comprising computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the steps of any one of the above-mentioned speech recognition methods.

[0040] By means of the above technical solution, the speech recognition method provided by the present application first obtains the speech to be recognized, that is, the target speech, and then inputs the target speech into the speech recognition model on the terminal side for speech recognition, and transmits the target speech to the cloud side so that the speech recognition model on the cloud side performs speech recognition on the target speech. After obtaining the speech recognition result on the terminal side, the large language model is used to determine whether the speech recognition result on the terminal side is reliable. If the speech recognition result on the terminal side is reliable, the speech recognition result on the terminal side is determined as the target speech recognition result. If the speech recognition result on the terminal side is unreliable, after obtaining the speech recognition result on the cloud side, the speech recognition result on the cloud side is determined as the target speech recognition result. Taking into account the limited capabilities of the end-side speech recognition model, this application proposes an end-cloud collaborative speech recognition solution, and also proposes a decision-making strategy based on the reliability of the end-side speech recognition results. That is, the target speech is provided to the end-side speech recognition model and the cloud-side speech recognition model at the same time for speech recognition. After obtaining the end-side speech recognition results, the large language model is used to judge the reliability of the end-side speech recognition results. According to the reliability judgment results of the end-side speech recognition results, the decision is made whether to finally adopt the end-side speech recognition results or the cloud-side speech recognition results. Compared with the pure end-side speech recognition solution, the speech recognition method provided by this application can effectively improve the speech recognition effect, thereby improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0042] Figure 1 A schematic diagram of a system architecture involved in this application;

[0043] Figure 2 A schematic diagram of the hardware structure of a terminal provided in an embodiment of the present application;

[0044] Figure 3 A schematic diagram of the hardware structure of a server provided in an embodiment of the present application;

[0045] Figure 4 A flowchart of a speech recognition method provided in an embodiment of the present application;

[0046] Figure 5 A flowchart of another speech recognition method provided in an embodiment of the present application;

[0047] Figure 6 A schematic diagram of the structure of a large language model provided in an embodiment of the present application;

[0048] Figure 7 A schematic diagram of the structure of the speech recognition device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0049] The following describes the embodiments of the present application in conjunction with the accompanying drawings. The terms used in the implementation methods of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0050] The embodiments of the present application are described below in conjunction with the accompanying drawings. Those skilled in the art will appreciate that, with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0051] The terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances, and this is merely a way of distinguishing the objects of the same attributes when describing them in the embodiments of the present application. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, so that the process, method, system, product or equipment comprising a series of units need not be limited to those units, but may include other units that are not clearly listed or inherent to these processes, methods, products or equipment.

[0052] In one possible implementation, Figure 1 As shown, the system architecture involved in this application may include a terminal 101 and a cloud-side server 102. The terminal 101 may interact with the cloud-side server 102 via a network (wired network or wireless network). The cloud-side server 102 may include one or more servers ( Figure 1 A server is included as an example for explanation).

[0053] Next, the product form of the above-mentioned terminal will be described.

[0054] The above-mentioned terminal may be an on-board device in a vehicle-mounted scenario. Of course, this embodiment is not limited thereto, and the above-mentioned terminal may also be a terminal in other scenarios.

[0055] Figure 2 A schematic diagram of an optional hardware structure of a terminal is shown.

[0056] refer to Figure 2As shown, the terminal may include a radio frequency unit 210, a memory 220, an input unit 230, a display unit 240, a camera 250 (optional), an audio circuit 260 (optional), a speaker 261 (optional), a microphone 262 (optional), an earphone jack 263 (optional), a processor 270, an external interface 280, a power supply 290 and other components. Those skilled in the art will understand that Figure 2 This is merely an example of a terminal and does not constitute a limitation on the terminal. The terminal may include more or fewer components than shown in the figure, or a combination of certain components, or different components.

[0057] The input unit 230 can be used to receive input digital or character information and generate key signal input related to the user settings and function control of the terminal. Specifically, the input unit 230 may include a touch screen 231 (optional) and / or other input devices 232. The touch screen 231 can collect user touch operations on or near it (for example, operations performed on or near the touch screen using a finger, joint, stylus, or any other suitable object) and drive corresponding connected devices according to a pre-set program. The touch screen can detect user touch actions on the touch screen, convert the touch actions into touch signals and transmit them to the processor 270, and can receive and execute commands sent by the processor 270; the touch signals include at least touch point coordinate information. The touch screen 231 can provide an input interface and an output interface between the terminal and the user. In addition, touch screens can be implemented using various types, such as resistive, capacitive, infrared, and surface acoustic wave. In addition to the touch screen 231, the input unit 230 may also include other input devices. Specifically, the other input devices 232 may include, but are not limited to, one or more of a physical keyboard, function keys (such as a volume control button, a switch button, etc.), a trackball, a mouse, a joystick, and the like.

[0058] The display unit 240 may be used to display information input by a user or information provided to a user, various menus of the terminal, an interactive interface, file display, and / or playback of any multimedia file.

[0059] Memory 220 can be used to store instructions and data. Memory 220 primarily includes an instruction storage area and a data storage area. The data storage area can store various data, such as multimedia files and text. The instruction storage area can store software units such as the operating system, applications, and instructions required for at least one function, or subsets or extensions thereof. It may also include non-volatile random access memory (RAM). It provides processor 270 with management functions for the hardware, software, and data resources within the computing and processing device, supporting control software and applications. It is also used to store multimedia files and running programs and applications.

[0060] The processor 270 is the terminal's control center, connecting all components of the terminal using various interfaces and circuits. By executing instructions stored in memory 220 and accessing data stored in memory 220, it performs various terminal functions and processes data, thereby providing overall control of the terminal. Optionally, the processor 270 may include one or more processing units. Preferably, the processor 270 integrates an application processor and a modem processor, with the application processor primarily processing the operating system, user interface, and application programs, while the modem processor primarily handles wireless communications. It is understood that the modem processor may not be integrated into the processor 270. In some embodiments, the processor and memory may be implemented on a single chip; in other embodiments, they may be implemented on separate chips. The processor 270 may also generate corresponding operational control signals and send them to corresponding components of the computing and processing device. It may also read and process data in the software, particularly the data and programs in memory 220, to enable the various functional modules therein to perform their corresponding functions, thereby controlling the corresponding components to operate as instructed.

[0061] Among them, the memory 220 can be used to store software codes related to the speech recognition method, the processor 270 can execute the software codes in the memory 220, and can also schedule other units (such as the above-mentioned input unit 230 and display unit 240) to implement corresponding functions.

[0062] The RF unit 210 (optional) can be used to send and receive information or receive and send signals during a call. For example, it receives downlink information from the base station and passes it to the processor 270 for processing; in addition, it sends uplink data to the base station. Typically, the RF unit 210 includes, but is not limited to, an antenna, at least one amplifier, a transceiver, a coupler, a low-noise amplifier (LNA), a duplexer, etc. Furthermore, the RF unit 210 can communicate with network devices and other devices via wireless communication. This wireless communication can use any communication standard or protocol, including but not limited to Global System of Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Long Term Evolution (LTE), email, Short Messaging Service (SMS), etc.

[0063] In the embodiment of the present application, the radio frequency unit 210 can send data to other devices and can also receive data sent by other devices. It should be understood that the radio frequency unit 210 is optional and can be replaced by other communication interfaces, such as a network port.

[0064] The terminal also includes a power supply 290 (such as a battery) for supplying power to various components. Preferably, the power supply can be logically connected to the processor 270 through a power management system, thereby implementing functions such as charging, discharging, and power consumption management through the power management system.

[0065] The terminal further includes an external interface 280 , which may be a standard Micro USB interface or a multi-pin connector, and may be used to connect the terminal to other devices for communication, or to connect a charger for charging the terminal.

[0066] Although not shown, the terminal may further include a flashlight, a wireless fidelity (WiFi) module, a Bluetooth module, sensors with different functions, etc., which will not be described in detail here.

[0067] Next, the product form of the above-mentioned cloud-side server is described.

[0068] Figure 3 A structural diagram of the above-mentioned cloud-side server is provided, such as Figure 3 As shown, the cloud-side server may include a bus 301 , a processor 302 , a communication interface 303 , and a memory 304 . The processor 302 , the memory 304 , and the communication interface 303 communicate with each other via the bus 301 .

[0069] The bus 301 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 3 Only one thick line is used in the diagram, but this does not mean that there is only one bus or one type of bus.

[0070] The processor 302 may be any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0071] The memory 304 may include volatile memory, such as random access memory (RAM). The memory 304 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0072] The memory 304 may be used to store software codes related to the speech recognition method. The processor 302 may call the software codes stored in the memory 304 or schedule other units to implement corresponding functions.

[0073] The processors in the above-mentioned terminals and cloud-side servers (such as processor 270 and processor 302) can be hardware circuits (such as application specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), general-purpose processors, digital signal processors (DSP), microprocessors or microcontrollers, etc.), or a combination of these hardware circuits. For example, the processor can be a hardware system with an instruction execution function, such as a CPU, DSP, etc., or a hardware system without an instruction execution function, such as an ASIC, FPGA, etc., or a combination of the above-mentioned hardware systems without an instruction execution function and hardware systems with an instruction execution function.

[0074] Given that the speech recognition model on the end side is usually small and may sometimes have poor recognition effects, the inventor of this case conducted research and came up with an end-cloud collaborative speech recognition solution during the research process. The initial end-cloud collaborative speech recognition solution was: input the target speech into the speech recognition model on the end side for speech recognition, and transmit the target speech to the cloud side so that the speech recognition model on the cloud side can perform speech recognition on the target speech. After obtaining the speech recognition result on the end side, the speech recognition result on the end side is matched with the standard text in a pre-built standard text database. If there is text in the standard text database that matches the speech recognition result on the end side (for example, the speech recognition result on the end side is "turn on the air conditioner", and "turn on the air conditioner" exists in the standard text database), the speech recognition result on the end side is determined as the target speech recognition result. If there is no text in the standard text database that matches the speech recognition result on the end side, the speech recognition result on the cloud side is determined as the target speech recognition result.

[0075] The inventors of this case studied the aforementioned end-cloud collaborative speech recognition solution and found that the aforementioned strategy for deciding whether to ultimately use the end-side speech recognition result or the cloud-side speech recognition result was too crude, which could have a certain impact on end-side performance (for example, the end-side speech recognition result should have been used, but the cloud-side speech recognition result was ultimately used due to the limitations of the aforementioned decision-making strategy). Given the shortcomings of the aforementioned end-cloud collaborative speech recognition solution, the inventors of this case continued their research and, through continuous research, ultimately proposed a more effective speech recognition method. The speech recognition method provided in this application is now introduced through the following examples.

[0076] See also Figure 4 , which shows a flow chart of a speech recognition method provided by an embodiment of the present application, and which may include:

[0077] Step S401: Acquire target speech.

[0078] The target speech is the speech to be recognized.

[0079] Step S402: The speech recognition model on the target speech input side performs speech recognition, and transmits the target speech to the cloud side so that the speech recognition model on the cloud side performs speech recognition on the target speech.

[0080] After obtaining the target voice, the target voice is provided to the client side for voice recognition and is also transmitted to the cloud side for voice recognition.

[0081] In this embodiment, the cloud-side speech recognition model is more capable than the device-side speech recognition model, especially for personalized scenarios. To ultimately achieve better speech recognition results, this application combines the device-side speech recognition capabilities with the cloud-side speech recognition capabilities.

[0082] Step S403: After obtaining the speech recognition result on the terminal side, the large language model is used to determine whether the speech recognition result on the terminal side is reliable.

[0083] Specifically, after obtaining the voice recognition result on the end side, the voice recognition result on the end side is input into the large language model, and the large language model determines the reliability of the voice recognition result on the end side. If the reliability of the voice recognition result on the end side is greater than or equal to the preset reliability threshold, the voice recognition result on the end side is determined to be reliable. If the reliability of the voice recognition result on the end side is less than the preset reliability threshold, the voice recognition result on the end side is determined to be unreliable.

[0084] The large language model in this embodiment is obtained by fine-tuning the pre-trained large language model on the end-cloud collaborative speech recognition task. The large language model has the ability to determine the reliability of speech recognition results.

[0085] Step S404a: If the speech recognition result on the terminal side is reliable, the speech recognition result on the terminal side is determined as the target speech recognition result.

[0086] If the speech recognition result on the terminal side is determined to be reliable, the speech recognition result on the terminal side is determined as the final speech recognition result, that is, the target speech recognition result.

[0087] Step S404b: If the speech recognition result on the terminal side is unreliable, after obtaining the speech recognition result on the cloud side, the speech recognition result on the cloud side is determined as the target speech recognition result.

[0088] When it is determined that the speech recognition result on the terminal side is unreliable, the speech recognition result on the cloud side is determined as the final speech recognition result, that is, the target speech recognition result.

[0089] This embodiment uses the natural language understanding capability of a large language model to determine the reliability of the speech recognition result on the end side. Then, based on the reliability determination result of the speech recognition result on the end side, a decision is made as to whether the speech recognition result on the end side or the speech recognition result on the cloud side will be used as the final speech recognition result. This strategy can more accurately determine whether the speech recognition result on the end side will be used as the final speech recognition result.

[0090] The speech recognition method provided in the embodiment of the present application first obtains the speech to be recognized, that is, the target speech, and then inputs the target speech into the speech recognition model on the terminal side for speech recognition, and transmits the target speech to the cloud side so that the speech recognition model on the cloud side performs speech recognition on the target speech. After obtaining the speech recognition result on the terminal side, the large language model is used to determine whether the speech recognition result on the terminal side is reliable. If the speech recognition result on the terminal side is reliable, the speech recognition result on the terminal side is determined as the target speech recognition result. If the speech recognition result on the terminal side is unreliable, the speech recognition result on the cloud side is determined as the target speech recognition result. Taking into account the limited capabilities of the end-side speech recognition model, the embodiment of the present application proposes an end-cloud collaborative speech recognition solution, and at the same time proposes a decision-making strategy based on the reliability of the end-side speech recognition results, that is, the target speech is provided to the end-side speech recognition model and the cloud-side speech recognition model at the same time for speech recognition. After obtaining the end-side speech recognition result, the large language model is used to judge the reliability of the end-side speech recognition result. According to the reliability judgment result of the end-side speech recognition result, the decision is made whether to finally adopt the end-side speech recognition result or the cloud-side speech recognition result. Compared with the pure end-side speech recognition solution, the speech recognition method provided by the embodiment of the present application can effectively improve the speech recognition effect, thereby improving the user experience. In addition, the decision-making strategy based on the reliability of the end-side speech recognition result provided by the embodiment of the present application can more accurately determine whether to use the end-side speech recognition result as the final speech recognition result.

[0091] In order to improve the voice recognition effect on the terminal side, the embodiment of the present application provides another voice recognition method, such as Figure 5 As shown, the speech recognition method includes, in addition to steps S401 to S404 in the above embodiment, the following steps:

[0092] Step S405b: extract entity words from the speech recognition results on the cloud side, and determine the similarity between the speech recognition results on the terminal side and the speech recognition results on the cloud side.

[0093] If the speech recognition results on the client side are unreliable, a large language model can be used to extract entity words from the speech recognition results on the cloud side and determine the similarity between the speech recognition results on the client side and the cloud side.

[0094] The large language model in this embodiment has the ability to identify entity words from speech recognition results and determine the similarity between two speech recognition results. The pre-trained large language model can be fine-tuned using the training dataset corresponding to the end-cloud collaborative speech recognition task, enabling it to determine the reliability of speech recognition results, identify entity words from speech recognition results, and determine the similarity between two speech recognition results.

[0095] Step S406b: Based on the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side, it is determined whether the hot word library on the terminal side needs to be updated immediately.

[0096] In this embodiment, the process of determining whether the hot word library on the terminal side needs to be updated immediately based on the similarity between the voice recognition result on the terminal side and the voice recognition result on the cloud side may include: determining whether the similarity between the voice recognition result on the terminal side and the voice recognition result on the cloud side is greater than or equal to a preset similarity threshold; if the similarity between the voice recognition result on the terminal side and the voice recognition result on the cloud side is greater than or equal to the preset similarity threshold, determining that the hot word library on the terminal side does not need to be updated immediately; if the similarity between the voice recognition result on the terminal side and the voice recognition result on the cloud side is less than the preset similarity threshold, determining that the hot word library on the terminal side needs to be updated immediately.

[0097] If the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side is greater than or equal to the preset similarity threshold, it means that the difference between the speech recognition result on the terminal side and the speech recognition result on the cloud side is not large, and it is determined that the hot word library on the terminal side does not need to be updated immediately. If the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side is less than the preset similarity threshold, it means that the difference between the speech recognition result on the terminal side and the speech recognition result on the cloud side is large, and it is determined that the hot word library on the terminal side needs to be updated immediately.

[0098] It should be noted that the device-side speech recognition model uses the device-side hotword library during speech recognition, and the quality of the hotword library affects the device-side speech recognition performance. To improve the device-side speech recognition performance, this embodiment updates the entity words extracted from the cloud-side speech recognition results to the device-side hotword library when there is a significant difference between the device-side speech recognition results and the cloud-side speech recognition results.

[0099] Taking into account the limited resources on the terminal side, in order to alleviate the resource consumption on the terminal side, this embodiment does not directly add the entity words to the hot word library on the terminal side after extracting the entity words from the speech recognition results on the cloud side. Instead, it first determines whether the hot word update conditions are currently met, that is, determines the similarity between the speech recognition results on the terminal side and the speech recognition results on the cloud side, and then determines whether the hot word library on the terminal side needs to be updated immediately based on the similarity between the speech recognition results on the terminal side and the speech recognition results on the cloud side. When it is determined that the hot word library on the terminal side needs to be updated immediately, the entity words extracted from the speech recognition results on the cloud side are added to the hot word library on the terminal side.

[0100] Of course, this embodiment is not limited to updating the hot word library on the terminal side when the hot word update conditions are met. After the entity words are extracted from the speech recognition results on the cloud side, the extracted entity words can be immediately added to the hot word library on the terminal side.

[0101] Updating the hot word library on the client side can improve the recognition accuracy of the speech recognition model on the client side the next time it performs speech recognition.

[0102] Step S407b-a: If it is determined that the hot word database on the terminal side does not need to be updated immediately, the extracted entity words are stored in the entity word database.

[0103] When it is determined that the hot word library on the terminal side does not need to be updated immediately, the entity words extracted from the speech recognition results on the cloud side are temporarily stored in the entity word database.

[0104] In one possible implementation, after the entity words extracted from the speech recognition results on the cloud side are stored in the entity word database, the number of entity words in the entity word database can be counted to determine whether the number of entity words in the entity word database is greater than or equal to a preset entity word number threshold. If the number of entity words in the entity word database is greater than or equal to the preset entity word number threshold, the entity words in the entity word database are added to the hot word library on the end side, and the entity words in the entity word database are deleted. If the number of entity words in the entity word database is less than the preset entity word number threshold, the hot word library on the end side is not updated.

[0105] Step S407b-b: If it is determined that the hot word library on the terminal side needs to be updated immediately, the extracted entity words and the entity words in the entity word database are added to the hot word library on the terminal side.

[0106] It should be noted that, when it is determined that the hot word library on the end side needs to be updated immediately, the entity words extracted from the speech recognition results on the cloud side are added to the hot word library on the end side. In addition to adding the entity words extracted from the speech recognition results on the cloud side to the hot word library on the end side, the entity words already in the entity word database are also added to the hot word library on the end side. After the entity words in the entity word database are added to the hot word library on the end side, the entity words in the entity word database are deleted.

[0107] The speech recognition method provided by the embodiment of the present application first obtains the speech to be recognized, that is, the target speech, and then inputs the target speech into the speech recognition model on the terminal side for speech recognition, and transmits the target speech to the cloud side so that the speech recognition model on the cloud side performs speech recognition on the target speech. After obtaining the speech recognition result on the terminal side, the large language model is used to determine whether the speech recognition result on the terminal side is reliable. If the speech recognition result on the terminal side is reliable, the speech recognition result on the terminal side is determined as the target speech recognition result. If the speech recognition result on the terminal side is unreliable, the speech recognition result on the cloud side is determined as the target speech recognition result, and entity words are extracted from the speech recognition result on the cloud side. and determining the similarity between the voice recognition result on the terminal side and the voice recognition result on the cloud side, and judging whether it is necessary to immediately update the hot word library on the terminal side according to the similarity between the voice recognition result on the terminal side and the voice recognition result on the cloud side; if it is determined that the hot word library on the terminal side does not need to be updated immediately, the extracted entity words are stored in the entity word database; when the number of entity words in the entity word database is greater than or equal to a preset entity word number threshold, the entity words in the entity word database are updated to the hot word library on the terminal side; if it is determined that the hot word library on the terminal side needs to be updated immediately, the entity words extracted from the voice recognition result on the cloud side and the entity words in the entity word database are updated to the hot word library on the terminal side. Taking into account the limited capabilities of the speech recognition model on the end side, the embodiment of the present application proposes an end-cloud collaborative speech recognition solution, and at the same time proposes a decision-making strategy based on the reliability of the speech recognition results on the end side, that is, the target speech is provided to the speech recognition model on the end side and the speech recognition model on the cloud side for speech recognition. After obtaining the speech recognition results on the end side, the large language model is used to judge the reliability of the speech recognition results on the end side. According to the reliability judgment results of the speech recognition results on the end side, it is decided whether to finally adopt the speech recognition results on the end side or the speech recognition results on the cloud side. Compared with the pure end-side speech recognition solution, the speech recognition method provided by the embodiment of the present application can effectively improve the speech recognition effect, thereby improving the user experience. In addition, according to the reliability judgment results of the speech recognition results on the end side, it is decided whether to use the speech recognition results on the end side or the speech recognition results on the cloud side as the final speech recognition results, which can more accurately determine whether to use the speech recognition results on the end side as the final speech recognition results. In order to improve the subsequent speech recognition accuracy on the terminal side, the embodiment of the present application proposes to update the entity words extracted from the speech recognition results on the cloud side to the hot word library on the terminal side. In order to alleviate the resource consumption on the terminal side, after the entity words are extracted from the speech recognition results on the cloud side, the embodiment of the present application does not directly add the entity words to the hot word library on the terminal side, but first determines whether the hot word update conditions are met. When the hot word update conditions are met, the extracted entity words are added to the hot word library on the terminal side.In summary, the speech recognition method provided in the embodiment of the present application has a good speech recognition effect, can improve the subsequent speech recognition accuracy on the terminal side, and can also alleviate the resource consumption on the terminal side to a certain extent.

[0108] The above embodiment mentioned that a large language model can be used to determine whether the speech recognition results on the terminal side are reliable. The large language model can be used to extract entity words from the speech recognition results on the cloud side and determine the similarity between the speech recognition results on the terminal side and the speech recognition results on the cloud side. The above process is introduced below.

[0109] In one possible implementation, Figure 6 As shown, the large language model may include: a feature acquisition module 601, a data distribution module 602, a reliability determination module 603a, an entity word recognition module 603b and a similarity determination module 603c.

[0110] The process of using a large language model to determine whether the speech recognition results on the client are reliable may include:

[0111] Step a1: Use the feature acquisition module 601 of the large language model to extract features from the speech recognition result on the terminal side to obtain the features of the speech recognition result on the terminal side.

[0112] The speech recognition results on the client side can be input into the feature acquisition module 601 of the large language model. The feature acquisition module 601 of the large language model extracts features from the input speech recognition results and outputs the features of the speech recognition results on the client side. Optionally, the feature acquisition module 601 can be, but is not limited to, a feature acquisition module based on a multi-head attention mechanism.

[0113] Step a2: Use the data distribution module 602 of the large language model to distribute the features of the speech recognition result on the terminal side to the reliability determination module 603a.

[0114] The features of the speech recognition result on the terminal side are input into the data distribution module 602 of the large language model, and the data distribution module 602 distributes the features of the speech recognition result on the terminal side to the reliability determination module 603a for processing.

[0115] Step a3: Utilize the reliability determination module 603a of the large language model to predict the reliability of the speech recognition result on the end side based on the characteristics of the speech recognition result on the end side.

[0116] The reliability determination module 603a predicts the reliability of the speech recognition result on the terminal side according to the characteristics of the speech recognition result on the terminal side.

[0117] Step a4: If the reliability of the speech recognition result on the terminal side is greater than or equal to the preset reliability threshold, the speech recognition result on the terminal side is determined to be reliable; if the reliability of the speech recognition result on the terminal side is less than the preset reliability threshold, the speech recognition result on the terminal side is determined to be unreliable.

[0118] The above process determines whether the device-side speech recognition result is reliable. If the device-side speech recognition result is unreliable, the device-side speech recognition result is used as the target speech recognition result. The large language model is used to extract entity words from the cloud-side speech recognition result, and the similarity between the device-side and cloud-side speech recognition results is determined. The following describes the specific implementation process of using the large language model to extract entity words from the cloud-side speech recognition results and determine the similarity between the device-side and cloud-side speech recognition results.

[0119] The process of extracting entity words from the cloud-side speech recognition results using a large language model and determining the similarity between the client-side speech recognition results and the cloud-side speech recognition results may include:

[0120] Step b1: Use the feature acquisition module 601 of the large language model to extract features from the speech recognition results on the terminal side and the speech recognition results on the cloud side, respectively, to obtain the features of the speech recognition results on the terminal side and the features of the speech recognition results on the cloud side.

[0121] The speech recognition results on the terminal side and the speech recognition results on the cloud side are input into the feature acquisition module 601 of the large language model. The feature acquisition module 601 extracts features from the speech recognition results on the terminal side and the speech recognition results on the cloud side respectively to obtain the features of the speech recognition results on the terminal side and the features of the speech recognition results on the cloud side.

[0122] Step b2: Use the data distribution module 602 of the large language model to distribute the features of the cloud-side speech recognition results to the entity word recognition module 603b of the large language model, and distribute the features of the terminal-side speech recognition results and the features of the cloud-side speech recognition results to the similarity determination module 603c of the large language model.

[0123] The features of the speech recognition results on the terminal side and the features of the speech recognition results on the cloud side are input into the data distribution module 602. The data distribution module 602 distributes data through a routing mechanism. Specifically, the features of the speech recognition results on the cloud side are distributed to the entity word recognition module 603b, and the features of the speech recognition results on the terminal side and the features of the speech recognition results on the cloud side are distributed to the similarity determination module 603c.

[0124] Step b3-a: Utilize the entity word recognition module 603b of the large language model to identify entity words in the speech recognition results on the cloud side based on the features of the speech recognition results on the cloud side.

[0125] Specifically, after the data distribution module 602 distributes the features of the speech recognition results on the cloud side to the entity word recognition module 603b, the entity word recognition module 603b predicts the probability that each word contained in the speech recognition results on the cloud side is an entity word based on the features of the speech recognition results on the cloud side. After obtaining the probability that each word contained in the speech recognition results on the cloud side is an entity word, the words with a probability greater than a preset probability threshold can be judged as entity words, so that the entity words in the speech recognition results on the cloud side can be obtained.

[0126] Step b3-b: Using the large language model similarity determination module 603c, based on the characteristics of the speech recognition result fed back from the cloud side and the characteristics of the speech recognition result on the terminal side, determine the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side.

[0127] Specifically, after the data distribution module 602 distributes the features of the speech recognition result on the terminal side and the features of the speech recognition result on the cloud side to the similarity determination module 603c, the similarity determination module 603c determines the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side based on the features of the speech recognition result on the terminal side and the features of the speech recognition result on the cloud side. More specifically, the similarity between the features of the speech recognition result on the terminal side and the features of the speech recognition result on the cloud side can be calculated, and the obtained similarity can be used as the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side.

[0128] The above embodiment mentioned that the pre-trained large language model can be fine-tuned using the training data set corresponding to the end-cloud collaborative speech recognition task to obtain a large language model with the ability to judge the reliability of speech recognition results, the ability to identify entity words from speech recognition results, and the ability to determine the similarity between two speech recognition results. The training data set corresponding to the end-cloud collaborative speech recognition task may include a number of first training data, a number of second training data, a number of third training data, and a number of fourth training data, wherein the first training data includes a training text with a first reliability label indicating that the text is reliable, the second training data includes a training text with a second reliability label indicating that the text is unreliable, the third training data includes two training texts with the same semantics, the two training texts with the same semantics have a hot word update label indicating that hot word update is not required immediately, and one of the two training texts with the same semantics has an entity word label indicating the location of the entity word. The fourth training data includes two training texts with different semantics, the two training texts with different semantics have a hot word update label indicating that hot word update is required immediately, and one of the two training texts with different semantics has an entity word label indicating the location of the entity word. The above-mentioned training data can be used to fine-tune the pre-trained large language model. When fine-tuning the pre-trained large language model, the goal is to enable the large language model to predict the corresponding label for the input training data.

[0129] The above describes the speech recognition method provided in the embodiments of the present application. The following describes the device corresponding to the above speech recognition method.

[0130] See also Figure 7 , Figure 7 This is a structural diagram of a speech recognition device provided in an embodiment of the present application. The speech recognition device may include: a speech acquisition module 701, a speech processing module 702, a terminal-side speech recognition result reliability determination module 703 and a target speech recognition result determination module 704.

[0131] The speech acquisition module 701 is used to acquire the target speech.

[0132] The speech processing module 702 is used to perform speech recognition on the speech recognition model on the target speech input side, and transmit the target speech to the cloud side so that the speech recognition model on the cloud side can perform speech recognition on the target speech.

[0133] The terminal-side speech recognition result reliability determination module 703 is used to use the large language model to determine whether the terminal-side speech recognition result is reliable after obtaining the terminal-side speech recognition result.

[0134] Among them, the large language model is obtained by fine-tuning the pre-trained large language model on the end-cloud collaborative speech recognition task. The large language model has the ability to judge the reliability of speech recognition results.

[0135] The target speech recognition result determination module 704 is used to determine the speech recognition result on the terminal side as the target speech recognition result when the speech recognition result on the terminal side is reliable, and to determine the speech recognition result on the cloud side as the target speech recognition result after obtaining the speech recognition result on the cloud side when the speech recognition result on the terminal side is unreliable.

[0136] In a possible implementation, the speech recognition device may further include: an entity word and similarity acquisition module, a hot word update determination module, a hot word storage module, and a hot word update module.

[0137] The entity word and similarity acquisition module is used to extract entity words from the cloud-side speech recognition results when the speech recognition results on the device side are unreliable, and to determine the similarity between the speech recognition results on the device side and the speech recognition results on the cloud side.

[0138] The hot word update determination module is used to determine whether the hot word library on the end side needs to be updated immediately based on the similarity between the speech recognition results on the end side and the speech recognition results on the cloud side.

[0139] The hot word storage module is used to store the extracted entity words in the entity word database when the hot word update determination module determines that the hot word library on the terminal side does not need to be updated immediately.

[0140] The hot word update module is used to add the extracted entity words and entity words in the entity word database to the hot word library on the terminal side when the hot word update judgment module determines that the hot word library on the terminal side needs to be updated immediately.

[0141] In one possible implementation, the large language model also has the ability to identify entity words from speech recognition results and determine the similarity between two speech recognition results.

[0142] The entity word and similarity acquisition module extracts entity words from the cloud-side speech recognition results and determines the similarity between the client-side speech recognition results and the cloud-side speech recognition results. Specifically, it is used to:

[0143] Using a large language model, entities are extracted from the cloud-side speech recognition results, and the similarity between the client-side speech recognition results and the cloud-side speech recognition results is determined.

[0144] In one possible implementation, the hot word update determination module determines whether the hot word library on the terminal side needs to be updated immediately based on the similarity between the speech recognition results on the terminal side and the speech recognition results on the cloud side. Specifically, it is used to:

[0145] If the similarity between the speech recognition result on the client side and the speech recognition result on the cloud side is greater than or equal to the preset similarity threshold, it is determined that there is no need to update the hot word library on the client side immediately;

[0146] If the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side is less than a preset similarity threshold, it is determined that the hot word library on the terminal side needs to be updated immediately.

[0147] In a possible implementation, the speech recognition device may further include: an entity word statistics module.

[0148] An entity word statistics module is used to count the number of entity words in the entity word database after the hot word storage module stores the extracted entity words in the entity word database;

[0149] The hot word update module is further configured to add entity words in the entity word database to the hot word library on the terminal side and delete entity words in the entity word database when the number of entity words in the entity word database is greater than or equal to a preset entity word number threshold.

[0150] In one possible implementation, the entity word and similarity acquisition module uses a large language model to extract entities from the cloud-side speech recognition results and determines the similarity between the client-side speech recognition results and the cloud-side speech recognition results. Specifically, it is used to:

[0151] The feature acquisition module of the large language model is used to extract features from the speech recognition results on the device side and the speech recognition results on the cloud side, respectively, to obtain the features of the speech recognition results on the device side and the features of the speech recognition results on the cloud side.

[0152] Using the data distribution module of the large language model, the features of the cloud-side speech recognition results are distributed to the entity word recognition module of the large language model, and the features of the terminal-side speech recognition results and the features of the cloud-side speech recognition results are distributed to the similarity determination module of the large language model.

[0153] The entity word recognition module of the large language model is used to identify entity words in the cloud-side speech recognition results based on the characteristics of the cloud-side speech recognition results.

[0154] The similarity determination module of the large language model is used to determine the similarity between the speech recognition results on the device and the cloud based on the characteristics of the speech recognition results fed back by the cloud and the characteristics of the speech recognition results on the device.

[0155] The speech recognition device provided in the embodiment of the present application provides the target speech to the speech recognition model on the terminal side and the speech recognition model on the cloud side for speech recognition at the same time. After obtaining the speech recognition result on the terminal side, the large language model is used to perform reliability judgment on the speech recognition result on the terminal side. According to the reliability judgment result of the speech recognition result on the terminal side, it is decided whether to finally adopt the speech recognition result on the terminal side or the speech recognition result on the cloud side. Compared with the pure speech recognition solution on the terminal side, the speech recognition device provided in the embodiment of the present application can effectively improve the speech recognition effect, thereby improving the user experience. In addition, according to the reliability judgment result of the speech recognition result on the terminal side, it is decided whether to use the speech recognition result on the terminal side or the speech recognition result on the cloud side as the final speech recognition result, which can more accurately determine whether to use the speech recognition result on the terminal side as the final speech recognition result. In order to improve the subsequent voice recognition accuracy on the end side, the voice recognition device provided in the embodiment of the present application updates the entity words extracted from the voice recognition results on the cloud side to the hot word library on the end side. In order to alleviate the resource consumption on the end side, the voice recognition device provided in the embodiment of the present application does not directly add the entity words to the hot word library on the end side after extracting the entity words from the voice recognition results on the cloud side, but first determines whether the hot word update conditions are met. When the hot word update conditions are met, the extracted entity words are added to the hot word library on the end side. In summary, the voice recognition device provided in the embodiment of the present application has a good voice recognition effect, can improve the subsequent voice recognition accuracy on the end side, and can also alleviate the resource consumption on the end side to a certain extent.

[0156] An embodiment of the present application also provides an electronic device, which may include: at least one processor, at least one communication interface, at least one memory and at least one communication bus.

[0157] In the embodiment of the present application, the number of the processor, the communication interface, the memory, and the communication bus is at least one, and the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0158] The processor may be a central processing unit (CPU), or an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement the embodiments of the present application;

[0159] The memory may include a high-speed RAM memory, and may also include a non-volatile memory, such as at least one disk memory;

[0160] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to implement the steps of the speech recognition method provided in the above embodiment.

[0161] An embodiment of the present application also provides a computer storage medium, which carries one or more computer programs. When the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the speech recognition method provided in the above embodiment.

[0162] An embodiment of the present application also provides a computer program product, including computer-readable instructions. When the computer-readable instructions are executed on an electronic device, the electronic device implements the steps of the speech recognition method provided in the above embodiment.

[0163] It should also be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided in this application, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines.

[0164] Through the description of the above embodiments, those skilled in the art can clearly understand that the present application can be implemented by means of software plus necessary general hardware, and of course can also be implemented by special hardware including application-specific integrated circuits, special CPUs, special memories, special components, etc. In general, all functions performed by computer programs can be easily implemented with corresponding hardware, and the specific hardware structures used to implement the same function can also be diverse, such as analog circuits, digital circuits or special circuits, etc. However, for the present application, software program implementation is a better implementation method in most cases. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art can be embodied in the form of a software product, which is stored in a readable storage medium, such as a computer's floppy disk, USB flash drive, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc., and includes a number of instructions to enable a computer device (which can be a personal computer, training equipment, or network equipment, etc.) to execute the methods described in each embodiment of the present application.

[0165] In the above embodiments, all or part of the embodiments may be implemented by software, hardware, firmware, or any combination thereof. When implemented by software, all or part of the embodiments may be implemented in the form of a computer program product.

[0166] The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a training device or a data center by wired (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (such as infrared, wireless, microwave, etc.) mode to another website, computer, training device or data center. The computer-readable storage medium can be any available medium that a computer can store or a data storage device such as a training device, a data center that includes one or more available media integrations. The available medium can be a magnetic medium, (for example, a floppy disk, a hard disk, a tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive (SSD)).

Claims

1. A speech recognition method, characterized in that: include: Get the target voice; Performing speech recognition on the target speech input terminal by the speech recognition model, and transmitting the target speech to the cloud side so that the speech recognition model on the cloud side performs speech recognition on the target speech; After obtaining the speech recognition result on the end side, a large language model is used to determine whether the speech recognition result on the end side is reliable, wherein the large language model is obtained by fine-tuning a pre-trained large language model on an end-cloud collaborative speech recognition task, and the large language model has the ability to determine the reliability of the speech recognition result. The training data set corresponding to the end-cloud collaborative speech recognition task includes training texts with a first reliability label indicating that the text is reliable, training texts with a second reliability label indicating that the text is unreliable, training text pairs with the same semantics, and training text pairs with different semantics, wherein the training text pairs with the same semantics have a hot word update label indicating that a hot word update does not need to be performed immediately, and the training text pairs with different semantics have a hot word update label indicating that a hot word update is required immediately; If the speech recognition result on the terminal side is reliable, determining the speech recognition result on the terminal side as the target speech recognition result; If the speech recognition result on the terminal side is unreliable, after obtaining the speech recognition result on the cloud side, determining the speech recognition result on the cloud side as the target speech recognition result; In the case where the speech recognition result on the terminal side is unreliable, using the large language model to determine the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side, wherein the large language model also has the ability to determine the similarity between two speech recognition results; According to the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side, it is determined whether the hot word library on the terminal side needs to be updated immediately.

2. The speech recognition method according to claim 1, wherein: Also includes: When the speech recognition result on the terminal side is unreliable, extracting entity words from the speech recognition result on the cloud side; If it is determined that the hot word library on the client side does not need to be updated immediately, the extracted entity words are stored in the entity word database; If it is determined that the hot word library on the terminal side needs to be updated immediately, the extracted entity words and the entity words in the entity word database are added to the hot word library on the terminal side.

3. The speech recognition method according to claim 2, wherein: The large language model also has the ability to identify entity words from speech recognition results; The extracting entity words from the speech recognition results on the cloud side includes: Utilizing the large language model, entity words are extracted from the speech recognition results on the cloud side.

4. The speech recognition method according to claim 1, wherein: The determining whether it is necessary to immediately update the hot word library on the terminal side according to the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side includes: If the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side is greater than or equal to a preset similarity threshold, it is determined that there is no need to immediately update the hot word library on the terminal side; If the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side is less than a preset similarity threshold, it is determined that the hot word library on the terminal side needs to be updated immediately.

5. The speech recognition method according to claim 2, wherein: Also includes: After storing the extracted entity words in the entity word database, counting the number of entity words in the entity word database; If the number of entity words in the entity word database is greater than or equal to a preset entity word number threshold, the entity words in the entity word database are added to the hot word library on the terminal side, and the entity words in the entity word database are deleted.

6. The speech recognition method according to claim 3, wherein: The extracting entity words from the cloud-side speech recognition result using the large language model, and determining the similarity between the terminal-side speech recognition result and the cloud-side speech recognition result using the large language model, include: Using the feature acquisition module of the large language model, extract features from the speech recognition result on the terminal side and the speech recognition result on the cloud side, respectively, to obtain features of the speech recognition result on the terminal side and features of the speech recognition result on the cloud side; Utilizing the data distribution module of the large language model, the features of the cloud-side speech recognition result are distributed to the entity word recognition module of the large language model, and the features of the client-side speech recognition result and the features of the cloud-side speech recognition result are distributed to the similarity determination module of the large language model; Using the entity word recognition module of the large language model, based on the characteristics of the speech recognition results on the cloud side, identify entity words in the speech recognition results on the cloud side; The similarity determination module of the large language model is used to determine the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side based on the characteristics of the speech recognition result fed back by the cloud side and the characteristics of the speech recognition result on the terminal side.

7. A speech recognition device, characterized in that: include: Speech acquisition module, speech processing module, end-side speech recognition result reliability determination module, target speech recognition result determination module, similarity acquisition module, and hot word update determination module; The speech acquisition module is used to acquire the target speech; The speech processing module is used to perform speech recognition on the speech recognition model on the target speech input side, and transmit the target speech to the cloud side so that the speech recognition model on the cloud side performs speech recognition on the target speech; The end-side speech recognition result reliability determination module is configured to, after obtaining the end-side speech recognition result, use a large language model to determine whether the end-side speech recognition result is reliable, wherein the large language model is obtained by fine-tuning a pre-trained large language model on an end-cloud collaborative speech recognition task, and the large language model has the ability to determine the reliability of the speech recognition result. The training data set corresponding to the end-cloud collaborative speech recognition task includes training texts having a first reliability label indicating that the text is reliable, training texts having a second reliability label indicating that the text is unreliable, training text pairs with the same semantics, and training text pairs with different semantics, wherein the training text pairs with the same semantics have a hot word update label indicating that a hot word update does not need to be performed immediately, and the training text pairs with different semantics have a hot word update label indicating that a hot word update is required immediately; The target speech recognition result determination module is configured to, if the speech recognition result on the terminal side is reliable, determine the speech recognition result on the terminal side as the target speech recognition result; and if the speech recognition result on the terminal side is unreliable, after obtaining the speech recognition result on the cloud side, determine the speech recognition result on the cloud side as the target speech recognition result; The similarity acquisition module is configured to determine the similarity between the speech recognition result on the device and the speech recognition result on the cloud using the large language model when the speech recognition result on the device is unreliable, wherein the large language model is further capable of determining the similarity between two speech recognition results; The hot word update determination module is used to determine whether the hot word library on the terminal side needs to be updated immediately based on the similarity between the speech recognition result on the terminal side and the speech recognition result on the cloud side.

8. An electronic device, characterized in that: comprising at least one processor and a memory connected to the processor, wherein: The memory is used to store computer programs; The processor is used to execute the computer program so that the electronic device can implement the steps of the speech recognition method according to any one of claims 1 to 6.

9. A computer storage medium, characterized in that The storage medium carries one or more computer programs, and when the one or more computer programs are executed by an electronic device, the electronic device can implement the steps of the speech recognition method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The method comprises computer-readable instructions, which, when executed on an electronic device, enable the electronic device to implement the steps of the speech recognition method according to any one of claims 1 to 6.

Images