Display device, and speech processing method for display device

Abstract

A display device (200), and a speech processing method for the display device (200), which relate to the technical field of display devices (200), and can improve the accuracy of speech processing. The display device (200) comprises: a receiver, which is configured to receive a voice instruction which is input by a user (S310); a memory, which is configured to store a computer instruction and / or data associated with the display device (200); and at least one processor (250), which is coupled to the receiver and the memory, wherein the at least one processor (250) is configured to execute the computer instruction to enable the display device (200) to execute: in response to the speech instruction, recognizing the speech instruction to obtain a plurality of candidate recognition results corresponding to the speech instruction (S320), on the basis of the candidate recognition results, determining the number of entities corresponding to the candidate recognition results (S330), and when the number of entities is less than or equal to a threshold number, analyzing scenario information of the speech instruction by using a scenario decision-making module so as to determine from the plurality of candidate recognition results a target recognition result of the speech instruction (S340).

Description

Display device and voice processing method for display device

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims priority to Chinese patent applications filed on August 11, 2023, with application number 202311014455.4; and filed on September 4, 2023, with application number 202311130206.1, the entire contents of which are incorporated by reference into this application. Technical Field

[0003] The present application relates to the technical field of display devices, and in particular to a display device and a voice processing method for the display device. Background Art

[0004] With the development of voice recognition technology, voice interaction applications are becoming increasingly common. For example, in applications such as smart TVs, smart cars, smart homes, and smart robots, users can use the voice function provided by the display device to input voice commands. The display device then performs semantic analysis of the voice commands to identify the intended operation, thereby controlling the display device to perform corresponding operations based on the voice command, such as running an application or searching for content corresponding to the voice command.

[0005] However, due to the diversity of languages, some different characters may have the same pronunciation. For example, "yue bing" has corresponding characters such as "yue bing" (parade soldiers) and "yue bing" (moon cake). This can easily lead to confusion in homophone recognition during speech processing, sometimes misidentifying characters with the same pronunciation as the wrong one, affecting the accuracy of speech processing.

[0006] Summary of the Invention

[0007] According to some embodiments of the present application, a display device may include: a receiver, which may be configured to receive voice instructions input by a user; a memory, which may be configured to store computer instructions and / or data associated with the display device; at least one processor coupled to the receiver and the memory, and the at least one processor may be configured to execute the computer instructions to cause the display device to perform: in response to the voice instruction, recognize the voice instruction to obtain multiple candidate recognition results corresponding to the voice instruction; determine the number of entities corresponding to the candidate recognition results; and when the number of entities is less than or equal to a quantity threshold, use a scene decision module to analyze the scene information of the voice instruction to determine a target recognition result of the voice instruction among the multiple candidate recognition results.

[0008] According to some embodiments of the present application, a voice processing method for a display device may include: receiving a voice instruction input by a user; in response to the voice instruction, recognizing the voice instruction to obtain multiple candidate recognition results corresponding to the voice instruction; determining the number of entities corresponding to the candidate recognition results; and when the number of entities is less than or equal to a quantity threshold, using a scene decision module to analyze the scene information of the voice instruction to determine a target recognition result of the voice instruction among the multiple candidate recognition results. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] FIG1 is a schematic diagram of interaction between a display device and a control device according to some embodiments of the present application;

[0010] FIG2 is a block diagram of a hardware configuration of a control device according to some embodiments of the present application;

[0011] FIG3 is a block diagram of a hardware configuration of a display device according to some embodiments of the present application;

[0012] FIG4 is a schematic diagram of software configuration of a display device according to some embodiments of the present application;

[0013] FIG5 is a flowchart of a voice processing method for a display device according to some embodiments of the present application;

[0014] FIG6 is a structural diagram of a speech processing system according to some embodiments of the present application;

[0015] FIG7 is a flowchart of another voice processing method for a display device according to some embodiments of the present application;

[0016] FIG8 is a flowchart of another voice processing method for a display device according to some embodiments of the present application;

[0017] FIG9 is a schematic diagram of a knowledge graph according to some embodiments of the present application;

[0018] FIG10 is a flowchart of another voice processing method for a display device according to some embodiments of the present application;

[0019] FIG11 is a flowchart of another voice processing method for a display device according to some embodiments of the present application;

[0020] FIG12 is a flowchart of processing a voice command according to some embodiments of the present application;

[0021] FIG13 is a flowchart illustrating another method for processing a voice command according to some embodiments of the present application;

[0022] FIG14 is a flowchart of another voice processing method for a display device according to some embodiments of the present application;

[0023] FIG15 is a flowchart of yet another method for voice processing for a display device according to some embodiments of the present application;

[0024] FIG16 is a schematic diagram of a screenshot according to some embodiments of the present application;

[0025] FIG17 is a flowchart of another voice processing method for a display device according to some embodiments of the present application;

[0026] FIG18 is a flowchart of another voice processing method for a display device according to some embodiments of the present application;

[0027] FIG19 is a flowchart of another voice processing method for a display device according to some embodiments of the present application;

[0028] FIG20 is a flowchart of another voice processing method for a display device according to some embodiments of the present application;

[0029] FIG21 is a schematic diagram of a display interface of a display device according to some embodiments of the present application;

[0030] FIG22 is a flowchart of another voice processing method for a display device according to some embodiments of the present application. DETAILED DESCRIPTION

[0031] In order to make the purpose and implementation of this application clearer, the exemplary implementation of this application will be clearly and completely described below in conjunction with the drawings in the exemplary embodiments of this application. Obviously, the exemplary embodiments described are only part of the embodiments of this application, not all of the embodiments. It should be noted that the brief description of the terms in this application is only for the convenience of understanding the implementation described below, and is not intended to limit the implementation of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0032] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.

[0033] The terms "comprises" and "comprising" and any variations thereof in this application are intended to cover but not exclude inclusion. For example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed, but may include other components not expressly listed or inherent to such products or devices.

[0034] According to some embodiments of the present application, the display device and the voice processing method for the display device can be a device with voice processing function, such as a mobile phone, a desktop computer, a tablet computer, a wearable device, etc. The user can operate the display device through the voice function of the display device, or can operate the display device through a control device. Figure 1 is a schematic diagram of the interaction between a display device and a control device according to some embodiments of the present application. As shown in Figure 1, the user can operate the display device 200 through the smart terminal 300 or the control device 100.

[0035] In some embodiments, the control device 100 may be a remote control, and the remote control and the display device 200 may communicate via an infrared protocol or a Bluetooth protocol, or the remote control may control the display device 200 wirelessly or via other wired methods. A user may control the display device 200 by inputting user commands via buttons on the remote control, voice input, a control panel, etc. For example, a user may control the display device 200 by using the up and down keys on the remote control to switch the displayed pages, control the video playback on the display device 200 by using the play and pause keys, and control the display device 200 by inputting voice commands via the voice input key.

[0036] In some embodiments, the user may also use a smart device 300, such as a mobile terminal, tablet computer, computer, laptop computer, or other smart device, to control the display device 200. For example, the user may control the display device 200 through an application installed on the smart device 300, which may be configured to provide the user with various controls in an intuitive user interface on a screen associated with the smart device 300.

[0037] In some embodiments, the smart terminal 300 can connect and communicate with the software applications installed on the display device 200 via a network communication protocol, achieving one-to-one control operations and data communication. For example, the smart terminal 300 can establish a control command protocol with the display device 200, synchronize a remote control keyboard with the smart terminal 300, and control the display device 200 by controlling the user interface on the smart terminal 300. Alternatively, content displayed on the smart terminal 300 can be transmitted to the display device 200 for synchronized display.

[0038] As shown in FIG1 , the display device 200 and the server 400 can communicate data through a variety of communication methods, allowing the display device 200 to communicate through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 can provide various content and interactions to the display device 200. For example, the display device 200 receives software program updates or accesses a remotely stored digital media library by sending and receiving messages and interacting with an electronic program guide (EPG). The server 400 can be a cluster or multiple clusters, and can include one or more types of servers.

[0039] The display device 200 may be a liquid crystal display, an organic light-emitting diode (OLED) display, a projection electronic device, a smart terminal such as a mobile phone, a tablet computer, a smart TV, a laser projection device, an electronic table, etc. The specific display device type, size, and resolution are not limited.

[0040] FIG2 is a block diagram of a control device according to some embodiments of the present application. As shown in FIG2 , the control device 100 may include at least one processor 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 may receive user input commands and convert them into commands that the display device 200 can recognize and respond to, acting as an interactive intermediary between the user and the display device 200.

[0041] FIG3 is a block diagram of the hardware configuration of a display device according to some embodiments of the present application. As shown in FIG3 , the display device 200 includes: a tuner / demodulator 210, a communication device 220, a detector 230, an external device interface 240, at least one processor 250, a display 260, an audio output interface 270, a user interface 280, and at least one of a memory, a power supply, and a user interface.

[0042] In some embodiments, the tuner-demodulator 210 may receive broadcast television signals via wired or wireless reception, and demodulate audio and video signals, such as EPG data signals, from a plurality of wireless or wired broadcast television signals.

[0043] In some embodiments, the detector 230 may be used to collect signals from the external environment or from interactions with the external environment.

[0044] In some embodiments, the communication device 220 may be a component for communicating with an external device or an external server according to various communication protocols. For example, the communication device may include at least one of a wireless fidelity (WiFi) chip, a Bluetooth communication protocol chip, a wired Ethernet communication protocol chip, or other network communication protocol chip or a near-field communication protocol chip, as well as an infrared receiver.

[0045] In some embodiments, the detector 230 can be used to collect signals from the external environment or external interactions, and may include a light receiver and a temperature sensor. The light receiver can be used as a sensor to collect ambient light intensity and adaptively adjust display parameters based on the ambient light intensity. The temperature sensor can be used to sense the ambient temperature, allowing the display device 200 to adaptively adjust the display color temperature of the image. For example, when the ambient temperature is high, the color temperature of the image displayed by the display device 200 can be adjusted to a cooler color, or when the ambient temperature is low, the color temperature of the image displayed by the display device 200 can be adjusted to a warmer color.

[0046] In some embodiments, the detector 230 may also include an image collector, such as a camera, a camera, etc., which can be used to collect external environment scenes, as well as to collect user attributes or gestures for interacting with the user. It can adaptively change display parameters and recognize user gestures to realize the function of interacting with the user.

[0047] In some embodiments, the detector 230 may further include a sound collector, such as a microphone, for receiving the user's voice. For example, the detector 230 may include a voice signal of a user's control command for controlling the display device 200, or may collect ambient sound for identifying the type of ambient scene, so that the display device 200 can adaptively adapt to the ambient noise.

[0048] In some embodiments, the external device interface 240 may include but is not limited to the following: any one or more interfaces such as a High Definition Multimedia Interface (HDMI), an analog or digital high-definition component input interface, a composite video input interface, a Universal Serial Bus (USB) input interface, an RGB port, or a composite input / output interface formed by multiple interfaces mentioned above.

[0049] As shown in FIG3 , the processor may include a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a random access memory (RAM), a read-only memory (ROM), and at least one of first to nth interfaces for input / output. A communication bus connects the various components.

[0050] In some embodiments, at least one processor 250 can control the operation of the display device and respond to user operations through various software control programs / computer instructions stored in an external memory. For example, a user can input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface. Alternatively, a user can input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

[0051] In some embodiments, the display 260 may be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and may also be a projection device and a projection screen.

[0052] In some embodiments, the display 260 may be used to display a display interface of an application program, which is any page in the application program.

[0053] In some embodiments, the display 260 may be configured to receive audio and video signals output by the audio processor and the video processor, display video content and images, play audio of the video content, and display components of a menu control interface.

[0054] In some embodiments, the display 260 may be used to present a user operation UI interface generated in the display device 200 and used to control the display device 200 .

[0055] In some embodiments, the display device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the content providing device through the communication device 220 .

[0056] In some embodiments, the display device 200 can receive control operations input by the user on the display interface by adding a receiver. For example, when the receiver is a touch component, the touch component and the display 260 can form a touch screen. On the touch screen, the user can input different control commands through touch operations, such as the user can input touch commands such as click, slide, long press, double click, etc. Different touch commands can represent different control functions.

[0057] To implement the above-mentioned different touch actions, the touch component can generate different electrical signals when the user inputs different touch actions, and send the generated electrical signals to at least one processor 250. The at least one processor 250 can extract features from the received electrical signals and determine the control function to be executed by the user based on the extracted features.

[0058] In some embodiments, the receiver may also be an external control component connected to the display device 200. For example, in the case of a desktop computer, the receiver may be a mouse, keyboard, or the like connected to the display. A user may input various control commands, such as click, slide, and switch, via the mouse or keyboard. For example, when a user performs various control operations on the external control component, the external control component may generate various control signals in response to the user's control operations and transmit the generated control signals to the at least one processor 250. The at least one processor 250 may extract features from the received control signals and determine the control function the user intends to execute based on the extracted features.

[0059] In some embodiments, the receiver may also be an external control component connected to the display device 200. For example, in the case of a television, the receiver may be a remote control. The user may input various control commands through the remote control, such as click, switch, voice input, and other operation commands. When the user presses the voice button on the remote control, the voice processing function of the television may be activated, and the sound collector of the television may then collect the voice command input by the user, so that at least one processor may recognize and analyze the voice command to execute the operation corresponding to the voice command.

[0060] In some embodiments, the display device 200 can receive user-entered voice commands via a receiver. For example, when the receiver is a touch-sensitive component, the touch-sensitive component and the display of the display device 200 can together form a touch screen. The user can trigger the voice input function of the display device 200 by touching the touch screen to input the voice command.

[0061] In some embodiments, the receiver may also be an external control component, such as a mouse or remote control that establishes a communication connection with the display device. When the user performs different control operations on the external control component, the external control component may generate different control signals in response to the user's control operations and send the generated control signals to the display device 200.

[0062] For example, when the user presses the voice button on the remote control, the remote control can start the voice input function. During the process of the user inputting the voice command, the remote control can synchronize the voice command to the display of the display device 200. At this time, the display can display the voice input mark to indicate that the user is inputting the voice command.

[0063] In some embodiments, the receiver can also be a control component connected to the display device 200. Taking a desktop computer as an example, the control component can be a keyboard connected to the display. The user can input different control instructions through the keyboard, such as click, switch, record voice and other operation instructions. The user can input voice instructions through the corresponding shortcut keys. For example, when the user selects the "Ctrl" key and the "V" key, at least one processor 250 can receive a key signal and determine that the user has triggered a voice input operation. Then, the receiver can receive the voice instruction input by the user and control the display device 200 to perform the corresponding operation according to the voice instruction.

[0064] Figure 4 is a schematic diagram of the software configuration of the display device in some embodiments of the present application. Referring to Figure 4, the system can be divided into four layers, from top to bottom, namely the application layer (abbreviated as "application layer"), the kernel layer and the hardware layer.

[0065] In some embodiments, at least one application is running in the application layer. These applications may be window programs, system settings programs, or clock programs that come with the operating system; or they may be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples. For example, the application layer may include a voice processing application (such as a voice assistant). The display device 200 may start the voice processing application in response to the operation of the voice processing application to activate the voice processing function of the display device.

[0066] The kernel layer can serve as software middleware between the hardware layer and the application layer to manage and control hardware and software resources.

[0067] In some embodiments, the kernel layer may include a microphone driver and a display driver. Among them, the microphone driver can call the interface of the microphone to collect the voice commands input by the user through the microphone (i.e., the sound collector), or call the interface of the microphone to set the microphone. The display driver can call the interface of the display to obtain multimedia resources, or call the interface of the display to set the display.

[0068] Speech recognition technology is a technology in which a machine receives, recognizes, and understands speech signals and converts them into corresponding digital signals. Due to the vast territory of our country, there are various types of languages, and the languages used in each region are mostly dialects of their own regions. At the same time, with the widespread application of the Internet, many homophonic words will have different meanings. For example, for the pinyin "lu yao zhi ma li", the corresponding words may include "Lu Yao Zhi Ma Li" and "路遥知马力" (路遥知马力 means as distance tests a horse's strength). Among them, "Lu Yao Zhi Ma Li" is the name of a certain TV drama. In view of this situation, it is very difficult to optimize the speech recognition model from the recognition engine side, resulting in speech being easily recognized as incorrect text. Therefore, the accuracy of speech processing is not high.

[0069] FIG. 5 is a flowchart of a speech processing method for a display device according to some embodiments of the present application. The display device 200 may include a receiver (also known as a sound collector), a memory, and at least one processor coupled to the receiver and the memory. Among them, the receiver can be used to receive the voice commands input by the user, the memory can be used to store computer instructions and / or data associated with the display device, and at least one processor can analyze the voice commands and control the display device to perform corresponding operations.

[0070] According to the speech processing method for a display device according to some embodiments of the present application, the display device can respond to the voice commands input by the user, recognize the voice commands, obtain multiple candidate recognition results corresponding to the voice commands, determine the number of entities corresponding to the candidate recognition results, and when the number of entities is less than or equal to the number threshold, use the scenario decision module to analyze the scenario information of the voice commands to determine the target recognition result of the voice commands.

[0071] Applying this solution, when multiple candidate recognition results corresponding to the voice commands are obtained and the number of entities of the candidate recognition results does not exceed the number threshold, the scenario decision module can be used to analyze the scenario information of the voice commands, so as to clarify and disambiguate each candidate recognition result, and the accuracy of speech processing can be improved.

[0072] As shown in FIG. 5, at least one processing controller may be configured to execute the computer instructions stored in the memory to enable the display device to perform some or all of the following steps S310 to S340:

[0073] Step S310: receiving a voice command input by a user.

[0074] Voice commands are sound data captured by a display device. They can be voice data input by a user through the display device's voice input function, or through an external control component of the display device, such as a remote control or microphone. In some cases, voice commands can also be sound data captured by other means, such as voice data downloaded from the Internet.

[0075] In some embodiments, the voice command may include scene information, which may include the display device and application information when the voice command is received, the time information when the voice command is triggered, the regional location of the display device, etc.

[0076] Step S320: In response to the voice instruction, the voice instruction is recognized to obtain a plurality of candidate recognition results corresponding to the voice instruction.

[0077] The candidate recognition result refers to the recognition result listed as a candidate object.

[0078] When receiving a voice command input by a user, the display device can recognize the voice command, such as by using hidden Markov models, artificial neural networks, support vector machines, machine learning and other methods to recognize the voice command and obtain multiple candidate recognition results corresponding to the voice command.

[0079] In some embodiments, Figure 6 is a structural diagram of a speech processing system according to some embodiments of the present application. As shown in Figure 6, the speech processing system 500 may mainly include four components: feature extraction and signal processing, acoustic model, language model and decoding search part.

[0080] Among them, feature extraction and signal processing are mainly used to take the audio signal of the input voice command as input, enhance the voice by eliminating noise and channel distortion, convert the signal from the time domain to the frequency domain, and extract suitable representative feature vectors for the subsequent acoustic model.

[0081] The acoustic model is the underlying model of the speech processing system 500. Its purpose is to calculate the distance between the speech feature vector sequence and each pronunciation template. In other words, the smaller the difference between the extracted speech features and a particular pronunciation, the more likely it is that pronunciation. Specifically, the acoustic model integrates knowledge of acoustics and phonetics, using the features generated by the feature extraction component as input and generating an acoustic model score for the variable-length feature sequence.

[0082] A language model can learn the relationships between words from a training corpus to estimate the likelihood of a hypothetical word sequence, also known as the language model score. Decoding search combines the acoustic model score with the language model score, selecting word sequences with a higher overall output probability as candidate recognition results.

[0083] In some embodiments, word sequences with an output probability above a minimum probability threshold can be determined as candidate recognition results. That is, among the word sequences corresponding to the voice instruction determined by the voice processing system, each word sequence with an output probability above the minimum probability threshold can be determined as a candidate recognition result. This allows word sequences with excessively low output probabilities to be screened out, reducing the amount of data for subsequent analysis.

[0084] By recognizing the voice command and determining multiple candidate recognition results corresponding to the voice command, multiple text recognition results corresponding to the voice command can be determined, thereby achieving preliminary recognition of the voice command.

[0085] Step S330: Determine the number of entities corresponding to the candidate recognition results.

[0086] An entity is a word or phrase in a candidate recognition result that represents a specific entity, such as a thing, person, place, or time. For example, in the candidate recognition result "I'm going to Hong Kong tomorrow," "tomorrow" and "Hong Kong" are entities, representing time and place, respectively. Another example is the candidate recognition result "watch a movie" has only one entity word, "movie," while the candidate recognition result "Zhang San's movie" has both "Zhang San" and "movie."

[0087] When multiple candidate recognition results are obtained, the display device can determine the number of entities in each candidate recognition result, thereby obtaining the number of entities corresponding to these candidate recognition results. Generally speaking, the number of entities in each candidate recognition result is equal. For example, when the number of entities in multiple candidate recognition results is different, the largest number of entities value can be determined as the number of entities corresponding to the candidate recognition result.

[0088] Step S340: When the number of entities is less than or equal to the number threshold, the scene information of the voice instruction is analyzed using the scene decision module to determine a target recognition result of the voice instruction from multiple candidate recognition results.

[0089] Among them, the quantity threshold can be set to a default value, such as 1, or it can be customized by the developer. The target recognition result refers to the final recognition result of the voice command, and the target recognition result is one of multiple candidate recognition results. The scene decision module is a module that analyzes the scene and environment in which the voice command occurs. It can be used to analyze the user's geographical location, time information, spatial information, environment when the voice command occurs, as well as the device type and application when the voice command is triggered, in order to achieve optimal recognition performance.

[0090] When the number of entities is less than or equal to the quantity threshold, it means that there are fewer specific entity words in the candidate recognition results. In order to determine the target recognition result from the candidate recognition results, the scene decision module can be used to analyze the scene, environment, time, space and other information of the voice command to determine the target recognition result of the voice command from multiple candidate recognition results.

[0091] Through the above method, the scenario decision module can be used to analyze the occurrence scenario of the voice command, thereby clarifying and disambiguating each candidate recognition result and improving the accuracy of voice processing.

[0092] In some embodiments, each candidate recognition result can be output as an array, and the entity in the candidate recognition result can include the entity name, the entity's pinyin, and the output probability. Table 1 below is an example of a candidate recognition result, as shown below:

[0093] Table 1

[0094] Among them, "Shu Nian" is the name of a TV series.

[0095] When there are a large number of candidate recognition results, clarifying and disambiguating each candidate recognition result will increase the recognition time of the voice command. Therefore, to improve the efficiency of voice processing, in some embodiments, as shown in FIG7 , at least one processor may be further configured to execute computer instructions to cause the display device to perform some or all of the following steps:

[0096] Step S510: Determine the output probability difference between any two candidate recognition results among the plurality of candidate recognition results.

[0097] In some embodiments, the output probability difference between each two candidate recognition results among the plurality of candidate recognition results may be calculated, that is, the absolute value of the output probability difference between the two candidate recognition results may be calculated.

[0098] Step S520: When the minimum output probability difference among the multiple output probability differences is greater than the difference threshold, the candidate recognition result with the maximum output probability among the multiple candidate recognition results is determined as the target recognition result.

[0099] The difference threshold can be customized by the developer or set to a default value, such as 0.05 or 0.1. Among multiple output probability differences, if the minimum output probability difference is greater than the difference threshold, it means that the differences between the output probability differences are large, and the difference between the maximum output probability and the minimum output probability is even greater. In this case, the candidate recognition result with the highest output probability among the multiple candidate recognition results can be directly determined as the target recognition result.

[0100] For example, assuming there are three candidate recognition results, the output probability of the first candidate recognition result is 0.96, the output probability of the second candidate recognition result is 0.83, and the output probability of the third candidate recognition result is 0.72. The difference threshold is 0.05. After calculation, the output probability differences between the above three output probabilities are 0.13, 0.09, and 0.24, respectively. The minimum output probability difference is 0.09 < the difference threshold 0.05. The difference between any two candidate recognition results is large, so the first candidate recognition result with an output probability of 0.96 can be directly determined as the target recognition result.

[0101] Step S530: When the maximum output probability difference among multiple output probability differences is less than or equal to the difference threshold, execute steps S330 and S340, that is, determine the number of entities corresponding to the candidate recognition results. When the number of entities is less than or equal to the number threshold, use the scene decision module to analyze the scene information of the voice command to determine the target recognition result of the voice command among multiple candidate recognition results.

[0102] In some embodiments, among multiple output probability differences, if the maximum output probability difference is less than or equal to the difference threshold, it can be said that the differences between the output probability differences are not large, that is, the output probability differences between the candidate recognition results are not large. At this time, the number of entities corresponding to the candidate recognition results can be determined, and then when the number of entities is less than or equal to the number threshold, the scene information of the voice command can be analyzed using the scene decision module to determine the target recognition result from multiple candidate recognition results, thereby achieving the purpose of further recognizing the voice command.

[0103] Through the above method, it is possible to determine whether it is necessary to use the scene decision module to analyze the scene of the voice command based on the output probability differences between the candidate recognition results, so as to further clarify and disambiguate the candidate recognition results, thereby determining the target recognition result and improving the efficiency of voice analysis.

[0104] In some embodiments, as shown in FIG8 , the at least one processor may be further configured to execute computer instructions to cause the display device to perform some or all of the following steps:

[0105] Step S610: When the number of entities is greater than the quantity threshold, the knowledge graph database is used to determine the relationship between multiple entities in each candidate recognition result.

[0106] A knowledge graph is a semantic network that represents the relationships between entities. In this embodiment, the knowledge graph database can be a knowledge graph of homophones built by developers, which can be used for homophone disambiguation and decision-making in speech processing. The knowledge graph database can include multiple pieces of knowledge, each of which can be represented as a triple: [subject: theme] → predicate → [object].

[0107] When the number of entities is greater than the quantity threshold, each candidate recognition result can be clarified and disambiguated based on the association relationship between the entities, that is, the knowledge graph database is used to determine the relationship between multiple entities in each candidate recognition result.

[0108] Figure 9 is a schematic diagram of a knowledge graph according to some embodiments of the present application. As shown in Figure 9, according to the knowledge graph, it can be seen that "movie" has entity association relationships with "actor", "director" and "song", and "actor" plays "movie", "director" directs "movie", the theme song corresponding to "movie" is "song", and the singer of "song" is "singer".

[0109] As can be seen, the relationships between entities are complex and intricate. Knowledge graphs can be used to express the associations and relationships between entities, intuitively demonstrating whether two entities are directly related and the nature of the relationship. Knowledge graph databases contain a large amount of relationship data between entities, which can be used to analyze the correct relationships between entities in candidate recognition results.

[0110] Taking the movie "The Truth" as an example, the director is Zhao. Using the knowledge graph database, we can find the entity association data between the movie "The Truth" and "Zhao", but we cannot find the entity association data between the movie "The Truth" and "Li". Therefore, it can be determined that there is an association relationship between the movie "The Truth" and "Zhao".

[0111] Step S620: According to the relationship between multiple entities in each candidate recognition result, determine the candidate recognition result that meets the first output condition among the multiple candidate recognition results as the target recognition result.

[0112] The first output condition refers to the condition for selecting a target recognition result from multiple candidate recognition results, which may be the presence of a correct entity relationship among the candidate recognition results. In other words, when a candidate recognition result having a correct entity relationship is determined from the knowledge graph database, the candidate recognition result having the correct entity relationship can be determined to meet the first output condition.

[0113] For example, in the above example, if the candidate recognition result is "I want to watch Zhao's movie "The Truth"," then the candidate recognition result can be determined to be the target recognition result.

[0114] For another example, among multiple candidate recognition results "Search for the idiom story "A long journey will reveal a horse's strength" and "Search for the idiom story "Lu Yao knows Mali", if the correct relationship between the entities can be determined as "Idiom story: A long journey will reveal a horse's strength" based on the knowledge graph database, then the candidate recognition result "Search for the idiom story "A long journey will reveal a horse's strength" can be determined as the final target recognition result.

[0115] Through the above method, when the number of entities is greater than the quantity threshold, the knowledge graph database can be used to check and verify the relationship between each entity, so as to select the target recognition result with the correct entity relationship from each candidate recognition result.

[0116] Since the knowledge graph database may be imperfect, if the knowledge graph database cannot be used to determine the target recognition result from multiple candidate recognition results, in some embodiments, at least one processor may also be configured to: execute computer instructions to enable the display device to perform the following operations: if multiple candidate recognition results do not meet the first output condition, use the scene decision module to analyze the scene information of the voice instruction to determine the target recognition result of the voice instruction from the above-mentioned multiple candidate recognition results.

[0117] That is to say, when the correct entity relationship cannot be determined using the knowledge graph database and therefore the target recognition result cannot be determined, the display device can continue to use the scene decision module to analyze the scene information of the voice command and determine the target recognition result in the candidate recognition results that matches the scene corresponding to the voice command.

[0118] In some embodiments, the scene information of the voice instruction may include at least one of the first scene information, the second scene information and the third scene information, and the scene decision module may include at least one of the environment perception database, the space-time perception database and the dialect perception database.

[0119] The environment perception database is used to store the entity word library corresponding to the first scenario information. For example, it can store the device information entity word library and the application entity word library. The device information entity word library may include entity word libraries corresponding to different device types, and the application entity word library may include entity word libraries corresponding to different applications. The spatiotemporal perception database can be used to store the entity word library corresponding to the second scenario information. For example, it can store the time information entity word library, which may include the entity word library for weekdays, the entity word library for weekends, the entity word library for holidays, etc. It can also store the entity word library corresponding to regional locations, such as the entity word library for the southeast region or the entity word library corresponding to a certain city.

[0120] The dialect-aware database is used to store the entity word library corresponding to the third scene information, and may include dialect pronunciation information of various regions. For example, the dialect entity word library of each region can be stored, which can be used to represent the entity words corresponding to each pronunciation.

[0121] It should be understood that the environment perception database, the spatiotemporal perception database, and the dialect perception database can be divided into vocabulary according to the corresponding data dimensions, and the degree of refinement of the vocabulary division can also be flexibly set.

[0122] Therefore, referring to FIG10 , at least one processor may be further configured to execute computer instructions to cause the display device to perform some or all of the following steps:

[0123] Step S810: Match each candidate recognition result with the information in the information library corresponding to the first scene information in the environment perception database to obtain a first matching result for each candidate recognition result.

[0124] The first scenario information may include device information and / or application information corresponding to the voice command. Device information may indicate the type of display device that triggered the voice command, such as a TV, smartphone, in-vehicle terminal, or air conditioner. Application information refers to the application that triggered the voice command, such as a video application, audio application, or home appliance management program. The first matching result may indicate whether each candidate recognition result successfully matches the information in the information library corresponding to the first scenario information in the environmental perception database.

[0125] In order to determine the hardware environment that triggers the voice command, the controller can determine the device type and / or application corresponding to the voice command based on the first scene information, and then match each candidate recognition result with the database corresponding to the device type and / or application in the environmental perception database to determine the first matching result of each candidate recognition result.

[0126] In some embodiments, the display device can determine the type of device that triggered the voice command based on the device information corresponding to the voice command, and then match each candidate recognition result with the device entity vocabulary corresponding to the device type in the environmental perception database to determine whether there is an entity that matches a candidate recognition result in each device entity vocabulary. It can also determine the application that triggered the voice command based on the application information corresponding to the voice command, and then match each candidate recognition result with the application entity vocabulary corresponding to the application in the environmental perception database to determine whether there is an entity that matches a candidate recognition result in each application entity vocabulary. For example, when it is determined that the device information corresponding to the voice command corresponds to a television device, the display device can match each candidate recognition result with an entity in the television device entity vocabulary. When it is determined that a candidate recognition result matches any entity in the television device entity vocabulary, it is determined that the candidate recognition result matches successfully.

[0127] Step S820: Match each candidate recognition result with the information in the information library corresponding to the second scene information in the spatiotemporal perception database to obtain a second matching result for each candidate recognition result.

[0128] The second scenario information may include time information and / or regional location information corresponding to the voice command. Time information refers to the time when the voice command is triggered, which can be used to determine whether the voice command is triggered on a weekday, a weekend, a holiday, etc.

[0129] In some embodiments, in order to determine the time and location when the voice command is triggered, the display device can determine the time and / or regional location corresponding to the voice command based on the second scene information, and then search the corresponding time entity vocabulary and / or regional location entity vocabulary to see whether there is an entity that matches each candidate recognition result. For example, when it is determined that the time of voice triggering is a holiday, the entity words in the time entity vocabulary corresponding to the holiday can be matched with each candidate recognition result. When it is determined that any entity word matches a candidate recognition result, it means that the candidate recognition result has been matched successfully.

[0130] Step S830: Match each candidate recognition result with the information in the information library corresponding to the third scene information in the dialect perception database to obtain a third matching result for each candidate recognition result.

[0131] The third scenario information may include dialect information of the voice command. For example, the pronunciation tones vary from region to region in my country, such as the tone change rule of Shaanxi dialect: "1 tone changes to 3 tone, 2 tone remains unchanged, 3 tone changes to 4 tone, and 4 tone changes to 1 tone."

[0132] Based on the third scenario information corresponding to the voice command, the display device can determine the dialect type of the voice command, thereby matching entity words in the dialect-aware database of the corresponding region that have the same pronunciation as the entity words in each candidate recognition result to determine whether each candidate recognition result is a match. If there is a candidate recognition result that has the same pronunciation as any entity word in the dialect-aware database, and the two entity words also have the same character, then the candidate recognition result is considered a match.

[0133] Step S840: According to at least one of the first matching result, the second matching result, and the third matching result, a candidate recognition result satisfying a second output condition among the plurality of candidate recognition results is determined as a target recognition result.

[0134] The second output condition refers to the condition for determining the target recognition result based on the first matching result, the second matching result and the third matching result. When any one or more matching results are successfully matched, it can be determined that each candidate recognition result meets the second output condition, and the successfully matched candidate recognition result is the target recognition result.

[0135] Through the above method, the utterance scene of the voice command can be identified based on the environmental perception database, the spatiotemporal perception database and the dialect perception database, so as to select the corresponding target recognition result from each candidate recognition result.

[0136] It should be noted that the execution order of the above steps S810 to S830 is not fixed, and the above steps S810 to S830 can be executed in full or in part, which is not specifically limited in this embodiment.

[0137] In some embodiments, if it is determined in step S840 that there are multiple candidate recognition results that meet the second output condition, the candidate recognition result with the maximum output probability can be determined as the target recognition result based on the output probability of the candidate recognition results that meet the second output condition.

[0138] In some embodiments, at least one processor can also be configured to execute computer instructions to cause the display device to perform the following operations: when the above-mentioned multiple candidate recognition results do not meet the second output condition, the candidate recognition result with the highest output probability is determined as the target recognition result based on the output probability of each candidate recognition result.

[0139] For example, when it is determined that none of the candidate recognition results can be matched successfully, it is determined that multiple candidate recognition results do not meet the second output condition. At this time, based on the output probability of each candidate recognition result, the candidate recognition result corresponding to the maximum output probability can be determined as the target recognition result.

[0140] FIG11 is a flowchart of another voice processing method for a display device according to some embodiments of the present application. As shown in FIG11 , the method may include some or all of the following steps:

[0141] Step S901: receiving a voice command input by a user.

[0142] Step S902: In response to the voice instruction, the voice instruction is recognized to obtain a plurality of candidate recognition results corresponding to the voice instruction.

[0143] Step S903: Determine whether the candidate recognition result is a single entity. If not, execute step S904; if yes, execute step S907.

[0144] In some embodiments, if the candidate recognition result is not a single entity, it means that there are multiple entity words in the candidate recognition result. At this time, the candidate recognition result can be clarified and disambiguated based on the relationship between the entity words. Therefore, step S904 can be executed to use the knowledge graph database to determine the relationship between multiple entities in each candidate recognition result.

[0145] In other embodiments, if the candidate recognition result is a single entity, it means that there is only one entity word representing a person, place, etc. in the candidate recognition result. At this time, in order to clarify and disambiguate the entity word, step S907 can be executed to match the first scene information of the voice command with the environmental perception database to obtain the first matching result of each candidate recognition result.

[0146] Step S904: Use the knowledge graph database to determine the relationship between multiple entities in each candidate recognition result.

[0147] Step S905: Determine whether each candidate recognition result meets the first output condition. If yes, execute step S906; if not, execute step S907.

[0148] The first output condition may be that there is a correct entity relationship in the candidate recognition result.

[0149] In some embodiments, if a candidate recognition result satisfies the first output condition, it indicates that the candidate recognition result has a correct entity relationship, and thus step S906 can be executed to determine the candidate recognition result that satisfies the first output condition as the target recognition result. In other embodiments, if none of the candidate recognition results satisfies the first output condition, it indicates that none of the candidate recognition results has a correct entity relationship, and thus step S907 can be executed to match the first scene information of the voice command with the environment perception database to obtain a first matching result for each candidate recognition result.

[0150] Step S906: Determine the candidate recognition result that meets the first output condition among the multiple candidate recognition results as the target recognition result.

[0151] Step S907: Match each candidate recognition result with the information in the information library corresponding to the first scene information in the environment perception database to obtain a first matching result for each candidate recognition result.

[0152] The first scenario information may include device information and / or application information corresponding to the voice command.

[0153] In some embodiments, the type of device that triggered the voice command can be determined based on the device information corresponding to the voice command, and then each candidate recognition result can be matched with the information in the device information library in the environmental perception database to determine whether there is information in the device information library that matches a candidate recognition result. If so, it indicates that the candidate recognition result has successfully matched. Alternatively, the application that triggered the voice command can be determined based on the application information corresponding to the voice command, and then each candidate recognition result can be matched with the information in the application information library in the environmental perception database to determine whether there is information in the application information library that matches a candidate recognition result. If so, it indicates that the candidate recognition result has successfully matched.

[0154] Step S908: Determine whether the first matching result is a successful match. If yes, execute step S914; if no, execute step S909.

[0155] In some embodiments, if any candidate recognition result is determined to be a successful match based on the first matching result, step S914 may be executed to determine that the candidate recognition result meets the second output condition and is determined as the target recognition result. If all candidate recognition results are determined to be unmatched based on the first matching result, step S909 may be executed to further analyze the scenario conditions of each candidate recognition result.

[0156] Step S909: Match each candidate recognition result with the information in the information library corresponding to the second scene information in the spatiotemporal perception database to obtain a second matching result for each candidate recognition result.

[0157] The second scene information may include time information and / or area location information corresponding to the voice command.

[0158] In some embodiments, when the time that triggered the voice command is determined to be the National Day holiday based on the time information corresponding to the voice command, each candidate recognition result can be matched with the information in the time information library corresponding to the National Day holiday in the spatiotemporal perception database to determine whether there is information in the time information library corresponding to the National Day holiday that matches a candidate recognition result. If so, it means that the candidate recognition result has successfully matched. Alternatively, the regional location that triggered the voice command can be determined based on the regional location information corresponding to the voice command, and then each candidate recognition result can be matched with the information in the regional location information library in the spatiotemporal perception database to determine whether there is information in the regional location information library that matches a candidate recognition result. If so, it means that the candidate recognition result has successfully matched.

[0159] Step S910: Determine whether the second matching result is a successful match. If yes, execute step S914; if not, execute step S911.

[0160] In some embodiments, if any candidate recognition result is determined to be a successful match based on the second matching result, step S914 may be executed to determine that the candidate recognition result meets the second output condition and is determined to be the target recognition result. In other embodiments, if all candidate recognition results are determined to be unmatched based on the second matching result, step S911 may be continued to analyze the scenario conditions of each candidate recognition result.

[0161] Step S911: Match each candidate recognition result with the information in the information library corresponding to the third scene information in the dialect perception database to obtain a third matching result of each candidate recognition result.

[0162] Among them, the third scene information may include dialect information of the voice command, such as the pronunciation entity vocabulary of each region.

[0163] In some embodiments, based on the dialect type of the voice instruction, entity words with the same pronunciation as the entity words of each candidate recognition result in the dialect perception database of the corresponding region can be matched to determine whether each candidate recognition result is successfully matched.

[0164] Step S912: Determine whether the third matching result is a successful match. If yes, execute step S914; if no, execute step S913.

[0165] In some embodiments, if any candidate recognition result is determined to be a successful match based on the third matching result, step S914 may be executed to determine that the candidate recognition result meets the second output condition and is determined as the target recognition result. In other embodiments, if all candidate recognition results are determined to be unmatched based on the third matching result, step S913 may be continued to determine the candidate recognition result with the highest output probability as the target recognition result.

[0166] Step S913: Determine the candidate recognition result with the highest output probability as the target recognition result.

[0167] Step S914: According to at least one of the first matching result, the second matching result, and the third matching result, a candidate recognition result that meets the second output condition among the multiple candidate recognition results is determined as a target recognition result.

[0168] The second output condition refers to the condition for determining the target recognition result based on the first matching result, the second matching result and the third matching result. When any one or more matching results are successfully matched, it can be determined that each candidate recognition result meets the second output condition, and the successfully matched candidate recognition result is the target recognition result.

[0169] After determining the target recognition result, in order to control the display device to output feedback corresponding to the voice command, for example, referring to FIG12 , when the voice command is recognized as “play music <123>”, the display device can respond to the voice command by outputting a feedback voice message such as “OK, I’ll play music <123> for you right away” and start playing music <123>. That is, in some embodiments, the at least one processor can be further configured to execute computer instructions to cause the display device to perform the following operations: determine the operation intention corresponding to the voice command based on the target recognition result, and execute the control operation corresponding to the operation intention.

[0170] For example, as shown in reference figure 13, after the user inputs a voice command, the voice processing method for a display device in some embodiments of the present application can be used to recognize the voice command, determine the target recognition result corresponding to the voice command, and then use a natural language processing method to perform semantic understanding of the target recognition result to determine the operation intention corresponding to the voice command, such as searching for the feedback operation corresponding to the target recognition result in the instruction library, thereby controlling the display device to perform the feedback operation and outputting a feedback instruction so that the user can know the operation progress of the display device on the voice command.

[0171] Through the above method, the display device can receive the voice command input by the user, and based on the recognition of the voice command, control the display device to output feedback on the voice command, so that the user can achieve the purpose of voice control of the display device.

[0172] To further illustrate the voice processing method for a display device according to some embodiments of the present application, FIG14 is a flowchart of another voice processing method for a display device according to some embodiments of the present application. As shown in FIG14 , the method may include the following steps:

[0173] Step S1210: Receive a voice command input by the user.

[0174] Step S1220: Recognize the voice instruction to obtain a candidate recognition result corresponding to the voice instruction.

[0175] In some embodiments, a speech processing system may be used to recognize speech instructions, calculate the output probability of each word sequence corresponding to the speech instruction, and determine a word sequence with a higher output probability as a candidate recognition result.

[0176] Step S1230: Determine whether there are multiple candidate recognition results. If yes, go to step S1240; if not, go to step S1260.

[0177] In some embodiments, if there are multiple candidate recognition results, in order to determine the final target recognition result, step S1240 may be executed to input the multiple candidate recognition results into the scenario decision module for processing. In another embodiment, if there is only one candidate recognition result, it may be directly determined as the target recognition result, and step S1260 may be executed to input the target recognition result into the semantic understanding system to determine the operational intent corresponding to the voice command.

[0178] Step S1240: Input multiple candidate recognition results into the scene decision module for processing.

[0179] Specifically, the method shown in FIG. 11 in the embodiment of the present application can be used to process multiple candidate recognition results. To avoid repetition, it will not be described here.

[0180] Step S1250: Determine the target recognition result among the candidate recognition results.

[0181] According to the matching results output by the scene decision module, the target recognition result can be determined from the candidate recognition results.

[0182] Step S1260: Input the target recognition result into the semantic understanding system to determine the operation intention corresponding to the voice command.

[0183] Semantic understanding systems are artificial intelligence technologies designed to understand and interpret the meaning and intent of natural language text or speech input. Through lexical analysis, syntactic analysis, entity recognition, semantic parsing, and intent recognition, semantic understanding systems can convert target recognition results into semantic representations that computers can understand and manipulate.

[0184] After determining the target recognition result, the target recognition result can be input into the semantic understanding system to understand and interpret the meaning and operation intention of the voice command.

[0185] Step S1270: Execute the control operation corresponding to the operation intention.

[0186] After determining the operation intention corresponding to the voice command, the display device can execute the control operation corresponding to the operation intention, such as opening a page, opening or closing an application, performing a setting function, etc.

[0187] Through the above method, based on the recognition and analysis of the voice command, the operation intention corresponding to the voice command can be determined to control the display device to execute the control operation, enabling human-computer interaction in the form of voice, and supporting functions such as intelligent dialogue, information retrieval, knowledge Q&A, and device control.

[0188] In summary, according to the voice processing method for a display device in some embodiments of the present application, the voice command can be recognized, the candidate recognition result corresponding to the voice command can be determined, and based on the number of entities in the candidate recognition result, the scene decision module can analyze the scene information of the voice command, thereby clarifying and disambiguating each candidate recognition result, and improving the accuracy of voice processing.

[0189] As mentioned above, since there is no one-to-one correspondence between Chinese characters and pinyin, in the process of voice processing, it is easy to cause confusion in the recognition of homophonic characters, affecting the accuracy of voice processing. For example, for "Zhang Shan" and "Zhang Shan", the pronunciation is both "zhang shan", and it is easy to be confused during voice processing, resulting in incorrect recognition. On the other hand, for new and popular content, it is usually difficult to obtain correct recognition results during the voice processing process.

[0190] According to the display device of some embodiments of the present application and another voice processing method for a display device, the display device may further include a display. Still referring to FIG1 , when the display device 200 is in the on state, the display device 200 may also obtain a screenshot of the display 260 in response to a voice command input by the user. The display device 200 may recognize the text in the screenshot and perform pinyin conversion on the text in the screenshot to obtain the pinyin corresponding to the text in the screenshot. The display device 200 may perform voice recognition on the voice command to obtain a voice recognition result. The display device 200 may perform word segmentation processing and pinyin conversion processing on the voice recognition result to obtain a recognized word and the pinyin corresponding to the recognized word. The display device 200 may compare the pinyin corresponding to the recognized word with the pinyin corresponding to the text in the screenshot to correct the first voice recognition result based on the information reflected in the screenshot, thereby obtaining a more accurate second voice recognition result, improving the accuracy of the voice recognition of the display device, and providing a better user experience.

[0191] Specifically, when the target pinyin (i.e., the pinyin that matches the pinyin corresponding to the recognized word) exists in the pinyin corresponding to the text in the screenshot, the speech recognition result is corrected according to the text corresponding to the target pinyin to obtain the corrected speech recognition result. The display device 200 can control the display 260 to display the multimedia resources corresponding to the corrected speech recognition result. In this way, the problem of easy confusion in homophone recognition and the high misrecognition rate of new and hot content can be solved, thereby improving the accuracy of the speech processing of the display device and providing a better user experience.

[0192] Figure 15 is a flowchart of another method for voice processing for a display device according to some embodiments of the present application. As shown in Figure 15, at least one processor may be further configured to execute computer instructions stored in the memory to cause the display device to perform steps including but not limited to the following steps 1510 to 1540.

[0193] Step 1510: Acquire a screenshot in response to the voice command.

[0194] In some embodiments, when the display device activates the voice processing function, the user can input voice commands to control the operation of the display device through voice commands. For example, the user can search for multimedia resources through voice commands. The above-mentioned voice commands can be commands input by the user that reflect the user's operation intention. When the display device receives the voice command, in response to the voice command, the display device can take a screenshot of the user interface currently displayed, obtain a screenshot, and then use the information indicated by the screenshot to correct the voice recognition result of the voice command.

[0195] The screenshot may be the user interface displayed by the display device upon receiving a user's voice command. Specifically, when the display device receives the user's voice command, the display device takes a screenshot of the user interface currently displayed, thereby obtaining the screenshot. The user interface displayed by the display device upon receiving the user's voice command may include multimedia resource information recommended based on the user's viewing habits, as well as information about currently popular multimedia resources.

[0196] In some embodiments, the user interface displayed by the display device upon receiving a user-input voice command may be an interface switched by the user (e.g., a "recommendation" interface, a "TV series" interface, etc.). For example, after the display device is powered on, the display device's user interface may include title information such as "My," "Cable TV," "Children's," "Recommendation," "Investiture of the Gods," "VIP," "TV series," "Movies," "Variety shows," etc. At this point, the user switches the display device's user interface to the "recommendation" interface, which may include information about new and popular multimedia resources (e.g., movies, TV series, variety shows, etc.), such as "Investiture of the Gods," "Avatar: The Way of Water," "Mature Years," "Kuangbiao," "Let's Farm," etc. At this point, upon receiving the user-input voice command, the display device takes a screenshot of its currently displayed user interface (i.e., the "recommendation" interface) to obtain a screenshot. As shown in FIG16 , the screenshot 60 may include the aforementioned new and popular multimedia resource information, such as "Investiture of the Gods," "Avatar: The Way of Water," "Shadow," "Mature Years," "Kuangbiao," "Let's Farm," etc.

[0197] In some embodiments, before taking a screenshot in response to a voice instruction, the at least one processor may be further configured to execute computer instructions to cause the display device to execute: in response to a wake-up instruction, start a voice processing function of the display device.

[0198] In some embodiments, when the display device is powered on and the user uses the voice processing function of the display device, the user can wake up the voice assistant of the display device by inputting a wake-up command (such as the wake-up word "xiaoju").

[0199] The wake-up command can be used to activate the voice processing function of the display device, so that the user can control the operation of the display device through voice commands. In other words, the wake-up command can be used to wake up a voice processing application (such as a voice assistant) installed on the display device to put the voice processing application of the display device into operation.

[0200] In some embodiments, the wake-up instruction can be a triggering operation by the user on the voice button of an external control component (such as a remote control) of the display device, or a triggering operation by the user on the voice processing control on the user interface of the display device, or a voice wake-up instruction. For example, taking the voice wake-up instruction as an example, the user can wake up the voice assistant of the display device, that is, start the voice processing function of the display device, by using a wake-up word (such as "Xiaoju").

[0201] Step 1520, perform image recognition and pinyin conversion on the screenshot to obtain first mapping data, where the first mapping data includes at least one first keyword and at least one first pinyin, and one first pinyin corresponds to one first keyword.

[0202] Among them, the first keyword can be the keyword corresponding to the text content in the screenshot, or the keyword corresponding to the person information indicated by the face image (such as an actor) in the screenshot. By recognizing the screenshot, one first keyword or multiple first keywords can be obtained. For example, taking the screenshot shown in FIG. 16 as an example, the screenshot can include multiple first keywords, namely "Fengshen", "Avatar: The Way of Water", "Shadow", "Mature Years", "The Knockout", "Farmers' Field" and so on.

[0203] In some embodiments, since the text content in the screenshot can consist of one character or multiple characters, and the person information indicated by the face image (such as an actor) in the screenshot can also consist of one character or multiple characters, the first keyword can be one character, one word, or one sentence, that is, the first keyword can include one first character or multiple first characters. For example, when the first keyword includes one first character, the first keyword can be "Shadow". Another example is that when the first keyword includes multiple first characters, the first keyword can be "Fengshen", "Avatar: The Way of Water", "Mature Years", "The Knockout", "Farmers' Field".

[0204] In some embodiments, the first pinyin can be the pinyin text corresponding to the first keyword, that is, multiple first keywords can correspond to multiple first pinyin. When the first keyword includes a single first character, the first pinyin can be the pinyin corresponding to the first character included in the first keyword. For example, if the first keyword is "shadow", the first pinyin corresponding to this first keyword is "ying". When the first keyword includes multiple first characters, the first pinyin can include the third pinyin corresponding to each of the multiple first characters. For example, if the first keyword is "Fengshen", the first pinyin corresponding to this first keyword can include the third pinyin of the two first characters, that is, the first pinyin includes the pinyin "feng" corresponding to "Feng" and the pinyin "shen" corresponding to "Shen". Another example is that if the first keyword is "Avatar: The Way of Water", the first pinyin corresponding to this first keyword can include the third pinyin of six first characters, that is, the first pinyin includes the pinyin "a" corresponding to "A", the pinyin "fan" corresponding to "Fan", the pinyin "da" corresponding to "Da", the pinyin "shui" corresponding to "Water", the pinyin "zhi" corresponding to "Zhi", and the pinyin "dao" corresponding to "Dao".

[0205] In some embodiments, the first mapping data can be data reflecting the corresponding relationship between different first keywords and the first pinyin corresponding to the first keywords. That is, the first mapping data can include at least one first keyword, the first pinyin corresponding to each first keyword in the at least one first keyword, and the corresponding relationship between each first keyword and the first pinyin corresponding to the first keyword. For example, the first mapping data can be a first mapping table.

[0206] It should be noted that for the case where the first keyword includes multiple first characters, the multiple third pinyin corresponding to the multiple first characters included in the first mapping data can be arranged in a first order, where the first order can be the order indicated by the positions of the multiple first characters in the first keyword.

[0207] For example, taking the screenshot shown in Figure 16 as an example, the first mapping data (i.e., the first mapping table) is as shown in Table 2 below:

[0208] Table 2

[0209] Among them, the multiple third pinyin corresponding to the multiple first characters included in any first keyword in the first mapping table are arranged in the order indicated by the positions of the multiple characters in the first keyword. For example, the order of the pinyin "feng" corresponding to "Feng" and the pinyin "shen" corresponding to "Shen" is "feng", "shen". For example, the order of the pinyin "zhong" corresponding to "Plant", the pinyin "di" corresponding to "Land", and the pinyin "ba" corresponding to "Bar" is "zhong", "di", "ba".

[0210] In some embodiments, referring to FIG. 17 , the screenshot is subjected to image recognition and pinyin conversion to obtain first mapping data. At least one processor may be further configured to execute computer instructions to enable the display device to execute steps including but not limited to steps 1710 to 1730 below.

[0211] Step 1710: perform feature extraction on the screenshot to obtain image features corresponding to the screenshot.

[0212] In some embodiments, a Feature Pyramid Network (FPN) can be used to extract features from screenshots to obtain image features at different scales. The image features at different scales can include global and local information of the screenshots, which can help recognize text at different scales.

[0213] In some embodiments, to improve the accuracy of screenshot recognition, before performing feature extraction on the screenshot to obtain image features corresponding to the screenshot, the at least one processor may be further configured to execute computer instructions to cause the display device to perform: pre-processing on the screenshot, where the pre-processing may include grayscale processing, binarization processing, and smoothing processing. After the pre-processing of the screenshot, feature extraction may be performed on the pre-processed screenshot to obtain image features corresponding to the screenshot.

[0214] Step 1720 : Perform text detection on image features corresponding to the screenshot based on a text detection algorithm to determine at least one text region in the screenshot.

[0215] The text detection algorithm can be a connectionist text proposal network (CTPN) or an efficient and accurate scene text detection algorithm (EAST).

[0216] In some embodiments, based on a text detection algorithm, text detection is performed on the image features corresponding to the screenshot to determine at least one text area in the screenshot, which may include but is not limited to: performing text area detection on the screenshot to obtain text edge information of each text area in the screenshot; and determining the text box of each text area based on the text edge information of each text area.

[0217] In some embodiments, in order to further improve the accuracy of screenshot recognition, after determining the text box of each text area based on the text edge information of each text area, each text area is corrected according to the text box of each text area to obtain at least one corrected text area.

[0218] Step 1730: Perform text recognition on at least one text area in the screenshot based on a text recognition algorithm to obtain at least one first keyword.

[0219] The text recognition algorithm may be an optical character recognition (OCR) algorithm or a deep learning algorithm, such as a convolutional recurrent neural network (CRNN).

[0220] Step 1740: Perform pinyin conversion on at least one first keyword to obtain at least one first pinyin corresponding to the at least one first keyword.

[0221] The pinyin conversion may be a process of converting the text corresponding to the first keyword into the corresponding pinyin. It should be noted that the pinyin may include syllables and tones.

[0222] In some embodiments, a feature pyramid network is used to extract features from the screenshot, and then, based on the extracted image features, at least one first keyword can be identified from the screenshot with high accuracy. Furthermore, in conjunction with subsequent steps, by utilizing the characteristic that homophones have the same pronunciation, the first mapping data obtained from the screenshot can be used to correct the second keyword included in the first speech recognition result, thereby obtaining a more accurate second speech recognition result.

[0223] Step 1530: Perform voice recognition on the voice instruction to obtain a first voice recognition result, where the first voice recognition result includes the second keyword.

[0224] In some embodiments, upon receiving a voice command input by a user, the display device may perform voice recognition on the voice command in response to the voice command to obtain a first voice recognition result. The first voice recognition result may include text content corresponding to the voice command, that is, the first voice recognition result may include at least one second keyword included in the text content corresponding to the voice command.

[0225] It should be noted that step 1530 can be executed after step 1520, step 1530 can also be executed before step 1520, and step 1530 and step 1520 can also be executed simultaneously. The embodiment of the present application does not limit the execution order of step 1530 and step 1520.

[0226] In some embodiments, referring to FIG. 18 , in the step of performing voice recognition on a voice instruction to obtain a first voice recognition result, at least one processor may be specifically configured to execute computer instructions to enable the display device to execute including but not limited to the following steps 1810 to 1820 .

[0227] Step 1810: extract features from the voice command to obtain audio features.

[0228] In some embodiments, any one of the linear predictive cepstrum coefficients (LPCC), mel frequency cepstrum coefficients (MFCC), perceptual linear prediction parameter method (PLP) and mel scale filter bank (FBANK) can be used to extract audio feature information in the voice command.

[0229] In some embodiments, in order to improve the accuracy of voice processing, before performing feature extraction on the voice instructions and obtaining audio features, the at least one processor may also be configured to execute computer instructions to cause the display device to perform operations including but not limited to the following: preprocessing the voice instructions, wherein the preprocessing may include noise reduction processing; performing feature extraction on the preprocessed voice instructions and obtaining audio features.

[0230] Step 1820: Process the audio features using the target acoustic model and the target language model to obtain a first speech recognition result.

[0231] In some embodiments, the audio feature information can be converted into text using a target acoustic model, and the converted text can be contextually processed and error-corrected using a target language model to obtain a first speech recognition result. The target acoustic model can be a Hidden Markov Model (HMM), a deep learning model (such as a recurrent neural network), or the like.

[0232] In some embodiments, a speech recognition algorithm may be used to perform speech recognition on the speech instruction to obtain a first speech recognition result. The speech recognition algorithm may be a speech recognition algorithm based on linguistics and acoustics, a stochastic model method, a speech recognition algorithm based on an artificial neural network, or a probabilistic grammar analysis.

[0233] In some embodiments, as shown in FIG. 19, after performing speech recognition on a voice command to obtain a first speech recognition result, at least one processor may further be configured to execute computer instructions to cause a display device to perform steps 1910 to 1920 including but not limited to the following.

[0234] Step 1910: Perform word segmentation on the first speech recognition result to obtain a second keyword.

[0235] Performing word segmentation on the first speech recognition result can divide the first speech recognition result into one or more groups of characters, that is, at least one second keyword is obtained. Among them, the second keyword may be a keyword included in the text content corresponding to the first speech recognition result. Performing word segmentation on the first speech recognition result may obtain one second keyword or multiple second keywords. For example, taking the first speech recognition result as "I want to see Fengshen", this first speech recognition result can be divided into three second keywords: "I", "want to see", and "Fengshen".

[0236] In some embodiments, the first speech recognition result (i.e., the voice command) may be composed of one character or multiple characters, that is, the first speech recognition result (i.e., the voice command) may be a single character, a word, or a sentence. Therefore, the second keyword obtained based on the first speech recognition result may include one second character or multiple second characters, that is, any second keyword may be a single character or a word. For example, when the second keyword includes one second character, the second keyword may be "I". Another example is that when the second keyword includes multiple second characters, the second keyword may be "want to see" and "Fengshen".

[0237] Step 1920: Perform pinyin conversion on the second keyword to obtain a second pinyin corresponding to the second keyword.

[0238] The second pinyin may be the pinyin text corresponding to the second keyword. In some embodiments, when the second keyword includes one second character, the second pinyin may be the pinyin corresponding to the second character included in the second keyword. For example, if the second keyword is "I", the second pinyin corresponding to this second keyword is "wo".

[0239] In some embodiments, when the second keyword includes multiple second characters, the second pinyin may include the fourth pinyin corresponding to each of the multiple second characters. Among them, the multiple fourth pinyins corresponding to the multiple second characters may be arranged in a second order, and a delimiter is used to separate adjacent two fourth pinyins. The second order is the order indicated by the positions of the multiple second characters in the second keyword. For example, when the second keyword is "Fengshen", the second pinyin corresponding to this second keyword includes the fourth pinyins of two second characters, that is, the second pinyin includes the pinyin "feng" corresponding to "风" and the pinyin "shen" corresponding to "神", that is, the second pinyin is "feng, shen".

[0240] It should be noted that the delimiter used to distinguish adjacent two pinyins may also be other symbols, such as "·", " / ", etc. The specific type of the delimiter is not limited in the embodiments of the present application.

[0241] In this embodiment, feature extraction is performed on the voice command to obtain an audio feature. Then, the audio feature is processed by a target acoustic model and a target language model, and a first recognition result can be obtained, which has high accuracy. Furthermore, word segmentation processing is performed on the first voice recognition result to obtain a second keyword, and pinyin conversion is performed on the second keyword to obtain the second pinyin corresponding to the second keyword. In this way, in combination with subsequent steps, using the characteristic that homophones have the same pronunciation, the second pinyin corresponding to each second keyword is compared with the pinyin in the first mapping data to correct the first voice recognition result by using the information included in the screen shot, and then a more accurate second voice recognition result is obtained.

[0242] In this embodiment, after performing pinyin conversion on the second keyword to obtain the second pinyin corresponding to the second keyword, second mapping data is determined according to the second pinyin corresponding to the second keyword.

[0243] The second mapping data may be data reflecting the correspondence between different second keywords and the second pinyins corresponding to the second keywords. That is, the second mapping data may include at least one second keyword, the second pinyins corresponding to each second keyword in at least one second keyword, and the correspondence between each second keyword and the second pinyin corresponding to the second keyword. Exemplarily, the second mapping data may be a second mapping table.

[0244] For example, continuing with the first voice recognition result being "I want to see Fengshen" as an example, the second mapping data (that is, the second mapping table) is shown in Table 3 below:

[0245] Table 3

[0246] Step 1540, when there is a target pinyin in the first mapping data, replace the second keyword in the first speech recognition result with the target keyword corresponding to the target pinyin, to obtain a second speech recognition result, where the target pinyin is a pinyin in at least one first pinyin that matches the second pinyin corresponding to the second keyword.

[0247] In some embodiments, after obtaining the second mapping data according to the first speech recognition result, the second pinyin corresponding to each second keyword in the second mapping data may be compared with the pinyin corresponding to each first character in each first keyword in the first mapping data (i.e., the first pinyin or the third pinyin), to determine whether there is a target pinyin in the first mapping data. When there is a target pinyin in the first mapping data, replace the second keyword in the first speech recognition result with the target keyword corresponding to the target pinyin, to obtain a corrected second speech recognition result.

[0248] The above-mentioned target pinyin is a pinyin in the first mapping data that matches the second pinyin corresponding to the second keyword. In some embodiments, when the second keyword includes one second character, the target pinyin is a pinyin in the first mapping data that is the same as the second pinyin corresponding to the second keyword. For example, if the second keyword is "樱" and the second pinyin corresponding to the second keyword is "ying", the target pinyin is the pinyin "ying" corresponding to "影" in the first mapping data.

[0249] In other embodiments, when the second keyword includes multiple second characters, each second character corresponds to a fourth pinyin, and the target pinyin may include pinyins that are the same as the fourth pinyins corresponding to each second character. That is, the target pinyin may include multiple third pinyins corresponding to multiple fourth pinyins, and the sorting of the multiple third pinyins corresponding to the multiple fourth pinyins is the same as the sorting of the multiple fourth pinyins corresponding to the multiple second characters. For example, if the second keyword is "风神" and the second pinyin corresponding to the second keyword is "feng, shen", the target pinyin is two consecutive pinyins "feng" and "shen" in the first mapping data, and "feng" is in front of "shen" in order.

[0250] In some embodiments, when the second keyword includes one second character, to determine whether there is a target pinyin in the first mapping data, at least one processor may be further configured to execute computer instructions to cause the display device to execute: when there is a pinyin in the first mapping data that is the same as the second pinyin corresponding to the second keyword (i.e., the fourth pinyin corresponding to the second character), determine that there is a target pinyin in the first mapping data. Where the target pinyin is a pinyin that is the same as the second pinyin corresponding to the second keyword.

[0251] In some embodiments, the second keyword may include multiple second characters, and the second pinyin may include multiple fourth pinyins arranged in a second order, the multiple fourth pinyins corresponding one-to-one to the multiple second characters, and the second order is the order indicated by the positions of the multiple second characters in the second keyword. To determine whether the target pinyin exists in the first mapping data, the at least one processor may be further configured to execute computer instructions to cause the display device to execute: if multiple third pinyins corresponding to the multiple fourth pinyins exist in the first mapping data, and the order of the multiple third pinyins is consistent with the order of the multiple fourth pinyins, determine that the target pinyin exists in the first mapping data, and the target pinyin includes the multiple third pinyins corresponding to the multiple fourth pinyins.

[0252] In this embodiment, when the second keyword includes multiple second characters, it can be determined whether the first mapping data includes the pinyin corresponding to each second character, and it is also necessary to determine whether the order of the pinyin corresponding to each second character included in the first mapping data meets the requirements, thereby further improving the accuracy of the second speech recognition result.

[0253] For example, if the first mapping data is the first mapping table (i.e., Table 2) and the first speech recognition result is "I want to see Fengshen" (i.e., the second mapping table), as shown in Table 3 above, the first speech recognition result may include three second keywords: "I", "I want to see", and "Fengshen", where the pinyin corresponding to "I" is "wo", the pinyin corresponding to "I want to see" is "xiang, kan", and the pinyin corresponding to "Fengshen" is "feng, shen".

[0254] For each of the above second keywords, the first mapping data is checked to see if there is a target pinyin that matches the second pinyin corresponding to the second keyword. In other words, the pinyin corresponding to "wo" (I) is compared with the pinyin corresponding to each first character in the first mapping table to determine whether the pinyin "wo" exists in the first mapping data (i.e., the same pinyin as the pinyin "wo" corresponding to "I"). Referring to Table 2 above, the pinyin "wo" (i.e., the same pinyin as the pinyin "wo" corresponding to "I") does not exist in the first mapping data.

[0255] After that, the next second keyword is detected. Taking the next second keyword "want to watch" as an example, the pinyin "xiang" in "xiang, kan" corresponding to "want to watch" is compared with the pinyin of each first character in the first mapping table to determine whether there is the pinyin "xiang" in the first mapping data (that is, the same pinyin as the pinyin "xiang" corresponding to "want"). Referring to Table 2 above, it can be determined that there is no such pinyin "xiang" in the first mapping data (that is, the same pinyin as the pinyin "xiang" corresponding to "want"). After that, the pinyin "kan" in "xiang, kan" corresponding to "want to watch" is compared with the pinyin of each first character in the first mapping table to determine whether there is the pinyin "kan" in the first mapping data (that is, the same pinyin as the pinyin "kan" corresponding to "watch"). Referring to Table 2 above, it can be determined that there is no such pinyin "kan" in the first mapping data (that is, the same pinyin as the pinyin "kan" corresponding to "watch").

[0256] After that, continue to detect the next second keyword. Compare the "feng" in the pinyin "feng, shen" corresponding to "Fengshen" with the pinyin of each first character in the first mapping table to determine whether there is the pinyin "feng" in the first mapping data (that is, the same pinyin as the pinyin "feng" corresponding to "wind"). Referring to Table 2 above, it can be determined that there is the pinyin "feng" in the first mapping data (that is, the same pinyin as the pinyin "feng" corresponding to "wind"). After that, compare the "shen" in the pinyin "feng, shen" corresponding to "Fengshen" with the pinyin of each first character in the first mapping table to determine whether there is the pinyin "shen" in the first mapping data (that is, the same pinyin as the pinyin "shen" corresponding to "god"). Referring to Table 2 above, it can be determined that there is the pinyin "shen" in the first mapping data (that is, the same pinyin as the pinyin "shen" corresponding to "god"). After that, determine whether "feng" and "shen" are two consecutive pinyins in the first mapping data, and whether the order of "feng" is before "shen". When it is determined that "feng" and "shen" are two consecutive pinyins in the first mapping data and the order of "feng" is before "shen", determine that there is a target pinyin in the first mapping data that matches the second pinyin corresponding to the second keyword "Fengshen", that is, the target pinyin includes "feng" and "shen". At this time, replace the "Fengshen" in the first speech recognition result "I want to watch Fengshen" with the target keyword corresponding to the target pinyin, that is, replace the "Fengshen" in the first speech recognition result "I want to watch Fengshen" with "Feng Shen", and then obtain the second speech recognition result "I want to watch Feng Shen". After traversing all the second keywords included in the first speech recognition result, the error correction process of the first speech recognition result is completed, that is, the second speech recognition result "I want to watch Feng Shen" is obtained.

[0257] In some embodiments, as shown in FIG. 20, after replacing the second keyword in the first speech recognition result with the target keyword corresponding to the target pinyin to obtain the second speech recognition result, the at least one processor may further be configured to execute computer instructions to cause the display device to perform steps 2010 to 2020 including but not limited to the following.

[0258] Step 2010, determine the multimedia resource corresponding to the second speech recognition result according to the second speech recognition result.

[0259] Step 2020, control the display to display the multimedia resource corresponding to the second speech recognition result.

[0260] For example, referring to Figure 21, taking the second voice recognition result of "I want to watch Fengshen" as an example, the display device can obtain multimedia resources related to "Fengshen" and control the display to display multimedia resources related to "Fengshen", such as "Fengshen Works", "Fengshen Romance", etc.

[0261] In this embodiment, after the first voice recognition result is corrected based on the information corresponding to the screenshot to obtain a second voice recognition result, the multimedia resource corresponding to the second voice recognition result can be displayed to the user, allowing the user to select the multimedia resource to be viewed, making it more convenient to use. Furthermore, based on the corrected second voice recognition result, the multimedia resource corresponding to the second voice recognition result is obtained and displayed, allowing the display device to accurately execute the operation intended by the voice instruction, providing a better user experience.

[0262] In some embodiments, after performing voice recognition on the voice instruction and obtaining a first voice recognition result, the at least one processor can also be configured to execute computer instructions to enable the display device to perform: when the target pinyin does not exist in the first mapping data, determine the multimedia resources corresponding to the first voice recognition result based on the first voice recognition result; and control the display to display the multimedia resources corresponding to the first voice recognition result.

[0263] In this embodiment, when the target pinyin does not exist in the first mapping data, the multimedia resources corresponding to the first speech recognition result can be displayed to the user, which is more convenient to use.

[0264] In some embodiments, since the user interface displayed by the display device may include multimedia resources recommended based on the user's usage habits, as well as new and popular multimedia resources, a screenshot can be obtained when a voice command input by the user is received, so that the display device can perform image recognition and pinyin conversion on the screenshot to obtain first mapping data reflecting the correspondence between at least one first keyword and the first pinyin corresponding to each first keyword. The display device can determine the first voice recognition result based on the voice command, and use the first mapping data obtained based on the screenshot to correct the second keyword included in the first voice recognition result to obtain a more accurate second voice recognition result. In this way, the problem of easy confusion in homophone recognition and the high misrecognition rate of new and popular content can be solved, thereby improving the accuracy of the voice recognition of the display device and providing a better user experience.

[0265] As a specific embodiment, as shown in FIG22 , another voice processing method for a display device according to some embodiments of the present application may include steps 2201 to 2215 .

[0266] Step 2201: Receive a voice command.

[0267] Step 2202: Acquire a screenshot in response to the voice command.

[0268] Step 2203: perform feature extraction on the screenshot to obtain image features corresponding to the screenshot.

[0269] Step 2204 : Perform text detection on the image features corresponding to the screenshot based on a text detection algorithm to determine at least one text region in the screenshot.

[0270] Step 2205: Based on a text recognition algorithm, perform text recognition on at least one text area in the screenshot to obtain at least one first keyword.

[0271] Step 2206: convert the at least one first keyword into pinyin to obtain at least one first pinyin corresponding to the at least one first keyword, that is, obtain first mapping data.

[0272] Step 2207: extract features from the voice command to obtain audio features.

[0273] Step 2208: Process the audio features using the target acoustic model and the target language model to obtain a first speech recognition result.

[0274] Step 2209: perform word segmentation processing on the first speech recognition result to obtain multiple second keywords.

[0275] Step 2210 , performing pinyin conversion on each of the plurality of second keywords to obtain a plurality of second pinyins corresponding to the plurality of second keywords.

[0276] Step 2211: Select the current keyword one by one from multiple second keywords.

[0277] Step 2212 , determine whether there is a target pinyin matching the second pinyin corresponding to the current keyword in the first mapping data; if so, execute step 2213 ; otherwise, execute step 2214 .

[0278] Step 2213: Replace the current keyword in the first speech recognition result with the target keyword corresponding to the target pinyin to obtain a second speech recognition result.

[0279] Step 2214 , determining whether the multiple second keywords in the first speech recognition result have been traversed; if so, executing step 2215 ; otherwise, returning to step 2211 .

[0280] Step 2215: Acquire and display the multimedia resources corresponding to the second speech recognition result.

[0281] Since the user interface displayed by the display device may include multimedia resources recommended based on the user's usage habits, as well as new and popular multimedia resources, a screenshot can be obtained when a voice command input by the user is received, so that the display device can perform image recognition and pinyin conversion on the screenshot to obtain first mapping data reflecting the correspondence between at least one first keyword and the first pinyin corresponding to each first keyword. At the same time, the display device can determine the first voice recognition result based on the voice command, and use the first mapping data obtained based on the screenshot to correct the second keyword included in the first voice recognition result to obtain a more accurate second voice recognition result. In other words, the voice processing method provided in the embodiment of the present application can solve the problem of easy confusion in homophone recognition and the high misrecognition rate of new and popular content, thereby improving the accuracy of the voice processing of the display device and providing a better user experience.

[0282] According to some embodiments of the present application, a voice processing device for a display device may include: a receiving module for receiving a voice instruction input by a user; a first recognition module for recognizing the voice instruction in response to the voice instruction to obtain multiple candidate recognition results corresponding to the voice instruction; a determination module for determining the number of entities corresponding to the candidate recognition results; and a second recognition module for analyzing the scene information of the voice instruction using a scene decision module when the number of entities is less than or equal to a quantity threshold, so as to determine a target recognition result of the voice instruction among multiple candidate recognition results.

[0283] According to some embodiments of the present application, another voice processing device for a display device may include: an acquisition module for acquiring a screenshot in response to a voice instruction; a processing module for performing image recognition and pinyin conversion on the screenshot to obtain first mapping data, the first mapping data may include at least one first keyword and at least one first pinyin, one first pinyin corresponding to one first keyword; a voice recognition module for performing voice recognition on the voice instruction to obtain a first voice recognition result, the first voice recognition result may include a second keyword; a voice recognition correction module for replacing the second keyword in the first voice recognition result with a target keyword corresponding to the target pinyin when a target pinyin exists in the first mapping data, to obtain a second voice recognition result, wherein the target pinyin is a pinyin that matches the second pinyin corresponding to the second keyword in the at least one first pinyin.

[0284] The specific details of each part of the above-mentioned voice processing device have been described in detail in the above-mentioned display device implementation. For undisclosed details, please refer to the implementation content of the display device part, and thus will not be repeated here.

[0285] According to a computer-readable non-volatile storage medium of some embodiments of the present application, the storage medium stores at least one executable instruction, which, when running on a display device / speech processing device, causes the display device / speech recognition device to perform the speech processing method in any of the above-mentioned method embodiments. The executable instruction can be specifically used to cause the display device / speech processing device to perform the above-mentioned speech processing method. In this embodiment, the computer-readable non-volatile storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it. More specific examples (non-exhaustive list) of computer-readable non-volatile storage media can include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0286] According to some embodiments of the present application, a computer program product may include a computer program that, when executed on a computer, causes the computer to implement the aforementioned speech processing method for a display device. It should be noted that the algorithms or displays provided herein are not inherently related to any particular computer, virtual system, or other device. Furthermore, the embodiments of the present application are not specific to any particular programming language.

[0287] For ease of explanation, the above description has been made in conjunction with specific embodiments. However, the above discussion of some embodiments is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Based on the above teachings, various modifications and variations can be obtained. The selection and description of the above embodiments are intended to better explain the principles and practical applications, so that those skilled in the art can better use the embodiments and various different variations of the embodiments suitable for specific use considerations.

Claims

1. A display device, comprising: A receiver configured to receive a voice command input by a user; a memory configured to store computer instructions and / or data associated with the display device; At least one processor coupled to the receiver and the memory, the at least one processor being configured to execute the computer instructions to cause the display device to perform: In response to the voice instruction, the voice instruction is recognized to obtain a plurality of candidate recognition results corresponding to the voice instruction; Determining the number of entities corresponding to the candidate recognition results; When the number of entities is less than or equal to the number threshold, the scene information of the voice instruction is analyzed using a scene decision module to determine a target recognition result of the voice instruction from among the multiple candidate recognition results.

2. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: When the number of entities is greater than the number threshold, using the knowledge graph database to determine the relationship between the multiple entities in each of the candidate recognition results; According to the relationship between the multiple entities in each of the candidate recognition results, a candidate recognition result that meets the first output condition among the multiple candidate recognition results is determined as the target recognition result.

3. The display device of claim 2, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: If the multiple candidate recognition results do not meet the first output condition, the scene information of the voice instruction is analyzed by using the scene decision module to determine the target recognition result of the voice instruction from the multiple candidate recognition results.

4. The display device according to any one of claims 1 to 3, wherein the scene information of the voice command comprises at least one of first scene information, second scene information and third scene information, the scene decision module comprises at least one of an environment perception database, a spatiotemporal perception database and a dialect perception database, and the at least one processor is configured to execute the computer instruction to enable the display device to execute: Matching each of the candidate recognition results with information in an information library corresponding to the first scene information in the environment perception database to obtain a first matching result for each of the candidate recognition results, wherein the first scene information includes device information and / or application information corresponding to the voice command; and / or, Matching each of the candidate recognition results with information in an information library corresponding to the second scene information in the spatiotemporal perception database to obtain a second matching result for each of the candidate recognition results, wherein the second scene information includes time information and / or regional location information corresponding to the voice command; and / or, Matching each of the candidate recognition results with information in an information base corresponding to the third scene information in the dialect perception database to obtain a third matching result for each of the candidate recognition results, wherein the third scene information includes dialect information of the voice command; According to at least one of the first matching result, the second matching result and the third matching result, a candidate recognition result among the plurality of candidate recognition results that meets a second output condition is determined as the target recognition result.

5. The display device of claim 4, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: When the plurality of candidate recognition results do not satisfy the second output condition, the candidate recognition result with the highest output probability is determined as the target recognition result according to the output probability of each of the candidate recognition results.

6. The display device according to any one of claims 1 to 3, wherein the at least one processor of the controller is further configured to execute the computer instructions to enable the display device to execute: Determine an output probability difference between any two candidate recognition results among the multiple candidate recognition results; When the minimum output probability difference among the plurality of output probability differences is greater than the difference threshold, determining the candidate recognition result with the maximum output probability among the plurality of candidate recognition results as the target recognition result; When the maximum output probability difference among the multiple output probability differences is less than or equal to the difference threshold, the number of entities corresponding to the candidate recognition results is determined; when the number of entities is less than or equal to the number threshold, the scene information of the voice command is analyzed using the scene decision module to determine the target recognition result of the voice command among the multiple candidate recognition results.

7. The display device according to any one of claims 1 to 3, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: The operation intention corresponding to the voice instruction is determined according to the target recognition result to execute the control operation corresponding to the operation intention.

8. The display device according to claim 1, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: In response to a voice command, taking a screenshot; Performing image recognition and pinyin conversion on the screenshot to obtain first mapping data, wherein the first mapping data includes at least one first keyword and at least one first pinyin, and one first pinyin corresponds to one first keyword; Performing voice recognition on the voice instruction to obtain a first voice recognition result, wherein the first voice recognition result includes a second keyword; When the target pinyin exists in the first mapping data, the second keyword in the first speech recognition result is replaced with the target keyword corresponding to the target pinyin to obtain a second speech recognition result, wherein the target pinyin is a pinyin that matches the second pinyin corresponding to the second keyword in at least one of the first pinyins.

9. The display device according to claim 8, wherein the first keyword includes a plurality of first characters, the first pinyin includes a plurality of third pinyins arranged in a first order, the plurality of third pinyins correspond to the plurality of first characters one by one, respectively, and the first order is an order indicated by positions of the plurality of first characters in the first keyword; The second keyword includes a plurality of second characters, the second pinyin includes a plurality of fourth pinyins arranged in a second order, the plurality of fourth pinyins correspond one-to-one to the plurality of second characters, and the second order is the order indicated by the positions of the plurality of second characters in the second keyword; The at least one processor is further configured to execute the computer instructions to cause the display device to perform: When there are multiple third pinyins corresponding to the multiple fourth pinyins in the first mapping data, and the order of the multiple third pinyins is consistent with the order of the multiple fourth pinyins, it is determined that there is a target pinyin in the first mapping data, and the target pinyin includes the multiple third pinyins corresponding to the multiple fourth pinyins.

10. The display device according to claim 8, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Performing feature extraction on the screenshot to obtain image features corresponding to the screenshot; Based on a text detection algorithm, performing text detection on image features corresponding to the screenshot to determine at least one text area in the screenshot; Based on a text recognition algorithm, performing text recognition on at least one text area in the screenshot to obtain the at least one first keyword; The at least one first keyword is converted into pinyin to obtain the at least one first pinyin corresponding to the at least one first keyword.

11. The display device according to claim 8, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Extracting features from the voice command to obtain audio features; The audio features are processed by a target acoustic model and a target language model to obtain the first speech recognition result.

12. The display device according to claim 8, wherein the at least one processor is further configured to execute the computer instructions to cause the display device to perform: Performing word segmentation processing on the first speech recognition result to obtain the second keyword; The second keyword is converted into pinyin to obtain a second pinyin corresponding to the second keyword.

13. The display device according to any one of claims 8 to 12, further comprising a display, wherein the at least one processor is further configured to execute the computer instructions to enable the display device to execute: Determining, according to the second speech recognition result, a multimedia resource corresponding to the second speech recognition result; Control the display to display the multimedia resources corresponding to the second speech recognition result.

14. The display device according to any one of claims 8 to 12, further comprising a display, wherein the at least one processor is further configured to execute the computer instructions to enable the display device to execute: In the case where the target pinyin does not exist in the first mapping data, determining the multimedia resource corresponding to the first speech recognition result according to the first speech recognition result; Control the display to display multimedia resources corresponding to the first speech recognition result.

15. A voice processing method for a display device, comprising: Receiving voice commands input by a user; In response to the voice instruction, the voice instruction is recognized to obtain a plurality of candidate recognition results corresponding to the voice instruction; Determining the number of entities corresponding to the candidate recognition results; When the number of entities is less than or equal to the number threshold, the scene information of the voice instruction is analyzed using a scene decision module to determine a target recognition result of the voice instruction from among the multiple candidate recognition results.

Images