Electronic equipment, voice processing method and device and storage medium

By employing a two-stage semantic integrity detection method, including speech-text matching and pre-trained model processing, the problem of low accuracy in speech command integrity detection is solved, achieving more efficient and accurate speech command recognition and feedback.

CN119007716APending Publication Date: 2024-11-22HISENSE VISUAL TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310569294.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-19
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The accuracy of voice command integrity detection in existing technologies is not high, which may result in voice commands being truncated too early or users having to wait too long for feedback.

Method used

A two-stage semantic integrity detection method is adopted. First, the speech command is converted into speech text and matched with the semantic detection database. Then, a pre-trained semantic integrity detection model is used for further detection, including processing of the BERT model, Transformer model, fully connected layer and activation function layer, to determine the semantic integrity of the speech command.

Benefits of technology

It improves the detection efficiency and accuracy of voice commands, ensuring that the semantic information of voice commands can be accurately recognized and promptly fed back, thereby enhancing the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119007716A_ABST
    Figure CN119007716A_ABST
Patent Text Reader

Abstract

The embodiment of the invention relates to the technical field of voice, and discloses electronic equipment, a voice processing method and device and a storage medium, and the electronic equipment comprises a receiver which is configured to receive a voice instruction; the controller is coupled with the receiver, and the controller is configured to convert the voice instruction into a voice text when the voice instruction is received; matching the voice text with voice features in a semantic detection database to determine a first detection result; based on the first detection result, performing semantic integrity detection on the voice text by adopting a pre-trained semantic integrity detection model, and determining a second detection result of the voice instruction; and determining semantic information corresponding to the voice instruction according to the first detection result or the second detection result, and executing the voice instruction based on the semantic information. By applying the technical scheme of the invention, the accuracy of detecting the semantic integrity of the voice instruction can be improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of speech technology, and in particular to an electronic device, a speech processing method, a device and a storage medium. Background Art

[0002] With the development of artificial intelligence technology, voice interaction has become an important way of human-computer interaction. For example, in various smart devices such as smartphones, smart TVs, and smart speakers, users can control smart devices to perform certain operations, such as playing TV programs or music, by entering voice commands.

[0003] As the number of functions that can be triggered by voice commands continues to increase, how to accurately identify the user's intention indicated by voice commands has become a research focus for smart device manufacturers. The key link is to determine whether the semantics indicated by the voice command is complete, that is, to determine whether the voice command has been input, or whether the semantic information represented by the voice command is complete. Summary of the invention

[0004] In view of the above problems, the embodiments of the present invention provide an electronic device, a voice processing method, an apparatus and a storage medium, which are used to solve the problem of low accuracy of voice command integrity detection in the prior art.

[0005] According to one aspect of an embodiment of the present invention, an electronic device is provided, comprising: a receiver configured to receive a voice instruction; a controller coupled to the receiver, the controller being configured to: upon receiving the voice instruction, convert the voice instruction into voice text; match the voice text with voice features in a semantic detection database to determine a first detection result; the first detection result is used to indicate whether the semantic information of the voice instruction is complete, the semantic information is incomplete, or no semantic information is detected; based on the first detection result, a semantic integrity detection model is used to perform a semantic integrity detection on the voice text to determine a second detection result of the voice instruction, the second detection result is used to indicate whether the semantic information of the voice instruction is complete, the semantic information is incomplete, or the semantic information is ambiguous; according to the first detection result or the second detection result, the semantic information corresponding to the voice instruction is determined, and the voice instruction is executed based on the semantic information.

[0006] In some embodiments, the controller is configured to: when the first detection result or the second detection result indicates that the semantic information of the voice instruction is complete, determine the semantic information corresponding to the voice instruction, and execute the voice instruction based on the semantic information; when the first detection result or the second detection result indicates that the semantic information of the voice instruction is incomplete or ambiguous, receive a continuation instruction of the voice instruction through the receiver to obtain an updated voice instruction, and based on the updated voice instruction, determine the updated semantic information corresponding to the updated voice instruction, and execute the updated voice instruction based on the updated semantic information.

[0007] In some embodiments, the controller is configured to: when the semantic information of the voice instruction is incomplete and no continuation instruction of the voice instruction is received within a first preset time length starting from a first moment, determine the semantic information corresponding to the voice instruction and execute the voice instruction based on the semantic information; when the semantic information of the voice instruction is ambiguous and no continuation instruction of the voice instruction is received within a second preset time length starting from a second moment, determine the semantic information corresponding to the voice instruction and execute the voice instruction based on the semantic information; wherein the first preset time length is greater than the second preset time length.

[0008] In some embodiments, the semantic detection database includes a complete semantic database and an incomplete semantic database, and the controller is configured to: match the voice text with the voice features in the complete semantic database, and when the voice text matches any voice feature in the complete semantic database, determine that the first detection result of the voice instruction indicates that the semantic information of the voice instruction is complete; match the voice text with the voice features in the incomplete semantic database, and when the voice text matches any voice feature in the incomplete semantic database, determine that the first detection result of the voice instruction indicates that the semantic information of the voice instruction is incomplete.

[0009] In some embodiments, the semantic integrity detection model includes a Bert model, a Transformer model, a fully connected layer and an activation function layer, and the controller is configured to: use the Bert model to process the speech text to obtain first feature data of the speech text; use the Transformer model to process the first feature data to obtain second feature data of the speech text; convert the second feature data through a fully connected layer and an activation function layer to obtain a second detection result of the voice instruction.

[0010] In some embodiments, the controller is further configured to: obtain multiple input sample data and a semantic integrity detection result corresponding to each input sample data; the semantic integrity detection result includes complete semantic information, incomplete semantic information or ambiguous semantic information; process the input sample data based on an initial semantic integrity detection model to obtain a semantic integrity prediction result; use the semantic integrity prediction result as the initial training output of the initial semantic integrity detection model, and use the semantic integrity detection result as supervision information, and iteratively train the initial semantic integrity detection model to obtain a trained semantic integrity detection model.

[0011] In some embodiments, the controller is further configured to: acquire multiple text data; perform word segmentation on each text data, and determine the word boundaries of each text data based on the word segmentation results of each text data; and disassemble and annotate each text data based on the word boundaries of each text data to obtain the multiple input sample data and the semantic integrity detection results corresponding to each input sample data.

[0012] According to another aspect of an embodiment of the present invention, a speech processing method is provided, the method comprising: upon receiving a voice instruction, converting the voice instruction into voice text; matching the voice text with voice features in a semantic detection database to determine a first detection result; the first detection result is used to indicate whether the semantic information of the voice instruction is complete, the semantic information is incomplete, or no semantic information is detected; based on the first detection result, using a pre-trained semantic integrity detection model to perform a semantic integrity detection on the voice text to determine a second detection result of the voice instruction, the second detection result is used to indicate whether the semantic information of the voice instruction is complete, the semantic information is incomplete, or the semantic information is ambiguous; according to the first detection result or the second detection result, determining the semantic information corresponding to the voice instruction, and executing the voice instruction based on the semantic information.

[0013] In some embodiments, determining the semantic information corresponding to the voice instruction according to the first detection result or the second detection result, and executing the voice instruction based on the semantic information, includes: when the first detection result or the second detection result indicates that the semantic information of the voice instruction is complete, determining the semantic information corresponding to the voice instruction, and executing the voice instruction based on the semantic information; when the first detection result or the second detection result indicates that the semantic information of the voice instruction is incomplete or ambiguous, receiving a continuation instruction of the voice instruction through the receiver to obtain an updated voice instruction, and based on the updated voice instruction, determining updated semantic information corresponding to the updated voice instruction, and executing the updated voice instruction based on the updated semantic information.

[0014] In some embodiments, determining the semantic information corresponding to the voice instruction based on the first detection result or the second detection result, and executing the voice instruction based on the semantic information, includes: when the semantic information of the voice instruction is incomplete and no continuation instruction of the voice instruction is received within a first preset time length starting from a first moment, determining the semantic information corresponding to the voice instruction and executing the voice instruction based on the semantic information; when the semantic information of the voice instruction is ambiguous and no continuation instruction of the voice instruction is received within a second preset time length starting from a second moment, determining the semantic information corresponding to the voice instruction and executing the voice instruction based on the semantic information; wherein, the first preset time length is greater than the second preset time length.

[0015] In some embodiments, the semantic detection database includes a complete semantic database and an incomplete semantic database, and the matching of the speech text with speech features in the semantic detection database to determine a first detection result includes: matching the speech text with speech features in the complete semantic database, and when the speech text matches any speech feature in the complete semantic database, determining that the first detection result of the speech instruction indicates that the semantic information of the speech instruction is complete; matching the speech text with speech features in the incomplete semantic database, and when the speech text matches any speech feature in the incomplete semantic database, determining that the first detection result of the speech instruction indicates that the semantic information of the speech instruction is incomplete.

[0016] In some embodiments, the semantic integrity detection model includes a Bert model, a Transformer model, a fully connected layer and an activation function layer. Based on the first detection result, a pre-trained semantic integrity detection model is used to perform a semantic integrity detection on the voice text to determine a second detection result of the voice instruction, including: using the Bert model to process the voice text to obtain first feature data of the voice text; using the Transformer model to process the first feature data to obtain second feature data of the voice text; and converting the second feature data through a fully connected layer and an activation function layer to obtain a second detection result of the voice instruction.

[0017] In some embodiments, the method further includes: obtaining multiple input sample data and a semantic integrity detection result corresponding to each input sample data; the semantic integrity detection result includes complete semantic information, incomplete semantic information or ambiguous semantic information; processing the input sample data based on an initial semantic integrity detection model to obtain a semantic integrity prediction result; using the semantic integrity prediction result as the initial training output of the initial semantic integrity detection model and the semantic integrity detection result as supervision information, iteratively training the initial semantic integrity detection model to obtain a trained semantic integrity detection model.

[0018] In some embodiments, the method further includes: acquiring multiple text data; performing word segmentation on each text data, and determining the word boundaries of each text data based on the word segmentation results of each text data; and disassembling and annotating each text data based on the word boundaries of each text data to obtain the multiple input sample data and the semantic integrity detection results corresponding to each input sample data.

[0019] According to another aspect of an embodiment of the present invention, a speech processing device is provided, the device comprising: a conversion module, for converting the speech instruction into speech text when receiving a speech instruction; a matching module, for matching the speech text with speech features in a semantic detection database to determine a first detection result; the first detection result is used to indicate whether the semantic information of the speech instruction is complete, the semantic information is incomplete, or the semantic information is not detected; a detection module, for performing a semantic integrity detection on the speech text based on the first detection result using a pre-trained semantic integrity detection model to determine a second detection result of the speech instruction, the second detection result is used to indicate whether the semantic information of the speech instruction is complete, the semantic information is incomplete, or the semantic information is ambiguous; a determination module, for determining the semantic information corresponding to the speech instruction based on the first detection result or the second detection result, and executing the speech instruction based on the semantic information.

[0020] According to another aspect of the embodiments of the present invention, a computer-readable storage medium is provided, wherein at least one executable instruction is stored in the storage medium, and when the executable instruction is executed on an electronic device, the electronic device executes the operation of the speech processing method as described above.

[0021] In summary, the electronic device, speech processing method, device and storage medium provided by the embodiments of the present invention can convert the voice instruction into speech text when receiving the voice instruction, and match the voice text with the speech features in the semantic detection database to determine a first detection result. Based on the first detection result, a pre-trained semantic integrity detection model is used to perform semantic integrity detection on the voice text to determine the second detection result of the voice instruction. According to the first detection result or the second detection result, the semantic information corresponding to the voice instruction is determined, and the voice instruction is executed based on the semantic information.

[0022] Through the above method, a two-stage semantic integrity detection method can be used to detect the semantic integrity of voice commands, that is, in the first stage, the voice text is matched with the voice features in the semantic detection database to determine the first detection result, and in the second stage, a pre-trained semantic integrity detection model is used to perform semantic integrity detection on the voice text to determine the second detection result of the voice command. This can realize progressive semantic integrity detection, achieve semantic integrity segmentation of voice commands, and improve the detection efficiency and accuracy of voice commands.

[0023] The above description is only an overview of the technical solution of the embodiment of the present invention. In order to more clearly understand the technical means of the embodiment of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiment of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are only used to illustrate the embodiments and are not to be considered as limiting the present invention. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings. In the accompanying drawings:

[0025] Figure 1 A schematic diagram showing the interaction between a display device and a control device provided by an embodiment of the present invention is shown;

[0026] Figure 2 A configuration block diagram of a control device in an embodiment of the present invention is shown;

[0027] Figure 3 A hardware configuration block diagram of a display device provided by an embodiment of the present invention is shown;

[0028] Figure 4 A flow chart of a speech processing method provided by an embodiment of the present invention is shown;

[0029] Figure 5 A flowchart of a semantic integrity detection method provided by an embodiment of the present invention is shown;

[0030] Figure 6A schematic diagram of a semantic database provided by an embodiment of the present invention is shown;

[0031] Figure 7 A flowchart of another semantic integrity detection method provided by an embodiment of the present invention is shown;

[0032] Figure 8 A flowchart of another semantic integrity detection method provided by an embodiment of the present invention is shown;

[0033] Fig. 9 A flowchart of a method for generating a semantic integrity detection model provided by an embodiment of the present invention is shown;

[0034] Fig.10 A flowchart of text data processing provided by an embodiment of the present invention is shown;

[0035] Fig.11 A flow chart of a speech processing method provided by an embodiment of the present invention is shown;

[0036] Fig.12 A flowchart of another speech processing method provided by an embodiment of the present invention is shown;

[0037] Fig.13 A structural schematic diagram of a speech processing device provided by an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0038] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the drawings, it should be understood that the present invention can be implemented in various forms and should not be limited to the embodiments set forth herein.

[0039] As electronic devices are becoming more advanced, they can support the operation of multiple applications. For example, a TV can play videos, play music, query information, and run various applications, such as sports and fitness, audio and video communications, and provide various services through applications. In different operating scenarios, the intentions of the voice commands entered by users are also different. Therefore, in order to meet the needs of users to control the TV with voice assistants, TV manufacturers need to focus on improving the intelligence of voice assistants to identify different intentions of users to control the TV.

[0040] In one solution of the related art, the end point of the voice command is detected by using acoustic VAD (Voice Activity Detection). When the voice command includes the end point, it means that the semantics of the voice command is complete. On the contrary, when the voice command does not include the end point, it means that the semantics of the voice command is incomplete. n, when the input duration of the voice command is less than t n , the voice command input is not completed, the semantics of the voice command is incomplete, and when the input duration of the voice command is not less than t n , then the voice command input is completed and the semantics of the voice command is complete.

[0041] However, due to t n is fixed, if t n If the setting is too small, the voice command input by the user will be truncated prematurely, which is called "early truncation problem". n If the setting is too large, the user will have to wait for a long time before the electronic device can give feedback, which is called the "user waiting problem". Therefore, setting the instruction time t n The method cannot accurately determine the end point of the voice command, that is, the accuracy of voice integrity detection is not high.

[0042] In view of one or more of the above problems, an embodiment of the present invention provides an electronic device and a voice processing method, which can be applied to the electronic device. Figure 1 FIG. 1 shows a schematic diagram of interaction between an electronic device and a control device provided by an embodiment of the present invention. Figure 1 As shown, the user can operate the electronic device 200 through the mobile terminal 300 or the control device 100. The control device 100 can be a remote controller, and the remote controller and the electronic device 200 can communicate through infrared protocol, Bluetooth protocol, or the remote controller can also control the electronic device 200 wirelessly or in other wired ways.

[0043] The user can control the electronic device 200 by inputting user commands through buttons on the remote controller, voice input, a control panel, etc. For example, the user can turn on the voice control function of the electronic device 200 through the voice key on the remote controller, and then enter a voice command so that the electronic device 200 can recognize the voice command and execute the function indicated by the voice command.

[0044] In some embodiments, the user may also use a mobile terminal, tablet computer, computer, laptop computer, and other smart devices to control the electronic device 200. For example, the user may control the electronic device 200 through an application installed on the smart device, which may be configured to provide the user with various controls in an intuitive user interface on a screen associated with the smart device.

[0045] In some embodiments, the mobile terminal 300 can connect and communicate with the software application installed on the electronic device 200 through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. For example, it is possible to establish a control instruction protocol with the mobile terminal 300 and synchronize the remote control keyboard to the mobile terminal 300, and control the user interface on the mobile terminal 300 to achieve the function of controlling the electronic device 200, or it is also possible to transmit the content displayed on the mobile terminal 300 to the electronic device 200 to achieve the function of synchronous display.

[0046] like Figure 1 As shown, the electronic device 200 and the server 400 can communicate data through a variety of communication methods, allowing the electronic device 200 to communicate and connect through a local area network (LAN), a wireless local area network (WLAN) and other networks. The server 400 can provide various content and interactions to the electronic device 200. For example, the electronic device 200 receives software program updates or accesses a remotely stored digital media library by sending and receiving messages and interacting with an electronic program guide (EPG). Among them, the server 400 can be a cluster or multiple clusters, and can include one or more types of servers.

[0047] The electronic device 200 may be a liquid crystal display with voice control function, an organic light-emitting diode (OLED) display, a projection electronic device, a smart terminal such as a mobile phone, a tablet computer, etc. The specific electronic device type, size, and resolution are not limited.

[0048] In some embodiments, the voice control function of the electronic device 200 can also be completed through an external device, that is, the electronic device 200 can be connected to a device with a voice input function, such as headphones, microphones, etc., and the user can enter voice commands through the external device, and the electronic device 200 can perform recognition and feedback of the voice commands.

[0049] Figure 2 1 shows a block diagram of a configuration of a control device 100 in an exemplary embodiment of the present invention, as shown in FIG. Figure 2 As shown, the control device 100 includes a controller 110, a communication interface 130, a user input / output interface 140, a memory, and a power supply. The control device 100 can receive operation instructions input by the user, and convert the operation instructions into instructions that the electronic device 200 can recognize and respond to, and play an interactive intermediary role between the user and the electronic device 200.

[0050] For example, taking the electronic device as a television, Figure 3 FIG. 2 shows a hardware configuration block diagram of an electronic device 200 provided by an embodiment of the present invention. Figure 3 As shown, the electronic device 200 includes: a tuner and demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a receiver 280 and at least one of a memory, a power supply, and a user interface.

[0051] The tuner demodulator 210 can receive broadcast television signals by wired or wireless reception, and demodulate audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals. The detector 230 can be used to collect signals from the external environment or interact with the outside.

[0052] In some embodiments, the frequency demodulated by the tuner-demodulator 210 is controlled by the controller 250, which can send a control signal according to the user's selection to make the modem respond to the TV signal frequency selected by the user and modulate and demodulate the TV signal carried by the frequency.

[0053] Broadcast television signals can be classified into terrestrial broadcast signals, cable broadcast signals, satellite broadcast signals or Internet broadcast signals according to different broadcasting formats of television signals. Or they can be classified into digital modulation signals, analog modulation signals, etc. according to different modulation types. Or they can be classified into digital signals, analog signals, etc. according to different signal types.

[0054] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0055] In some embodiments, the communicator 220 may be a component for communicating with an external device or an external server according to various communication protocol types. For example, the communicator may include at least one of a Wifi chip, a Bluetooth communication protocol chip, a wired Ethernet communication protocol chip, other network communication protocol chips or a near field communication protocol chip, and an infrared receiver.

[0056] In some embodiments, the detector 230 may be used to collect signals from the external environment or interact with the outside, and may include a light receiver and a temperature sensor, etc.

[0057] Among them, the light receiver can be used as a sensor to collect the intensity of ambient light, and to adaptively adjust display parameters according to the intensity of ambient light; the temperature sensor can be used to sense the ambient temperature, so that the electronic device 200 can adaptively adjust the display color temperature of the image. For example, when the ambient temperature is high, the color temperature of the image displayed by the electronic device 200 can be adjusted to a cooler tone, or when the ambient temperature is low, the color temperature of the image displayed by the electronic device 200 can be adjusted to a warmer tone.

[0058] In some embodiments, the detector 230 may include a sound collector, such as a microphone, which may be used to receive the user's voice, for example, a voice signal including a control instruction of the user controlling the electronic device 200, or collect environmental sounds for identifying the type of environmental scene, so that the electronic device 200 can adaptively adapt to the environmental noise.

[0059] In some embodiments, the detector 230 may also include an image collector, such as a camera, a webcam, etc., which can be used to collect external environment scenes, as well as to collect user attributes or gestures for interacting with the user. It can adaptively change display parameters and recognize user gestures to achieve the function of interacting with the user.

[0060] In some embodiments, the external device interface 240 may include, but is not limited to, any one or more interfaces such as a High Definition Multimedia Interface (HDMI) interface, an analog or digital high-definition component input interface, a composite video input interface, a Universal Serial Bus (USB) input interface, an RGB port, or a composite input / output interface formed by multiple interfaces mentioned above.

[0061] like Figure 3 As shown, the controller 250 may include a central processing unit, a video processor, an audio processor, a graphics processor, a random access memory (RAM), a read-only memory (ROM), and at least one of a first interface to an nth interface for input / output. A communication bus connects the various components.

[0062] In some embodiments, the controller 250 can control the operation of the electronic device and respond to the user's operation through various software control programs stored in the external memory. For example, the user can input a user command through a graphical user interface (GUI) displayed on the display 260, and the user input interface receives the user input command through the graphical user interface, or the user can input a user command by inputting a specific sound or gesture, and the user input interface recognizes the sound or gesture through a sensor to receive the user input command.

[0063] "User interface" is the medium interface for interaction and information exchange between applications or operating systems and users. It realizes the conversion between the internal form of information and the form acceptable to users. The commonly used form of user interface is graphical user interface, which refers to the user interface related to computer operation displayed in a graphical way. It can be an interface element such as an icon, window, control, etc. displayed on the display screen of an electronic device, where the control can include at least one of the visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, widgets (Web widgets, referred to as Widgets), etc.

[0064] In some embodiments, RAM can be used to store temporary data of the operating system or other running programs; ROM can be used to store various system startup instructions, for example, it can be used to store a basic input and output system, called a basic input and output system (BIOS) startup instruction. ROM can be used to complete the power-on self-test of the system, the initialization of each functional module in the system, the basic input / output driver of the system and boot the operating system.

[0065] In some embodiments, when a power-on signal is received, the power of the electronic device 200 starts to start, and the central processing unit runs the system startup instruction in the ROM, and copies the temporary data of the operating system stored in the memory to the RAM to start or run the operating system. When the operating system is started, the central processing unit copies the temporary data of various application programs in the memory to the RAM, and then starts or runs various application programs.

[0066] In some embodiments, the central processing unit can be used to execute operating system and application instructions stored in the memory, as well as execute various applications, data and content based on various interactive instructions received from external input, so as to ultimately display and play various audio and video content.

[0067] In some exemplary embodiments, the central processor may include multiple processors. The multiple processors may include a main processor and one or more sub-processors. The main processor is used to perform some operations of the electronic device 200 in the pre-power-on mode and / or display the screen in the normal mode. The one or more sub-processors are used for an operation in the standby mode and the like.

[0068] In some embodiments, the video processor can be configured to receive an external video signal and perform decompression, decoding, scaling, noise reduction, frame rate conversion, resolution conversion, transparency setting, image synthesis and other video processing according to the standard codec protocol of the input signal to obtain a signal that can be directly displayed or played on the electronic device 200.

[0069] In some embodiments, the video processor may include a demultiplexing module, a video decoding module, an image synthesis module, a frame rate conversion module, a display formatting module, etc.

[0070] Among them, the demultiplexing module is used to demultiplex the input audio and video data stream. For example, if the Moving Picture Experts Group-2 (MPEG-2) is input, the demultiplexing module demultiplexes it into video signals and audio signals, etc.; the video decoding module is used to process the demultiplexed video signal, including decoding and scaling, transparency setting, etc.

[0071] An image synthesis module, such as an image synthesizer, is used to superimpose and mix the GUI signal generated by the graphics generator according to the user input or itself with the scaled video image to generate an image signal for display. A frame rate conversion module is used to convert the input video frame rate, such as converting the 60Hz frame rate to 120Hz frame rate or 240Hz frame rate. The usual format is implemented by interpolation. A display formatting module is used to receive the video output signal after frame rate conversion and change the signal to conform to the display format signal, such as outputting an RGB data signal.

[0072] In some embodiments, the audio processor can be used to receive external audio signals, perform decompression and decoding according to the standard codec protocol of the input signal, as well as noise reduction, digital-to-analog conversion, and amplification processing to obtain a sound signal that can be played in a speaker.

[0073] In some embodiments, the video processor may include one or more chips. The audio processor may also include one or more chips. At the same time, the video processor and the audio processor may be a separate chip or may be integrated with the controller in one or more chips.

[0074] In some embodiments, the interface for input / output can be used for audio output, that is, receiving the sound signal output by the audio processor under the control of the controller 250, and outputting it to an external speaker such as a speaker, and in addition to the speaker carried by the electronic device 200 itself, it can be output to an external audio output terminal of a generating device of an external device, such as an external audio interface or a headphone interface. The audio output can also include a short-range communication module in the communication interface, for example, a Bluetooth module for sound output of a speaker connected to Bluetooth.

[0075] In some embodiments, the graphics processor may be used to generate various graphic objects, such as icons, operation menus, and user input command display graphics. The graphics processor may include an operator that performs operations by receiving various interactive instructions input by the user and displays various objects according to display attributes. The graphics processor may also include a renderer that renders various objects obtained based on the operator, and the rendered objects are used to be displayed on the display.

[0076] In some embodiments, the graphics processor and the video processor may be integrated or separately configured. When integrated, the graphics processor and the video processor may process graphics signals output to the display. When separately configured, the graphics processor and the video processor may respectively perform different functions, such as a graphics processor (GPU) + frame rate conversion technology (FRC) architecture.

[0077] The display 260 may be at least one of a liquid crystal display, an OLED display, a touch display, and a projection display, and may also be a projection device and a projection screen.

[0078] In some embodiments, the display 260 may be used to display a display interface of an application program, which is any page in the application program.

[0079] In some embodiments, the display 260 may be used to receive audio and video signals output by the audio processor and the video processor, display video content and images, play audio of video content, and display components of a menu control interface.

[0080] In some embodiments, the display 260 may be used to present a user operation UI interface generated in the electronic device 200 and used to control the electronic device 200 .

[0081] In some embodiments, the electronic device 200 may establish transmission and reception of control signals and data signals with the external control device 100 or the content providing device through the communicator 220 .

[0082] In some embodiments, the memory may include various software modules for driving the electronic device 200. For example, the various software modules stored in the first memory include at least one of a basic module, a detection module, a communication module, a display control module, a browser module, and various service modules.

[0083] The basic module is a low-level software module used for signal communication between various hardware in the electronic device 200 and sending processing and control signals to the upper-level module. The detection module is a management module used to collect various information from various sensors or user input interfaces, and perform digital-to-analog conversion and analysis management.

[0084] For example, the speech recognition module may include a speech analysis module and a speech instruction database module. The display control module may be used to control the display to display image content, and may be used to play multimedia image content and UI interface information. The communication module may be used to control and communicate data with external devices. The browser module may be used to perform data communication between browsing servers. The service module may be used to provide various services and various application programs. At the same time, the memory may also be used to store and receive external data and user data, images of various items in various user interfaces, and visual effects of focus objects.

[0085] In some embodiments, the user interface may be used to receive an infrared control signal sent by the control device 100 , such as an infrared remote controller.

[0086] The power supply can provide power to the electronic device 200 through power input from an external power source under the control of the controller 250 .

[0087] In some embodiments, the electronic device 200 can receive a voice signal input by a user by adding a receiver 280. The receiver 280 can be an external control component connected to the display 260. Taking a desktop computer as an example, the receiver 280 can be a headset connected to the display 260. The user can input different voice commands through the headset, such as sound adjustment, playback control and other operation commands.

[0088] Accordingly, when the user performs different control operations on the external control component, the external control component can generate different control signals in response to the user's control operations, and send the generated control signals to the controller 250. The controller 250 can identify the received control signal, and determine the control function that the user wants to execute according to the identification result.

[0089] For example, when the user connects the external control component to the display 260, the voice control function of the display 260 is triggered. The user can start the recording of the voice command by controlling the external control component, and the display 260 can receive the voice command recorded by the external control component, and perform text conversion and recognition on the voice command to determine the operation intention of the voice command, and then control the display 260 to output the content indicated by the voice command or perform the control operation indicated by the voice command.

[0090] In order to improve the recognition accuracy of voice commands, the electronic device provided by an embodiment of the present invention can convert the voice command into voice text when receiving the voice command, and then match the voice text with the voice features in the semantic detection database to determine a first detection result. Based on the first detection result, a pre-trained semantic integrity detection model is used to perform a semantic integrity detection on the voice text to determine a second detection result of the voice command. According to the first detection result or the second detection result, the semantic information corresponding to the voice command is determined, and the semantic command is executed based on the semantic information.

[0091] The following is combined with Figure 4 The speech processing method based on an electronic device provided by an embodiment of the present invention is described in detail.

[0092] The speech processing method provided by the embodiment of the present invention can be applied to Figure 3 The electronic device 200 shown in the figure may include a receiver 280 and a controller 250 coupled to the receiver 280. The receiver may be used to receive voice commands. Figure 4 As shown, the controller 250 is configured to perform the following steps S410-S440:

[0093] Step S410, when a voice instruction is received, convert the voice instruction into voice text.

[0094] The voice command is the sound data acquired by the controller, which may be the voice data recorded by the user through a recording device, such as the voice data recorded by the user through a microphone configured on a remote control, a smart phone, etc. In some cases, the voice command may also be the voice data recorded in advance by the user, or may also be the sound data acquired by other means, such as the voice command data downloaded from the network.

[0095] When receiving a voice command, the controller may convert the voice command into voice text in order to analyze the semantics of the voice command. For example, the voice command may be preprocessed, feature extracted, and pattern matched to generate a voice text corresponding to the voice command.

[0096] Preprocessing can include removing noise, removing duplications, and repairing missing parts of voice commands. This process can ensure the command of the voice signal, making subsequent processing more accurate and effective. Next, feature extraction can be performed on the preprocessed voice command, that is, through a series of algorithms and techniques, the sound features in the voice signal, such as frequency, energy, spectrum, intonation, pitch, etc., are extracted. Finally, pattern matching is performed on the sound features, that is, the extracted sound features are matched with the voice model in the voice library. Since the voice library stores a large number of correspondences between text and voice, the sound features can be compared with the voice model through pattern matching to find the text content that best matches it, that is, the voice text corresponding to the voice command is obtained.

[0097] In order to improve the conversion efficiency of voice instructions, in some embodiments, the voice instructions can also be converted into voice text through a voice conversion tool.

[0098] Through the above method, when a voice instruction is received, the voice instruction can be converted into voice text that can be analyzed.

[0099] Step S420, matching the speech text with speech features in a semantic detection database to determine a first detection result.

[0100] Among them, the first detection result is used to indicate that the semantic information of the voice command is complete, the semantic information is incomplete, or no semantic information is detected. Complete semantic information means that the voice has a clear semantic orientation, that is, the intention of the voice expression is clear. For example, for the voice text "What day of the week is October 1, 2023", it has a clear time and inquiry intention, so the semantic information is complete. Incomplete semantic information means that the semantic orientation of the voice is unclear, that is, the intention of the voice expression is unclear. For example, for the voice text "I want to watch a TV series", since it does not specify which TV series to watch and which episode, the semantic information is incomplete. No semantic information is detected means that the semantic information of the voice text cannot be determined.

[0101] The semantic detection database refers to a database for detecting the semantic integrity of text, which may include multiple speech features, which may be speech features with complete semantic information or speech features with incomplete semantic information. The speech features in the semantic detection database may be text features, which may be represented by text sentences, characters, character strings, etc.

[0102] In some embodiments, the semantic detection database can be configured according to the application scenario of the voice command. For example, for a voice assistant on a TV, the semantic detection database can be a voice feature database configured for an application on the TV.

[0103] After obtaining the speech text, in order to determine whether the speech information of the speech text is complete, the speech text can be matched with the speech features in the semantic detection database to determine the first detection result, that is, to determine whether the semantic information of the speech text is complete or whether the semantic information is not detected.

[0104] Taking the semantic detection database as an example of a speech feature database with complete speech information, the speech text can be matched with each speech feature in the semantic detection database. If any speech feature matches the speech text, it means that the semantic information of the speech text is complete; if no speech feature matches the speech text, it can be determined that the semantic information of the speech text has not been detected.

[0105] In some embodiments, the semantic detection database may also only include a speech feature database with incomplete speech information, and match the speech text with each speech feature in the semantic detection database. If any speech feature matches the speech text, it means that the semantic information of the speech text is incomplete; if no speech feature matches the speech text, it can be determined that the semantic information of the speech text has not been detected.

[0106] In the above method, the matching of speech features and speech text means that the speech features and speech text are completely identical. For example, the speech features and speech text can be determined to match when the character string corresponding to the speech features and the character string corresponding to the speech text are completely identical.

[0107] In some embodiments, the semantic detection database may include a complete semantic database and an incomplete semantic database at the same time, wherein a complete semantic database means that the speech features in the database are all speech features with complete semantic information; an incomplete semantic database means that the speech features in the database are all speech features with incomplete semantic information.

[0108] Therefore, if Figure 5 As shown, step S420 can be implemented by the following method:

[0109] Step S510, matching the voice text with the voice features in the complete semantic database, and when the voice text matches any voice feature in the complete semantic database, determining that the first detection result of the voice instruction indicates that the semantic information of the voice instruction is complete.

[0110] Step S520, matching the voice text with the voice features in the incomplete semantic database, and when the voice text matches any voice feature in the incomplete semantic database, determining that the first detection result of the voice instruction indicates that the semantic information of the voice instruction is incomplete.

[0111] That is to say, the voice text is matched with the voice features in the complete semantic database and the voice features in the incomplete semantic database respectively. When the voice text matches any voice feature in the complete semantic database, it means that the first detection result of the voice command is that the semantic information is complete. When the voice text matches any voice feature in the incomplete semantic database, it means that the first detection result of the voice command is that the semantic information is incomplete. If the voice text does not match every voice feature in the complete semantic database and does not match every voice feature in the incomplete semantic database, it means that the first detection result of the voice command is that no semantic information is detected.

[0112] Figure 6 A schematic diagram of a semantic database provided by an embodiment of the present invention is shown. Figure 6 As shown, the semantic database includes a complete semantic database and an incomplete semantic database. Among them, the complete semantic database may include non-interface word instructions and interface word instructions, the non-interface word instructions may include "I want to watch a movie", "I want to listen to music", "XX music", etc., the interface word instructions may include "power off", "turn off the TV", "return", "exit", "return to home page", etc.; the incomplete semantic database may include "I", "I want", "I want to watch", "watch", "play", "I want to listen", "search" and "adjust", etc.

[0113] It can be seen that the complete semantic database and the incomplete data database can be divided into one or more types according to the instruction type, and the database can be set to voice features with a high number of occurrences, while the number of such voice features is generally small. Therefore, the semantic detection results of voice instructions can be quickly and accurately identified through the database matching method.

[0114] In some embodiments, a multi-level indexing architecture may be used to improve the matching efficiency of semantic text with a complete semantic database and an incomplete data database, thereby improving the detection efficiency of semantic instructions.

[0115] In order to improve the perfection of the complete semantic database and the incomplete semantic database, in some embodiments, the complete semantic database and the incomplete semantic database may be updated regularly to adapt them to the latest hotspot query data.

[0116] Through the above method, the first detection result of the voice text can be determined by matching the complete semantic database and the incomplete semantic database with the voice text, and the preliminary detection of the semantic integrity of the voice text can be achieved. At the same time, since the complete semantic database and the incomplete semantic database have the characteristics of few types and high number of occurrences, the database matching method can handle common voice command detection with high accuracy.

[0117] Step S430: Based on the first detection result, a pre-trained semantic integrity detection model is used to perform a semantic integrity detection on the voice text to determine a second detection result of the voice instruction.

[0118] The second detection result is used to indicate whether the semantic information of the voice command is complete, incomplete or ambiguous. Ambiguous semantic information means that the voice text has semantic orientation, but the semantic orientation is relatively ambiguous. For example, for the voice text "I want to watch Peppa Pig Season 3", although it can express the user's intention and the range of the cartoons he wants to watch, the range is not specific to a certain episode, so the semantic information is ambiguous.

[0119] It should be understood that the standard for whether the voice information is complete or ambiguous can vary according to the application scenario, for example, in the above example, if the user does not specify the number of episodes, and if the user is assumed to want to watch videos starting from the first episode, then the semantic information of the voice text "I want to watch Peppa Pig Season 3" can also be considered complete. Therefore, whether the semantic information is complete or ambiguous can be adjusted according to actual needs, and is not fixed.

[0120] The semantic integrity detection model refers to a classification model that detects whether the semantic information of a speech text is complete or ambiguous, which may include any semantic analysis model, such as a recurrent neural network detection model, a GPT (Generative Pre-Trained Transformer) detection model, and the like.

[0121] According to the first detection result, it can be determined whether the semantic information of the voice text is complete or whether the semantic information is detected. Exemplarily, when the first detection result is that the semantic information is not detected, a pre-trained semantic integrity detection model can be further used to perform a semantic integrity detection on the voice text to determine the second detection result of the voice command, that is, to determine whether the semantic information of the voice text is complete or ambiguous.

[0122] Since the semantic integrity detection model is trained with a large amount of speech text data, the semantic integrity of the speech text can be determined by performing semantic integrity detection on the speech text using the pre-trained semantic integrity detection model, and a higher detection accuracy rate can be provided.

[0123] In some embodiments, the semantic integrity detection model may include a Bert model (Bidirectional Encoder Representations from Transformers, a bidirectional semantic encoding representation model built on Transformer), a Transformer model (converter model), a fully connected layer, and an activation function layer.

[0124] Among them, the Bert model is a pre-trained model that can use the context information of each word in the process of encoding speech text, so it has a strong ability to extract semantic information. The Transformer model is a neural network that learns context and thus semantics by tracking the relationship in sequence data. The main function of the fully connected layer is to extract and transform the input data to achieve tasks such as classification, regression, and clustering of the data. The activation function layer can be a Softmax activation function layer.

[0125] Therefore, if Figure 7 As shown, step S430 can be implemented by the following method:

[0126] Step S710: Process the speech text using the Bert model to obtain first feature data of the speech text.

[0127] By processing the speech text using the Bert model, the first feature data of the speech text can be extracted. For example, by processing the speech text using the Bert model, the speech text can be converted into a feature vector representation to obtain a first feature vector.

[0128] Step S720: Use the Transformer model to process the first feature data to obtain second feature data of the speech text.

[0129] The Transformer model is used to perform deep feature representation on the first feature data to obtain second feature data of the speech text.

[0130] Step S730: convert the second feature data through a fully connected layer and an activation function layer to obtain a second detection result of the voice command.

[0131] After obtaining the second feature data, the second feature data can be mapped into categories through a fully connected layer and an activation function layer to obtain a second detection result of the voice command, that is, to determine the classification result of the semantic command.

[0132] Figure 8 A flow chart of semantic detection of speech text provided by an embodiment of the present invention is shown, Figure 8 As shown, the following steps may be included:

[0133] Step S810: Process the speech text using the Bert model to obtain first feature data of the speech text.

[0134] Step S820: Process the first feature data using a fully connected layer, a Dropout layer, and a ReLU activation function layer to obtain processed first feature data.

[0135] Among them, the Dropout layer can be used to prevent overfitting problems, and the ReLU activation function layer is an activation function layer that can remove negative values ​​in the convolution results and keep the positive values ​​unchanged.

[0136] After obtaining the first feature data, the first feature data may be further transformed using a fully connected layer, a dropout layer, and a ReLU activation function layer to obtain processed first feature data.

[0137] Step S830: Use the Transformer model to process the processed first feature data to obtain second feature data of the speech text.

[0138] Step S840: Process the second feature data using a fully connected layer, a Dropout layer, and a ReLU activation function layer to obtain processed second feature data.

[0139] After obtaining the second feature data, the second feature data can be further transformed using a fully connected layer, a Dropout layer, and a ReLU activation function layer to obtain processed second feature data.

[0140] Step S850: Use a fully connected layer and a Softmax layer to convert the processed second feature data to obtain a second detection result of the voice command.

[0141] It should be noted that the fully connected layers in steps S820, S840 and S850 may have the same or different numbers of neurons, which is not specifically limited in this embodiment.

[0142] Through the above method, the second detection result of the voice command can be obtained while ensuring the accuracy of the detection result.

[0143] In this embodiment, the semantic integrity detection model can be trained by a large amount of training data to have better performance, that is, a pre-trained semantic integrity detection model can be obtained. Fig. 9 As shown, a pre-trained semantic integrity detection model can be generated by the following method:

[0144] Step S910: Acquire multiple input sample data and the semantic integrity detection result corresponding to each input sample data.

[0145] The semantic integrity detection result includes complete semantic information, incomplete semantic information or ambiguous semantic information. The input sample data is the input data used to train the semantic integrity detection model, and the semantic integrity detection result corresponding to each input sample data refers to the semantic integrity detection label.

[0146] The input sample data can be obtained through the corresponding public datasets, or by collecting datasets in the corresponding field and setting labels.

[0147] Step S920: Process the input sample data based on the initial semantic integrity detection model to obtain a semantic integrity prediction result.

[0148] The initial semantic integrity detection model refers to a semantic integrity detection model that has not been trained.

[0149] The input sample data is input into the initial semantic integrity detection model for processing, and the predicted semantic integrity prediction result predicted by the model is obtained.

[0150] Step S930, using the semantic integrity prediction result as the initial training output of the initial semantic integrity detection model and the semantic integrity detection result as supervision information, iteratively training the initial semantic integrity detection model to obtain a trained semantic integrity detection model.

[0151] The initial semantic integrity prediction and detection model is supervised and trained using the semantic integrity prediction results and the semantic integrity detection results. The initial semantic integrity detection model is iteratively updated until the performance of the initial semantic integrity detection model reaches the best, thereby obtaining a trained semantic integrity detection model.

[0152] Through the above method, the initial semantic integrity detection model can be trained using the input sample data to generate a pre-trained semantic integrity detection model.

[0153] In order to increase the amount of training data and improve the training effect, in some embodiments, such as Fig.10 As shown, the controller can generate the above input sample data by the following method:

[0154] Step S1010, obtaining multiple text data.

[0155] The text data may be acquired general text data or dedicated text data for a specific field or application scenario.

[0156] For example, the text data can be obtained by acquiring a public text data set, or by collecting a text data set for a specific field or application scenario. For example, in a TV terminal, the server can obtain the user's voice command data for a period of time, such as the last six months, to obtain multiple text data.

[0157] Step S1020 , segment each text data into words, and determine the word boundary of each text data according to the segmentation result of each text data.

[0158] In order to facilitate the analysis of text data, each text data can be segmented to split it into multiple words or characters. For example, the text data can be processed using a corresponding segmentation tool to obtain the segmentation results corresponding to the text data. It should be noted that when selecting a segmentation tool, the segmentation granularity needs to be considered to avoid the segmentation granularity being too fine or too coarse.

[0159] Considering that users may pause when recording voice commands, the pause points generally do not appear in the middle of words. For example, for the voice command "play Zhang's songs Beijing", the possible pause points may be "play... Zhang's songs Beijing", "play Zhang's... songs Beijing", "play Zhang's... songs Beijing", "play Zhang's... songs Beijing", but it is unlikely to be "play Zhang... songs Beijing", "play Zhang's... songs Beijing" and "play Zhang's... songs Beijing".

[0160] Therefore, in order to determine the pause point in the text data, after obtaining the word segmentation result, the word boundaries of the text data can be further determined according to the word segmentation result, etc. For example, for the text data "I want to watch Zhang's TV series", the word segmentation results are "I", "want", "watch", "Zhang", "TV series", so the word boundaries of the text data are "I", "want", "watch", "Zhang", "someone", "TV" and "TV series".

[0161] Step S1030 , based on the word boundaries of each text data, each text data is disassembled and annotated to obtain a plurality of input sample data and a semantic integrity detection result corresponding to each input sample data.

[0162] According to the word boundaries of each text data, the text data can be disassembled and annotated. For example, according to the word segmentation result, the characters in the word are not annotated, but only the characters at the word boundaries are annotated. Table 1 below shows an example of annotating text data, as shown below:

[0163] Table 1

[0164] Text Example 1 I / think / look / open / certain / of / electricity film Semantic integrity 1 1 1 - 2 1 - 0 Text Example 2 I / think / look / Small pig wear / strange / th three set Semantic integrity 1 1 1 - - - 0 - - 0 Text Example 3 broadcast put / No want money / of / special work Team Semantic integrity - 1 - - 1 1 - - 0

[0165] Among them, "0" indicates that the semantic information is complete, "1" indicates that the semantic information is incomplete, "2" indicates that the semantic information is ambiguous, and " / " indicates a word boundary.

[0166] In some cases, the pause point in the voice command is also likely to be located in the middle of a word, for example, "play the third episode...", "turn off...", in which case, the annotations in some special words need to be modified, as shown in Table 2 below:

[0167] Table 2

[0168] Shutdown Play / Episode 3 -0 -1--0 1 0 -1 1-0

[0169] Furthermore, for a text data, it can be decomposed according to word boundaries to decompose the text data into multiple input sample data, and the semantic integrity detection result corresponding to each input sample data is set. For example, "I want to watch Peppa Pig Season 3 Episode 2 5 minutes 30 seconds" can be decomposed into multiple input sample data as shown in Table 3 below.

[0170] Table 3

[0171]

[0172]

[0173] It should be noted that when defining whether the semantic information of a voice command is complete, it can be considered that the semantic information is complete when the voice command is entered. At this time, if it is considered that "I want to watch Peppa Pig Season 3 Episode 2, Fifth Minute and Thirty Seconds" is a complete voice command, while other voice commands are incomplete or vague, this is obviously unreasonable. Therefore, it is necessary to formulate and select category boundaries of appropriate granularity to achieve a better human-computer interaction experience.

[0174] Through the above method, multiple input sample data can be generated according to multiple text data, and the semantic integrity detection result corresponding to each input sample data can be set to obtain the training data of the semantic integrity detection model.

[0175] Step S440, determining semantic information corresponding to the voice instruction according to the first detection result or the second detection result, and executing the voice instruction based on the semantic information.

[0176] After obtaining the first detection result and the second detection result, the semantic information corresponding to the voice instruction can be determined according to the first detection result or the second detection result, and the operation corresponding to the semantic information can be performed. For example, when the first detection result indicates that the semantic information of the voice instruction is complete or incomplete, the operation corresponding to the semantic information of the voice instruction can be performed. At this time, when the semantic information is complete, the semantic information of the voice instruction is complete semantic information, and when the semantic information is incomplete, the semantic information of the voice instruction is incomplete semantic information.

[0177] For another example, based on the first detection result and the second detection result, when the first detection result and the second detection result are inconsistent, either one of them can be used as the basis to determine the semantic information corresponding to the voice command and execute the operation corresponding to the semantic information.

[0178] In order to improve the execution accuracy of voice commands, in some embodiments, such as Fig.11 As shown, the controller can also execute the following methods:

[0179] Step S1110, when the first detection result or the second detection result indicates that the semantic information of the voice instruction is complete, determine the semantic information corresponding to the voice instruction, and execute the voice instruction based on the semantic information.

[0180] That is, when the first detection result or the second detection result indicates that the semantic information of the voice command is complete, it means that the user has completed the entry of the voice command, or the voice command has a clear intention. At this time, the controller can determine the semantic information corresponding to the voice command and execute the operation corresponding to the semantic information.

[0181] Step S1120, when the first detection result or the second detection result indicates that the semantic information of the voice instruction is incomplete or ambiguous, a continuation instruction of the voice instruction is received through the receiver to obtain an updated voice instruction, based on the updated voice instruction, the updated semantic information corresponding to the updated voice instruction is determined, and the updated voice instruction is executed based on the updated semantic information.

[0182] When the first detection result or the second detection result indicates that the semantic information of the voice command is incomplete or ambiguous, it may be that the user has not yet completed the entry of the voice command. At this time, the receiver can continue to receive the continuation instruction of the voice command to obtain an updated voice command. For example, when the voice command entered by the user is "I want to watch Peppa Pig", and it is determined that the semantic information of the voice command is incomplete or ambiguous, the receiver can continue to wait and receive the continuation instruction, such as "... Season 3", to obtain the updated voice command "I want to watch Peppa Pig Season 3".

[0183] After obtaining the updated voice instruction, the updated semantic information corresponding to the updated voice instruction may be re-determined, and the operation corresponding to the updated semantic information may be performed.

[0184] Through the above method, according to whether the semantic information of the voice command is complete, incomplete or ambiguous, the operation corresponding to the voice command can be executed or the continuation command can be continued to be received to obtain a voice command with complete semantic information, which can improve the feedback accuracy of the voice command and enhance the user experience.

[0185] In order to improve the feedback accuracy of voice commands and reduce the waiting time of users, in some embodiments, Fig.12 As shown, the controller can also execute the following methods:

[0186] Step S1210, when the semantic information of the voice instruction is incomplete and no continuation instruction of the voice instruction is received within a first preset time period starting from the first moment, determine the semantic information corresponding to the voice instruction, and execute the voice instruction based on the semantic information.

[0187] The first moment may be the moment when the voice command is received, or the moment when the semantic information of the voice command is determined.

[0188] When the semantic information of the voice command is incomplete, it means that the semantic information of the voice command is not clear. At this time, you can continue to receive continuation instructions of the voice command. If no continuation instructions of the voice command are received within a first preset time length starting from the first moment, you can execute a subsequent method, that is, determine the semantic information corresponding to the voice information, and execute the operation corresponding to the semantic information.

[0189] Step S1220, when the semantic information of the voice instruction is ambiguous and no continuation instruction of the voice instruction is received within a second preset time period starting from the second moment, determine the semantic information corresponding to the voice instruction and execute the voice instruction based on the semantic information.

[0190] Among them, the first preset duration is greater than the second preset duration. The first preset duration and the second preset duration can be set according to actual needs, and this embodiment does not specifically limit this. Correspondingly, the second moment can also be the moment when the voice command is received, or the moment when the semantic information of the voice command is determined.

[0191] When the semantic information of the voice command is vague, it means that the semantic information of the voice command is very unclear. At this time, you can continue to receive continuation instructions of the voice command. If you do not receive continuation instructions of the voice command within a second preset time period starting from the second moment, the user may have completed the entry of the voice command, or there is no possibility of entering the continuation instruction temporarily. Therefore, the semantic information corresponding to the voice information can be determined, and the operation corresponding to the semantic information can be executed.

[0192] Through the above method, the feedback accuracy of voice commands can be improved, and the waiting time of users can also be reduced.

[0193] In summary, according to the electronic device and voice processing method provided in this embodiment, when a voice command is received, the voice command can be converted into voice text for matching the voice text with voice features in a semantic detection database, and a first detection result can be determined. Based on the first detection result, a pre-trained semantic integrity detection model is used to perform semantic integrity detection on the voice text to determine a second detection result of the voice command. According to the first detection result or the second detection result, the semantic information corresponding to the voice command is determined, and the voice command is executed based on the semantic information.

[0194] Through the above method, a two-stage semantic integrity detection method can be used to detect the semantic integrity of voice commands, that is, in the first stage, the voice text is matched with the voice features in the semantic detection database to determine the first detection result, and in the second stage, a pre-trained semantic integrity detection model is used to perform semantic integrity detection on the voice text to determine the second detection result of the voice command. This can realize progressive semantic integrity detection, achieve semantic integrity segmentation of voice commands, and improve the detection efficiency and accuracy of voice commands.

[0195] Fig.13 FIG. 2 shows a schematic diagram of the structure of a speech processing device provided by an embodiment of the present invention. Fig.13 As shown, the speech processing device 1300 includes: a conversion module 1310, which is used to convert the voice instruction into voice text when receiving the voice instruction; a matching module 1320, which is used to match the voice text with the voice features in the semantic detection database to determine a first detection result; the first detection result is used to indicate that the semantic information of the voice instruction is complete, the semantic information is incomplete, or the semantic information is not detected; a detection module 1330, which is used to perform a semantic integrity detection on the voice text based on the first detection result using a pre-trained semantic integrity detection model to determine a second detection result of the voice instruction, and the second detection result is used to indicate that the semantic information of the voice instruction is complete, the semantic information is incomplete, or the semantic information is ambiguous; a determination module 1340, which is used to determine the semantic information corresponding to the voice instruction according to the first detection result or the second detection result, and execute the voice instruction based on the semantic information.

[0196] In some embodiments, the determination module 1340 is used to determine the semantic information corresponding to the voice instruction when the first detection result or the second detection result indicates that the semantic information of the voice instruction is complete, and execute the voice instruction based on the semantic information; when the first detection result or the second detection result indicates that the semantic information of the voice instruction is incomplete or ambiguous, receive a continuation instruction of the voice instruction through the receiver to obtain an updated voice instruction, and based on the updated voice instruction, determine the updated semantic information corresponding to the updated voice instruction, and execute the updated voice instruction based on the updated semantic information.

[0197] In some embodiments, the determination module 1340 is used to determine the semantic information corresponding to the voice instruction and execute the voice instruction based on the semantic information when the semantic information of the voice instruction is incomplete and no continuation instruction of the voice instruction is received within a first preset time length starting from a first moment; determine the semantic information corresponding to the voice instruction and execute the voice instruction based on the semantic information when the semantic information of the voice instruction is ambiguous and no continuation instruction of the voice instruction is received within a second preset time length starting from a second moment; wherein the first preset time length is greater than the second preset time length.

[0198] In some embodiments, the semantic detection database includes a complete semantic database and an incomplete semantic database. The matching module 1320 is used to match the speech text with the speech features in the complete semantic database. When the speech text matches any speech feature in the complete semantic database, it is determined that the first detection result of the speech instruction indicates that the semantic information of the speech instruction is complete; and to match the speech text with the speech features in the incomplete semantic database. When the speech text matches any speech feature in the incomplete semantic database, it is determined that the first detection result of the speech instruction indicates that the semantic information of the speech instruction is incomplete.

[0199] In some embodiments, the semantic integrity detection model includes a Bert model, a Transformer model, a fully connected layer and an activation function layer. The detection module 1330 is used to process the speech text using the Bert model to obtain first feature data of the speech text; process the first feature data using the Transformer model to obtain second feature data of the speech text; and convert the second feature data through the fully connected layer and the activation function layer to obtain a second detection result of the voice command.

[0200] In some embodiments, the detection module 1330 is used to obtain multiple input sample data and the semantic integrity detection results corresponding to each input sample data; the semantic integrity detection results include complete semantic information, incomplete semantic information or ambiguous semantic information; the input sample data is processed based on the initial semantic integrity detection model to obtain a semantic integrity prediction result; the semantic integrity prediction result is used as the initial training output of the initial semantic integrity detection model, and the semantic integrity detection result is used as the supervision information, and the initial semantic integrity detection model is iteratively trained to obtain a trained semantic integrity detection model.

[0201] In some embodiments, the detection module 1330 is used to obtain multiple text data; segment each text data, and determine the word boundaries of each text data based on the segmentation results of each text data; based on the word boundaries of each text data, disassemble and annotate each text data to obtain multiple input sample data and the semantic integrity detection results corresponding to each input sample data.

[0202] Correspondingly, the specific details of each part of the above-mentioned voice processing device have been described in detail in the above-mentioned electronic device implementation method. The undisclosed details can be found in the implementation method of the electronic device part, and thus will not be repeated here.

[0203] An embodiment of the present invention provides a computer-readable storage medium, which stores at least one executable instruction. When the executable instruction is executed on an electronic device / apparatus, the electronic device / apparatus executes the speech processing method in any of the above method embodiments.

[0204] The executable instructions may be used to enable the electronic device / apparatus to perform the following operations:

[0205] When a voice command is received, the voice command is converted into voice text; the voice text is matched with voice features in a semantic detection database to determine a first detection result; the first detection result is used to indicate whether the semantic information of the voice command is complete, the semantic information is incomplete, or no semantic information is detected; based on the first detection result, a pre-trained semantic integrity detection model is used to perform a semantic integrity detection on the voice text to determine a second detection result of the voice command, the second detection result is used to indicate that the semantic information of the voice command is complete, the semantic information is incomplete, or the semantic information is ambiguous; according to the first detection result or the second detection result, the semantic information corresponding to the voice command is determined, and the voice command is executed based on the semantic information.

[0206] In this embodiment, computer readable storage media may be any tangible media that can contain or store a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0207] More specific examples (a non-exhaustive list) of computer-readable storage media may include: an electrical connection having one or more conductors, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), optical fibers, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0208] The algorithms or displays provided herein are not inherently related to any particular computer, virtual system or other device. In addition, the embodiments of the present invention are not directed to any particular programming language.

[0209] In the description provided herein, a large number of specific details are described. However, it is understood that embodiments of the present invention can be practiced without these specific details. Similarly, in order to simplify the present invention and help understand one or more of the various inventive aspects, in the above description of exemplary embodiments of the present invention, the various features of the embodiments of the present invention are sometimes grouped together into a single embodiment, figure, or description thereof. Wherein, the claims that follow the specific embodiment are hereby expressly incorporated into the specific embodiment, wherein each claim itself is a separate embodiment of the present invention.

[0210] Those skilled in the art will appreciate that the modules in the devices in the embodiments may be adaptively changed and arranged in one or more devices different from the embodiments. The modules or units or components in the embodiments may be combined into one module or unit or component, and further may be divided into a plurality of submodules or subunits or subcomponents. Except that at least some of such features and / or processes or units are mutually exclusive.

[0211] It should be noted that the above embodiments illustrate the present invention rather than limit it, and that those skilled in the art may devise alternative embodiments without departing from the scope of the appended claims. In the claims, any reference symbol between brackets shall not be construed as a limitation on the claims. The word "comprising" does not exclude the presence of elements or steps not listed in the claims. The word "one" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention may be implemented by means of hardware comprising a number of different elements and by means of a suitably programmed computer. In a unit claim enumerating a number of devices, several of these devices may be embodied by the same hardware item. The use of the words first, second, and third, etc. does not indicate any order. These words may be interpreted as names. The steps in the above embodiments, unless otherwise specified, should not be understood as limitations on the order of execution.

Claims

1. An electronic device, characterized in that: include: a receiver configured to receive a voice command; A controller coupled to the receiver, the controller being configured to: When receiving the voice command, converting the voice command into voice text; Matching the voice text with voice features in a semantic detection database to determine a first detection result; the first detection result is used to indicate whether the semantic information of the voice instruction is complete, the semantic information is incomplete, or no semantic information is detected; Based on the first detection result, a pre-trained semantic integrity detection model is used to perform a semantic integrity detection on the voice text to determine a second detection result of the voice instruction, wherein the second detection result is used to indicate whether the semantic information of the voice instruction is complete, incomplete, or ambiguous; According to the first detection result or the second detection result, semantic information corresponding to the voice instruction is determined, and the voice instruction is executed based on the semantic information.

2. The electronic device according to claim 1, characterized in that: The controller is configured to: When the first detection result or the second detection result indicates that the semantic information of the voice instruction is complete, determining the semantic information corresponding to the voice instruction, and executing the voice instruction based on the semantic information; When the first detection result or the second detection result indicates that the semantic information of the voice instruction is incomplete or ambiguous, a continuation instruction of the voice instruction is received by the receiver to obtain an updated voice instruction, and based on the updated voice instruction, updated semantic information corresponding to the updated voice instruction is determined, and the updated voice instruction is executed based on the updated semantic information.

3. The electronic device according to claim 2, characterized in that: The controller is configured to: When the semantic information of the voice instruction is incomplete and no continuation instruction of the voice instruction is received within a first preset time period starting from the first moment, determining the semantic information corresponding to the voice instruction and executing the voice instruction based on the semantic information; When the semantic information of the voice instruction is ambiguous and no continuation instruction of the voice instruction is received within a second preset time period starting from the second moment, determining the semantic information corresponding to the voice instruction and executing the voice instruction based on the semantic information; Among them, the first preset time length is greater than the second preset time length.

4. The electronic device according to claim 1, characterized in that: The semantic detection database includes a complete semantic database and an incomplete semantic database, and the controller is configured to: Matching the voice text with the voice features in the complete semantic database, and when the voice text matches any voice feature in the complete semantic database, determining that the first detection result of the voice instruction indicates that the semantic information of the voice instruction is complete; The voice text is matched with the voice features in the incomplete semantic database, and when the voice text matches any voice feature in the incomplete semantic database, it is determined that the first detection result of the voice instruction indicates that the semantic information of the voice instruction is incomplete.

5. The electronic device according to claim 1, characterized in that: The semantic integrity detection model includes a Bert model, a Transformer model, a fully connected layer and an activation function layer, and the controller is configured as follows: Processing the speech text using the Bert model to obtain first feature data of the speech text; Processing the first feature data using the Transformer model to obtain second feature data of the speech text; The second feature data is converted through a fully connected layer and an activation function layer to obtain a second detection result of the voice command.

6. The electronic device according to claim 5, characterized in that: The controller is also configured to: Acquire multiple input sample data and a semantic integrity detection result corresponding to each input sample data; the semantic integrity detection result includes complete semantic information, incomplete semantic information or ambiguous semantic information; Processing the input sample data based on the initial semantic integrity detection model to obtain a semantic integrity prediction result; The semantic integrity prediction result is used as the initial training output of the initial semantic integrity detection model, and the semantic integrity detection result is used as supervision information, and the initial semantic integrity detection model is iteratively trained to obtain a trained semantic integrity detection model.

7. The electronic device according to claim 6, characterized in that: The controller is also configured to: Get multiple text data; Performing word segmentation on each text data, and determining the word boundary of each text data according to the word segmentation result of each text data; Based on the word boundaries of each of the text data, each of the text data is disassembled and annotated to obtain the multiple input sample data and the semantic integrity detection result corresponding to each of the input sample data.

8. A speech processing method, characterized in that: The method comprises: When receiving a voice command, convert the voice command into voice text; Matching the voice text with voice features in a semantic detection database to determine a first detection result; the first detection result is used to indicate whether the semantic information of the voice instruction is complete, the semantic information is incomplete, or no semantic information is detected; Based on the first detection result, a pre-trained semantic integrity detection model is used to perform a semantic integrity detection on the voice text to determine a second detection result of the voice instruction, wherein the second detection result is used to indicate whether the semantic information of the voice instruction is complete, incomplete, or ambiguous; According to the first detection result or the second detection result, semantic information corresponding to the voice instruction is determined, and the voice instruction is executed based on the semantic information.

9. A speech processing device, characterized in that: The device comprises: A conversion module, used for converting the voice instruction into voice text when receiving the voice instruction; A matching module, used for matching the voice text with the voice features in the semantic detection database to determine a first detection result; the first detection result is used to indicate whether the semantic information of the voice instruction is complete, the semantic information is incomplete, or no semantic information is detected; a detection module, configured to perform a semantic integrity detection on the voice text based on the first detection result using a pre-trained semantic integrity detection model, and determine a second detection result of the voice instruction, wherein the second detection result is used to indicate whether the semantic information of the voice instruction is complete, incomplete, or ambiguous; A determination module is used to determine semantic information corresponding to the voice instruction according to the first detection result or the second detection result, and execute the voice instruction based on the semantic information.

10. A computer-readable storage medium, characterized in that: The storage medium stores at least one executable instruction, and when the executable instruction is executed on the electronic device, the electronic device executes the operation of the speech processing method as claimed in claim 8.