Display device and data processing method

Abstract

Some embodiments of the present application show a display device and a data processing method, the method comprising: a voice acquisition service obtaining a first protocol message, and sending the first protocol message to a voice recognition service through local socket communication. The voice acquisition service and the voice recognition service are independent services, the voice acquisition service is constructed based on a first architecture, the voice recognition service is constructed based on a second architecture, and the word length of the first architecture is smaller than that of the second architecture; and the voice recognition service obtains the state information in the first protocol message or obtains the environment audio in the first protocol message so as to perform voice recognition on the user voice in the environment audio. According to the embodiment of the invention, the voice acquisition service and the voice recognition service are designed to be independent services, and the voice acquisition service still adopts an architecture with a relatively small word length, so that the system compatibility is improved. And the voice recognition service adopts a framework with a relatively large word length, so that the CPU occupancy rate is reduced. The two services communicate with each other through a local socket, so that the communication delay can be reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of display devices, in particular to a display device and a data processing method. BACKGROUND

[0002] The scheme of the television far-field function is often to couple the voice collection service and the voice recognition service in the same process. Since the voice recognition service is suitable for running in a computationally intensive high-load module, the voice collection service is suitable for running in an input / output (I / O) intensive low-load module, and the voice collection service depends on a Tiny Advanced Linux Sound Architecture (TinyALSA) driver with extensive horizontal and vertical dependencies. The horizontal dependency is on other audio services, and the vertical dependency is on the kernel audio driver. From the economic point of view of television consumer electronics, continuous optimization is needed to reduce Central Processing Unit (CPU) occupation and reduce system dependencies. Usually, optimization options such as vector instructions and inline expansion are enabled during the construction process, but optimization from the level of processor architecture is not performed.

[0003] The 64-bit architecture of the core processor is more efficient in processing computationally intensive tasks than the 32-bit architecture, which can reduce CPU occupation. However, the TinyALSA has extensive software and hardware dependencies. If the entire far-field voice service module is migrated to a 64-bit architecture, it will bring huge compatibility overhead. SUMMARY

[0004] Some embodiments of the present application provide a display device and a data processing method. The voice collection service and the voice recognition service are designed as independent services. The voice collection service still uses an architecture with a smaller word length, which improves system compatibility. The voice recognition service uses an architecture with a larger word length, which reduces CPU occupation. The services communicate through local sockets, which reduces communication delay.

[0005] In a first aspect, some embodiments of the present application provide a display device, comprising: a display; a sound collector; a controller coupled to the display and configured to: control a voice collection service to obtain a first protocol message, the first protocol message comprising state information of the sound collector or environmental audio collected by the sound collector; control the voice collection service to send the first protocol message to a voice recognition service through local socket communication, the voice collection service and the voice recognition service being independent services, the voice collection service being built based on a first architecture, the voice recognition service being built based on a second architecture, a word length of the first architecture being smaller than a word length of the second architecture, the word length being used to represent data processing capability; The voice recognition service is controlled to obtain the state information in the first protocol message to update the state information of the sound collector in the voice recognition service, or to obtain the environmental audio in the first protocol message to perform voice recognition on the user voice in the environmental audio.

[0006] The above technical solution has the following advantages or beneficial effects: the voice collection service and the voice recognition service are designed as independent services, the voice collection service still adopts an architecture with a small word length, and the system compatibility is improved. The voice recognition service adopts an architecture with a large word length, and the CPU occupancy rate is reduced. The voice collection service and the voice recognition service communicate through a local socket, and the communication delay can be reduced.

[0007] In some embodiments, the controller, which controls the voice recognition service to obtain the environmental audio in the first protocol message to perform voice recognition on the user voice in the environmental audio, is further configured to: control the voice recognition service to obtain the physical length of the first protocol message; control the voice recognition service to identify the first protocol message as a buffer message if the physical length is a first preset length; control the voice recognition service to parse the buffer message to obtain a service type and the environmental audio; control the voice recognition service to perform voice recognition on the user voice in the environmental audio if the service type matches a preset type.

[0008] The above technical solution has the following advantages or beneficial effects: the rapid classification of the physical layer is realized through the length of the protocol message, only one parsing is performed, the business logic routing is realized through the service type verification, the security is guaranteed through the double verification, the performance close to the hardware limit is obtained, and the interaction delay of the independent services is further reduced.

[0009] In some embodiments, after the voice recognition service performs voice recognition on the user voice in the environmental audio, the controller is further configured to: control the voice recognition service to generate a recognition feedback message and send the recognition feedback message to the voice collection service through local socket communication; control the voice collection service to send a second protocol message to the voice recognition service through local socket communication after receiving the recognition feedback message, the second protocol message being obtained before the voice collection service receives the recognition feedback message.

[0010] The above technical solution has the following advantages or beneficial effects: the service design has strict communication timing. The sender must wait for the receiver to process the buffer message and return the identification feedback message before it can send the next protocol message. This ensures that only one direction of data is transmitted at any given time. This can meet the needs of remote voice communication while effectively avoiding the packet merging and splitting problems of streaming communication.

[0011] In some embodiments, the controller executes a control speech recognition service to obtain status information from a first protocol message to update the status information of the sound collector in the speech recognition service, which is further configured as follows: Control the physical length of the first protocol message obtained by the speech recognition service; When the physical length is a second preset length, the voice recognition service is controlled to recognize the first protocol message as a status message; The speech recognition service is controlled to parse status messages to obtain service type and status information. When the business type matches the preset type, the speech recognition service updates the status information of the sound collector in the speech recognition service.

[0012] The above technical solution has the following advantages or beneficial effects: it achieves fast classification at the physical layer by using protocol message length and only requires parsing once; it achieves business logic routing by using business type verification; and it ensures security through dual verification while achieving performance close to the hardware limit, further reducing the interaction latency of independent services.

[0013] In some embodiments, after the speech recognition service updates the status information of the sound collector in the speech recognition service, the controller is further configured to: The speech recognition service is controlled to generate status feedback messages, and these messages are sent to the speech acquisition service via local socket communication. After receiving the status feedback message, the voice acquisition service sends the second protocol message to the voice recognition service via local socket communication. The second protocol message is obtained before the voice acquisition service receives the status feedback message.

[0014] The above technical solution has the following advantages or beneficial effects: the service design has strict communication timing, and the sender must wait for the receiver to process the status message and return the status response before sending the next protocol message, ensuring that only one direction of data is transmitted at any given time. This can meet the needs of remote voice communication while effectively avoiding the packet merging and splitting problems of streaming communication.

[0015] In some embodiments, the controller executes a control voice acquisition service to send a first protocol message to a voice recognition service via local socket communication, and is further configured to: The voice acquisition service detects the on / off status of the voice remote control, and the on / off status is used to determine the recipient of the first protocol message; When the control voice acquisition service detects that the switch is off, it sends the first protocol message to the voice recognition service of the display device via local socket communication.

[0016] The above technical solution has the following advantages or beneficial effects: users can actively turn off the function of remotely controlling other devices by voice, so that the received voice controls the local device, avoiding the transmission of voice data or voice commands to other devices, which can improve privacy and security and enhance the user's control over the display device.

[0017] In some embodiments, after the control voice acquisition service detects the on / off state of the voice remote control, the controller is further configured to: When the control voice acquisition service detects that the switch is on, it obtains the Internet Protocol address of the controlled terminal device from the address list. The control voice acquisition service sends the first protocol message to the voice recognition service of the controlled terminal device corresponding to the Internet Protocol address through remote socket communication, so that the voice recognition service of the controlled terminal device can process the first protocol message.

[0018] The above technical solution has the following advantages or beneficial effects: users can actively enable the function of remotely controlling other devices by voice, and the voice acquisition service can actively obtain the Internet protocol address of the target device and send the first protocol message to the designated controlled terminal to realize remote control of the controlled terminal.

[0019] In some embodiments, before the control voice acquisition service detects the on / off state of the voice remote control, the controller is further configured to: In response to a user's input command to activate remote voice control, the on / off state of the remote voice control is set to the 'on' state. Acquire terminal devices that are on the same network as the display device; Control the display to show the device controls corresponding to the terminal device; In response to the user's instruction to select a device control, the Internet Protocol address of the terminal device corresponding to the device control is obtained and stored in the address list.

[0020] The above technical solution has the following advantages or beneficial effects: users can complete device binding through simple operations, without having to manually enter internet protocol addresses or complex configurations. The graphical interface displays voice-controllable devices, making it intuitive and easy to use, thus lowering the barrier to entry.

[0021] In some embodiments, the controller executes a control speech recognition service to parse buffered messages to obtain the service type and ambient audio, which is further configured as follows: The speech recognition service is controlled to parse the buffered message to obtain the service type, ambient audio, and reference audio. The reference audio is the original signal of the audio played by the display device. The controller executes a speech recognition service to perform speech recognition on the user's voice in the ambient audio, and is further configured as follows: The speech recognition service uses an echo cancellation algorithm to remove the echo caused by playing reference audio from the ambient audio, obtains the user's speech, and then recognizes the user's speech.

[0022] The above technical solution has the following advantages or beneficial effects: While playing audio, the microphone picks up the sound emitted by the speaker, i.e., echo, which interferes with the user's speech. By filtering out the echo of the played content from the ambient audio through echo cancellation, and retaining the pure user speech, it is possible to effectively prevent the recognition system from misinterpreting the sound played by the device as user commands.

[0023] Secondly, some embodiments of this application provide a data processing method, including: The voice acquisition service obtains a first protocol message, which includes the status information of the voice acquisition device or the ambient audio acquired by the voice acquisition device. The voice acquisition service sends the first protocol message to the voice recognition service through local socket communication. The voice acquisition service and the voice recognition service are independent services. The voice acquisition service is built on the first architecture, and the voice recognition service is built on the second architecture. The word length of the first architecture is smaller than that of the second architecture. The word length is used to characterize the data processing capability. The speech recognition service obtains status information or ambient audio from the first protocol message.

[0024] The above technical solution has the following advantages or beneficial effects: designing the voice acquisition service and the voice recognition service as independent services, with the voice acquisition service still using a smaller word length architecture to improve system compatibility, and the voice recognition service using a larger word length architecture to reduce CPU utilization, and communicating between the voice acquisition service and the voice recognition service via local sockets to reduce communication latency.

[0025] In the technical solution provided in this application embodiment, the voice acquisition service obtains a first protocol message, which includes the status information of the sound acquisition device or the environmental audio collected by the sound acquisition device. The voice acquisition service sends the first protocol message to the voice recognition service through local socket communication. The voice acquisition service and the voice recognition service are independent services. The voice acquisition service is built based on a first architecture, and the voice recognition service is built based on a second architecture. The word length of the first architecture is smaller than that of the second architecture, and the word length is used to characterize data processing capabilities. The voice recognition service obtains the status information or environmental audio from the first protocol message. In this application embodiment, the voice acquisition service and the voice recognition service are designed as independent services. The voice acquisition service still adopts an architecture with a smaller word length to improve system compatibility. The voice recognition service adopts an architecture with a larger word length to reduce CPU utilization. The voice acquisition service and the voice recognition service communicate with each other through local sockets, which can reduce communication latency. Attached Figure Description

[0026] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0027] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application; Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application; Figure 3 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application; Figure 4 A schematic diagram illustrating a far-field voice service communication provided for some embodiments of this application; Figure 5 A flowchart illustrating a data processing method provided in some embodiments of this application; Figure 6 A schematic diagram illustrating the composition of protocol messages provided in some embodiments of this application; Figure 7 A schematic diagram illustrating a communication timing constraint provided for some embodiments of this application; Figure 8 A schematic diagram illustrating communication between a voice acquisition service and a voice recognition service, provided for some embodiments of this application; Figure 9 A schematic diagram illustrating device communication provided for some embodiments of this application; Figure 10A schematic diagram of interface data provided for some embodiments of this application; Figure 11 This is a timing diagram of a data processing method provided in some embodiments of this application. Detailed Implementation

[0028] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0029] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0030] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0031] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0032] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0033] In this application embodiment, display device 200 generally refers to a device with screen display and data processing capabilities. For example, display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0034] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1As shown, a user can operate the display device 200 via touch operation, a mobile terminal 300, and a control device 100. The control device 100 receives user input commands and converts them into control commands that the display device 200 can recognize and respond to. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc.

[0035] The mobile terminal 300 can function as a control device for human-computer interaction between the user and the display device 200. It can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can have software applications installed on it and communicate with the display device 200 via network communication protocols to achieve one-to-one control and data communication. Furthermore, it can transmit audio and video content displayed on the mobile terminal 300 to the display device 200 for synchronized display.

[0036] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application that controls the display device 200.

[0037] like Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. This allows the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0038] Display device 200 can provide broadcast television reception function, and can also be equipped with intelligent network television function that provides computer support function, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0039] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0040] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0041] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0042] In this embodiment, the sound acquisition device interfaces with a voice acquisition service. The voice acquisition service and the voice recognition service are connected. The voice acquisition service and the voice recognition service are independent services; the voice acquisition service is built using a 32-bit architecture, and the voice recognition service is built using a 64-bit architecture.

[0043] For example, the voice acquisition service can acquire ambient audio collected or recorded by a sound acquisition device and send the ambient audio to the speech recognition service via a local socket. The speech recognition service can then perform speech recognition on the user's voice within the ambient audio.

[0044] In this embodiment, the sound acquisition device may include a echo microphone board. An echo microphone board refers to a microphone acquisition board with echo functionality. Echo acquisition refers to acquiring or copying an original, clean audio signal before sending it to a speaker for playback, and using it as an echo reference signal.

[0045] The echo microphone board can acquire two types of audio signals. One is ambient audio picked up from the environment, which includes the user's voice, echoes from the speaker array, and ambient noise. The other is the raw audio signal acquired through circuitry and will be sent to the speaker array for playback. For example, the sound acquisition unit may include two echo microphone boards, and the voice acquisition service can acquire the ambient audio from both boards, as well as a reference signal transmitted through five data channels.

[0046] In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0047] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the display device 200 supports Bluetooth connectivity, it needs to have a communication device 220 with Bluetooth functionality.

[0048] The communication device 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0049] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200.

[0050] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0051] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface receives user input commands through the graphical user interface (GUI).

[0052] In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200, i.e., a speaker array of the entire device, or it can be an external audio output device of the display device 200, i.e., an external speaker array of the display device 200. For the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the display device 200 to output sound from the display device 200.

[0053] In some embodiments, the user input interface 280 can be used to receive instructions from user input. For example, the user input interface 280 can receive confirmation instructions from the user regarding controls in the user interface. The user input interface 280 can also receive voice instructions input by the user.

[0054] In some embodiments, to enable user interaction, the display device 200 may run an operating system. An operating system is a computer program that manages and controls the hardware and software resources of the display device 200. The operating system can control the display device to provide a user interface; for example, the operating system can directly control the display device to provide a user interface, or it can provide a user interface by running applications. The operating system also allows users to interact with the display device 200.

[0055] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for display devices.

[0056] An operating system can be divided into different modules or levels based on the functions it implements, for example... Figure 3 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the System Library layer, and the Kernel layer.

[0057] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0058] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0059] like Figure 3As shown, the application framework layer in this embodiment includes a view system, managers, and content providers. The view system designs and implements the application's interface and interactions, and includes lists, grids, text boxes, and buttons. The managers include at least one of the following modules: an activity manager for interacting with all running activities in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0060] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling changes to the display window, such as shrinking the display window, shaking the display, or distorting the display.

[0061] In some embodiments, the system runtime library layer can provide support for the framework layer. When the framework layer is used, the operating system runs the instruction library contained in the system runtime library layer, such as the C / C++ instruction library, to implement the functions to be performed by the framework layer.

[0062] In some embodiments, the kernel layer is a functional layer situated between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. For example, ... Figure 3 As shown, hardware drivers can be configured in the kernel layer. The kernel layer can contain at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0063] It should be noted that the above examples are merely a simple division of operating system functions and do not limit the specific form of the operating system of the display device 200 in this application embodiment. Depending on the function of the display device, the type of operating system, and other factors, the number of levels and the specific level type of the operating system may be expressed in other forms.

[0064] Solutions for far-field television functionality often couple voice acquisition and voice recognition services into the same process. For example... Figure 4 As shown, the speaker array is used to play or provide the audio played on the television, i.e., the reference audio. The echo microphone board is used to pick up ambient audio and acquire the raw signal of the audio played on the television. The kernel peripheral driver is the bridge between the operating system kernel and the hardware. The far-field voice service integrates the core functions of the far-field voice module, such as sound pickup and recognition, to enable users to interact naturally with their voices and execute commands in complex environments at greater distances. The far-field voice service includes TinyALSA, a voice acquisition service, and a voice recognition service. TinyALSA provides an Application Programming Interface (API) for upper-layer applications to operate the kernel audio driver and acquire reference and ambient audio. The voice acquisition service, i.e., the TinyALSA data acquisition task, continuously reads the ambient and reference audio captured by the microphone from the kernel driver and sends the ambient and reference audio to the voice recognition service, i.e., the wake word recognition speech model. The voice recognition service removes speaker echoes and ambient noise from the ambient sound and recognizes the user's voice.

[0065] Because speech recognition services are suitable for running on computationally intensive, high-load modules, while speech capture services are suitable for running on I / O-intensive, low-load modules, and the TinyALSA driver upon which speech capture services depend has extensive horizontal and vertical dependencies. Horizontally, it depends on other audio services, such as media playback and call services. Vertically, it depends on the kernel-level audio driver, which is tightly bound to the hardware. From an economic perspective for consumer electronics television, continuous optimization is needed to reduce CPU usage and system dependencies. This typically involves enabling vector instructions to accelerate computation and using function inlining to reduce call overhead during the build process, rather than optimizing at the processor architecture level.

[0066] While the 64-bit architecture of the core processor is more efficient than the 32-bit architecture in handling computationally intensive tasks and can reduce CPU usage, TinyALSA has extensive software and hardware dependencies. Migrating the entire far-field voice service module to a 64-bit architecture would incur significant compatibility overhead. For example, the kernel driver might need to be upgraded to a 64-bit compatible version, and other audio applications or services that depend on TinyALSA might not yet support 64-bit. Cross-process communication may also fail in a mixed 32 / 64-bit environment.

[0067] To reduce CPU usage and improve system compatibility, this application provides a display device 200. The structure and functions of each part of the display device 200 can be found in the above embodiments. Furthermore, based on the display device 200 shown in the above embodiments, this embodiment further improves some functions of the display device 200. For example... Figure 5 As shown, controller 250 is configured to perform the following steps: Step S501: Control the voice acquisition service to obtain the first protocol message.

[0068] In some embodiments, the voice acquisition service can acquire or generate a first protocol message. The first protocol message may include status information of the voice acquisition device. The status information may include the current operating status and setting parameters of the voice acquisition device. The operating status may include start or enable acquisition, cancel acquisition, and terminate acquisition, etc. The operating status of the voice acquisition device helps the voice acquisition service accurately capture valid segments of the user's voice. The setting parameters may include the number of microphones for acquisition and the number of data channels for transmitting reference audio, etc. The number of microphones for acquisition and the number of data channels for transmitting reference audio can be used to calculate the verification length of the protocol message, i.e., a first preset length.

[0069] For example, after receiving a user's instruction to disable the far-field communication, the settings page sends the far-field disabling instruction to the voice acquisition service. The voice acquisition service then sends a disabling instruction to the sound acquisition device. After executing the disabling instruction, the sound acquisition device sends its current disabling status information to the voice acquisition service. The voice acquisition service generates a first protocol message based on this disabling status information, ensuring that the first protocol message carries the disabling status information of the sound acquisition device.

[0070] In some embodiments, the first protocol message may include ambient audio collected by the sound acquisition device. The first protocol message may also include the ambient audio collected by the sound acquisition device and reference audio transmitted through multiple data channels. A data channel refers to a dedicated digital path between the audio playback service and the voice acquisition service for transmitting the raw audio signal being played by a speaker. The reference audio is a clean digital audio signal that has not been physically played by a speaker. The voice acquisition service can directly obtain the reference audio from the audio playback service. The echo microphone board can obtain the reference audio from the audio playback service and send the ambient audio along with the reference audio to the voice acquisition service. The voice acquisition service can generate the first protocol message based on the ambient audio and the reference audio, so that the first protocol message carries both the ambient audio and the reference audio.

[0071] It should be noted that when the speaker array of the display device 200 is not playing any audio, the first protocol message may include ambient audio collected by the sound acquisition device, which may include user voice and ambient noise. When the speaker array of the display device 200 is playing audio, the first protocol message may include ambient audio and reference audio. Ambient audio may include user voice, speaker array echo, and ambient noise.

[0072] In some embodiments, the first protocol message may be a status message.

[0073] In some embodiments, the first protocol message may be a buffer message. If the speaker array of the current display device 200 is not playing audio, the buffer message may include ambient audio picked up by the sound collector. If the speaker array of the current display device 200 is playing audio, the buffer message may include ambient audio picked up by the sound collector and reference audio transmitted through multiple data channels.

[0074] This application's embodiments employ a custom data protocol, abstracting all services into two types of protocol messages: State messages (StateMsg) and Buffer messages (BufferMsg). State messages are used to transmit status information, such as enabling acquisition, while Buffer messages are used to transmit audio data, such as ambient audio acquired by the sound acquisition device and reference audio played by the speaker array.

[0075] The protocol message consists of the following: Figure 6 As shown in Table 1.

[0076] Table 1

[0077] Depend on Figure 6As shown in Table 1, the State message (StateMsg) includes the State Message Common Attribute Header (MsgCommonHeader). The State Message Common Attribute Header (MsgCommonHeader) includes the message type or business type (msg_type), the message integer data type (msg_value), and the message integer data type (msg_value_str). The Buffer message (BufferMsg) includes the Buffer Message Attribute Header (BufferMsgHeader) and the Buffer Data Body (data_body). The Buffer Message Attribute Header (BufferMsgHeader) includes the Buffer Message Common Attribute Header (MsgCommonHeader), the Buffer Message Data Length (data_length), and the Buffer Type (buffer_type). The Buffer Message Common Attribute Header (MsgCommonHeader) includes the message type (msg_type), the message integer data type (msg_value), and the message integer data type (msg_value_str).

[0078] Step S502: Control the voice acquisition service to send the first protocol message to the voice recognition service through local socket communication.

[0079] The voice acquisition service and the voice recognition service are independent services. The voice acquisition service is built on the first architecture, while the voice recognition service is built on the second architecture. The word length of the first architecture is shorter than that of the second architecture. Word length is used to characterize data processing capabilities. Word length can define the processor's data processing capabilities, memory addressing range, and data type size, and affects the operating system, software architecture, and overall system performance. The voice recognition service can be used to recognize far-field wake words.

[0080] The architecture can be either 32-bit or 64-bit, meaning the word length can be either 32-bit or 64-bit. A 32-bit processor core is characterized by using 32-bit wide registers, a data bus, and an address bus. It can process 32 bits of data in a single operation and can directly address 4GB of memory. A 64-bit processor core is characterized by using 64-bit wide registers, a data bus, and an address bus. Compared to the 32-bit architecture, it offers significant improvements in performance, memory addressing capabilities, and data processing efficiency.

[0081] For example, the voice acquisition service can be built using a 32-bit architecture, while the voice recognition service can be built using a 64-bit architecture. The 64-bit architecture offers a larger address space, higher data throughput, richer register resources, and more efficient instruction set extensions, enabling it to handle computationally intensive tasks more efficiently than the 32-bit architecture. By separating the voice acquisition and voice recognition services into two independent services, with the voice acquisition service still built using a 32-bit architecture, the dependencies and compatibility of the original system are not affected. That is, the kernel audio driver does not need to be upgraded to a 64-bit compatible version, and other audio applications or services that rely on TinyALSA do not need to support 64-bit compatible versions. The voice recognition service is built using a 64-bit architecture to leverage the computational and resource advantages of the 64-bit architecture, further reducing CPU usage and improving cost-effectiveness.

[0082] The voice acquisition service and the voice recognition service communicate via local sockets. Local sockets are a mechanism for efficient and reliable communication between different processes on the same device. Unlike network protocol-based sockets, local sockets do not go through the network protocol stack; data does not need to be packaged into network data packets or transmitted through the network card. It directly utilizes the inter-process communication mechanism of the operating system kernel, using socket files as communication endpoints in the file system.

[0083] This application's embodiments utilize local sockets, enabling direct data transfer within the device without network protocol overhead. This avoids network latency and bandwidth limitations, resulting in significantly faster communication speeds compared to network sockets. Furthermore, communication is confined to local processes and cannot be accessed from external networks, inherently providing a degree of security. File system permission control is supported, such as read and write permissions, allowing restrictions on which processes can connect. Low-latency local socket communication between services meets the data latency requirements of Artificial Intelligence (AI) voice models, ensuring that normal wake-up rates are not affected even in service separation scenarios.

[0084] Step S503: Control the speech recognition service to obtain the status information in the first protocol message to update the status information of the sound collector in the speech recognition service, or obtain the environmental audio in the first protocol message to perform speech recognition on the user's speech in the environmental audio.

[0085] In some embodiments, one implementation of the speech recognition service obtaining state information or ambient audio from a first protocol message may include: after receiving the first protocol message, the speech recognition service can parse it word by word to obtain the state information or ambient audio. Word-by-word parsing allows for inspection at the earliest stage of data processing, enabling real-time detection without loading the entire message into the buffer at once. However, word-by-word parsing logic is complex, and improper state machine design can easily lead to errors.

[0086] In some embodiments, one implementation of the speech recognition service obtaining status information or ambient audio from the first protocol message may include: after receiving the first protocol message, the speech recognition service may first receive and parse the 4-byte message type in the message general attribute header. After confirming that the message type is correct, it then receives and parses the remaining part of the first protocol message to extract the status information or ambient audio. This embodiment requires two parsing steps to obtain the content of the first protocol message.

[0087] In some embodiments, one implementation of the speech recognition service obtaining status information or ambient audio from the first protocol message may include: after receiving the first protocol message, the speech recognition service obtains the physical length of the first protocol message. It then determines whether the physical length of the first protocol message is a pre-set physical length for the protocol message. If the physical length of the first protocol message is the pre-set physical length, it can be determined that the first layer of verification of the first protocol message has passed. If the physical length of the first protocol message is not the pre-set physical length, it can be determined that the first layer of verification of the first protocol message has failed.

[0088] Protocol messages include two types: status messages and buffered messages. Status messages only include a general message attribute header, which contains the message type, message integer type data, and message integer type data. The physical length of a status message, i.e., the second preset length, is 136 bytes.

[0089] A buffered message consists of a message attribute header and a buffered data body. The physical length of the buffered message, i.e., the first preset length, is the length of the message attribute header and the buffered data body. The message attribute header has a fixed composition of 144 bytes, which is the sum of the general message attribute header (136 bytes), the buffered message data length (4 bytes), and the buffer type (4 bytes). The physical length of the buffered data is related to the number of sampling microphones, the number of reference channels, the audio frame length, and the number of single sampling points.

[0090] Taking 2 backsampling microphone boards and 5 reference channels, N=7 as an example: TinyALSA acquisition parameters: frame length 1024, single sampling point 16 bits, physical length of buffered messages, i.e., the first preset length is 1024. (16 / 8) 7 + 144 = 14480 bytes.

[0091] In some embodiments, one implementation of the speech recognition service obtaining ambient audio from a first protocol message to perform speech recognition on user speech in the ambient audio may include: the speech recognition service obtaining the physical length of the first protocol message. If the physical length is a first preset length, the first protocol message is determined to be a buffered message, and then the buffered message is parsed to obtain the service type and ambient audio. If the service type matches a preset type, speech recognition is performed on the user speech in the ambient audio.

[0092] After obtaining the physical length of the first protocol message, it is determined whether the physical length is a first preset length. If the physical length is the first preset length, the first protocol message is identified as a buffered message. If the physical length is not the first preset length, it is determined whether the physical length is a second preset length. If the physical length is the second preset length, the first protocol message is identified as a status message. If the physical length is not the second preset length, the first protocol message is identified as an error message.

[0093] In this case, after determining that the first protocol message is an error message, the protocol message may not be parsed or processed, and an error message may be sent to the voice acquisition service.

[0094] After identifying the first protocol message as a buffer message, the buffer message can be parsed, and the service type and the environmental audio stored in the buffer data body can be obtained directly through a single parsing.

[0095] The preset type can be a service category identifier that is pre-defined during the system design or configuration phase and is expected to be received or processed. When the service type carried in the buffer message is the same as one of several preset types, speech recognition can be performed on the user's voice in the ambient audio.

[0096] For example, the identifiers corresponding to the preset types are {1, 2, 3, and 4}. If the buffer message carries an identifier of service type 2, it can be determined that the service type matches the preset type. If the buffer message carries an identifier of service type 6, it can be determined that the service type does not match the preset type.

[0097] If it is determined that the business type does not match the preset type, the protocol message can be left unprocessed and an error message can be sent to the voice acquisition service.

[0098] In some embodiments, the ambient audio does not include echoes caused by audio played by the display device 200. One implementation of speech recognition of user speech in ambient audio may include: removing noise from the ambient audio to obtain the user speech, and then recognizing the user speech using a speech recognition model. If the recognized text includes a wake-up word, the voice control service is activated, and a wake-up prompt tone can be played to remind the user to input voice control commands.

[0099] In some embodiments, the ambient audio includes an echo caused by audio played by the display device 200, and the first protocol message also includes a reference audio. The step of the speech recognition service parsing the buffer message to obtain the service type and ambient audio includes: the speech recognition service parsing the buffer message to obtain the service type, ambient audio, and reference audio, where the reference audio is the original signal of the audio played by the display device provided by the audio playback service or the echo acquisition microphone board. One implementation of speech recognition of user speech in the ambient audio may include: the speech recognition service using an echo cancellation algorithm to remove the echo from the ambient audio based on the reference audio to obtain the user speech, and then using a speech recognition model to recognize the user speech.

[0100] The core principle of echo cancellation algorithms is to simulate and cancel the reflection path of sound. When a speaker plays distant speech, the sound travels through space and is picked up again by the microphone, forming an echo. AEC records the played audio signal, i.e., the reference audio, and uses an adaptive filter to estimate the acoustic channel response of the sound from the speaker to the microphone in real time, generating a predicted echo signal. This predicted echo signal is then subtracted from the mixed signal collected by the microphone, i.e., the ambient audio, thus preserving the user's speech.

[0101] In some embodiments, after the speech recognition service performs speech recognition on the user's speech in the ambient audio, it can generate a recognition feedback message and send the recognition feedback message to the speech acquisition service via local socket communication. After receiving the recognition feedback message, the speech acquisition service sends a second protocol message to the speech recognition service via local socket communication. The second protocol message is acquired or generated before the speech acquisition service receives the recognition feedback message.

[0102] It is important to emphasize that the voice acquisition service and the voice recognition service are designed with a strict and orderly synchronous communication sequence. 1) Strict timing control: The sender must wait for the receiver to complete processing and return a response before sending the next message. 2) Simplex communication: Data is transmitted in only one direction at a time. 3) Protocol freedom: Protocol messages (Msg) and feedback messages (Ack) can both be status messages or buffered message types.

[0103] Communication timing as follows Figure 7 As shown, the sender sends protocol message 1 (Msg-1) at time t0. If protocol message 2 (Msg-2) is received between t0 and t1, it cannot send Msg-2 immediately. Instead, it must wait until it receives the feedback message 1 (Ack-1) corresponding to Msg-1 before it can send Msg-2.

[0104] The communication model designed in this application adopts streaming transmission, and the data has no predetermined boundaries, which can meet the execution scenarios of single task and multi-task.

[0105] In multi-task scenarios, such as audio data transmission tasks, if a user closes the far field in a zero-one task, the process of closing the far field must wait until the current frame is completely transmitted before it can be processed. The above provisions enable the communication model to meet the needs of remote voice communication while effectively avoiding the packet merging and splitting problems of streaming communication.

[0106] In some embodiments, one implementation of the speech recognition service obtaining status information from a first protocol message to update the status information of the sound collector in the speech recognition service may include: the speech recognition service obtaining the physical length of the first protocol message. If the physical length is a second preset length, the speech recognition service determines the first protocol message as a status message, then parses the status message to obtain the service type and status information. If the service type matches a preset type, the speech recognition service updates the status information of the sound collector.

[0107] The speech recognition service can also generate a status feedback message after updating the status information of the sound acquisition device, and send the status feedback message to the speech acquisition service via local socket communication. After receiving the status feedback message, the speech acquisition service sends a second protocol message to the speech recognition service via local socket communication. The second protocol message is generated or acquired before the speech acquisition service receives the status feedback message.

[0108] In terms of protocol parsing, this application embodiment differs from the method of parsing the 4-byte message type first and then receiving the remaining part in word-by-word parsing and buffer parsing. This application embodiment achieves fast classification of the physical layer by using the message length of the two types of messages, performs parsing only once, implements business logic routing by business type (msg_type), and ensures security by combining memory copy and dual authentication, while achieving performance close to the hardware limit and further reducing the interaction latency of independent services.

[0109] In some embodiments, such as Figure 8 As shown, the voice acquisition service built on a 32-bit architecture includes TinyALSA and a data protocol encapsulation and transmission service. The voice recognition service built on a 64-bit architecture includes a protocol reception and voice recognition service. TinyALSA obtains the ambient audio and reference audio acquired by the acquisition microphone board through the kernel peripheral driver, and sends the acquisition microphone board status, or the ambient audio and reference audio, to the data protocol encapsulation and transmission service. The data protocol encapsulation and transmission service generates a protocol message and sends the protocol message to the protocol reception and voice recognition service. The protocol reception and voice recognition service receives and processes the protocol message.

[0110] This application separates the voice acquisition service and voice recognition service of the far-field television function into independent services. A 32-bit processor architecture is used to construct the data acquisition service, and a 64-bit processor architecture is used to construct the voice recognition service. The mixed-width services communicate via low-latency local sockets, employing a custom data protocol and timing model. This further reduces system load while maintaining the far-field wake-up rate. Compared to a single process, the two independent services require low-latency local socket communication to avoid affecting the latency requirements of the voice recognition model for data processing and to maintain the wake-up rate.

[0111] In some embodiments, after the voice acquisition service generates a first protocol message based on the status information of the sound acquisition device or the environmental audio collected by the sound acquisition device, the voice acquisition service may first detect the on / off status of the voice remote control.

[0112] When the switch is detected to be off, i.e., the voice remote control flag is 0, the first protocol message is sent to the voice recognition service of the display device via local socket communication.

[0113] When the switch is detected to be on (i.e., the voice remote control flag is 1), the IP address of the controlled terminal device is obtained from the address list. Then, a first protocol message is sent to the voice recognition service of the controlled terminal device corresponding to the Internet Protocol (IP) address via Remote Procedure Call Socket (RPC Socket) communication, so that the voice recognition service of the controlled terminal device can process the first protocol message. The controlled terminal device is configured to process the first protocol message in the same way as the display device 200.

[0114] For example, such as Figure 9 As shown, when the voice remote control switch is detected to be off, the local 32-bit voice acquisition service sends protocol messages to the local 64-bit voice recognition service via a local socket. When the voice remote control switch is detected to be on, the local 32-bit voice acquisition service sends protocol messages to the 64-bit voice recognition service of another terminal device via a remote call socket.

[0115] In some embodiments, remote voice control can be turned on or off before the control voice acquisition service detects the on / off state of remote voice control. Remote voice control is the function of controlling other devices by inputting voice commands from the current device.

[0116] In some embodiments, one implementation of disabling remote voice control may include: in response to a user input command to disable remote voice control, setting the on / off state of the voice remote control, i.e., setting the voice remote control flag to 0.

[0117] In some embodiments, enabling remote voice control may include: in response to a user inputting an instruction to enable remote voice control, setting the remote voice control switch to an "on" state, i.e., setting the remote voice control identifier to 1; then acquiring terminal devices on the same network as the display device 200 and controlling the display to show the device control corresponding to the terminal device. In response to a user's instruction to select a device control, acquiring the IP address of the terminal device corresponding to the device control and storing the IP address in an address list. The address list stores the IP addresses of at least one controlled terminal device.

[0118] Since the speech recognition service is unaware of changes in the data source, the multi-bit-width separation service in this application embodiment can be extended to meet distributed application scenarios, such as one TV providing voice data to another TV for wake-up. Figure 10 As shown, different communication methods are provided based on the same communication base interface (ICommuBase): local sockets and remote call sockets.

[0119] LocalSocket is used on the same display device 200. It has low communication latency. For example, in the physical length verification, the latency is in the microsecond range for N=7 channels, which fully meets the requirements of the speech recognition model.

[0120] Remote Procedure Call (RPCSocket) sockets use Transmission Control Protocol / Internet Protocol (TCP / IP) sockets to connect different terminal devices. The network latency requirements are as follows: If TinyALSA has a sampling rate of 16kHz, a frame length of 1024 points, a single point of 16 bits, and a sampling duration of 1 / 16000... 1024 = 64ms. If the current network bandwidth is 100Mbps, then the transmission time is 1024. 16 / 100000000 7 = 1.14ms, rounded up to 2ms, the maximum network latency is: 64ms – 2ms = 62ms. To allow time for jitter and processing, the network latency should be less than 60ms. This condition can be met for WANs or LANs in different countries and regions.

[0121] In some embodiments, the timing diagram of the data processing method can be as follows: Figure 11 As shown. The voice acquisition service (32-bit) obtains the status information of the sound acquisition device or the ambient audio collected by the sound acquisition device, and generates a protocol message based on the status information or ambient audio. Then, it determines whether the remote voice control switch is on. If the remote voice control switch is off, the protocol message is sent to the local speech recognition service via local socket communication. The local speech recognition service (64-bit) first obtains the physical length of the protocol message. If the physical length is a first preset length, the protocol message is identified as a buffered message, and then the buffered message is parsed to obtain the service type and ambient audio. If the service type matches the preset type, speech recognition is performed on the user's voice in the ambient audio. If the physical length is a second preset length, the protocol message is identified as a status message, and then the status message is parsed to obtain the service type and status information. If the service type matches the preset type, the status information is updated.

[0122] The speech recognition service generates a feedback message and sends it to the speech acquisition service via local socket communication. The speech acquisition service then sends the next protocol message to the speech recognition service via the local socket.

[0123] If the remote voice control switch is enabled, the protocol message is sent to the voice recognition service of the designated terminal device via remote socket communication. The voice recognition service of the terminal device processes the protocol message in the same way as the display device.

[0124] In this embodiment, the speech acquisition service is built using a 32-bit architecture, which does not affect the existing system dependencies and improves system compatibility. The speech recognition service, built using a 64-bit architecture, further reduces CPU utilization by about 4% after enabling vector instructions and optimization, and improves the smoothness of operation in high-resource scenarios when enabling far-field functionality. The acquisition and recognition services are separated; only the architecture of the acquisition service is replaced, without affecting the recognition service. Local socket communication can be replaced with remote call socket communication to support distributed deployment. Custom data protocols and communication timing constraints are used to achieve near-hardware parsing performance with a single parsing operation, meeting model latency requirements and improving system compatibility and interconnectivity.

[0125] Some embodiments of this application also provide a computer-readable storage medium that can store a program. When the computer storage medium is configured in a display device or server, the program, when executed, can include the program steps involved in the data processing method described in the above embodiments. The computer storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0126] This application provides an electronic device, which includes a processor and a memory for storing processor-executable instructions. The processor is configured to read executable instructions from the memory and execute the instructions to implement the data processing method described in the above embodiments.

[0127] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

Claims

1. A display device, characterized in that, include: monitor; Sound acquisition device; The controller is configured as follows: The voice acquisition service is controlled to obtain a first protocol message, the first protocol message including the status information of the sound acquisition device or the ambient audio acquired by the sound acquisition device; The voice acquisition service is controlled to send the first protocol message to the voice recognition service through local socket communication. The voice acquisition service and the voice recognition service are independent services. The voice acquisition service is built based on a first architecture, and the voice recognition service is built based on a second architecture. The word length of the first architecture is smaller than that of the second architecture. The word length is used to characterize data processing capabilities. The speech recognition service is controlled to obtain the status information in the first protocol message to update the status information of the sound collector in the speech recognition service, or to obtain the environmental audio in the first protocol message to perform speech recognition on the user's speech in the environmental audio.

2. The display device according to claim 1, characterized in that, The controller, which executes control over the speech recognition service to acquire the environmental audio from the first protocol message, and performs speech recognition on the user's speech in the environmental audio, is further configured to: Control the speech recognition service to obtain the physical length of the first protocol message; When the physical length is a first preset length, the speech recognition service is controlled to recognize the first protocol message as a buffer message; The speech recognition service is controlled to parse the buffered message to obtain the service type and ambient audio. The speech recognition service is controlled to perform speech recognition on the user's voice in the ambient audio when the service type matches the preset type.

3. The display device according to claim 2, characterized in that, After controlling the speech recognition service to perform speech recognition on the user's speech in the ambient audio, the controller is further configured to: The speech recognition service is controlled to generate a recognition feedback message, and the recognition feedback message is sent to the speech acquisition service via local socket communication; After receiving the recognition feedback message, the voice acquisition service sends a second protocol message to the voice recognition service via local socket communication. The second protocol message is acquired before the voice acquisition service receives the recognition feedback message.

4. The display device according to claim 2, characterized in that, The controller, which executes control over the speech recognition service to obtain the status information in the first protocol message to update the status information of the sound collector in the speech recognition service, is further configured to: Control the speech recognition service to obtain the physical length of the first protocol message; When the physical length is a second preset length, the speech recognition service is controlled to recognize the first protocol message as a status message; The speech recognition service is controlled to parse the status message to obtain the service type and status information; When the service type matches a preset type, the speech recognition service updates the status information of the sound collector in the speech recognition service.

5. The display device according to claim 4, characterized in that, After controlling the speech recognition service to update the status information of the sound collector in the speech recognition service, the controller is further configured to: The speech recognition service is controlled to generate a status feedback message, and the status feedback message is sent to the speech acquisition service via local socket communication; After receiving the status feedback message, the voice acquisition service sends a second protocol message to the voice recognition service via local socket communication. The second protocol message is acquired before the voice acquisition service receives the status feedback message.

6. The display device according to claim 1, characterized in that, The controller, which controls the voice acquisition service to send the first protocol message to the voice recognition service via local socket communication, is further configured to: The voice acquisition service is controlled to detect the on / off state of the voice remote control, and the on / off state is used to determine the recipient of the first protocol message; When the voice acquisition service detects that the switch is off, it sends a first protocol message to the voice recognition service of the display device via local socket communication.

7. The display device according to claim 6, characterized in that, After the control voice acquisition service detects the on / off state of the voice remote control, the controller is further configured to: When the voice acquisition service detects that the switch is on, it obtains the Internet Protocol address of the controlled terminal device from the address list. The voice acquisition service is controlled to send a first protocol message to the voice recognition service of the controlled terminal device corresponding to the Internet Protocol address via remote socket communication, so that the voice recognition service of the controlled terminal device can process the first protocol message.

8. The display device according to claim 7, characterized in that, Before controlling the voice acquisition service to detect the on / off state of voice remote control, the controller is further configured to: In response to a user's input command to activate remote voice control, the on / off state of the remote voice control is set to the 'on' state. Obtain terminal devices that are on the same network as the display device; Control the display to show the device controls corresponding to the terminal device; In response to a user's instruction to select the device control, the Internet Protocol address of the terminal device corresponding to the device control is obtained and stored in the address list.

9. The display device according to claim 2, characterized in that, The controller executes control over the speech recognition service to parse the buffered message, obtain the service type and ambient audio, and is further configured as follows: The speech recognition service is controlled to parse the buffered message to obtain the service type, ambient audio, and reference audio, wherein the reference audio is the original signal of the audio played by the display device; The controller, which controls the speech recognition service to perform speech recognition on the user's speech in the ambient audio, is further configured to: The speech recognition service is controlled to use an echo cancellation algorithm to remove the echo caused by playing the reference audio from the ambient audio, thereby obtaining the user's speech and recognizing the user's speech.

10. A data processing method, characterized in that, include: The voice acquisition service obtains a first protocol message, which includes the status information of the voice acquisition device or the ambient audio acquired by the voice acquisition device. The voice acquisition service sends the first protocol message to the voice recognition service through local socket communication. The voice acquisition service and the voice recognition service are independent services. The voice acquisition service is built on a first architecture, and the voice recognition service is built on a second architecture. The word length of the first architecture is smaller than that of the second architecture. The word length is used to characterize data processing capabilities. The speech recognition service obtains the status information or the ambient audio from the first protocol message.

Images