Projection equipment and far-field speech recognition method

Abstract

The invention relates to projection equipment and a far-field speech recognition method. The projection equipment comprises a first sound collector, a wireless transmission module, a controller, a signal processing module and an audio output device, wherein the audio output device is configured to output media sound and send recovery reference data corresponding to the media sound to the signal processing module; the signal processing module is configured to send recovery reference data to the wireless transmission module; the wireless transmission module is configured to send the recovery reference data to the controller; the first sound collector is configured to collect first external sound data; the controller is configured to: signal align the recovery reference data and the first external sound data; according to the recovery reference data after signal alignment, performing echo cancellation on media sound in the first external sound data after signal alignment to obtain far-field voice data; and performing far-field speech recognition on the far-field speech data. By adopting the projection equipment, the accuracy of far-field speech recognition can be improved.

Description

Technical Field

[0001] The present application relates to the technical field of projection equipment, and in particular to a projection equipment and a far-field speech recognition method. Background Art

[0002] With the rapid development of smart devices and IoT technology, more and more laser TVs are beginning to be equipped with far-field voice functions, which can greatly improve the convenience of laser TV operation. In addition, many high-end laser TVs currently have audio output devices placed in the projection screen to enhance the audio and video experience.

[0003] In traditional technology, screen-sounding laser TVs collect far-field voice and perform far-field voice recognition through the sound collector in the projection host.

[0004] However, when performing far-field speech recognition, the media sound output by the audio output device in the projection screen can easily interfere with the user's far-field speech, or even mask the user's far-field speech, resulting in reduced accuracy of far-field speech recognition. Summary of the Invention

[0005] The present application provides a projection device and a far-field speech recognition method to solve the problem of low accuracy in far-field speech recognition.

[0006] In a first aspect, some embodiments provide a projection device, comprising: a projection screen and a projection host, the projection host being configured to project a display image onto the projection screen;

[0007] The projection host includes a first sound collector, a wireless transmission module and a controller, and the projection screen includes a signal processing module and an audio output device, wherein:

[0008] The projection screen is configured to display a user interface;

[0009] The audio output device is configured to output media sound and, when outputting the media sound, send the corresponding acquisition reference data of the media sound to the signal processing module;

[0010] The signal processing module is configured to send the retrieved reference data to the wireless transmission module;

[0011] A wireless transmission module is configured to receive the back-collecting reference data and send the back-collecting reference data to the controller;

[0012] a first sound collector configured to collect first external sound data and send the first external sound data to the controller;

[0013] The controller is configured as:

[0014] When the retrieved reference data and the first external sound data are received, signal alignment is performed on the retrieved reference data and the first external sound data;

[0015] performing echo cancellation on the media sound in the first external sound data after signal alignment according to the echoed reference data after signal alignment to obtain far-field voice data;

[0016] Perform far-field speech recognition on far-field speech data.

[0017] Technical effect: On the one hand, the audio output device in the projection screen transmits back the echo reference data corresponding to the media sound while outputting the media sound, and transmits it to the projection host via wireless transmission. On the other hand, the media sound output by the audio output device in the projection screen can be captured by the first sound collector in the projection host; in this way, the projection host can obtain the echo reference data corresponding to the media sound, as well as the first external sound data mixed with the media sound and the far-field voice, and then eliminate the phase difference caused by the different transmission times of the two through signal alignment, thereby improving the echo cancellation effect, and then use the far-field voice with the media sound eliminated to perform far-field voice recognition, which can effectively reduce the interference of the media sound on the far-field voice recognition and improve the accuracy of far-field voice recognition.

[0018] In a second aspect, some embodiments further provide a far-field speech recognition method, which is applied to a projection host in a projection device, wherein the projection device further includes a projection screen, and the projection host is configured to project a display image onto the projection screen; the method comprising:

[0019] Collecting first external sound data and receiving retrieved reference data, the retrieved reference data being wirelessly transmitted to the projection host by the projection screen when outputting media sound;

[0020] Performing signal alignment on the retrieved reference data and the first external sound data;

[0021] performing echo cancellation on the media sound in the first external sound data after signal alignment according to the echoed reference data after signal alignment to obtain far-field voice data;

[0022] Perform far-field speech recognition on far-field speech data.

[0023] Technical effect: On the one hand, the audio output device in the projection screen transmits back the echo reference data corresponding to the media sound while outputting the media sound, and transmits it to the projection host via wireless transmission. On the other hand, the media sound output by the audio output device in the projection screen can be captured by the first sound collector in the projection host; in this way, the projection host can obtain the echo reference data corresponding to the media sound, as well as the first external sound data mixed with the media sound and the far-field voice, and then eliminate the phase difference caused by the different transmission times of the two through signal alignment, thereby improving the echo cancellation effect, and then use the far-field voice with the media sound eliminated to perform far-field voice recognition, which can effectively reduce the interference of the media sound on the far-field voice recognition and improve the accuracy of far-field voice recognition.

[0024] In a third aspect, some embodiments further provide a far-field speech recognition method, which is applied to a projection screen in a projection device, wherein the projection device further includes a projection host configured to project a display image onto the projection screen; the method comprising:

[0025] Output media sound, and when outputting the media sound, wirelessly transmit the collected reference data corresponding to the media sound to the projection host, so that the projection host performs signal alignment on the collected reference data and the collected first external sound data, and performs echo cancellation on the media sound in the first external sound data after signal alignment according to the collected reference data after signal alignment to obtain far-field voice data, and performs far-field voice recognition on the far-field voice data.

[0026] Technical effect: On the one hand, the audio output device in the projection screen transmits back the echo reference data corresponding to the media sound while outputting the media sound, and transmits it to the projection host via wireless transmission. On the other hand, the media sound output by the audio output device in the projection screen can be captured by the first sound collector in the projection host; in this way, the projection host can obtain the echo reference data corresponding to the media sound, as well as the first external sound data mixed with the media sound and the far-field voice, and then eliminate the phase difference caused by the different transmission times of the two through signal alignment, thereby improving the echo cancellation effect, and then use the far-field voice with the media sound eliminated to perform far-field voice recognition, which can effectively reduce the interference of the media sound on the far-field voice recognition and improve the accuracy of far-field voice recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0028] Figure 1 A schematic diagram of an operation scenario between a projection device and a control device provided in some embodiments of the present application;

[0029] Figure 2 A schematic diagram of the hardware configuration of a projection device provided in some embodiments of the present application;

[0030] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0031] Figure 4 A schematic diagram of the software configuration of a projection device provided in some embodiments of the present application;

[0032] Figure 5 A schematic diagram of the structure of a projection device provided in some embodiments of the present application;

[0033] Figure 6 A timing diagram illustrating a far-field speech recognition function implemented through a projection device in some embodiments of the present application;

[0034] Figure 7 A schematic diagram of an audio signal of voice data to be tested provided in some embodiments of the present application;

[0035] Figure 8 A schematic diagram of a process for far-field speech detection provided in some embodiments of the present application;

[0036] Figure 9 Schematic diagram of audio and reverse audio provided in some other embodiments of the present application;

[0037] Figure 10 A schematic diagram of a flow chart of decoding speed feedback adjustment provided in other embodiments of the present application;

[0038] Figure 11 A flowchart of a method for implementing far-field speech recognition provided in some embodiments of the present application. DETAILED DESCRIPTION

[0039] The following embodiments are described in detail, with examples illustrated in the accompanying drawings. When the following description refers to the drawings, identical numbers in different figures represent identical or similar elements unless otherwise indicated. The embodiments described in the following embodiments are not intended to represent all possible implementations consistent with the present application. They are merely examples of systems and methods consistent with certain aspects of the present application, as detailed in the claims.

[0040] It should be noted that the brief descriptions of terms in this application are only for the purpose of facilitating the understanding of the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their ordinary and usual meanings.

[0041] In the specification and claims of this application and the accompanying drawings, the terms "first," "second," "third," etc. are used to distinguish similar or similar objects or entities, and are not necessarily intended to limit a particular order or sequence, unless otherwise noted. It should be understood that the terms used in this manner are interchangeable under appropriate circumstances.

[0042] The terms "comprise," "include," and "have," and any variations thereof, are intended to cover but not exclude inclusion; for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0043] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functionality associated with that element.

[0044] In the embodiments of the present application, the projection device 200 generally refers to a device capable of displaying images and processing data. For example, the projection device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, and the like that use laser light sources.

[0045] Figure 1 This is a schematic diagram of an operation scenario between a projection device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG, a user can operate the projection device 200 through touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0046] The mobile terminal 300 can function as a control device for user interaction with the projection device 200. It can also function as a communication device, establishing a communication connection with the projection device 200 and exchanging data. In some embodiments, the mobile terminal 300 can install software applications with the projection device 200, enabling communication via a network communication protocol for one-to-one control and data communication. Audio and video content displayed on the mobile terminal 300 can also be transmitted to the projection device 200 for synchronized display.

[0047] like Figure 1 As shown in FIG, the projection device 200 also communicates data with the server 400 through various communication methods. The projection device 200 may be allowed to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0048] The projection device 200 may provide a broadcast receiving television function, and may also additionally provide an intelligent network television function with computer support functions, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0049] Figure 2 Some embodiments of this application provide Figure 1 FIG. 2 is a block diagram of the hardware configuration of the projection device 200.

[0050] In some embodiments, the projection device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a projection screen 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0051] In some embodiments, a projection device includes a projection host and a projection screen, wherein the projection host is configured to project a display image onto the projection screen. The projection screen includes at least a signal processing module and an audio output device, and the projection host includes at least a first sound collector, a wireless transmission module, and a controller. The projection host may also include at least one of a tuner / demodulator, a communication device, a detector, a device interface, a controller, a projection screen, an audio output device, a memory, a power supply, and a user input interface.

[0052] In some embodiments, detector 230 is used to collect signals from the external environment or external interactions. For example, detector 230 may include a light receiver, such as a sensor for collecting ambient light intensity; or an image collector, such as a camera, for collecting external environmental scenes, user attributes, or user interaction gestures; or a sound collector, such as a microphone, for receiving external sounds.

[0053] In some embodiments, projection screen 260 includes a display component for presenting images and a driver component for driving image display. Projection screen 260 is configured to receive image signals output from controller 250 for display. For example, projection screen 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces.

[0054] In some embodiments, the communication device 220 is a component used to communicate with the first external device or server 400 according to various communication protocols. The projection device 200 can be equipped with multiple communication devices 220 depending on the communication methods supported. For example, if the projection device 200 supports wireless network communication, the projection device 200 can be equipped with a communication device 220 that includes WiFi functionality. If the projection device 200 supports Bluetooth communication, the projection device 200 also needs to be equipped with a communication device 220 that includes Bluetooth functionality.

[0055] The communication device 220 can establish a communication connection between the projection device 200 and the first external device or server 400 via a wireless or wired connection. A wired connection can use components such as a data cable and an interface to display the projection device 200 and the personalized recommendation reasons. A wireless connection can use wireless signals or a wireless network to display the projection device 200 and the personalized recommendation reasons. The projection device 200 can establish a direct connection with the first external device or indirectly establish a connection through a gateway, router, or connection device.

[0056] In some embodiments, controller 250 may include at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processor, and a power processor, as well as first to nth interfaces for input / output. Controller 250 controls the operation of the projection device and responds to user operations using various software control programs stored in memory. Controller 250 controls the overall operation of projection device 200.

[0057] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0058] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the projection screen 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0059] In some embodiments, the audio output device 270 may be a speaker of the projection device 200, or an external audio output device connected to the projection device 200. For the external audio output device connected to the projection device 200, the projection device 200 may further be provided with an external audio output terminal, through which the audio output device may be connected to the projection device 200 to output the sound of the projection device 200.

[0060] In some embodiments, the user input interface 280 may be configured to receive instructions from a user.

[0061] Figure 3 Some embodiments of this application provide Figure 1 The hardware configuration diagram of the control device in the figure is as follows. Figure 3 As shown, the control device 100 may include: a controller 110, a communication interface 130, a user input / output interface, a memory, and a power supply.

[0062] The control device 100 is configured to control the projection device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the projection device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the projection device 200 .

[0063] In some embodiments, the control device 100 may be a smart device. For example, the control device 100 may be installed with various applications for controlling the projection device 200 according to user needs.

[0064] In some embodiments, as Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the projection device 200 .

[0065] The controller 110 includes a processor 112, RAM 113, ROM 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between internal components and external and internal data processing functions.

[0066] Under the control of the controller 110, the communication interface 130 communicates control signals and data signals with the projection device 200. The communication interface 130 may include at least one of a WiFi chip 131, a Bluetooth module 132, an NFC module 133, or other near field communication modules.

[0067] The user input / output interface 140 includes at least one of a microphone 141 , a touch panel 142 , a sensor 143 , a button 144 and other input interfaces.

[0068] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with the communication interface 130, such as a WiFi, Bluetooth, or NFC module, to encode user input commands via the WiFi protocol, Bluetooth protocol, or NFC protocol and transmit them to the projection device 200.

[0069] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0070] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0071] To facilitate user interaction, in some embodiments, projection device 200 may run an operating system. An operating system is a computer program used to manage and control the hardware and software resources of projection device 200. The operating system can provide a user interface (to control the projection device), allowing user interaction with projection device 200 and supporting the execution of various application programs.

[0072] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for projection equipment.

[0073] The operating system can be divided into different modules or layers according to the functions implemented, e.g. Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom: the application layer (abbreviated as "application layer"), the application framework layer (abbreviated as "framework layer"), the system library layer and the kernel layer.

[0074] In some embodiments, the application layer provides services and interfaces for applications, enabling projection device 200 to run applications and interact with users based on these applications. The application layer can host at least one application, which can include built-in window programs, system settings programs, clock programs, and other applications provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0075] The framework layer provides applications with an application programming interface (API) and programming framework. The application framework layer includes predefined functions. The application framework layer acts as a processing center, determining the actions taken by applications in the application layer. Through the API, applications can access system resources and services during execution.

[0076] like Figure 4As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to the system location service; a package manager for retrieving various information related to the application packages currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0077] In some embodiments, the activity manager is used to manage the lifecycle of each application and common navigation back functions, such as controlling application exit, opening, and back. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, taking screenshots, and controlling changes in display windows, such as shrinking, shaking, or distorting the display window.

[0078] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0079] In some embodiments, the kernel layer is a functional layer between the hardware and software of the projection device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 4 As shown, the kernel layer can be configured with hardware drivers, and the drivers included in the kernel layer can be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0080] It should be noted that the above example is only a simple division of the operating system functions and does not limit the specific operating system form of the projection device 200 in the embodiment of the present application. Depending on factors such as the function of the projection device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0081] To enhance the audiovisual experience of laser TVs, a special form of screen-generating laser TV has been developed. This technology uses a vibrating screen to propel air in front of and behind the screen, generating sound, utilizing the entire screen to produce sound and enhancing the sound quality. This technology provides users with a "sound-visual integration" experience, seamlessly blending picture and sound in three dimensions, similar to the experience in a top-tier cinema. These screen-generating laser TVs incorporate an audio output device within the projection screen. To streamline the design of the entire projection unit and projection screen, the signal cable connecting the projection unit and the screen is eliminated. The projection unit wirelessly drives the audio output device in the projection screen to play audio. With the rapid development of smart devices and the Internet of Things (IoT), more and more laser TVs are now equipped with far-field voice capabilities, which significantly enhance their ease of use. Traditionally, screen-generating laser TVs use a sound collector within the projection unit to capture and perform far-field voice recognition.

[0082] However, echo cancellation usually requires a certain amount of computing power, so it is usually implemented by the projection host. However, when the screen-sounding laser TV plays multimedia content, the external sound collected by the sound collector in the projection host includes the media sound output by the audio output device in the projection screen, and the echo signal of the media sound cannot be transmitted back to the projection host in time, making it difficult for the projection host to effectively eliminate the media sound in the external sound. As a result, the far-field voice is easily affected by the media sound or even masked, resulting in reduced accuracy of far-field voice recognition.

[0083] In some embodiments, a projection device is provided, the projection device comprising a projection screen and a projection host, the projection host being configured to project a display image onto the projection screen;

[0084] The projection host includes a first sound collector, a wireless transmission module and a controller, and the projection screen includes a signal processing module and an audio output device, wherein:

[0085] The projection screen is configured to display a user interface;

[0086] The audio output device is configured to output media sound and, when outputting the media sound, send the corresponding acquisition reference data of the media sound to the signal processing module;

[0087] The signal processing module is configured to send the retrieved reference data to the wireless transmission module;

[0088] A wireless transmission module is configured to receive the back-collecting reference data and send the back-collecting reference data to the controller;

[0089] a first sound collector configured to collect first external sound data and send the first external sound data to the controller;

[0090] The controller is configured as:

[0091] When the retrieved reference data and the first external sound data are received, signal alignment is performed on the retrieved reference data and the first external sound data;

[0092] performing echo cancellation on the media sound in the first external sound data after signal alignment according to the echoed reference data after signal alignment to obtain far-field voice data;

[0093] Perform far-field speech recognition on far-field speech data.

[0094] The projection device may refer to a screen-sounding laser projection device, such as a screen-sounding laser TV, etc. The screen-sounding laser projection device may refer to a projection device that combines laser display technology and screen-sounding technology.

[0095] In some embodiments, as Figure 5 As shown, the projection device includes at least a projection screen 510 and a projection host 520. The projection host 520 is configured to project a display image onto the projection screen 510. The projection screen 510 includes at least a signal processing module 511 and an audio output device 512. The projection host 520 includes at least a first sound collector 521, a wireless transmission module 523, and a controller 522. The projection host 520 may also include at least one of a tuner / demodulator, a communication device, a detector, a device interface, a controller, a display, an audio output device, a memory, a power supply, and a user input interface. It is understood that the projection host may be equipped with more powerful processors and functional modules, with greater computing power and heat dissipation capabilities, to perform more complex processing tasks. To maintain its thinness and aesthetics, the projection screen is equipped with relatively limited hardware resources and computing power, as well as relatively limited heat dissipation capabilities. Therefore, it is primarily used for display and simple processing tasks such as audio driving and local image enhancement.

[0096] Media sound may refer to the sound of multimedia content. When a projection device plays multimedia content, it may simultaneously display the image of the multimedia content through a projection screen and display the sound of the multimedia content through an audio output device.

[0097] The retrieved reference data may refer to a data copy of the media sound output by the audio output device.

[0098] The first external sound data may refer to the sound in the environment that can be collected by the first sound collector, and may include media sound output by the audio output device, far-field voice, and other sounds that may exist in the environment, such as rain, wind, knocking, walking, animal calls, etc.

[0099] Far-field voice can refer to voice commands issued by the user that can be captured by the sound collector of the projection device.

[0100] Far-field voice data may refer to the sound data after the media sound is eliminated from the first external sound data. It can be understood that, in addition to far-field voice, the far-field voice data may also include other interfering sounds in the environment. Compared with media sounds, other interfering sounds have less impact on the accuracy of far-field voice recognition. Media sounds may include human voices, and media sounds are emitted by the projection screen. The distance between the projection screen and the projection host is usually smaller than the distance between the user and the projection host. Therefore, regardless of the volume or sound type, the media sound has a greater impact on the accuracy of far-field voice recognition. Eliminating media sounds can effectively improve the accuracy of far-field voice recognition.

[0101] Acoustic echo refers to the sound output by an audio output device that propagates through multiple paths and is picked up by a sound collector. Because the sound is reflected by multiple paths, echoes with varying delays are generated. Acoustic echoes can include direct echoes and indirect echoes. Direct echoes refer to the sound broadcast by the audio output device that enters the sound collector directly without any reflection. This type of echo has the shortest delay and is directly related to factors such as the distance and angle between the audio output device and the sound collector. Indirect echoes refer to the collection of sounds broadcast by the audio output device that enter the sound collector after one or more reflections from different paths. Any movement of objects in space will alter the echo path. Therefore, indirect echoes are characterized by multipath and time-varying characteristics. Echo cancellation in this embodiment can include at least one of direct echo cancellation and indirect echo cancellation. Both direct and indirect echo cancellation require signal alignment between the echoed reference data and the first external sound data to accurately cancel the media sound from the first external sound data, achieving echo cancellation.

[0102] In some feasible implementations, signal alignment methods may include fixed delay estimation, adaptive filter, frequency domain alignment, feature point matching alignment, machine learning or deep learning, etc. The specific selection can be made according to actual conditions, and this embodiment does not limit this.

[0103] For example, Figure 6As shown, during the process of playing multimedia content, the projection device displays the media image of the multimedia content through the projection screen, and synchronously outputs the media sound of the multimedia content through the audio output device. The output media sound can be transmitted to the user's ears through the air, or it can be collected by the first sound collector after being transmitted through the air. The first sound collector can collect various sounds in the environment including media sounds and far-field voices, generate first external sound data, and transmit the generated first external sound data to the controller; while outputting the media sound, the audio output device can also back up the media sound data and use the backup copy as a reference for re-collection. The data is sent to the signal processing module, and the signal processing module further wirelessly transmits the retrieved reference data to the wireless transmission module in the projection host. After receiving the retrieved reference data, the wireless transmission module transmits the retrieved reference data to the controller; after receiving the first external sound data and the retrieved reference data, the controller can first align the signals of the first external sound data and the retrieved reference data, and then use the aligned retrieved reference data to perform echo cancellation on the first external sound data after signal alignment, eliminate the interference of the media sound in the first external sound data on the far-field voice, obtain the far-field voice data, and then perform far-field voice recognition based on the far-field voice data.

[0104] In some feasible implementations, the process of the audio output device outputting media sound of multimedia content may include: receiving media sound data sent by the projection host, performing sound effect processing, enhancement processing, etc. on the received media sound data, and then outputting the processed media sound data.

[0105] In some feasible implementations, the process of performing far-field speech recognition on far-field speech data may include: using a preset acoustic model to extract features of the far-field speech data to obtain far-field speech features, and performing similarity detection between the far-field speech features and the preset wake-up word features. When the similarity is higher than a preset similarity threshold, it is determined that the far-field speech wake-up function is triggered, a communication connection can be established with the cloud, and audio streaming can be transmitted to the cloud in real time for the cloud to perform speech recognition through a speech recognition model, and the speech recognition results can be sent to the projection device. The projection device can display the speech recognition results through a projection screen or an audio output device.

[0106] In this embodiment, on the one hand, the audio output device in the projection screen transmits back the echo reference data corresponding to the media sound while outputting the media sound, and transmits it to the projection host via wireless transmission. On the other hand, the media sound output by the audio output device in the projection screen can be captured by the first sound collector in the projection host. In this way, the projection host can obtain the echo reference data corresponding to the media sound, as well as the first external sound data mixed with the media sound and the far-field voice, and then eliminate the phase difference caused by the different transmission times of the two through signal alignment, thereby improving the echo cancellation effect, and then use the far-field voice with the media sound eliminated to perform far-field voice recognition, which can effectively reduce the interference of the media sound on the far-field voice recognition and improve the accuracy of far-field voice recognition.

[0107] In some embodiments, during the process of sending the retrieved reference data to the wireless transmission module, the signal processing module is further configured to:

[0108] When the far-field voice function is triggered, the reference data is sent back to the wireless transmission module.

[0109] It should be noted that since the acquired reference data needs to be transmitted in real time, transmitting it wirelessly from the projection screen to the projection host in real time increases the burden of wireless signal transmission and processing, and consumes a lot of energy. Furthermore, the projection host will also continuously perform echo cancellation processing due to the transmission of the acquired reference data, which consumes the projection host's computing power. However, echo cancellation is usually only required when the user uses the far-field voice function. If the far-field voice function is not used, there is no need for echo cancellation, which will result in unnecessary waste of resources.

[0110] Among them, the far-field voice function can refer to the function of the projection device to effectively receive and process the user's voice input at a longer distance. It allows the user to interact with the projection device through natural language without having to be close to or very close to the microphone.

[0111] Exemplarily, the audio output device can continuously transmit the playback reference data corresponding to the media sound to the signal processing module when outputting the media sound. After receiving the playback reference data, the signal processing module can perform operations such as storage, processing or transmission; during the operation of the signal processing module, the signal processing module can detect whether the far-field voice function is triggered; before the far-field voice function is detected to be triggered, the signal processing module can perform operations such as storage or processing after receiving the playback reference data, but will not send the playback reference data to the projection host; after the far-field voice function is not detected to be triggered, the signal processing module can perform operations such as storage or processing after receiving the playback reference data, and can also send the playback reference data to the projection host.

[0112] In some feasible implementations, the method for detecting whether the far-field voice function has been triggered may include: the projection screen independently determining whether the far-field voice function has been triggered, or the projection host determining whether the far-field voice function has been triggered. After the projection host determines that the far-field voice function has been triggered, it sends a far-field voice function activation instruction to the projection screen. After receiving the far-field voice function activation instruction sent by the projection host, the projection screen determines that the far-field voice function has been triggered. The method for determining whether the far-field voice function has been triggered may include voice recognition, voice wake-up, human voice recognition, key triggering, etc. The specific method can be determined based on actual circumstances and is not limited in this embodiment.

[0113] In this embodiment, whether the far-field voice function is triggered is determined by the signal processing module on the projection screen. Only when the far-field voice function is triggered is the echo reference data sent to the projection host. This can effectively reduce unnecessary transmission of echo reference data and echo cancellation processing, thereby effectively saving communication resources and computing power resources, and ensuring the normal operation of other functions of the projection device.

[0114] In some embodiments, the projection screen includes a second sound collector, which is configured to collect second external sound data; when the far-field voice function is triggered, in the process of sending the collected reference data to the wireless transmission module, the signal processing module is further configured to:

[0115] Acquiring second external sound data and retrieved reference data;

[0116] Performing echo cancellation on the second external sound data according to the retrieved reference data to obtain the voice data to be tested;

[0117] When far-field voice data exists in the voice data to be tested, it is determined that the far-field voice function is triggered, and the collected reference data is sent to the wireless transmission module.

[0118] It should be noted that compared to the projection screen, the projection host has higher computing power, so the projection host has more options for detecting whether the far-field voice function is triggered. However, after detecting that the far-field voice function is triggered, the projection host still needs to communicate with the projection screen, which not only occupies communication resources, but also causes a delay in the startup time of echo cancellation, which may affect the effectiveness and accuracy of the far-field voice function.

[0119] Detecting whether the far-field voice function has been triggered may include detecting whether a human voice exists in the second external sound data. However, media audio often also contains human voices. If the second external sound data contains both far-field voice and human voices in the media audio, detecting human voices may result in erroneous triggering of the far-field voice function.

[0120] The second sound collector may be a sound collector deployed in the projection screen. In some feasible embodiments, compared to the projection host, the projection screen has greater space and resource constraints for reasons such as aesthetics and thinness. Therefore, the first sound collector set in the projection host may be a microphone array to ensure comprehensive and accurate sound collection, while the second sound collector set in the projection screen may be a single microphone, effectively balancing the actual needs of the projection screen and the actual needs of sound collection.

[0121] Exemplarily, during the operation of the projection screen, the second sound collector can continuously collect second external sound data and transmit the second external sound data to the signal processing module. Synchronously, the audio output device can transmit the collected reference data corresponding to the media sound to the signal processing module while outputting the media sound. Because the second external sound data collected by the second sound collector and the collected reference data have not been wirelessly transmitted before being compared, there is no problem of audio delay. After receiving the second external sound data and the collected reference data, the signal processing module can perform echo cancellation on the second external sound data according to the collected reference data to obtain the voice data to be tested, and detect whether there is far-field voice data in the voice data to be tested. If there is far-field voice data in the voice data to be tested, it can be determined that the far-field voice function is triggered, and then the collected reference data can be started to be sent to the wireless transmission module. If no far-field voice data is detected in the voice data to be tested, there is no need to send the collected reference data to the wireless transmission module, and the detection of whether there is far-field voice data in the voice data to be tested can be continued.

[0122] In this embodiment, by setting a second sound collector on the projection screen, the projection screen can accurately determine whether the far-field voice function is triggered. This can effectively reduce the communication resources required to determine whether the far-field voice function is triggered, improve the timeliness of echo cancellation, and thus improve the effectiveness and accuracy of the far-field voice function.

[0123] In some embodiments, when far-field voice data exists in the voice data to be tested, in the process of determining whether the far-field voice function is triggered, the signal processing module is further configured to:

[0124] When it is detected that the energy amplitude of the voice data to be measured exceeds a preset energy amplitude threshold, it is determined that far-field voice data exists in the voice data to be measured, and it is determined that the far-field voice function is triggered.

[0125] It should be noted that the external sounds collected by the sound collector may include not only media sounds and far-field voices, but also environmental interference sounds such as footsteps, rain, wind, and noise. When the external sounds do not contain far-field voices, after echo cancellation using the collected reference data, the voice data to be tested still contains environmental interference sounds and noise that are not far-field voices. It is possible to determine whether the voice data to be tested contains human voices through methods such as timbre recognition or voice recognition, thereby determining whether the far-field voice function is triggered.

[0126] However, the methods of timbre recognition or voice recognition require high computing power, and the frequency of detecting whether there is far-field voice data in the voice data to be tested is high. Frequent timbre recognition or voice recognition may place a heavy burden on the projection screen and affect the normal use of other functions of the projection screen.

[0127] Energy amplitude refers to the instantaneous or average intensity level of the audio signal corresponding to the voice data being tested. Amplitude is a key parameter that describes the characteristics of an audio waveform, indicating the degree to which the audio signal deviates from zero at a given moment. The energy amplitude of environmental interference sounds and noise is typically low, while the energy amplitude of far-field speech is typically high. Therefore, by setting an energy amplitude threshold, we can effectively distinguish far-field speech from environmental interference sounds and noise, enabling detection of far-field speech data.

[0128] For example, after echo cancellation obtains the voice data to be tested, the energy amplitude of the audio signal corresponding to the voice data to be tested can be detected to determine whether the energy amplitude exceeds a preset energy amplitude threshold. If the energy amplitude is detected to exceed the preset energy threshold, it is determined that far-field voice data exists in the voice data to be tested. From the moment the energy amplitude is detected to exceed the preset energy threshold, it is determined that the far-field voice function is triggered, and the back-transmission of the back-sampled reference data begins from that moment. For example, in a living room scenario, the volume of the user's voice reaching the microphone is approximately 65dB, corresponding to an amplitude of approximately 1000smpl. In this case, 1000smpl can be set as the energy amplitude threshold. When the energy amplitude is detected to have reached the energy amplitude threshold, the back-transmission of the back-sampled reference data begins.

[0129] In some possible implementations, such as Figure 7 As shown, the energy amplitude of the audio signal of the voice data to be measured reaches the preset energy amplitude threshold at time t1, so the sampling reference data can be transmitted back starting from time t1.

[0130] In this embodiment, by limiting the energy amplitude threshold, far-field speech can be effectively distinguished from environmental interference sounds and noise, thereby realizing the detection of far-field speech data and improving the accuracy of triggering the far-field speech function. Reference data is retrieved and transmitted back only after the far-field speech data is detected, which can reduce unnecessary data transmission and data processing and improve resource utilization.

[0131] In some embodiments, the signal processing module includes a bandpass filter; when detecting that the energy amplitude of the voice data to be tested exceeds a preset energy amplitude threshold, in the process of determining that far-field voice data exists in the voice data to be tested, the signal processing module is further configured to:

[0132] The voice data to be tested is filtered through a bandpass filter to obtain the target human voice data corresponding to the preset human voice signal frequency band;

[0133] When it is detected that the energy amplitude of the target human voice data exceeds a preset energy amplitude threshold, it is determined that far-field voice data exists in the voice data to be tested.

[0134] It should be noted that for louder environmental interference sounds and noises, such as the sound of musical instruments and objects falling, it is difficult to distinguish them by limiting the energy amplitude threshold, which may still lead to false triggering of the far-field voice function.

[0135] A bandpass filter may refer to an electronic filter that allows signals within a specific frequency range to pass through while attenuating frequency components outside of this range. The bandpass filter in this embodiment may allow only signals within a predetermined vocal frequency band to pass through while attenuating signals in other frequency bands, thereby filtering out sounds other than the human voice.

[0136] For example, after the voice data to be tested is obtained by echo cancellation, the signals in the signal frequency bands other than the preset human voice signal frequency band in the voice data to be tested can be first subjected to a band-pass filter to obtain the target human voice data corresponding to the preset human voice signal frequency band; then the energy amplitude of the audio signal corresponding to the target human voice data is detected to determine whether the energy amplitude exceeds the preset energy amplitude threshold. When it is detected that the energy amplitude exceeds the preset energy threshold, it is determined that far-field voice data exists in the target human voice data. From the moment when the energy amplitude is detected to exceed the preset energy threshold, it is determined that the far-field voice function is triggered, and the reference data is transmitted back from this moment.

[0137] In some possible implementations, such as Figure 8As shown, the second sound collector collects the second external sound data and transmits it to the signal processing module. At the same time, the audio output device transmits the corresponding collected reference data of the output media sound back to the signal processing module. The signal processing module can receive the second external sound data and the collected reference data, record the second external sound data as audio A, record the collected reference data as audio B, and subtract audio B from audio A to obtain audio C. Audio C still contains far-field speech and ambient noise. The implementation process of audio A subtracting audio B includes: Figure 9 As shown, each frequency point of audio B is first multiplied by -1 to obtain the inverse audio B*; then audio A and the inverse audio B* are added to cancel the energy of the frequency points in audio A that are the same as those in audio B, and obtain audio C.

[0138] Human voice usually generates energy in the human voice signal frequency band between 500Hz and 4kHz, while other sounds such as noise and musical instruments are usually distributed in different frequency bands. Therefore, a bandpass filter can be designed to filter out the non-human voice signal frequency band below 500Hz and above 4kHz. According to the cutoff frequency formula:

[0139]

[0140] Where F is the cutoff frequency; R is the resistance of the resistor in the bandpass filter; C is the capacitance of the capacitor in the bandpass filter; 2π refers to twice the pi, which is used to convert the time constant into frequency.

[0141] The high-pass filter filters audio below 500Hz. Set F = 500Hz and select appropriate resistor values ​​and capacitor capacitance. The low-pass filter filters audio above 4kHz. Set F = 4kHz and select appropriate resistor values ​​and capacitor capacitance. After filtering audio C, audio D is obtained. Energy determination is performed on audio D. When the energy amplitude of audio D reaches the energy amplitude threshold, the reference data transmission is retrieved.

[0142] In this embodiment, by filtering out sounds in signal frequency bands other than the human voice signal frequency band through a bandpass filter, the interference of environmental interference sounds and noise can be effectively eliminated, the detection accuracy of far-field voice can be improved, and thus the accuracy of triggering the far-field voice function can be improved.

[0143] In some embodiments, during the process of performing signal alignment on the retrieved reference data and the first external sound data, the controller is further configured to:

[0144] Decoding the retrieved reference data to obtain a retrieved reference signal, and decoding the first external sound data to obtain a first external sound signal;

[0145] Obtaining a first timestamp in the retrieved reference signal and a second timestamp in the first external sound signal, wherein the first timestamp is used to indicate the time when the retrieved reference signal is emitted from the audio output device, and the second timestamp is used to indicate the time when the first sound collector collects the first external sound signal;

[0146] Signal alignment is performed on the retrieved reference data and the first external sound data according to a preset sound propagation time range and a difference value between the second timestamp and the first timestamp.

[0147] It should be noted that there is a signal delay in the wireless transmission process of the retrieved reference data, which makes it difficult to align the first external sound data collected by the first sound collector with the retrieved reference data in the time domain, thereby affecting the echo cancellation effect.

[0148] Among them, the first timestamp is used to characterize the time when the collected reference signal is emitted from the audio output device. The second timestamp is used to characterize the time when the first sound collector collects the first external sound signal. In the absence of signal delay, the first timestamp and the second timestamp can be aligned according to the sound propagation duration, so that the first external sound data and the collected reference data are difficult to align in the time domain. However, during the wireless transmission process, the collected reference data may have a signal delay, resulting in the time difference between the first external sound data and the collected reference data not only being caused by sound propagation, but the signal delay caused by wireless transmission is not stable and will change with the current actual situation, resulting in poor echo cancellation effect of wireless transmission of collected reference data.

[0149] The sound propagation duration range can be determined based on the sound propagation duration. It can be equal to the sound propagation duration, or the sound propagation duration can be added to or subtracted from a preset error value to determine the sound propagation market range. For example, if the sound propagation duration is t, the sound propagation duration range can be t or t±t', where t' can be a preset error value.

[0150] Exemplarily, after receiving the echo reference data and the first external sound data, the controller can first decode the echo reference data into a echo reference signal, and decode the first external sound data into a first external sound signal, extract the first timestamp from the echo reference signal, and extract the second timestamp from the first external sound signal; then, calculate the difference between the first timestamp and the second timestamp, and detect whether the difference falls within the preset sound propagation time range. If it is detected that the difference falls within the preset sound propagation time range, it can be said that the echo reference data and the first external sound data are signal-aligned, and echo cancellation can be performed; if it is detected that the difference does not fall within the preset sound propagation time range, it can be said that the echo reference data and the first external sound data are signal-aligned, and echo cancellation can be performed; The reference data and the first external sound data are not signal-aligned, and the echo cancellation effect is poor at this time. In this case, you can wait until the signal-aligned retrieved reference data or the first external sound data is decoded, and then use the signal-aligned retrieved reference signal and the first external sound signal to perform echo cancellation. You can also feedback-adjust the decoding speed of the retrieved reference data and the first external sound data to make the subsequent retrieved reference data and the first external sound data signals aligned, and continue to monitor whether the retrieved reference data and the first external sound data are signal-aligned, until the retrieved reference data and the first external sound data signals are aligned, and then use the signal-aligned retrieved reference data and the first external sound data to perform echo cancellation.

[0151] In this embodiment, based on the preset sound propagation time range and the difference value between the second timestamp and the first timestamp, it is possible to accurately determine whether there is a signal delay in the wireless transmission process of the retrieved reference data, and guide the signal alignment, which can improve the accuracy of the signal alignment and thus improve the echo cancellation effect.

[0152] In some embodiments, during the process of performing signal alignment on the retrieved reference data and the first external sound data based on the preset sound propagation duration range and the difference between the second timestamp and the first timestamp, the controller is further configured to perform at least one of the following:

[0153] When the difference between the second timestamp and the first timestamp is greater than a preset sound propagation duration range, the decoding speed of the first external sound data is slowed down according to the difference;

[0154] When the difference between the second timestamp and the first timestamp is smaller than the preset sound propagation time range, the decoding speed of the retrieved reference data is slowed down according to the difference.

[0155] It should be noted that although the method of aligning the decoded echo reference signal and the first external sound signal and then performing echo cancellation can achieve signal alignment, it will cause the decoded echo reference signal and the first external sound signal to accumulate, occupying storage resources and resulting in reduced resource utilization.

[0156] For example, Figure 10 As shown, let the first timestamp be T0 and the second timestamp be Tn. If it is detected that the difference between the first timestamp and the second timestamp does not fall within the preset sound propagation time range, it can be determined that the retrieved reference data is not signal-aligned with the first external sound data, and the echo cancellation effect is poor at this time. In this case, based on the order of the first timestamp and the second timestamp, it can be determined whether the retrieved reference data is delayed or the retrieved reference data is transmitted too fast, thereby guiding the feedback adjustment of the decoding speed. If the difference between the second timestamp and the first timestamp is greater than the preset sound propagation time range, it indicates that the retrieved reference data is delayed, and the decoding speed of the first external sound data can be slowed down based on the difference. If the difference between the second timestamp and the first timestamp is less than the preset sound propagation time range, it indicates that the retrieved reference data is transmitted too fast, and the decoding speed of the retrieved reference data can be slowed down based on the difference. Until the difference between the first timestamp and the second timestamp falls within the preset sound propagation time range, it is determined that the retrieved reference data is signal-aligned with the first external sound data, and echo cancellation can be performed.

[0157] In this embodiment, by performing feedback adjustment on the decoding speed, not only can signal alignment be effectively achieved, but additional resources do not need to be occupied.

[0158] In some embodiments, the audio output device is configured to output a test sound; the first sound collector is further configured to collect third external sound data corresponding to the test sound; the projection screen includes a second sound collector, and the second sound collector is configured to collect fourth external sound data corresponding to the test sound; before performing signal alignment on the retrieved reference data and the first external sound data based on a preset sound propagation duration range and a difference between the second timestamp and the first timestamp, the controller is further configured to:

[0159] detecting a third time stamp of the target signal from the third external sound data, and detecting a fourth time stamp of the target signal from the fourth external sound data;

[0160] Calculating a time difference between the third timestamp and the fourth timestamp, and calculating a sound propagation duration based on a distance between the audio output device and the second sound collector;

[0161] The sound propagation duration range is determined based on the sum of the time difference and the sound propagation duration.

[0162] It should be noted that since the distance between the projection host and the projection screen installed in each user's home is not necessarily the same, this may lead to inaccurate estimation of the distance between the media sound from the audio output device to the first sound collector, and the accuracy of the estimation of the sound propagation time range is reduced, thereby reducing the detection accuracy of signal alignment and worsening the echo cancellation effect.

[0163] For example, after placing the projection host and projection screen, the user can first test the sound propagation time range. Specifically, a sweep frequency wave can be output through the audio output device, for example, a sweep frequency wave of 100 Hz to 8 kHz. Each frequency point of the sweep frequency wave appears only once during playback and is unique. After the sweep frequency wave propagates through the air, it can be collected by the first sound collector and the second sound collector. The sound data collected by the first sound collector is recorded as the third external sound data, and the sound data collected by the second sound collector is recorded as the fourth external sound data. Furthermore, the third timestamp of a target signal of any target frequency in the third external sound data and the fourth timestamp of the target signal in the fourth external sound data can be detected. The time difference between the third timestamp and the fourth timestamp can then be calculated. Since the distance between the second sound collector and the audio output device in the projection screen is fixed, the sound propagation time of the target signal from the audio output device to the second sound collector can be calculated. The sound propagation time range can be determined by adding the sound propagation time to the time difference between the third timestamp and the fourth timestamp.

[0164] In this embodiment, by outputting a test sound and utilizing the fixed distance between the second sound collector in the projection screen and the audio output device, the sound propagation time range can be accurately determined, thereby improving the detection accuracy of signal alignment and improving the echo cancellation effect.

[0165] In some embodiments, in the process of performing echo cancellation on the media sound in the first external sound data after signal alignment based on the echo reference data after signal alignment to obtain the far-field voice data, the controller is further configured to:

[0166] By using a preset acoustic model, echo prediction is performed based on the reference data after signal alignment to obtain echo estimation data;

[0167] Echo cancellation is performed on the media sound in the first external sound data after signal alignment according to the echo estimation data to obtain far-field voice data.

[0168] It's important to note that indirect echo refers to the collection of sounds broadcast by an audio output device that enter the sound collector after one or more reflections along different paths. Any movement of objects in the space will alter the echo path, so indirect echoes are characterized by multipath and time-varying characteristics.

[0169] Among them, the acoustic model may refer to a mathematical model used to simulate and predict the reflection path of the indirect echo so that the indirect echo component can be subtracted or suppressed from the received external sound data. The echo estimation data may refer to the sound data collected by the first sound collector after the media sound is reflected in the space, which is predicted by the acoustic model based on the echo reference data. After the audio output device plays the media sound, the first sound collector can simultaneously collect the far-field sound and the media sound played by the audio output device. Since the media sound played by the audio output device is transmitted through multiple paths, the correlation between the speaker signal and the multipath echo generated by the media sound can be used to extract the characteristic parameters of the echo path, thereby establishing a corresponding acoustic model. The echo is estimated through the acoustic model, and the filter coefficients are continuously modified to make the echo estimation data closer to the real echo. Then, the echo estimation data is subtracted from the first external sound data to achieve echo cancellation.

[0170] Exemplarily, after signal alignment, the echo reference data after signal alignment can be input into the acoustic model, and the echo estimation data corresponding to the echo reference data can be predicted by the acoustic model. Then, based on the echo estimation data, the media sound in the first external sound data after signal alignment is echo eliminated to obtain far-field voice data.

[0171] In this embodiment, indirect echo cancellation is performed through an acoustic model, which can further improve the echo cancellation effect and thus improve the accuracy of far-field speech recognition.

[0172] In some embodiments, a far-field speech recognition method is provided, such as Figure 11 As shown, the method is applied to a projection host in a projection device, the projection device also includes a projection screen, and the projection host is configured to project a display image on the projection screen. The method includes:

[0173] Step 1102: collecting first external sound data and receiving back-collected reference data, the back-collected reference data being wirelessly transmitted to the projection host by the projection screen when outputting media sound;

[0174] Step 1104: performing signal alignment on the retrieved reference data and the first external sound data;

[0175] Step 1106 , performing echo cancellation on the media sound in the first external sound data after signal alignment based on the echoed reference data after signal alignment to obtain far-field voice data;

[0176] Step 1108: Perform far-field speech recognition on the far-field speech data.

[0177] In some embodiments, performing signal alignment on the retrieved reference data and the first external sound data includes:

[0178] Decoding the retrieved reference data to obtain a retrieved reference signal, and decoding the first external sound data to obtain a first external sound signal;

[0179] Obtaining a first timestamp in the retrieved reference signal and a second timestamp in the first external sound signal, wherein the first timestamp is used to indicate the time when the retrieved reference signal is emitted from the audio output device, and the second timestamp is used to indicate the time when the first sound collector collects the first external sound signal;

[0180] Signal alignment is performed on the retrieved reference data and the first external sound data according to a preset sound propagation time range and a difference value between the second timestamp and the first timestamp.

[0181] In some embodiments, signal alignment is performed on the retrieved reference data and the first external sound data according to a preset sound propagation duration range and a difference between the second timestamp and the first timestamp, including:

[0182] When the difference between the second timestamp and the first timestamp is greater than a preset sound propagation duration range, the decoding speed of the first external sound data is slowed down according to the difference;

[0183] When the difference between the second timestamp and the first timestamp is smaller than the preset sound propagation time range, the decoding speed of the retrieved reference data is slowed down according to the difference.

[0184] In some embodiments, before performing signal alignment on the retrieved reference data and the first external sound data based on a preset sound propagation duration range and a difference between the second timestamp and the first timestamp, the method further includes:

[0185] collecting third external sound data corresponding to a test sound, wherein the test sound is output by an audio output device;

[0186] Detecting a third timestamp of the target signal from the third external sound data, and obtaining a fourth timestamp of the target signal from the fourth external sound data, where the fourth external sound data is obtained by collecting the test sound by the second sound collector in the projection screen;

[0187] Calculating a time difference between the third timestamp and the fourth timestamp, and calculating a sound propagation duration based on a distance between the audio output device and the second sound collector;

[0188] The sound propagation duration range is determined based on the sum of the time difference and the sound propagation duration.

[0189] In some embodiments, performing echo cancellation on the media sound in the first external sound data after signal alignment based on the echoed reference data after signal alignment to obtain far-field voice data includes:

[0190] By using a preset acoustic model, echo prediction is performed based on the reference data after signal alignment to obtain echo estimation data;

[0191] Echo cancellation is performed on the media sound in the first external sound data after signal alignment according to the echo estimation data to obtain far-field voice data.

[0192] In some embodiments, a far-field speech recognition method is provided, such as Figure 11 As shown, the method is applied to a projection screen in a projection device, the projection device further comprising a projection host configured to project a display image onto the projection screen, the method comprising:

[0193] Output media sound, and when outputting the media sound, wirelessly transmit the collected reference data corresponding to the media sound to the projection host, so that the projection host performs signal alignment on the collected reference data and the collected first external sound data, and performs echo cancellation on the media sound in the first external sound data after signal alignment according to the collected reference data after signal alignment to obtain far-field voice data, and performs far-field voice recognition on the far-field voice data.

[0194] In some embodiments, wirelessly transmitting the retrieved reference data corresponding to the media sound to the projection host includes:

[0195] When the far-field voice function is triggered, the corresponding reference data of the media sound is wirelessly transmitted to the projection host.

[0196] In some embodiments, the projection screen includes a second sound collector, which is configured to collect second external sound data; when the far-field voice function is triggered, wirelessly transmit the collected reference data corresponding to the media sound to the projection host, including:

[0197] Acquiring the second external sound data and the corresponding recapture reference data of the media sound;

[0198] Performing echo cancellation on the second external sound data according to the retrieved reference data to obtain the voice data to be tested;

[0199] When far-field voice data exists in the voice data to be tested, it is determined that the far-field voice function is triggered, and the collected reference data is wirelessly transmitted to the projection host.

[0200] In some embodiments, when far-field voice data exists in the voice data to be tested, determining that the far-field voice function is triggered and wirelessly transmitting the retrieved reference data to the projection host includes:

[0201] When it is detected that the energy amplitude of the voice data to be measured exceeds a preset energy amplitude threshold, it is determined that far-field voice data exists in the voice data to be measured, and it is determined that the far-field voice function is triggered.

[0202] In some embodiments, the projection screen includes a bandpass filter; when it is detected that the energy amplitude of the voice data to be tested exceeds a preset energy amplitude threshold, determining that far-field voice data exists in the voice data to be tested, and determining that the far-field voice function is triggered, includes:

[0203] The voice data to be tested is filtered through a bandpass filter to obtain the target human voice data corresponding to the preset human voice signal frequency band;

[0204] When it is detected that the energy amplitude of the target human voice data exceeds a preset energy amplitude threshold, it is determined that far-field voice data exists in the voice data to be tested.

[0205] In an exemplary embodiment, a computer device is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the method of the above embodiment when executing the computer program.

[0206] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the method of the above embodiment is implemented.

[0207] In one embodiment, a computer program product is provided, including a computer program. When the computer program is executed by a processor, the method of the above embodiment is implemented.

[0208] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0209] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. In particular, any reference to a memory, database, or other medium used in the embodiments provided in this application can include at least one of a non-volatile memory and a volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), quantum computing-based data processing logic devices, artificial intelligence (AI) processors, and the like.

[0210] The technical features of the above embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0211] The above embodiments merely illustrate several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art may make various modifications and improvements without departing from the spirit of the present invention, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. A projection device, characterized in that: It includes a projection screen and a projection host, wherein the projection host is configured to project a display image onto the projection screen; The projection host includes a first sound collector, a wireless transmission module and a controller, and the projection screen includes a signal processing module and an audio output device, wherein: The projection screen is configured to display a user interface; The audio output device is configured to output media sound, and when outputting the media sound, sends the collected reference data corresponding to the media sound to the signal processing module; The signal processing module is configured to send the retrieved reference data to the wireless transmission module; The wireless transmission module is configured to receive the back-collecting reference data and send the back-collecting reference data to the controller; The first sound collector is configured to collect first external sound data and send the first external sound data to the controller; The controller is configured to: When the retrieved reference data and the first external sound data are received, signal alignment is performed on the retrieved reference data and the first external sound data; According to the echoed reference data after signal alignment, echo cancellation is performed on the media sound in the first external sound data after signal alignment to obtain far-field voice data; Perform far-field speech recognition on the far-field speech data.

2. The projection device according to claim 1, characterized in that: In the process of sending the retrieved reference data to the wireless transmission module, the signal processing module is further configured to: When the far-field voice function is triggered, the retrieved reference data is sent to the wireless transmission module.

3. The projection device according to claim 2, characterized in that: The projection screen includes a second sound collector, and the second sound collector is configured to collect second external sound data; In the process of sending the retrieved reference data to the wireless transmission module when the far-field voice function is triggered, the signal processing module is further configured to: Acquire the second external sound data and the retrieved reference data; According to the echo-collected reference data, the second external sound data is echo-cancelled to obtain the voice data to be tested; In the case that far-field voice data exists in the voice data to be tested, it is determined that the far-field voice function is triggered, and the collected reference data is sent to the wireless transmission module.

4. The projection device according to claim 3, characterized in that: In the process of determining that the far-field voice function is triggered when there is far-field voice data in the voice data to be tested, the signal processing module is further configured to: When it is detected that the energy amplitude of the voice data to be tested exceeds a preset energy amplitude threshold, it is determined that far-field voice data exists in the voice data to be tested, and it is determined that the far-field voice function is triggered.

5. The projection device according to claim 4, characterized in that: The signal processing module includes a bandpass filter; in the process of determining that there is far-field voice data in the voice data to be tested when the energy amplitude of the voice data to be tested exceeds a preset energy amplitude threshold, the signal processing module is further configured to: The voice data to be tested is filtered by a bandpass filter to obtain target human voice data corresponding to a preset human voice signal frequency band; When it is detected that the energy amplitude of the target human voice data exceeds a preset energy amplitude threshold, it is determined that far-field voice data exists in the voice data to be tested.

6. The projection device according to claim 1, characterized in that: In the process of performing signal alignment on the retrieved reference data and the first external sound data, the controller is further configured to: Decoding the retrieved reference data to obtain a retrieved reference signal, and decoding the first external sound data to obtain a first external sound signal; Obtaining a first timestamp in the sampling reference signal and a second timestamp in the first external sound signal, wherein the first timestamp is used to indicate the time when the sampling reference signal is sent from the audio output device, and the second timestamp is used to indicate the time when the first sound collector collects the first external sound signal; According to a preset sound propagation time range and a difference value between the second timestamp and the first timestamp, signal alignment is performed on the retrieved reference data and the first external sound data.

7. The projection device according to claim 6, characterized in that: In the process of performing signal alignment on the retrieved reference data and the first external sound data according to the preset sound propagation time range and the difference between the second timestamp and the first timestamp, the controller is further configured to perform at least one of the following: When the difference between the second timestamp and the first timestamp is greater than a preset sound propagation duration range, the decoding speed of the first external sound data is slowed down according to the difference; When the difference between the second timestamp and the first timestamp is smaller than a preset sound propagation duration range, the decoding speed of the retrieved reference data is slowed down according to the difference.

8. The projection device according to claim 6, characterized in that: The audio output device is configured to output a test sound; The first sound collector is further configured to collect third external sound data corresponding to the test sound; The projection screen includes a second sound collector, and the second sound collector is configured to collect fourth external sound data corresponding to the test sound; Before performing signal alignment on the retrieved reference data and the first external sound data according to the preset sound propagation time range and the difference between the second timestamp and the first timestamp, the controller is further configured to: detecting a third time stamp of the target signal from the third external sound data, and detecting a fourth time stamp of the target signal from the fourth external sound data; Calculating a time difference between the third timestamp and the fourth timestamp, and calculating a sound propagation duration according to a distance between the audio output device and the second sound collector; The sound propagation duration range is determined according to the sum of the time difference and the sound propagation duration.

9. A far-field speech recognition method, characterized in that: The method is applied to a projection host in a projection device, the projection device further comprising a projection screen, and the projection host is configured to project a display image onto the projection screen; the method comprises: Collecting first external sound data and receiving collected reference data, wherein the collected reference data is wirelessly transmitted to the projection host by the projection screen when outputting media sound; Performing signal alignment on the retrieved reference data and the first external sound data; According to the echoed reference data after signal alignment, echo cancellation is performed on the media sound in the first external sound data after signal alignment to obtain far-field voice data; Perform far-field speech recognition on the far-field speech data.

10. A far-field speech recognition method, characterized in that: The method is applied to a projection screen in a projection device, wherein the projection device further comprises a projection host, and the projection host is configured to project a display image onto the projection screen; the method comprises: Output media sound, and when outputting the media sound, wirelessly transmit the collected reference data corresponding to the media sound to the projection host, so that the projection host performs signal alignment on the collected reference data and the collected first external sound data, and performs echo cancellation on the media sound in the first external sound data after signal alignment according to the collected reference data after signal alignment to obtain far-field voice data, and performs far-field voice recognition on the far-field voice data.

Images