Display device, server and audio noise reduction and model training method thereof

Abstract

The invention relates to a display device, a server and an audio noise reduction and model training method thereof, and relates to the technical field of intelligent voice. The display equipment comprises a WiFi module which is configured to perform network search in different working modes; the microphone and the WiFi module are arranged on the same circuit board, and the microphone is configured to collect audio data; the at least one controller is connected with the microphone and the WiFi module and is configured to control the microphone to collect first audio data; the first audio data at least comprises interference audio of the WiFi module in the current working mode; taking the audio interference feature of the WiFi module in the current working mode as a target interference feature; wherein the characteristic frequencies or the characteristic intensities of the audio interference characteristics in different working modes are different; and according to the feature frequency and the feature intensity of the target interference feature, performing noise reduction processing on the first audio data to obtain the second audio data, thereby improving the definition of the second audio data.

Description

Technical Field

[0001] The present application relates to the field of intelligent voice technology, and in particular to a display device, a server, and an audio noise reduction and model training method thereof. Background Art

[0002] With the rapid development of intelligent voice technology, there are more and more products with intelligent voice functions, such as display devices. In order to save product costs and internal space, it is usually a new development trend to use all-in-one modules to replace multiple single hardware modules. For example, in a display device, a WiFi (Wireless Fidelity) module, a light sensor module, a Bluetooth module, and a voice module are set on a circuit board.

[0003] However, since the WiFi module is close to the microphone, when the WiFi module is working, the scanning signal it transmits is reflected into the microphone circuit through the internal structure, which will interfere with the data collected by the microphone.

[0004] In traditional technology, in order to overcome the above problems, shielding cloth is usually added around the microphone circuit to shield the radiation signal, or the relative position relationship between the WiFi module and the microphone in the circuit board is adjusted to avoid the radiation signal. However, adding shielding cloth will greatly increase the cost, and adjusting the distribution of modules in the circuit board requires a lot of manpower and time costs to analyze the radiation paths of multiple antennas in WiFi, which is also highly complex and has poor operability. Summary of the invention

[0005] The present application provides a display device, a server, and an audio noise reduction and model training method thereof to solve the influence of the WiFi module on the voice collection function of the microphone in the traditional technology, while taking into account the cost investment and operability.

[0006] In a first aspect, some embodiments provide a display device, including:

[0007] The WiFi module is configured to perform network search in different working modes;

[0008] A microphone is disposed on the same circuit board as the WiFi module and is configured to collect audio data;

[0009] At least one controller, connected to the microphone and the WiFi module, configured to:

[0010] Controlling the microphone to collect first audio data; wherein the first audio data at least includes interference audio of the WiFi module in the current working mode;

[0011] The audio interference feature of the WiFi module in the current working mode is used as the target interference feature; wherein the characteristic frequency or characteristic strength of the audio interference feature in different working modes is different;

[0012] The first audio data is subjected to noise reduction processing according to the characteristic frequency and characteristic intensity of the target interference feature to obtain second audio data.

[0013] In some embodiments, the target interference feature of the network search performed by the WiFi module in the current working mode is obtained by the controller in the display device, and the first audio data collected by the microphone is subjected to noise reduction processing according to the characteristic frequency and characteristic strength of the target interference feature to obtain the second audio data, so that the obtained second audio data can deduct the radiation interference generated by the WiFi module, thereby improving the clarity of the second audio data. Since the above-mentioned noise reduction process adopts a pure software method, there is no need to set other hardware structures or materials such as shielding cloth, which reduces the hardware cost, and there is no need to adjust the relative position relationship between the modules set in the circuit board, so there is no need to perform complicated radiation path analysis, which reduces the manpower and time cost, and has better operability and versatility. In addition, since the scanning frequency and / or scanning intensity of the WiFi module in different working modes are different, the characteristic frequency or characteristic intensity of the audio interference feature in different working modes also has certain differences. Therefore, the audio interference feature of the WiFi module in the current working mode is used as the target interference feature, rather than using a unified audio interference feature, which can improve the accuracy of the selected target interference feature, thereby avoiding the situation where the wrong target interference feature is selected, resulting in the loss of key information or the noise is not completely deducted, resulting in poor accuracy of the second audio data.

[0014] In a second aspect, some embodiments further provide a server, characterized in that it includes:

[0015] A communication device, communicatively connected to the sample display device;

[0016] At least one processor is connected to the communication device and is configured to:

[0017] Acquire a first audio sample generated during the operation of the sample display device when the indoor noise intensity is lower than a first intensity threshold, and a second audio sample generated during the operation of the sample display device after the WiFi driver is uninstalled;

[0018] determining a first difference feature between the first audio sample and the second audio sample;

[0019] Using the first audio sample as input data and the first difference feature as a label, training the feature extraction network to be trained until the difference between the trained feature extraction network and the standard feature extraction network meets a preset difference condition;

[0020] The feature extraction network is used to extract audio interference features of audio generated by a WiFi module in a display device during a network search;

[0021] Among them, the standard feature extraction network is trained in the following way:

[0022] Acquire a third audio sample generated during the operation of the sample display device in the silencing environment, and a fourth audio sample generated during the operation of the sample display device after uninstalling the WiFi driver;

[0023] determining a second difference feature between the third audio sample and the fourth audio sample;

[0024] The third audio sample is used as input data and the second difference feature is used as a label to train the standard feature extraction network to be trained.

[0025] In some embodiments, a server is used to obtain a first audio sample generated during the operation of a sample display device when the indoor noise intensity is lower than a first intensity threshold, and a second audio feature generated during the operation of the sample display device after the WiFi driver is uninstalled; the first audio sample is used as input data, and the first difference feature between the first audio sample and the second audio sample is used as a label to train the feature extraction network until the difference between the trained feature extraction network and the standard feature extraction network meets the preset difference condition. Since the standard feature extraction network is trained with the third audio sample generated during the operation of the sample display device in the anechoic environment as input data, and the second difference feature between the third audio sample and the fourth audio sample generated during the operation of the sample display device in the anechoic environment after the WiFi driver is uninstalled as a label, the standard feature extraction network has better feature extraction capabilities. Since the standard feature extraction network has better feature extraction capabilities in an anechoic room environment and is not suitable for a conventional indoor environment, the standard feature extraction network is used as a reference for the feature extraction network training process applied to a conventional indoor environment. When the difference between the trained feature extraction network and the standard feature extraction network meets the preset difference condition, the training of the feature extraction network is terminated, ensuring that the feature extraction capabilities of the feature extraction network are equivalent to those of the standard feature extraction network.

[0026] In a third aspect, some embodiments further provide an audio noise reduction method, which is applied to a display device, comprising:

[0027] Acquire first audio data collected by a microphone of a display device; wherein the first audio data at least includes interference audio of the WiFi module in a current working mode; and,

[0028] Acquire the audio interference feature of the WiFi module of the display device when performing network search in the current working mode as the target interference feature; wherein the characteristic frequency or characteristic strength of the audio interference feature in different working modes is different;

[0029] The first audio data is subjected to noise reduction processing according to the characteristic frequency and characteristic intensity of the target interference feature to obtain second audio data.

[0030] In some embodiments, by obtaining the target interference feature of the network search performed by the WiFi module in the current working mode, and according to the characteristic frequency and characteristic strength of the target interference feature, the first audio data collected by the microphone is subjected to noise reduction processing to obtain the second audio data, so that the obtained second audio data can deduct the radiation interference generated by the WiFi module, thereby improving the clarity of the second audio data. Since the above-mentioned noise reduction process adopts a pure software method, there is no need to set other hardware structures or materials such as shielding cloth, which reduces the hardware cost, and there is no need to adjust the relative position relationship between the modules set in the circuit board, so there is no need to perform complicated radiation path analysis, which reduces the manpower and time cost, and has better operability and versatility. In addition, since the scanning frequency and / or scanning intensity of the WiFi module in different working modes are different, the characteristic frequency or characteristic intensity of the audio interference feature in different working modes also has certain differences. Therefore, the audio interference feature of the WiFi module in the current working mode is used as the target interference feature, rather than using a unified audio interference feature, which can improve the accuracy of the selected target interference feature, thereby avoiding the situation where the wrong target interference feature is selected, resulting in the loss of key information or the noise is not completely deducted, resulting in poor accuracy of the second audio data.

[0031] In a fourth aspect, some embodiments further provide a model training method, which is applied to a server and includes:

[0032] Acquire a first audio sample generated during the operation of the sample display device when the indoor environment meets a preset quiet condition, and a second audio sample generated during the operation of the sample display device after the WiFi driver is uninstalled;

[0033] determining a first difference feature between the first audio sample and the second audio sample;

[0034] Using the first audio sample as input data and the first difference feature as a label, training the feature extraction network to be trained until the difference between the trained feature extraction network and the standard feature extraction network meets a preset difference condition;

[0035] The feature extraction network is used to extract audio interference features of the audio generated by the WiFi module in the display device during network search;

[0036] Among them, the standard feature extraction network is trained in the following way:

[0037] Acquire a third audio sample generated during the operation of the sample display device in the silencing environment, and a fourth audio sample generated during the operation of the sample display device after uninstalling the WiFi driver;

[0038] determining a second difference feature between the third audio sample and the fourth audio sample;

[0039] The third audio sample is used as input data and the second difference feature is used as a label to train the standard feature extraction network to be trained.

[0040] In some embodiments, a server is used to obtain a first audio sample generated during the operation of a sample display device when the indoor noise intensity is lower than a first intensity threshold, and a second audio feature generated during the operation of the sample display device after the WiFi driver is uninstalled; the first audio sample is used as input data, and the first difference feature between the first audio sample and the second audio sample is used as a label to train the feature extraction network until the difference between the trained feature extraction network and the standard feature extraction network meets the preset difference condition. Since the standard feature extraction network is trained with the third audio sample generated during the operation of the sample display device in the anechoic environment as input data, and the second difference feature between the third audio sample and the fourth audio sample generated during the operation of the sample display device in the anechoic environment after the WiFi driver is uninstalled as a label, the feature extraction capability of the standard feature extraction network is better. Since the feature extraction capability of the standard feature extraction network is better in the anechoic room environment and is not suitable for the conventional indoor environment, the standard feature extraction network is used as a reference for the feature extraction network training process applied to the conventional indoor environment. When the difference between the trained feature extraction network and the standard feature extraction network meets the preset difference condition, the training of the feature extraction network is terminated, thereby ensuring that the feature extraction capability of the feature extraction network is equivalent to that of the standard feature extraction network. Correspondingly, the audio interference features of the audio generated by the WiFi module in the display device during network search are extracted based on the trained feature extraction network, thereby improving the accuracy of the extracted audio interference features.

[0041] In a fifth aspect, some embodiments further provide an audio noise reduction device, which is configured on a display device and includes:

[0042] A first acquisition module, used to acquire first audio data collected by a microphone of a display device; wherein the first audio data at least includes interference audio of the WiFi module in a current working mode;

[0043] The first acquisition module is further used to acquire the audio interference feature of the WiFi module of the display device when performing network search in the current working mode as the target interference feature; wherein the characteristic frequency or characteristic intensity of the audio interference feature in different working modes is different;

[0044] The noise reduction module is used to perform noise reduction processing on the first audio data according to the characteristic frequency and characteristic intensity of the target interference feature to obtain second audio data.

[0045] In some embodiments, the target interference feature of the network search performed by the WiFi module in the current working mode is obtained by the first acquisition module, and the first audio data collected by the microphone is subjected to noise reduction processing according to the characteristic frequency and characteristic strength of the target interference feature by the noise reduction module to obtain the second audio data, so that the obtained second audio data can deduct the radiation interference generated by the WiFi module, thereby improving the clarity of the second audio data. Since the above-mentioned noise reduction process adopts a pure software method, there is no need to set other hardware structures or materials such as shielding cloth, which reduces the hardware cost, and there is no need to adjust the relative position relationship between the modules set in the circuit board, so there is no need to perform complicated radiation path analysis, which reduces the manpower and time cost, and has better operability and versatility. In addition, since the scanning frequency and / or scanning intensity of the WiFi module in different working modes are different, the characteristic frequency or characteristic intensity of the audio interference feature in different working modes also has certain differences. Therefore, the audio interference feature of the WiFi module in the current working mode is used as the target interference feature, rather than using a unified audio interference feature, which can improve the accuracy of the selected target interference feature, thereby avoiding the situation where the wrong target interference feature is selected, resulting in the loss of key information or the noise is not completely deducted, resulting in poor accuracy of the second audio data.

[0046] In a sixth aspect, some embodiments further provide a model training device, which is configured on a server and includes:

[0047] A second acquisition module is used to acquire a first audio sample generated during the operation of the sample display device when the indoor noise intensity is lower than a first intensity threshold, and a second audio sample generated during the operation of the sample display device after the WiFi driver is uninstalled;

[0048] A determination module, configured to determine a first difference feature between the first audio sample and the second audio sample;

[0049] A training module, used to train the feature extraction network to be trained using the first audio sample as input data and the first difference feature as a label, until the difference between the trained feature extraction network and the standard feature extraction network meets a preset difference condition;

[0050] The feature extraction network is used to extract audio interference features of the audio generated by the WiFi module in the display device during network search;

[0051] Among them, the standard feature extraction network is trained in the following way:

[0052] Acquire a third audio sample generated during the operation of the sample display device in the silencing environment, and a fourth audio sample generated during the operation of the sample display device after uninstalling the WiFi driver;

[0053] determining a second difference feature between the third audio sample and the fourth audio sample;

[0054] The third audio sample is used as input data and the second difference feature is used as a label to train the standard feature extraction network to be trained.

[0055] In some embodiments, a second acquisition module is set to acquire a first audio sample generated during the operation of the sample display device when the indoor noise intensity is lower than a first intensity threshold, and a second audio feature generated during the operation of the sample display device after the WiFi driver is uninstalled; a determination module is set to determine a first difference feature between the first audio sample and the second audio sample; a training module is set to use the first audio sample as input data and the first difference feature as a label to train the feature extraction network until the difference between the trained feature extraction network and the standard feature extraction network meets a preset difference condition. Since the standard feature extraction network uses the third audio sample generated during the operation of the sample display device in a silent environment as input data, and uses the second difference feature between the third audio sample and the fourth audio sample generated during the operation of the sample display device in a silent environment after the WiFi driver is uninstalled as a label, the standard feature extraction network to be trained is obtained, so the feature extraction capability of the standard feature extraction network is better. Since the standard feature extraction network has better feature extraction capability in an anechoic room environment and is not suitable for a conventional indoor environment, the standard feature extraction network is used as a reference for the feature extraction network training process applied to a conventional indoor environment. When the difference between the trained feature extraction network and the standard feature extraction network meets a preset difference condition, the training of the feature extraction network is terminated, thereby ensuring that the feature extraction capability of the feature extraction network is equivalent to that of the standard feature extraction network. Accordingly, the audio interference features of the audio generated by the WiFi module in the display device during network search are displayed based on the trained feature extraction network, thereby improving the accuracy of the extracted audio interference features.

[0056] In the seventh aspect, some embodiments further provide a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method provided in some embodiments of the third aspect or the fourth aspect are implemented.

[0057] In the eighth aspect, some embodiments further provide a computer program product, comprising a computer program, which, when executed by a processor, implements the steps of the method provided in some embodiments of the third aspect or the fourth aspect. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0059] Figure 1 A schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application;

[0060] Figure 2 A schematic diagram of the hardware configuration of a display device provided in some embodiments of the present application;

[0061] Figure 3 A schematic diagram of the hardware configuration of a control device provided in some embodiments of the present application;

[0062] Figure 4 A schematic diagram of software configuration of a display device provided in some embodiments of the present application;

[0063] Figure 5A This is a schematic diagram of the circuit board evolution process in some embodiments of the present application;

[0064] Figure 5B A schematic diagram of the hardware structure of a display device in some embodiments of the present application;

[0065] Figure 5C This is a spectrum diagram of interference of the WiFi module to the microphone in some embodiments of the present application;

[0066] Figure 6 A flowchart of an audio noise reduction method provided in some embodiments of the present application;

[0067] Figure 7 A schematic flow chart of steps for acquiring target interference features provided in some embodiments of the present application;

[0068] Figure 8 A schematic diagram of the trimming results of audio samples provided in some embodiments of the present application;

[0069] Fig. 9 A flowchart of a model training method provided in some embodiments of the present application;

[0070] Fig.10 A flowchart of an audio noise reduction method provided in some embodiments of the present application;

[0071] Fig.11 A structural block diagram of an audio noise reduction device provided in some embodiments of the present application;

[0072] Fig.12 A structural block diagram of a model training device provided in some embodiments of the present application;

[0073] Fig.13 FIG. 4 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0074] The following embodiments are described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The implementations described in the following embodiments do not represent all implementations consistent with the present application. They are only examples of systems and methods consistent with some aspects of the present application as detailed in the claims.

[0075] It should be noted that the brief description of terms in this application is only for the convenience of understanding the embodiments described below, and is not intended to limit the embodiments of this application. Unless otherwise specified, these terms should be understood according to their common and usual meanings.

[0076] The terms "first", "second", "third", etc. in the specification and claims of this application and the above drawings are used to distinguish similar or similar objects or entities, and do not necessarily mean to limit a specific order or sequence, unless otherwise noted. It should be understood that the terms used in this way can be interchangeable under appropriate circumstances.

[0077] The terms "comprises," "comprising," and "having," and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device comprising a list of components is not necessarily limited to all the components expressly listed but may include other components not expressly listed or inherent to such product or device.

[0078] The term "module" refers to any known or later developed hardware, software, firmware, artificial intelligence, fuzzy logic or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0079] In the embodiment of the present application, the display device 200 generally refers to a device with image display and data processing capabilities. For example, the display device 200 includes but is not limited to a smart TV, a mobile terminal, a computer, a monitor, an advertising screen, a wearable device, a virtual reality device, an augmented reality device, etc.

[0080] Figure 1 This is a schematic diagram of an operation scenario between a display device and a control device provided in some embodiments of the present application. Figure 1 As shown in FIG. 1 , the user can operate the display device 200 through touch operation, the mobile terminal 300 and the control device 100. For example, the control device 100 can be a remote controller, a stylus pen, a handle, etc.

[0081] The mobile terminal 300 can be used as a control device for performing human-computer interaction between the user and the display device 200. The mobile terminal 300 can also be used as a communication device for establishing a communication connection with the display device 200 and performing data interaction. In some embodiments, the mobile terminal 300 can install software applications with the display device 200, and achieve connection and communication through a network communication protocol to achieve the purpose of one-to-one control operation and data communication. The audio and video content displayed on the mobile terminal 300 can also be transmitted to the display device 200 to achieve a synchronous display function.

[0082] like Figure 1 As also shown in FIG. 1 , the display device 200 also communicates data with the server 400 through various communication methods. The display device 200 may be allowed to communicate and connect through a local area network (LAN), a wireless local area network (WLAN) and other networks.

[0083] The display device 200 can provide a broadcast receiving television function, and can also additionally provide an intelligent network television function with a computer support function, including but not limited to network television, smart television, Internet Protocol Television (IPTV), etc.

[0084] Figure 2 Some embodiments of the present application provide Figure 1 2 is a block diagram of the hardware configuration of the display device 200.

[0085] In some embodiments, the display device 200 may include at least one of a tuner 210 , a communication device 220 , a detector 230 , a device interface 240 , a controller 250 , a display 260 , an audio output device 270 , a memory, a power supply, and a user input interface 280 .

[0086] In some embodiments, the detector 230 is used to collect signals of the external environment or external interaction. For example, the detector 230 includes a light receiver, a sensor for collecting the intensity of ambient light; or, the detector 230 includes an image collector, such as a camera, which can be used to collect external environment scenes, user attributes or user interaction gestures; or, the detector 230 includes a sound collector, such as a microphone, etc., which is used to receive external sounds, such as collecting user voice data.

[0087] In some embodiments, the display 260 includes a display function component for presenting a picture, and a driving component for driving an image display. The display 260 is used to receive an image signal output from the controller 250 for display. For example, the display 260 can be used to display video content, image content, and components of a menu control interface and a user control UI interface (User Interface user interface).

[0088] In some embodiments, the communication device 220 is a component for communicating with an external device or server 400 according to various communication protocol types. The display device 200 may be provided with a plurality of communication devices 220 according to different supported communication modes. For example, when the display device 200 supports wireless network communication, the display device 200 may be provided with a communication device 220 including a WiFi (Wireless Fidelity) function. When the display device 200 supports Bluetooth connection communication, the display device 200 needs to be provided with a communication device 220 including a Bluetooth function.

[0089] The communication device 220 can communicate with the display device 200 and the external device or server 400 by wireless or wired connection. Among them, the wired connection can connect the display device 200 to the external device through components such as data cables and interfaces. The wireless connection can connect the display device 200 to the external device through wireless signals or wireless networks. The display device 200 can establish a connection relationship with the external device directly, or indirectly establish a connection relationship through a gateway, a router, a connection device, etc.

[0090] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first interface to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to the user's operation through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200.

[0091] In some embodiments, the controller 250 and the tuner-demodulator 210 may be located in different separate devices, that is, the tuner-demodulator 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0092] In some embodiments, the user may input a user command through a graphical user interface (GUI) displayed on the display 260 , and the user input interface receives the user input command through the graphical user interface (GUI).

[0093] In some embodiments, the audio output device 270 may be a local speaker of the display device 200, or may be an external audio output device of the display device 200. In particular, for the external audio output device of the display device 200, the display device 200 may also be provided with an external audio output terminal, and the audio output device may be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0094] Figure 3 Some embodiments of the present application provide Figure 1 The hardware configuration diagram of the control device in the figure. Figure 3 As shown, the control device 100 may include: a controller 110 , a communication interface 130 , a user input / output interface 140 , a power supply 180 , and a memory 190 .

[0095] The control device 100 is configured to control the display device 200 , and can receive user input operation instructions, and convert the operation instructions into instructions that the display device 200 can recognize and respond to, playing the role of an interactive intermediary between the user and the display device 200 .

[0096] In some embodiments, the control device 100 may be a smart device, for example, the control device 100 may be installed with various applications for controlling the display device 200 according to user needs.

[0097] In some embodiments, Figure 1 As shown, the mobile terminal 300 or other intelligent electronic devices can play a similar function as the control device 100 after installing the application for controlling the display device 200 .

[0098] The controller 110 includes a processor 112, a RAM (Random Access Memory) 113, a ROM (Read-Only Memory) 114, a communication interface 130, and a communication bus. The controller 110 is used to control the operation and operation of the control device 100, as well as the communication and cooperation between the internal components and the external and internal data processing functions.

[0099] The communication interface 130 implements communication of control signals and data signals with the display device 200 under the control of the controller 110. The communication interface 130 may include at least one of other near field communication modules such as a WiFi chip 131, a Bluetooth module 132, and an NFC (Near Field Communication) module 133.

[0100] The user input / output interface 140 , wherein the input interface includes at least one of other input interfaces such as a microphone 141 , a touch panel 142 , a sensor 143 , and a button 144 .

[0101] In some embodiments, the control device 100 includes at least one of a communication interface 130 and an input / output interface 140. The control device 100 is configured with a communication interface 130, such as a WiFi, Bluetooth, NFC or other module, and can encode the user input command through the WiFi protocol, Bluetooth protocol, or NFC protocol and send it to the display device 200.

[0102] The memory 190 is used to store various operating programs, data and applications for driving and controlling the control device 100 under the control of the controller. The memory 190 can store various control signal instructions input by the user.

[0103] The power supply 180 is used to provide operating power support for each component of the control device 100 under the control of the controller.

[0104] In order to perform user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program for managing and controlling hardware resources and software resources in the display device 200. The operating system may provide a user interface (control the display device), allow the user to interact with the display device 200, and support the running of various application programs.

[0105] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system deeply customized based on a specific operating platform, or an independent operating system specially developed for the display device.

[0106] The operating system can be divided into different modules or layers according to the functions implemented, such as Figure 4 As shown, in some embodiments, the system is divided into four layers, from top to bottom, namely, the application layer (Applications) layer (referred to as "application layer"), the application framework layer (Application Framework) layer (referred to as "framework layer"), the system library layer and the kernel layer.

[0107] In some embodiments, the application layer is used to provide services and interfaces for applications so that the display device 200 can run applications and interact with users based on the applications. At least one application can be run in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. that come with the operating system; they can also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0108] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Through the API interface, applications can access system resources and obtain system services during execution.

[0109] like Figure 4 As shown, in the embodiment of the present application, the application framework layer includes a view system, managers, content providers, etc., wherein the view system can design and implement the interface and interaction of the application, and the view system includes lists, grids, text boxes, buttons, etc. The manager includes at least one of the following modules: an activity manager for interacting with all activities running in the system; a location manager for providing system services or applications with access to system location services; a package manager for retrieving various information related to the application package currently installed on the device; a notification manager for controlling the display and clearing of notification messages; and a window manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0110] In some embodiments, the activity manager is used to manage the life cycle of each application and the usual navigation back function, such as controlling the exit, opening, and back of the application. The window manager is used to manage all window programs, such as obtaining the display screen size, determining whether there is a status bar, locking the screen, capturing the screen, and controlling the display window changes, for example, reducing the display window, shaking the display, distorting the display, etc.

[0111] In some embodiments, the system runtime layer can provide support for the framework layer. When the framework layer is used, the operating system will run the instruction library contained in the system runtime layer, such as the C / C++ instruction library, to implement the functions to be implemented by the framework layer.

[0112] In some embodiments, the kernel layer is a functional layer between the hardware and software of the display device 200. The kernel layer can implement functions such as hardware abstraction, multitasking, and memory management. Figure 4 As shown, the kernel layer may be configured with hardware drivers, and the drivers included in the kernel layer may be at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB (Universal Serial Bus) driver, HDMI (High-Definition Multimedia Interface) driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver, etc.

[0113] It should be noted that the above example is only a simple division of the operating system functions and does not constitute a limitation on the specific operating system form of the display device 200 in the embodiment of the present application. Depending on factors such as the function of the display device and the type of operating system, the number of levels and specific level types contained in the operating system may be expressed in other forms.

[0114] See also Figure 5A With the trend of product circuit miniaturization, the display device 200 has gradually evolved from the traditional microphone module (that is, the aforementioned voice module) and the WiFi module being separately arranged on different circuit boards to the microphone module and the WiFi module being arranged on the same small circuit board to obtain an all-in-one module.

[0115] Combination Figure 5B As shown in the hardware structure diagram of the display device, the controller 110 sends a first control signal to the WiFi module to control the WiFi module to send a detection request signal at a certain frequency to search the network; and the controller 110 sends a second control signal to the microphone to control the microphone to collect audio signals and send the audio signals to the controller 110 for subsequent processing. Since the detection request signal is reflected by the structure, if it is reflected into the microphone circuit, it will mix and interfere with the microphone signal. Figure 5CAs shown in the interference spectrum of the WiFi module to the microphone, since the transmission time of the detection request signal is usually relatively fixed (often set by software), the time of occurrence of such interference corresponds to the time when the WiFi module sends the detection request signal. It is worth noting that since the scanning frequency of the WiFi module is different in different working modes, and the signal strength of the transmitted detection request signal is also different, therefore, under the same environment, due to the different working modes of the WiFi module, its audio interference characteristics may also be different.

[0116] In order to reduce the audio interference of the WiFi module, the controller 110 can pre-acquire the target interference characteristics of the WiFi module in the current working mode and deduct them from the audio data collected by the microphone, thereby improving the clarity of the audio data and then improving the voice performance, reducing the impact on the voice function of the display device 200.

[0117] In some alternative embodiments, see Figure 6 , provides an audio noise reduction method, characterized in that the controller 110 applied to the display device 200 includes:

[0118] S610, obtaining first audio data collected by a microphone; wherein the first audio data at least includes interference audio of the WiFi module in a current working mode.

[0119] The controller 110 can control the microphone to collect the first audio data in real time and obtain the first audio data transmitted by the microphone. Since the WiFi module periodically transmits a detection request signal at a set frequency for network search when working, the first audio data collected by the microphone carries interference noise introduced by the detection request signal, that is, interference audio.

[0120] Optionally, the controller 110 can control the microphone to collect the first audio data in real time. Alternatively, the controller 110 can also control the microphone to collect the first audio data only when the voice function is turned on. The advantage of doing so is that when the voice function is not turned on, the hardware loss caused by irrelevant data collection and the unnecessary occupation of computing resources are reduced.

[0121] S620, obtaining audio interference features when the WiFi module of the display device performs network search in the current working mode as target interference features.

[0122] The audio interference feature is used to characterize the interference noise introduced by the WiFi module, and carries the key features of the interference noise. The characteristic frequency or characteristic strength of the audio interference feature in different working modes is different.

[0123] In some embodiments, the audio interference characteristics corresponding to the network search performed by the WiFi module itself in different working modes can be stored in the memory 190 of the display device 200, and when audio noise reduction is required, the audio interference characteristics of the WiFi module in the current working mode are obtained to obtain the target interference characteristics.

[0124] Exemplarily, since the WiFi module can work in different working modes, and the scanning properties of the WiFi module in different working modes (such as scanning frequency and the strength of the detection request signal, etc.) may have certain differences, such as the sending frequency of the detection request signal sent, that is, the scanning frequency is different, or the signal strength of the detection signal request sent, that is, the signal strength is different, etc. Therefore, the audio interference features corresponding to the WiFi module in different working modes also have certain differences. Therefore, for different working modes, audio interference features that match the corresponding working mode can be stored to avoid the impact caused by the mismatch of the working mode. Accordingly, the target interference features can be searched and obtained based on the current working mode of the WiFi module.

[0125] Among them, the working mode may include a self-scanning mode and an application scanning mode, etc. Among them, the self-scanning mode can be understood as a working mode entered by performing a network search when the display device 200 enters a wireless network coverage area and is not connected to the network. The application scanning mode can be understood as a working mode entered by performing a network search when a client providing services based on the network is started or running. Of course, the working mode of the WiFi module can also be configured in other modes as needed according to the WiFi model or settings, etc., and this embodiment does not make any limitation to this.

[0126] Exemplarily, when the controller 110 controls the WiFi module to perform a network search, the working mode of the WiFi module is preset, so the current working mode of the WiFi module can be directly used as the current working mode.

[0127] Optionally, the working mode identifier of the WiFi module can be obtained from the memory or the local cache, and the working mode corresponding to the working mode identifier is used as the current working mode.

[0128] The audio interference features in different working modes are pre-stored in the memory 190 of the display device 200. Accordingly, the controller 110 searches the memory 190 for the audio interference feature corresponding to the current working mode as the target interference feature.

[0129] It should be noted that the audio interference features under different working modes can also be stored in the server; accordingly, when it is necessary to obtain the audio interference features, the controller 110 can send an access request based on the current working mode to the server through its own communication device 220, and receive the audio interference features corresponding to the current working mode fed back by the server as target interference features for subsequent use.

[0130] It is worth noting that by storing audio interference features corresponding to different working modes in the memory 190, there is no need for the network side to access an external server to obtain the target interference features, which improves the convenience and efficiency of obtaining the target interference features, and also reduces the dependence on network functions, so that the target interference features can still be obtained when the display device 200 is not connected to the network, further improving the versatility and convenience of obtaining the target interference features. S630, according to the characteristic frequency and characteristic intensity of the target interference feature, the first audio data is subjected to noise reduction processing to obtain the second audio data.

[0131] Exemplarily, the target interference feature can be deducted from the first audio data, thereby achieving noise reduction processing on the first audio data to obtain the second audio data. It can be understood that since the noise interference of the WiFi module has been removed from the second audio data, the target audio is clearer than the first audio data.

[0132] In an optional embodiment, the target interference feature can characterize the noise feature of a single working cycle of the WiFi module. Accordingly, the first audio data can be trimmed according to the transmission time of the detection request signal of the WiFi module during the first audio data acquisition process to obtain at least one first audio segment; for each first audio segment, the target interference feature in the first audio segment is deducted to obtain a second audio segment corresponding to the first audio segment; and the corresponding second audio segments are spliced ​​according to the segmentation order of the first audio segments to obtain second audio data.

[0133] In the above optional embodiment, by obtaining the target interference feature of the network search performed by the WiFi module in the current working mode, and according to the characteristic frequency and characteristic intensity of the target interference feature, the first audio data collected by the microphone is subjected to noise reduction processing to obtain the second audio data, so that the obtained second audio data can deduct the radiation interference generated by the WiFi module, thereby improving the clarity of the second audio data. Since the above noise reduction process adopts a pure software method, there is no need to set other hardware structures or materials such as shielding cloth, which reduces the hardware cost, and there is no need to adjust the relative position relationship between the modules set in the circuit board, so there is no need to perform complicated radiation path analysis, which reduces the manpower and time cost, and has better operability and versatility. In addition, since the scanning frequency and / or scanning intensity of the WiFi module in different working modes are different, the characteristic frequency or characteristic intensity of the audio interference feature in different working modes also has certain differences. Therefore, the audio interference feature of the WiFi module in the current working mode is used as the target interference feature, rather than using a unified audio interference feature, which can improve the accuracy of the selected target interference feature, thereby avoiding the situation where the wrong target interference feature is selected, resulting in the loss of key information or the noise is not completely deducted, resulting in poor accuracy of the second audio data.

[0134] Based on the technical solutions of the above embodiments, in some embodiments, the steps for acquiring target interference features are refined.

[0135] See also Figure 7 The steps for obtaining the target interference characteristics shown include:

[0136] S710: When the indoor noise intensity is lower than a first intensity threshold, control the microphone to collect interference audio of the WiFi module in the current working mode.

[0137] The first intensity threshold may be set by technicians according to needs or experience, or may be determined repeatedly through a large number of experiments.

[0138] The interference audio is used to characterize the ambient noise of the current indoor environment of the display device 200, including the interference noise generated by the WiFi module performing network search in the current working mode. The current indoor environment can be understood as the current environment of the space where the display device 200 is installed.

[0139] Exemplarily, when the indoor environment meets the preset quiet condition, the controller 110 controls the microphone to collect audio data to obtain interference audio.

[0140] Optionally, audio collection indication information is output through the user output interface 140 or the display interface of the display 260; the controller 110 controls the microphone to collect audio data in response to the audio collection operation based on the audio collection indication information; the collected audio data is analyzed to identify whether the indoor environment meets the preset quiet condition, that is, whether the indoor noise intensity is lower than the first intensity threshold; when the preset quiet condition is met, the collected audio data is used as interference audio.

[0141] Optionally, whether the indoor environment meets the preset quiet condition can be determined by the collected indoor noise intensity of the indoor environment. Specifically, when the indoor noise intensity is lower than a first intensity threshold, it is determined that the indoor environment meets the preset quiet condition.

[0142] In some embodiments, the controller 110 may control the microphone to collect interference audio when the display device 200 is currently started for the first time.

[0143] Since the interference audio collected by the WiFi module of the same display device changes little over a long period of time (for example, within a year) when the indoor environment remains relatively unchanged (for example, the change in the set position is less than a preset distance threshold), in other optional embodiments, the controller 110 can also control the microphone to collect the interference audio of the WiFi module in the current working mode according to a certain collection cycle when the indoor environment meets the preset quiet condition, that is, when the indoor noise intensity is lower than the first intensity threshold, without the need to frequently collect the interference audio.

[0144] As different display devices wear out and age to different degrees over time, the magnitude of the change in interference audio is also different. Therefore, in an optional embodiment, the controller 110 can also obtain the device type of the display device from the memory 190; according to the collection period corresponding to the device type, the microphone is controlled to collect the interference audio of the WiFi module in the current working mode when the indoor environment meets the preset quiet conditions. For example, for air-conditioning equipment, the collection period can be 2 years; for television equipment, the collection period can be 1 year, etc.

[0145] S720: Based on the trained feature extraction network, extract features of the interference audio to obtain target interference features.

[0146] Among them, the feature extraction network can be understood as a network model dedicated to audio interference feature extraction, which can be implemented based on a traditional machine learning model or a deep learning model. This embodiment does not impose any limitation on the network structure of the feature extraction network.

[0147] In some embodiments, a call request for a pre-trained feature extraction network can be sent to a server so that the server can extract features of the interference audio based on the trained feature extraction network; and obtain feature extraction results fed back by the server; wherein the feature extraction results include target interference features.

[0148] Exemplarily, the controller 110 can generate a call request for a trained feature extraction network based on the interference audio; send the call request to the server through the communication device 220; the server responds to the call request, extracts features from the interference audio based on the trained feature extraction network, and feeds back the feature extraction results to the display device 200; the communication device 220 of the display device 200 receives the feature extraction results fed back by the server; the controller 110 parses the feature extraction results to obtain the target interference features carried.

[0149] It can be understood that by setting the trained feature extraction network on a server outside the display device 200, the server replaces the display device 200 to extract the target interference features through external interface calls, etc., thereby reducing the computing power requirements for the display device 200.

[0150] In other embodiments, the memory 190 of the display device 200 may also store a trained feature extraction network; accordingly, the controller 110 runs the trained feature extraction network stored in the memory 190 to extract target interference features in the interference audio.

[0151] It is understandable that by running the feature extraction network of the display device 200 this time, the interference audio is feature extracted to obtain the target interference feature without relying on external devices, thereby reducing the use of communication bandwidth. In addition, since the target interference feature of the WiFi module of the same display device in the same working mode changes less over a longer period of time (e.g., within a year), the target interference feature can be extracted periodically according to the above time period, reducing the occupation of the computing power resources of the display device 200 and improving the convenience of extracting the target interference feature.

[0152] It is worth noting that no matter which method is used to extract the target interference features, the extracted target interference features can be stored in the memory 190 of the display device 200 so that the target interference features can be searched and obtained during the audio noise reduction process before the next target interference feature extraction.

[0153] The above optional embodiment, by introducing a trained feature extraction network to extract target interference features, can determine the target interference features in real time or at a scheduled time, thereby effectively coping with the aging of the WiFi module or the repair or replacement of the WiFi module, etc., which leads to changes in the relative position between the WiFi module and the microphone circuit, and the situation where the target interference features are greatly different. This avoids the influence of using unchanging target interference features on the audio noise reduction effect and improves the flexibility of audio noise reduction.

[0154] In some embodiments, the feature extraction network may be trained locally on the display device 200 or on other devices outside the display device 200, such as a server.

[0155] In some embodiments, the initial audio samples generated during the operation of the sample display device can be obtained when the indoor environment meets the preset quiet condition, that is, the indoor noise intensity is lower than the first intensity threshold, and the reference audio samples generated during the operation of the sample display device after the WiFi driver is uninstalled; the difference data between the initial audio sample and the reference audio sample can be determined; the initial audio sample is used as input data, and the difference data is used as a label to train the feature extraction network to be trained until the training cutoff condition is met. Among them, the training cutoff condition may include that the number of training samples is greater than a preset number threshold, the number of iterations meets a preset number threshold, or the feature extraction network tends to converge, etc. Among them, the preset number threshold and the preset number threshold, etc., can be set or adjusted by technical personnel according to needs or experience, or repeatedly determined through a large number of experiments, and this embodiment does not impose any restrictions on this.

[0156] The first intensity threshold can be set by technicians according to needs or experience, or can be determined repeatedly through a large number of experiments. It is worth noting that the indoor environment here corresponds to the indoor environment, and the indoor environment corresponding to the aforementioned indoor environment can be the same or have the same spatial structure, and this embodiment does not make any limitation on this.

[0157] The sample display device may be a display device dedicated to model training, and the device type of the display device may be the same as or different from the aforementioned display device, and this embodiment does not impose any limitation on this. The number of sample display devices may be at least one, and usually, in order to ensure the performance of the trained feature extraction network, the number of sample display devices is usually multiple.

[0158] Among them, the initial audio sample can be understood as the audio data collected by the microphone in the sample display device when the indoor environment meets the preset quiet conditions and the WiFi module of the sample display device works normally. The audio data may carry the noise generated by the WiFi module during operation, and also include the ambient audio of the indoor environment, etc. After the sample display device uninstalls the WiFi driver, the WiFi module will no longer work, that is, it will no longer periodically send detection request signals. In this case, the reference audio samples collected by the microphone of the sample display device during operation will no longer carry the noise generated by the WiFi module during operation, and only include the ambient audio of the indoor environment, etc.

[0159] The difference data is used to characterize the difference between the initial audio sample and the reference audio sample, and the difference between the two can be numerically quantified by difference. Since the initial audio sample carries the noise generated by the operation of the WiFi module and the ambient audio, and the reference audio sample carries only the ambient audio, the difference data between the two only carries the noise generated by the operation of the WiFi module.

[0160] Accordingly, the feature extraction network to be trained can be trained with the initial audio sample as input data and the difference data as labels to adjust the network parameters of the feature extraction network so that the feature extraction network gradually has the ability to extract the noise generated by the WiFi module carried in the input data.

[0161] It is worth noting that, when the acquisition period of the initial audio sample corresponds to multiple working periods of the WiFi module, the transmission time of the detection request signal of the WiFi module can be used to roughly locate the interference in the audio; then, according to the difference data, the interference-free position with a value close to 0 is removed, so that the difference data is cut into a single-cycle feature sequence corresponding to the working period of the WiFi module. Figure 8 The trimming result diagram shows that the initial audio sample can also be trimmed according to the transmission time of the detection request signal of the WiFi module to obtain a single-cycle initial audio sample. Accordingly, the single-cycle initial audio sample is used as input data and the single-cycle difference data is used as a label to train the feature extraction network to be trained.

[0162] In order to improve the versatility of the trained feature extraction network, the initial audio samples can be audio data collected when the WiFi module works in different working modes, thereby improving the sample diversity of the initial audio samples, which in turn helps to improve the versatility and robustness of the trained feature extraction network.

[0163] It is worth noting that for different types of display devices (such as different categories or models, etc.), training of feature extraction networks suitable for corresponding types can be performed respectively. However, this training method requires a lot of manpower and time costs and has poor universality. In an optional embodiment, a universal feature extraction network can also be used to train a network structure suitable for extracting audio interference features of different types of display devices.

[0164] In some alternative embodiments, see Fig. 9 , provides a model training method, applied to the server, including:

[0165] S910, obtaining a first audio sample generated during the operation of the sample display device when the indoor noise intensity is lower than a first intensity threshold, and a second audio sample generated during the operation of the sample display device after the WiFi driver is uninstalled.

[0166] Among them, the first audio sample can be understood as the audio data collected by the microphone in the sample display device when the indoor noise intensity is lower than the first intensity threshold and the WiFi module in the sample display device is working normally. The audio data may carry the noise generated by the WiFi module during operation, and also include the ambient audio of the indoor environment, etc.

[0167] Among them, the second audio sample can be understood as the audio data collected by the microphone in the sample display device when the indoor noise intensity is lower than the first intensity threshold and the WiFi module in the sample display device is no longer working. The audio data no longer carries the noise generated by the WiFi module during operation, and only includes other audio except the above noise, such as ambient audio of the indoor environment.

[0168] It is worth noting that in order to improve the accuracy of the acquired audio samples, multiple first audio samples under the same conditions can be collected multiple times, and the interference of random noise can be reduced by taking the average value, and the determined average value is used as the first audio sample finally used. Similarly, multiple second audio samples under the same conditions can be collected multiple times, and the interference of random noise can be reduced by taking the average value, and the determined average value is used as the second audio sample finally used.

[0169] S920: Determine a first difference feature between the first audio sample and the second audio sample.

[0170] Among them, the first difference feature is used to measure the difference in audio data between the two situations where the WiFi module in the sample display device works normally and the WiFi module no longer works when the indoor environment meets the preset quiet conditions.

[0171] Exemplarily, the first difference feature between the first audio sample and the second audio sample may be determined by performing a difference.

[0172] S930, using the first audio sample as input data and the first difference feature as a label, training the feature extraction network to be trained until the difference between the trained feature extraction network and the standard feature extraction network meets a preset difference condition.

[0173] Among them, the feature extraction network is used to extract the target interference characteristics of the audio generated by the WiFi module in the display device during network search.

[0174] Exemplarily, the feature extraction network to be trained can be trained with the first audio sample as input data and the first difference feature as a label to adjust the network parameters of the feature extraction network so that the feature extraction network gradually has the ability to extract the noise generated by the WiFi module carried in the input data.

[0175] It is worth noting that when the acquisition period of the first audio sample corresponds to multiple working periods of the WiFi module, the transmission time of the detection request signal of the WiFi module can be used for rough positioning to determine the position of the interference in the audio; then, the interference-free position whose value tends to 0 is removed according to the first difference feature, so as to cut the first difference feature into a single-cycle feature sequence corresponding to the working period of the WiFi module. Accordingly, the first audio sample can also be trimmed according to the transmission time of the detection request signal of the WiFi module to obtain a single-cycle first audio sample. Accordingly, the single-cycle first audio sample is used as input data, and the single-cycle first difference feature is used as a label to train the feature extraction network to be trained.

[0176] In order to improve the versatility of the trained feature extraction network, the first audio sample can be audio data collected when the WiFi module works in different working modes, thereby improving the sample diversity of the first audio sample, which in turn helps to improve the versatility and robustness of the trained feature extraction network.

[0177] In some embodiments, the standard feature extraction network is trained in the following manner: obtaining a third audio sample generated during the operation of the sample display device in a sound-proof environment, and a fourth audio sample generated during the operation of the sample display device after uninstalling the WiFi driver; determining a second difference feature between the third audio sample and the fourth audio sample; using the third audio sample as input data and the second difference feature as a label to train the standard feature extraction network to be trained until the difference between the trained feature extraction network and the standard feature extraction network meets a preset difference condition.

[0178] Wherein, satisfying the preset difference condition may be that the similarity of the output results of the two is less than a preset similarity threshold. Wherein, the similarity can be determined by at least one of the traditional methods, which is not limited in this embodiment. Wherein, the preset similarity threshold can be set by a technician according to needs or experience, or determined repeatedly through a large number of experiments, which is not limited in this embodiment.

[0179] The anechoic environment is an indoor environment in which the ambient sound intensity does not exceed the second intensity threshold. The second intensity threshold is less than the first intensity threshold; the second intensity threshold can be set by technicians according to needs or experience, or determined repeatedly through a large number of experiments.

[0180] Among them, the third audio sample can be understood as the audio data collected by the microphone in the sample display device when the WiFi module in the sample display device works normally in a soundproofing environment. The audio data may carry the noise generated by the WiFi module during operation, as well as the ambient audio of the indoor environment.

[0181] Among them, the fourth audio sample can be understood as the audio data collected by the microphone in the sample display device in a sound-proof environment when the WiFi module in the sample display device is no longer working. The audio data no longer carries the noise generated by the WiFi module during operation, and only includes other audio except the above noise, such as ambient audio in the indoor environment.

[0182] It is worth noting that in order to improve the accuracy of the acquired audio samples, multiple third audio samples under the same conditions may be collected multiple times, and the interference of random noise may be reduced by taking the average value, and the determined average value may be used as the third audio sample finally used. Similarly, multiple fourth audio samples under the same conditions may be collected multiple times, and the interference of random noise may be reduced by taking the average value, and the determined average value may be used as the fourth audio sample finally used.

[0183] The second difference feature is used to measure the difference in audio data between two situations in which the WiFi module in the sample display device works normally and the WiFi module no longer works in a soundproof environment.

[0184] Optionally, the second difference feature between the third audio sample and the fourth audio sample may be determined by performing a difference.

[0185] Exemplarily, the third audio sample can be used as input data and the second difference feature can be used as a label to train the standard feature extraction network to be trained so as to adjust the network parameters of the standard feature extraction network until the training cutoff condition is met, so that the standard feature extraction network gradually has the ability to extract the noise generated by the WiFi module carried in the input data.

[0186] The training cutoff condition may include that the number of training samples is greater than a preset number threshold, the number of iterations meets a preset number threshold, or the feature extraction network tends to converge, etc. The preset number threshold and the preset number threshold, etc., may be set or adjusted by technicians according to needs or experience, or determined repeatedly through a large number of experiments, and this embodiment does not impose any limitation on this.

[0187] It is worth noting that the standard feature extraction network here can be understood as a network model dedicated to audio interference feature extraction, which can be implemented based on a traditional machine learning model or a deep learning model. This embodiment does not impose any restrictions on the network structure of the standard feature extraction network. Optionally, the network structure of the standard feature extraction network is the same as or at least partially different from the network structure of the aforementioned feature extraction network.

[0188] It can be understood that, when the acquisition period of the third audio sample corresponds to multiple working periods of the WiFi module, the transmission time of the detection request signal of the WiFi module can be used for rough positioning to determine the position of the interference in the audio; then, the interference-free position with a value approaching 0 is removed according to the second difference feature, so that the second difference feature is cut into a single-cycle feature sequence corresponding to the working period of the WiFi module. Correspondingly, the third audio sample can also be trimmed according to the transmission time of the detection request signal of the WiFi module to obtain a single-cycle third audio sample. Accordingly, the single-cycle third audio sample is used as input data, and the single-cycle second difference feature is used as a label to train the standard feature extraction network to be trained.

[0189] It should be noted that the execution device used in the training process of the standard feature extraction network and the execution device used in the aforementioned feature extraction network may be the same or different, and this embodiment does not impose any limitation on this.

[0190] Since the audio samples collected by the standard feature extraction network correspond to a soundproofing environment, while the audio samples collected by the aforementioned feature extraction network correspond to a relatively quiet indoor environment, the standard feature extraction network has better feature extraction capabilities, but has more stringent requirements on the working environment of the display device, and is not suitable for conventional device usage scenarios. Therefore, directly using the trained standard feature extraction network to extract audio interference features will cause the extracted audio interference features to not match the working environment of the display device, thereby affecting the accuracy and clarity of the audio noise reduction results. In order to eliminate different interferences in the indoor environment, the above embodiment uses the standard feature extraction network with better feature extraction capabilities as a benchmark, and trains a feature extraction network suitable for extracting interference features in a relatively quiet indoor environment until the trained feature extraction network has the same feature extraction capabilities as the standard feature extraction network, thereby helping to improve the indoor versatility and feature extraction capabilities of the feature extraction network.

[0191] In order to improve the versatility of the trained standard feature extraction network, the third audio sample can be audio data collected when the WiFi module works in different working modes, thereby improving the sample diversity of the third audio sample, which in turn helps to improve the versatility and robustness of the trained standard feature extraction network.

[0192] In some embodiments, the aforementioned trained feature extraction network may be migrated to a server that provides interference noise extraction services. The server may be the same as or different from the server that performs the feature extraction network training process, and this embodiment does not impose any limitation on this.

[0193] Optionally, if the two are the same server, the server can also obtain a call request from the display device to a trained feature extraction network; in response to the call request, perform feature extraction on the interference audio based on the trained feature extraction network; obtain the target interference feature; send the target interference feature to the display device, so that the display device performs noise reduction processing on the first audio data according to the feature frequency and feature intensity of the target interference feature to obtain second audio data; wherein the first audio data includes at least the interference audio of the WiFi module in the current working mode.

[0194] Exemplarily, a call request of the display device 200 to the trained feature extraction network is obtained through a communication device in the server; wherein the call request includes the interference audio of the WiFi module in the display device in the current working mode; in response to the call request, feature extraction is performed on the interference audio based on the trained feature extraction network to obtain a target interference feature; the target interference feature is fed back to the display device 200 through the communication device in the server, so that the display device performs noise reduction processing on the first audio data according to the feature frequency and feature strength of the target interference feature to obtain second audio data; wherein the first audio data at least includes the interference audio of the WiFi module in the current working mode.

[0195] In other embodiments, the aforementioned trained feature extraction network can be directly migrated to the memory 190 of the display device 200 for local calling by the controller 110 of the display device 200, thereby improving the extraction efficiency and convenience of the target interference features.

[0196] Based on the technical solutions of the above embodiments, in some optional embodiments, the audio noise reduction process is described by taking the interaction process between the display device and the server as an example.

[0197] See also Fig.10 The audio noise reduction method shown comprises:

[0198] S1001, the display device obtains the device type of the display device;

[0199] S1002, the display device controls the microphone to collect interference audio in different working modes when the indoor environment meets the preset quiet condition according to the collection cycle corresponding to the device type;

[0200] S1003, the display device sends a call request for the trained feature extraction network to the server;

[0201] S1004, the server extracts features of interference audio in different working modes based on the trained feature extraction network to obtain audio interference features in each working mode;

[0202] S1005, the server sends the audio interference characteristics in different working modes to the display device;

[0203] S1006, the display device stores the audio interference features in different working modes in a local memory in an associated manner;

[0204] S1007, when the voice function is enabled, the display device controls the microphone to collect first audio data;

[0205] S1008, the display device searches the memory for an audio interference feature in the current working mode of the WiFi module as a target interference feature;

[0206] S1009, the display device divides the first audio data according to the detection request signal transmission time of the WiFi module when the first audio data is collected to obtain a first audio segment;

[0207] S1010, the display device deducts the matched target interference feature from each first audio segment to obtain a second audio segment;

[0208] S1011: The display device combines the second audio segments according to the segmentation order of the corresponding first audio segments to obtain second audio data.

[0209] Based on the same inventive concept, the embodiment of the present application also provides an audio noise reduction device for implementing the audio noise reduction method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more audio noise reduction device embodiments provided below can refer to the limitations of the audio noise reduction method above, and will not be repeated here.

[0210] In an exemplary embodiment, Fig.11 As shown, an audio noise reduction device is provided, including: a first acquisition module 1110 and a noise reduction module 1120. Among them,

[0211] A first acquisition module 1110 is used to acquire first audio data collected by a microphone of a display device; wherein the first audio data at least includes interference audio of a WiFi module in a current working mode;

[0212] The first acquisition module 1110 is further used to acquire the audio interference feature of the WiFi module of the display device when performing network search in the current working mode as the target interference feature; wherein the characteristic frequency or characteristic strength of the audio interference feature in different working modes is different;

[0213] The noise reduction module 1120 is used to perform noise reduction processing on the first audio data according to the characteristic frequency and characteristic intensity of the target interference feature to obtain second audio data.

[0214] In some embodiments, the first acquisition module 1110 is also used to control the microphone to collect interference audio of the WiFi module in the current working mode when the indoor noise intensity is lower than the first intensity threshold; the device also includes: a first extraction module, which is used to extract features of the interference audio based on a trained feature extraction network to obtain target interference features.

[0215] In some embodiments, the first extraction module includes: a first sending unit, used to send a call request for a trained feature extraction network to a server, so that the server performs feature extraction on the interference audio based on the trained feature extraction network; a first acquisition unit, used to obtain the feature extraction result fed back by the server; wherein the feature extraction result includes the target interference feature.

[0216] In some embodiments, the first extraction module includes: an operation unit, which is used to extract the target interference features in the interference audio by using the trained feature extraction network stored in the memory during operation.

[0217] In some embodiments, the first acquisition module includes: a second acquisition unit, used to obtain the device type of the display device from the memory; a first control unit, used to control the microphone to collect interference audio when the indoor environment meets the preset quiet conditions according to the collection period corresponding to the device type.

[0218] In some embodiments, the first acquisition module 1110 includes: a second control unit, configured to control the microphone to collect the first audio data when the voice function is turned on.

[0219] Based on the same inventive concept, the embodiment of the present application also provides a model training device for implementing the model training method involved above. The implementation scheme for solving the problem provided by the device is similar to the implementation scheme recorded in the above method, so the specific limitations in one or more model training device embodiments provided below can refer to the limitations on the model training method above, and will not be repeated here.

[0220] In an exemplary embodiment, Fig.12 As shown, a model training device is provided, including: a second acquisition module 1210, a determination module 1220 and a training module 1230. Among them,

[0221] The second acquisition module 1210 is used to acquire a first audio sample generated during the operation of the sample display device when the indoor noise intensity is lower than a first intensity threshold, and a second audio sample generated during the operation of the sample display device after the WiFi driver is uninstalled;

[0222] A determination module 1220, configured to determine a first difference feature between the first audio sample and the second audio sample;

[0223] A training module 1230 is used to train the feature extraction network to be trained using the first audio sample as input data and the first difference feature as a label until the difference between the trained feature extraction network and the standard feature extraction network meets a preset difference condition;

[0224] Among them, the feature extraction network is used to extract the target interference features of the audio generated by the WiFi module in the display device during the network search;

[0225] Among them, the standard feature extraction network is trained in the following way:

[0226] Acquire a third audio sample generated during the operation of the sample display device in the silencing environment, and a fourth audio sample generated during the operation of the sample display device after uninstalling the WiFi driver;

[0227] determining a second difference feature between the third audio sample and the fourth audio sample;

[0228] The third audio sample is used as input data and the second difference feature is used as a label to train the standard feature extraction network to be trained.

[0229] In some embodiments, the device also includes: a third acquisition module, used to obtain a call request for a trained feature extraction network sent by a display device; wherein the call request includes the interference audio of the WiFi module in the display device in the current working mode; a request response module, used to respond to the call request, extract features of the interference audio based on the trained feature extraction network, and obtain target interference features; a feedback module, used to send the target interference features to the display device, so that the display device performs noise reduction processing on the first audio data according to the feature frequency and feature strength of the target interference features to obtain second audio data; wherein the first audio data at least includes the interference audio of the WiFi module in the current working mode.

[0230] Each module in the above-mentioned audio noise reduction device / model training device can be implemented in whole or in part by software, hardware and a combination thereof. Each of the above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory in the computer device in the form of software, so that the processor can call and execute the operations corresponding to each of the above modules.

[0231] In an exemplary embodiment, a computer device is provided. The computer device may be a terminal, and its internal structure diagram may be as shown in FIG. Fig.13As shown. The computer device includes a processor, a memory, an input / output interface, a communication interface, a display unit and an input device. Among them, the processor, the memory and the input / output interface are connected through a system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input / output interface. Among them, the processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The input / output interface of the computer device is used to exchange information between the processor and the external device. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be implemented through WIFI, a mobile cellular network, near field communication (Near Field Communication, NFC) or other technologies. When the computer program is executed by the processor, an audio noise reduction method is implemented. The display unit of the computer device is used to form a visually visible picture, which can be a display screen, a projection device or a virtual reality imaging device. The display screen can be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device can be a touch layer covering the display screen, or a button, trackball or touchpad set on the computer device shell, or an external keyboard, touchpad or mouse.

[0232] Those skilled in the art will understand that Fig.13 The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0233] In an alternative embodiment, Fig.13 The computer device shown may be the aforementioned projection device, for example, a laser TV.

[0234] In an exemplary embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps in the above-mentioned method embodiments when executing the computer program. In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and the computer program implements the steps in the above-mentioned method embodiments when executed by the processor.

[0235] In one embodiment, a computer program product is provided, including a computer program, which implements the steps in the above method embodiments when executed by a processor.

[0236] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0237] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to the memory, database or other medium used in the embodiments provided in the present application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The database involved in each embodiment provided in this application may include at least one of a relational database and a non-relational database. Non-relational databases may include distributed databases based on blockchains, etc., but are not limited to this. The processor involved in each embodiment provided in this application may be a general-purpose processor, a central processing unit, a graphics processor, a digital signal processor, a programmable logic device, a data processing logic device based on quantum computing, an artificial intelligence (AI) processor, etc., but are not limited to this.

[0238] The technical features of the above embodiments may be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0239] The above embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. A display device, characterized in that: include: A wireless fidelity WiFi module configured to perform network searches in different operating modes; A microphone is disposed on the same circuit board as the WiFi module and is configured to collect audio data; At least one controller, connected to the microphone and the WiFi module, configured to: Controlling the microphone to collect first audio data; wherein the first audio data at least includes interference audio of the WiFi module in the current working mode; The audio interference feature of the WiFi module in the current working mode is used as the target interference feature; wherein the characteristic frequency or characteristic strength of the audio interference feature in different working modes is different; The first audio data is subjected to noise reduction processing according to the characteristic frequency and characteristic intensity of the target interference feature to obtain second audio data.

2. The display device according to claim 1, characterized in that The controller is also configured to: When the indoor noise intensity is lower than a first intensity threshold, controlling the microphone to collect interference audio of the WiFi module in the current working mode; Based on the trained feature extraction network, feature extraction is performed on the interference audio to obtain target interference features.

3. The display device according to claim 2, characterized in that The display device further includes: A communication device, connected to the server for communication; The controller is configured to: Sending a call request for the trained feature extraction network to the server, so that the server extracts features of the interference audio based on the trained feature extraction network; Acquire the feature extraction result fed back by the server; wherein the feature extraction result includes the target interference feature.

4. The display device according to claim 2, characterized in that: The memory is further configured to store the trained feature extraction network; The controller is configured to: The trained feature extraction network stored in the memory is run to extract target interference features in the interference audio.

5. The display device according to claim 2, characterized in that The memory is further configured to store a device type of the display device; When the controller controls the microphone to collect interference audio when the indoor noise intensity is lower than the first intensity threshold, the controller is configured to: Acquire a device type of the display device from the memory; According to the collection period corresponding to the device type, the microphone is controlled to collect interference audio when the indoor noise intensity is lower than a first intensity threshold.

6. The display device according to any one of claims 1 to 5, characterized in that: When the controller controls the microphone to collect the first audio data, the controller is configured to: When the voice function is turned on, the microphone is controlled to collect first audio data.

7. A server, characterized in that: include: A communication device, communicatively connected to the sample display device; At least one processor is connected to the communication device and is configured to: Acquire a first audio sample generated during the operation of the sample display device when the indoor noise intensity is lower than a first intensity threshold, and a second audio sample generated during the operation of the sample display device after the WiFi driver is uninstalled; determining a first difference feature between the first audio sample and the second audio sample; Using the first audio sample as input data and the first difference feature as a label, training the feature extraction network to be trained until the difference between the trained feature extraction network and the standard feature extraction network meets a preset difference condition; Wherein, the feature extraction network is used to extract audio interference features of the audio generated by the WiFi module in the display device during network search; The standard feature extraction network is trained in the following way: Acquire a third audio sample generated during the operation of the sample display device in the silencing environment, and a fourth audio sample generated during the operation of the sample display device after the WiFi driver is uninstalled; determining a second difference feature between the third audio sample and the fourth audio sample; The third audio sample is used as input data and the second difference feature is used as a label to train the standard feature extraction network to be trained.

8. The server according to claim 7, characterized in that: The server is also configured to: Obtaining a call request for a trained feature extraction network sent by the display device; wherein the call request includes interference audio of a WiFi module in the display device in a current working mode; In response to the call request, extracting features of the interference audio based on a trained feature extraction network to obtain target interference features; The target interference feature is sent to the display device so that the display device performs noise reduction processing on the first audio data according to the characteristic frequency and characteristic intensity of the target interference feature to obtain second audio data; wherein the first audio data at least includes the interference audio of the WiFi module in the current working mode.

9. An audio noise reduction method, characterized in that: Applied to display devices, including: Acquire first audio data collected by a microphone of a display device; wherein the first audio data at least includes interference audio of the WiFi module in a current working mode; and, Acquire the audio interference feature of the WiFi module of the display device when performing network search in the current working mode as the target interference feature; wherein the characteristic frequency or characteristic strength of the audio interference feature in different working modes is different; The first audio data is subjected to noise reduction processing according to the characteristic frequency and characteristic intensity of the target interference feature to obtain second audio data.

10. A model training method, characterized in that: Applicable to servers, including: Acquire a first audio sample generated during the operation of the sample display device when the indoor noise intensity is lower than a first intensity threshold, and a second audio sample generated during the operation of the sample display device after the WiFi driver is uninstalled; determining a first difference feature between the first audio sample and the second audio sample; Using the first audio sample as input data and the first difference feature as a label, training the feature extraction network to be trained until the difference between the trained feature extraction network and the standard feature extraction network meets a preset difference condition; Wherein, the feature extraction network is used to extract audio interference features of the audio generated by the WiFi module in the display device during network search; The standard feature extraction network is trained in the following way: Acquire a third audio sample generated during the operation of the sample display device in the silencing environment, and a fourth audio sample generated during the operation of the sample display device after the WiFi driver is uninstalled; determining a second difference feature between the third audio sample and the fourth audio sample; The third audio sample is used as input data and the second difference feature is used as a label to train the standard feature extraction network to be trained.

Images