Display devices and sound processing methods

Abstract

This application discloses a display device and a sound effect processing method. The display device includes a system-on-a-chip (SoC), a sound effect chip, and a controller. The SoC of the display device performs first sound effect processing on initial audio data to determine intermediate audio data. Further, the controller determines the acquisition frequency of the intermediate audio data. After the SoC transmits the intermediate audio data to the sound effect chip, it can determine the adjustment stage of the intermediate audio data based on the acquisition frequency and the preset audio delay conditions of the application corresponding to the initial audio data. Further, based on the adjustment stage, it can determine the functional permission to perform second sound effect processing on the intermediate audio data and trigger the sound effect chip to perform corresponding second sound effect processing on the intermediate audio data according to the functional permission. This allows the sound effect chip to perform sound effect data processing according to the specific situation of the intermediate audio data, avoiding delays caused by sound effect processing and improving the user experience.

Description

Technical Field

[0001] This application relates to the field of display technology, and in particular to a display device and a sound processing method. Background Technology

[0002] With the development of technology, display devices are becoming increasingly diversified, offering users a wider range of functions. Display devices include smart TVs, smartphones, and other products with display screens. Taking smart TVs as an example, they not only serve as devices for watching video programs but also provide users with functions such as video calls, karaoke, games, and learning.

[0003] In related technologies, display devices primarily use chips to process audio data for various applications. The chip processes each audio data item sequentially according to the order in the thread. However, chip resources are limited. If there is too much audio data to process, or if the application has high real-time requirements for audio processing (for example, in a karaoke application, the voice data received by the chip in the display device requires high real-time audio processing to match the voice data with the accompaniment data provided by the application), delays in audio data processing will occur, affecting the user experience. Summary of the Invention

[0004] This application provides a display device and a sound effect processing method, which can be used to solve the technical problem of delay in the processing of sound effect data for various applications by the display device through the chip.

[0005] In a first aspect, some embodiments of this application provide a display device, including a display, a system-on-a-chip (SoC), an audio chip, and a controller, wherein:

[0006] The system-on-a-chip is configured to perform first sound effect processing on the initial audio data to determine the intermediate audio data;

[0007] The audio chip is configured to perform secondary audio processing on intermediate audio data;

[0008] The controller communicates with the system-on-a-chip and the audio chip. The controller is configured as follows:

[0009] Determine the acquisition frequency of the intermediate audio data;

[0010] After the system-on-a-chip transmits the intermediate audio data to the audio effects chip, the adjustment stage of the intermediate audio data is determined based on the sampling frequency and the preset audio delay conditions of the application corresponding to the initial audio data.

[0011] Based on the adjustment phase, the function permission to perform second sound effect processing on the intermediate audio data is determined, and the sound effect chip is triggered to perform the corresponding second sound effect processing on the intermediate audio data according to the function permission.

[0012] Secondly, some embodiments of this application provide a sound effect processing method applied to a display device. The display device includes a system-on-a-chip, a sound effect chip, and a controller. The sound effect processing method includes the following steps:

[0013] The acquisition frequency of the intermediate audio data is determined by the system-on-a-chip performing the first sound effect processing on the initial audio data;

[0014] After the system-on-a-chip transmits the intermediate audio data to the audio effects chip, the adjustment stage of the intermediate audio data is determined based on the sampling frequency and the preset audio delay conditions of the application corresponding to the initial audio data.

[0015] Based on the adjustment phase, the function permission to perform second sound effect processing on the intermediate audio data is determined, and the sound effect chip is triggered to perform the corresponding second sound effect processing on the intermediate audio data according to the function permission.

[0016] Some embodiments of this application provide a display device and an audio processing method. The system-on-a-chip (SoC) of the display device performs first audio processing on initial audio data to determine intermediate audio data. Further, it determines the acquisition frequency of the intermediate audio data. After the SoC transmits the intermediate audio data to the audio processing chip, it can determine the adjustment stage of the intermediate audio data based on the acquisition frequency and the preset audio delay conditions of the application corresponding to the initial audio data. Further, based on the adjustment stage, it can determine the functional permission to perform second audio processing on the intermediate audio data and trigger the audio processing chip to perform corresponding second audio processing on the intermediate audio data according to the functional permission. This allows the audio processing chip to implement audio processing according to the specific situation of the intermediate audio data, avoiding delays caused by audio processing and improving the user experience. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This application illustrates operational scenarios between a display device and a control device according to some embodiments;

[0019] Figure 2 A hardware configuration block diagram of a display device 200 according to some embodiments of this application is shown;

[0020] Figure 3 The present application illustrates a software configuration diagram in a display device according to some embodiments;

[0021] Figure 4 The illustration shows a schematic flowchart of a sound effect processing method in a display device according to some embodiments of this application;

[0022] Figure 5 A schematic flowchart of another sound effect processing method in a display device according to some embodiments of this application is shown;

[0023] Figure 6 This application shows a schematic diagram illustrating the process of determining the adjustment stage in a display device according to some embodiments;

[0024] Figure 7 This application shows a schematic diagram illustrating the process of determining the adjustment stage in a display device according to some embodiments;

[0025] Figure 8 The following is a flowchart illustrating the sound effect processing method for a karaoke application in a display device according to some embodiments of this application;

[0026] Figure 9 This application illustrates a schematic diagram of the software configuration in a display device for a karaoke application, as shown in some embodiments of the present application.

[0027] Figure 10 A schematic flowchart of another sound effect processing method in a display device according to some embodiments of this application is shown;

[0028] Figure 11 The diagram shows a flowchart of another sound effect processing method in a display device according to some embodiments of this application. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the exemplary embodiments of this application clearer, the technical solutions in the exemplary embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described exemplary embodiments are only some embodiments of this application, and not all embodiments.

[0030] Based on the exemplary embodiments shown in this application, all other embodiments obtained by those skilled in the art without inventive effort are within the scope of protection of this application. Furthermore, although the disclosures in this application are presented by way of one or more exemplary examples, it should be understood that each aspect of these disclosures can constitute a complete technical solution on its own.

[0031] It should be understood that the terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate, for example, to allow implementation in orders other than those given in the embodiments illustrated or described in this application.

[0032] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclusively include, for example, a product or device that includes a series of components is not necessarily limited to those that are explicitly listed, but may include other components that are not explicitly listed or that are inherent to such product or device.

[0033] The display device provided in this application can have various implementation forms, such as a television, smart television, smartphone, tablet computer, computer, laser projection device, monitor, electronic bulletin board, electronic table, and products with display screens. Figure 1 This is one specific embodiment of the display device of this application.

[0034] Figure 1 This is a schematic diagram illustrating the operational scenario between the display device and the control unit according to the embodiment. Figure 1 As shown, the user can operate the display device 200 through the smart device 300 or the control device 100.

[0035] In some embodiments, the control device 100 may be a remote control. Communication between the remote control and the display device includes infrared protocol communication, Bluetooth protocol communication, and other short-range communication methods, controlling the display device 200 wirelessly or via wired means. Users can control the display device 200 by inputting user commands through buttons on the remote control, voice input, control panel input, etc.

[0036] In some embodiments, a smart device 300 (such as a mobile terminal, tablet computer, computer, laptop computer, etc.) may also be used to control the display device 200. For example, an application running on the smart device may be used to control the display device 200.

[0037] In some embodiments, the display device may receive instructions not through the aforementioned smart devices or control devices, but through touch or gestures.

[0038] In some embodiments, the display device 200 can also be controlled in ways other than the control device 100 and the smart device 300. For example, it can be controlled by directly receiving the user's voice commands through a module configured inside the display device 200 for acquiring voice commands, or it can be controlled by receiving the user's voice commands through a voice control device set outside the display device 200.

[0039] In some embodiments, the display device 200 also communicates with the server 400. The display device 200 may communicate via a local area network (LAN), wireless local area network (WLAN), and other networks. The server 400 may provide various content and interactive features to the display device 200. The server 400 may be a cluster or multiple clusters, and may include one or more types of servers.

[0040] like Figure 2 The display device 200 includes at least one of the following: a tuner 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0041] In some embodiments, the controller includes a processor, a video processor, an audio processor, a graphics processor, RAM, ROM, and a first interface to an nth interface for input / output.

[0042] The display device 200 also includes a system-on-a-chip (SoC) and an audio chip. Both the SoC and the audio chip can process audio data. In some embodiments, the SoC can perform basic audio processing on the audio data, while the audio chip can perform basic audio processing or specific audio processing. Basic audio processing includes echo cancellation, sound enhancement, and noise reduction; specific audio processing includes echo cancellation, sound enhancement, noise reduction, bass enhancement, bass effects, automatic gain control, and feedback suppression.

[0043] The display 260 includes a display screen assembly for presenting images, a driving assembly for driving image display, a component for receiving image signals from the controller output, and a user control UI interface for displaying video content, image content, menu control interface, and user control UI interface.

[0044] The display 260 can be an LCD display, an OLED display, or a projection display, and can also be a projection device and a projection screen.

[0045] The communicator 220 is a component used to communicate with external devices or servers according to various communication protocol types. For example, the communicator may include at least one of the following: a Wi-Fi module, a Bluetooth module, a wired Ethernet module, other network communication protocol chips or near-field communication protocol chips, and an infrared receiver. The display device 200 can establish the transmission and reception of control signals and data signals with the control device 100 or the server 400 through the communicator 220.

[0046] The user interface can be used to receive control signals from the control device 100 (such as an infrared remote control).

[0047] Detector 230 is used to collect signals from the external environment or to interact with the external environment. For example, detector 230 includes a light receiver, a sensor for collecting ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to collect external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0048] The external device interface 240 may include, but is not limited to, one or more of the following: High Definition Multimedia Interface (HDMI), analog or high-definition component input interface (component), composite video input interface (CVBS), USB input interface (USB), RGB port, etc. It may also be a composite input / output interface formed by multiple interfaces mentioned above.

[0049] The tuner / demodulator 210 receives broadcast television signals via wired or wireless means, and demodulates audio and video signals, such as EPG data signals, from multiple wireless or wired broadcast television signals.

[0050] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0051] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command to select a UI object to display on the monitor 260, the controller 250 can execute operations related to the object selected by the user command.

[0052] In some embodiments, the controller includes at least one of a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), RAM (random access memory), ROM (read-only memory), a first to an nth interface for input / output, a communication bus, etc.

[0053] Users can input commands through a graphical user interface (GUI) displayed on the monitor 260, and the user input interface receives the user input commands through the GUI. Alternatively, users can input commands by entering specific sounds or gestures, and the user input interface receives the user input commands by recognizing the sounds or gestures through sensors.

[0054] A "user interface" is the medium through which an application or operating system interacts and exchanges information with the user. It converts information from its internal form to a form that the user can accept. A common form of user interface is the graphical user interface (GUI), which refers to a user interface related to computer operation displayed graphically. It can be an icon, window, control, or other interface element displayed on the screen of an electronic device. Controls can include visual interface elements such as icons, buttons, menus, tabs, text boxes, dialog boxes, status bars, navigation bars, and widgets.

[0055] like Figure 3 In some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "Application Layer"), the Application Framework layer (referred to as the "Framework Layer"), the Android Runtime and System Library layer (referred to as the "System Runtime Layer"), and the kernel layer.

[0056] In some embodiments, at least one application runs in the application layer. These applications may be Windows programs, system settings programs, or clock programs that come with the operating system; they may also be applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0057] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0058] like Figure 3 As shown, the application framework layer in this embodiment includes managers, content providers, etc., wherein the managers include at least one of the following modules: ActivityManager, which interacts with all activities running in the system; LocationManager, which provides access to system location services for system services or applications; PackageManager, which retrieves various information related to application packages currently installed on the device; NotificationManager, which controls the display and clearing of notification messages; and WindowManager, which manages user interface elements including icons, windows, toolbars, wallpapers, and desktop widgets.

[0059] In some embodiments, the Activity Manager manages the lifecycle of individual applications and common navigation and back functions, such as controlling application exit, opening, and back actions. The Window Manager manages all window programs, such as obtaining the screen size, determining if a status bar is present, locking the screen, capturing the screen, and controlling display window changes (e.g., shrinking the display window, shaking the display, distorting the display, etc.).

[0060] In some embodiments, the system runtime library layer provides support for the upper layer, namely the framework layer. When the framework layer is used, the Android operating system runs the C / C++ libraries contained in the system runtime library layer to implement the functions that the framework layer needs to perform.

[0061] In some embodiments, the kernel layer is a layer between hardware and software. For example... Figure 3 As shown, the kernel layer includes at least one of the following drivers: audio driver, display driver, Bluetooth driver, camera driver, WIFI driver, USB driver, HDMI driver, sensor driver (such as fingerprint sensor, temperature sensor, pressure sensor, etc.), and power driver.

[0062] Display devices are becoming increasingly diverse in function, providing users with a wider range of features and enhancing the user experience. As these functions become more diverse, the types and number of applications requiring sound effects processing also increase.

[0063] For example, in karaoke applications, the display device needs to process the received vocal data through a chip to match the vocal data with the accompaniment data for output; in media playback applications, the display device can process the audio data in media assets (video, audio, etc.) through a chip to provide users with different audio experience.

[0064] It should be noted that the chip can be a system-on-a-chip (SoC) with sound effect adjustment function in the display device, or it can be a standalone sound effect chip, meaning that the chip only needs to have sound effect adjustment performance, or it can have both SoC and sound effect chip functions.

[0065] When a display device includes both a main control chip and an audio processing chip for audio effects, the audio processing requires collaboration between the two. Since each chip has limited resources, if there is a large amount of audio data to be processed, or if the application has high real-time requirements for audio processing (for example, in a karaoke application, the vocal data received by the chip in the display device requires high real-time audio processing to match the accompaniment data provided by the application), poor collaboration between the two chips can lead to delayed sound output. Therefore, dynamic processing of audio data based on the latency requirements of various applications enables dynamic adjustment of the audio effects.

[0066] It should be understood that different applications have different latency requirements. Some applications are sensitive to latency in audio processing; some applications are not very sensitive to latency in audio processing, but improving audio quality within the acceptable latency requirements of the application will enhance the user experience; some applications are not sensitive to latency in audio processing, but have high requirements for audio quality; and so on.

[0067] To address the latency issue in the chip-based audio data processing of display devices for various applications, Figure 4This document illustrates a flowchart of an audio processing method in a display device according to some embodiments of this application. The embodiment provides an audio processing method applied to a display device, which includes a display, a system-on-a-chip (SoC), an audio chip, and a controller. The controller is communicatively connected to the SoC and the audio chip. The SoC of the display device performs first audio processing on initial audio data to determine intermediate audio data. It further determines the acquisition frequency of the intermediate audio data. After the SoC transmits the intermediate audio data to the audio chip, it determines an adjustment stage for the intermediate audio data based on the acquisition frequency and a preset audio delay condition corresponding to the initial audio data. Furthermore, based on the adjustment stage, it determines the functional permission to perform second audio processing on the intermediate audio data and triggers the audio chip to perform corresponding second audio processing on the intermediate audio data according to the functional permission. This allows the audio chip to implement audio processing according to the specific situation of the intermediate audio data, avoiding delays caused by audio processing and improving the user experience.

[0068] To facilitate a further understanding of the technical solutions in the embodiments of this application, the following detailed description of each step of the sound effect processing method in the display device is provided in conjunction with some embodiments and accompanying drawings. Figure 4 As shown, this sound effect processing method includes the following steps:

[0069] S110: In response to the sound processing command for application startup, receive the initial audio data of the application.

[0070] It should be noted that the sound effect processing commands for each application are triggered based on the user's interaction with the application, and the timing and form of triggering are determined by the application's functional settings. For example, in media playback applications, the sound effect processing command might be initiated when the media is clicked to play; in karaoke applications, the sound effect processing command might be initiated when the recording control is clicked; and so on.

[0071] Each application provides its own initial audio data. It should be noted that the initial audio data can be obtained directly by the application or through a cache. For example, if the application is a media playback application, its initial audio data is obtained from the corresponding media assets. If the application is a karaoke application, its initial audio data can be human voice data, obtained through the microphone on the display device or an external microphone. In this case, the initial audio data needs to be stored in an audio cache.

[0072] s120. Perform the first sound effect processing on the initial audio data to determine the intermediate audio data.

[0073] The initial sound effect processing of the initial audio data is implemented by the system-on-a-chip (SoC) in the display device. The SoC's initial sound effect processing of the initial audio data is mainly determined based on the application's processing requirements for the initial audio data.

[0074] The first audio processing step determines the intermediate audio data, which is now output through speakers, headphones, and other players. However, for increasingly feature-rich display devices, the intermediate audio data can undergo a second audio processing step via an audio chip. Before the audio chip performs the second audio processing on the intermediate audio data, the following steps are taken to further determine its quality, ensuring timely output and good sound quality.

[0075] In some embodiments, during the first sound effect processing of the initial audio data by the system-on-chip (SoC), there may be delays. The system-on-chip's sound effect processing cache resources can be used to determine whether to perform the first sound effect processing on the initial audio data. If the system-on-chip's sound effect processing cache resources meet the application's sound effect processing requirements, the first sound effect processing is performed, that is, the initial audio data is determined as intermediate audio data after the first sound effect processing. If the system-on-chip's sound effect processing cache resources do not meet the application's sound effect processing requirements, the system-on-chip does not perform the first sound effect processing on the initial audio data, that is, the initial audio data is used as intermediate audio data.

[0076] Figure 5 This application illustrates a flowchart of another sound effect processing method in a display device according to some embodiments, such as... Figure 5 As shown, before the system-on-a-chip performs the first sound effect processing on the initial audio data in step 120, the determination of the sound effect processing cache resources in the system-on-a-chip is also included. The specific process is as follows:

[0077] s1201 Determine whether the audio processing cache resources in the system-on-a-chip meet the audio processing requirements of the application.

[0078] For system-on-a-chip (SoC), the cache resources used for audio processing are limited. In order to improve the collaboration of various chips in the display device for audio data and audio effects processing, the performance of audio processing of SoC is assessed, and issues such as audio processing latency are further improved.

[0079] If the audio processing cache resources of the system-on-a-chip meet the audio processing requirements of the application, step 120 is executed to perform the first audio processing on the initial audio data to determine the intermediate audio data.

[0080] If the audio processing cache resources of the system-on-a-chip do not meet the audio processing requirements of the application, the system-on-a-chip will not perform the first audio processing on the initial audio data, and will execute S1202 to use the initial audio data as intermediate audio data.

[0081] In other words, the second audio effect processing will be performed on the intermediate audio data (which is directly determined by the initial audio data) through the audio effect chip.

[0082] In some embodiments, when the audio processing cache resources of the system-on-a-chip meet the audio processing requirements of the application, the first audio processing of the initial audio data by the system-on-a-chip may further include: determining the processing thread corresponding to each initial audio data by determining the priority of the initial audio data or the order in which the initial audio data is acquired; and the intermediate audio data obtained by each thread having different degrees of delay due to the busyness of each thread during the processing of the initial audio data.

[0083] In some embodiments, the intermediate audio data may have different degrees of delay at different times of use within the same application; of course, the delay of the intermediate audio data may be the same or different for different applications.

[0084] For example, in a karaoke application, at the first moment, after the system-on-a-chip (SoC) processes the initial audio data with its first sound effect, the resulting intermediate audio data has a 10ms delay. At the second moment, after the SoC processes the initial audio data with its first sound effect, the resulting intermediate audio data has a 30ms delay. At the third moment, after the SoC processes the initial audio data with its first sound effect, the resulting intermediate audio data has a 2ms delay. At the fourth moment, after the SoC processes the initial audio data with its first sound effect, the resulting intermediate audio data has a 30ms delay. In other words, for the same application, the threads executing the SoC's first sound effect processing of the initial audio data may obtain intermediate audio data with different or the same delay. The processing procedure of the audio chip also differs for the intermediate audio data at different stages.

[0085] like Figure 4 As shown, it also includes: s130, determining the acquisition frequency of intermediate audio data.

[0086] During the process of transmitting intermediate audio data to the audio effects chip for processing, it is necessary to determine the sampling frequency of the intermediate audio data. The sampling frequency of the intermediate audio data can be determined in one of the following ways:

[0087] If the initial audio data is provided by the karaoke application, i.e., data obtained from the cache, the sampling frequency of the intermediate audio data can be determined by the sampling frequency of the initial audio data; the sampling frequency of the intermediate audio data is determined based on the difference between the sampling period corresponding to the sampling frequency of the initial audio data and the application latency time; wherein, the sampling frequency of the intermediate audio data is greater than or equal to the sampling frequency of the initial audio data.

[0088] In some embodiments, after the system-on-a-chip (SoC) performs the first sound effect processing on the initial audio data, if the obtained intermediate audio data has a long delay, due to the limitation of the total time available for sound effect adjustment in the corresponding application, the acquisition frequency of the intermediate audio data can be increased and the acquisition cycle reduced, thus reserving more time for the sound effect chip to perform the second sound effect processing. If the obtained intermediate audio data has a short delay, the acquisition frequency of the intermediate audio data can be maintained or increased, thus reserving effective or more processing time for the sound effect chip to perform the second sound effect processing.

[0089] If the initial audio data is provided by a media playback application, that is, obtained directly through the application, then the acquisition frequency of the intermediate audio data can be directly obtained.

[0090] If the intermediate audio data is determined directly from the initial audio data without undergoing the first audio processing through the system-on-a-chip, its acquisition frequency can be the acquisition frequency of the initial audio data.

[0091] In some embodiments, other audio data storage methods may also exist. The determination of the sampling frequency varies for different audio data storage methods, and will not be limited here.

[0092] like Figure 4 As shown, after the system-on-a-chip transmits the intermediate audio data to the audio effects chip, the process also includes: S140, determining the adjustment stage of the intermediate audio data based on the sampling frequency and the preset audio delay conditions of the application corresponding to the initial audio data.

[0093] In some embodiments, intermediate audio data carrying its acquisition frequency can be transmitted via middleware or kernel, enabling the intermediate audio data to be transmitted from the system-on-a-chip to the audio chip.

[0094] The preset audio delay conditions for each application are determined by the performance of each application.

[0095] For example, in karaoke apps, if there is a delay in the matching process between vocal data and accompaniment data, it is easily perceived by users and affects their user experience. Such apps are sensitive to delay, and the time requirements corresponding to their preset audio delay conditions are relatively short.

[0096] If the user can perceive a delay of 40ms, then the maximum time in the corresponding preset audio delay conditions must be less than 40ms.

[0097] For example, media asset playback applications use audio data provided by the corresponding media assets. During audio processing, they are not sensitive to audio data latency. For instance, if the media asset is video, both audio and image data are provided by the corresponding media asset. Audio and image data output can be synchronized through control, preventing mismatches caused by latency in audio processing. If users have high demands for audio quality, audio effects processing can be used to improve sound quality and enhance the user experience. These applications are less or less sensitive to latency, and their preset audio latency conditions correspond to longer time requirements than those of latency-sensitive applications.

[0098] If the user can accept a delay time of 60ms, then the maximum time in the corresponding preset audio delay conditions must be less than 60ms.

[0099] In other words, the preset audio delay conditions of each application are different. Therefore, the fourth and fifth durations in the preset audio delay conditions of each application are different. Furthermore, the fourth duration is shorter than the fifth duration, and the fifth duration is less than or equal to the longest duration in the preset audio delay conditions.

[0100] Figure 6 The following is a schematic diagram illustrating the process of determining the adjustment stage in a display device according to some embodiments of this application, such as... Figure 6 As shown, in step 140, based on the acquisition frequency and the preset audio delay conditions corresponding to the initial audio data, the adjustment stage of the intermediate audio data is determined, including the following steps:

[0101] S1401. If the sampling period corresponding to the sampling frequency is less than the fourth duration in the preset audio delay conditions, the adjustment stage is the first stage.

[0102] The first stage can be determined as the processing time for performing all sound effects processing.

[0103] For example, executable sound effects processing may include A, B, C, and D. If its adjustment stage is determined to be the first stage, the processing time for executing sound effects processing may include A, B, C, and D.

[0104] In some embodiments, the first stage also represents a stage where the latency requirement of the application corresponding to the intermediate audio data is no greater than the first duration.

[0105] It should be noted that the first duration and the fourth duration can be equal or unequal. The first duration is used to determine the delay requirement, while the fourth duration is the data of the limit duration in the preset audio delay conditions.

[0106] S1402. If the sampling period corresponding to the sampling frequency is greater than or equal to the fourth duration, and the sampling period corresponding to the sampling frequency is less than the fifth duration in the preset audio delay conditions, the adjustment stage is the second stage.

[0107] The second stage can be identified as the processing time for performing some sound effects processing.

[0108] For example, executable sound effects processing can include A, B, C, and D. If the adjustment stage is determined to be the second stage, the processing time for which sound effects processing can be executed can only include A and C, or only include B, etc.

[0109] In some embodiments, the second stage also represents a stage where the application's latency requirement is no greater than a second duration, which is shorter than the first duration. That is, the processing time for sound effects processing at this stage is shorter than the processing time for sound effects processing in the first stage.

[0110] It should be noted that the second duration and the fifth duration may be equal or unequal.

[0111] S1403. If the acquisition period corresponding to the acquisition frequency is greater than or equal to the fifth duration, the adjustment stage is the third stage.

[0112] The third stage can be determined as a processing time that does not include sound effect processing.

[0113] In some embodiments, the third stage also represents a stage where the application's latency requirement is no greater than a third duration, which is less than the second duration.

[0114] It should be noted that for different applications, even if the intermediate audio data acquisition frequency is the same, the corresponding adjustment stages will differ due to the different preset audio delay conditions for each application. For the same application, different acquisition frequencies will result in different adjustment stages.

[0115] For example, the intermediate audio data acquisition frequency of a karaoke app is K, and its corresponding preset audio delay conditions include a fourth duration of M and a fifth duration of N; while the intermediate audio data acquisition frequency of a music app is K, and its corresponding preset audio delay conditions include a fourth duration of O and a fifth duration of P. At this time, although the intermediate audio data acquisition frequency of the two apps is the same, the preset audio delay conditions are different, and the corresponding adjustment stages are also different.

[0116] For example, in a karaoke app's preset audio delay conditions, the fourth duration is M and the fifth duration is N. If the sampling frequency of the karaoke app's intermediate audio data is X, and the sampling period corresponding to sampling frequency X is less than the fourth duration M, the adjustment phase of the intermediate audio data is the first phase, which represents the processing time for performing all sound effects processing. If the sampling frequency of the karaoke app's intermediate audio data is Y, and the sampling period corresponding to sampling frequency Y is greater than or equal to the fourth duration M and less than the fifth duration N, the adjustment phase of the intermediate audio data is the second phase, which represents the processing time for performing some sound effects processing. If the sampling frequency of the karaoke app's intermediate audio data is Z, and the sampling period corresponding to sampling frequency Z is greater than or equal to the fourth duration M and less than the fifth duration N, the adjustment phase of the intermediate audio data is the second phase, which represents the processing time for performing some sound effects processing.

[0117] For example, if the perceived latency for a karaoke app user is 40ms, the maximum time in the preset audio latency conditions must be less than 40ms, with the fourth duration being 15ms and the fifth duration being 25ms. If the intermediate audio data obtained after the first sound effect processing of the initial audio data in the karaoke app has a 10ms latency, meaning that to ensure the user experience, the processing time for the second sound effect processing of the audio chip is at most 30ms, then if step 140 determines the adjustment stage of the intermediate audio data as the first stage based on the sampling frequency and preset audio latency conditions, the sampling frequency of the intermediate audio data does not need to be adjusted. If the intermediate audio data obtained after the first sound effect processing of the initial audio data in the karaoke app has a 20ms latency, meaning that to ensure the user experience, the processing time for the second sound effect processing of the audio chip is at most 20ms, then by increasing the sampling frequency, the adjustment stage of the intermediate audio data can be changed from the second stage to the first stage through step 140, thereby increasing the degree of second sound effect processing on the intermediate audio data.

[0118] In some embodiments, different functions of the same application may have different preset audio delay conditions. For example, in a music application, the preset audio delay conditions for music playback and recording functions may be different.

[0119] In some embodiments, since the second sound effect processing is not performed when the adjustment stage is the third stage, it can be determined first by judging whether the adjustment stage is the third stage, and then the function permission for performing the second sound effect processing on the intermediate audio data based on the adjustment stage can be reduced.

[0120] In some embodiments, if the adjustment stage is the third stage, it can also be determined by whether a sound effect chip is needed. That is, in the third stage, the sound effect chip is not needed to perform the second sound effect processing.

[0121] like Figure 4 As shown, it also includes: S150, determining the function permission to perform second sound effect processing on intermediate audio data based on the adjustment phase.

[0122] The corresponding secondary sound effect processing function permissions differ for different adjustment stages. Based on each adjustment stage, the corresponding function permissions are turned off or on.

[0123] Figure 7 The following is a schematic diagram illustrating the process of determining the adjustment stage in a display device according to some embodiments of this application, such as... Figure 7 As shown, in step 150, based on the adjustment phase, the function permission to perform second sound effect processing on the intermediate audio data is determined, including the following steps:

[0124] S1501. If the adjustment stage of the intermediate audio data is the first stage, the function permission is to perform all sound effect processing functions on the intermediate audio data.

[0125] The first stage can be characterized by the processing time for performing all sound effects processing; the first stage can also be characterized by the latency requirement of the application corresponding to the intermediate audio data being no greater than the first duration, which can be understood as a stage that is not sensitive to latency.

[0126] In other words, the functional permissions at this time can include all sound effect processing. That is, for services that do not have high latency requirements but have high sound quality requirements (such as video playback and music playback), all sound effect processing can be performed to give full play to the role of the sound chip and improve the user experience.

[0127] S1502. If the adjustment stage of the intermediate audio data is the second stage, the function permission is determined to be the sound effect processing function whose function requirement weight for executing the application on the intermediate audio data is greater than the preset threshold.

[0128] The first stage can represent the processing time of performing some sound effects processing; the second stage represents the stage where the application's latency requirement is no greater than the second duration, and the second duration is less than the first duration.

[0129] In other words, the functional permissions at this time may include some sound effect processing. That is, for applications with a second duration delay, or for applications that have a second duration delay corresponding to the second stage, the user experience can be improved by performing some sound effect processing.

[0130] Among them, the functional permissions for some audio effects processing refer to audio effects processing functions whose functional requirements for executing applications on intermediate audio data have a weight greater than a preset threshold.

[0131] For each application, the functional requirements weight of the second sound effect processing of the sound effect chip can be determined through the function-weight mapping table of each application. For example, Table 1 below is an example of a function-weight mapping table.

[0132] Table 1 Function-Weight Mapping Table

[0133] Karaoke app Bass sound effect requirements 9 Karaoke app Automatic gain audio requirements 7 Karaoke app Stereo sound requirements 5 Media asset playback application Noise suppression sound effect requirements 9 Media asset playback application Suppressing the need for feedback sound effects 8 Media asset playback application Bass sound effect requirements 7 Media asset playback application Bass sound effect requirements 5

[0134] For example, when the preset threshold is 8, the sound effect processing function corresponding to the function permission of the karaoke application is the bass sound effect function, and the sound effect processing function corresponding to the media playback application is the noise suppression sound effect function.

[0135] If the application's functional requirement weight is greater than the preset threshold for bass sound effect requirements, the sound effect processing function performs bass sound effect processing on the intermediate audio data to enhance the depth and richness of the bass.

[0136] If the functional requirement weight in the application is greater than the preset threshold, it is an echo cancellation sound effect requirement. The sound effect processing function performs echo cancellation sound effect processing on the intermediate audio data to improve the overall sound effect experience.

[0137] If the functional requirement weight in the application is greater than the preset threshold, the requirement is for noise suppression sound effects. The sound effect processing function performs stereo noise suppression processing on the intermediate audio data to improve the stereo sound experience.

[0138] The audio effects that the audio chip can process are not limited to those described in the above embodiments, but also include other audio effects (e.g., automatic enhancement, active noise reduction, feedback suppression, etc.), which are not limited here.

[0139] like Figure 7 As shown, it also includes: S1503, if the adjustment stage of the intermediate audio data is the third stage, determine the function permission to not perform sound effect processing function on the intermediate audio data.

[0140] The latency requirement for the third stage of the application is no greater than the third duration, which is less than the second duration.

[0141] In other words, the intermediate audio data of this application is sensitive to latency. By setting the function permissions to not perform sound effect processing on the intermediate audio data, the latency loss can be reduced and the playback time of the intermediate audio data can be increased.

[0142] S160 triggers the sound effect chip to perform corresponding second sound effect processing on the intermediate audio data according to the function permissions.

[0143] Based on the above process, by selectively restricting the audio processing of the audio chip, the overall latency of the audio data is ensured to meet the application standard; at the same time, while ensuring that the latency meets the standard, the quality of the audio data is improved as much as possible to ensure fast and good sound output and improve the user experience.

[0144] In a display device that uses a dual-chip (system-on-a-chip and audio chip) collaboration, the system-on-a-chip, after processing the initial audio data, transmits the determined acquisition frequency of the intermediate audio data to the audio chip. Based on the transmission acquisition frequency (i.e., time) and the preset audio delay conditions of each application, the intermediate audio data is processed in stages according to the adjustment phase, ensuring that both sound effects and timing are taken into account when the audio data is output.

[0145] Figure 8 The following is a flowchart illustrating the sound effect processing method for a karaoke application in a display device according to some embodiments of this application, such as... Figure 8 As shown, this sound effect processing method includes the following steps:

[0146] S310: In response to the user's audio processing command to open the karaoke application, receive the initial audio data of the application.

[0147] The initial audio data at this point is human voice data, and the initial audio data is stored in the audio buffer.

[0148] S320: Read initial audio data from the audio buffer, perform first sound effect processing on the initial audio data, and determine intermediate audio data.

[0149] Karaoke apps require vocal data and need to read initial audio data from the audio buffer. However, music playback apps and other applications that do not require vocal data do not need to read initial audio data from the audio buffer and can obtain the initial audio data directly from the application.

[0150] S330, Determine the acquisition frequency of intermediate audio data.

[0151] The sampling frequency is the frequency at which initial audio data is read from the audio buffer, and each time data is read, all data in the audio buffer is retrieved.

[0152] S340, intermediate audio data carrying its acquisition frequency can be transmitted through middleware and kernel, enabling intermediate audio data to be transmitted from the system-on-a-chip to the sound chip.

[0153] After the system-on-a-chip transmits the intermediate audio data to the audio effects chip, the S350 determines the adjustment stage of the intermediate audio data based on the sampling frequency and the preset audio delay conditions of the application corresponding to the initial audio data.

[0154] S360, Based on the adjustment phase, determine the function permission to perform second sound effect processing on intermediate audio data.

[0155] After receiving the audio data, the audio chip performs sound effect processing based on the sampling frequency carried by the intermediate audio data and the preset audio delay conditions of the application corresponding to the initial audio data. If the corresponding adjustment stage is the first stage, it means that there was no delay during the intermediate audio acquisition and the audio data quality is relatively good. After receiving the intermediate audio data, the audio chip can perform the second sound effect processing to achieve the best effect. If the corresponding adjustment stage is the second stage, it means that there is some time consumption during acquisition. When passing through the audio chip, in order to ensure that the overall delay meets the standard, some functions of the second sound effect processing are required to improve some audio effects. If the corresponding adjustment stage is the third stage, it means that there is already a large time consumption during acquisition. When passing through the audio chip, in order to ensure that the overall delay meets the standard, the audio chip does not perform the second sound effect processing.

[0156] Figure 9 This application illustrates a schematic diagram of the software configuration in a display device for a karaoke application, as shown in some embodiments of the present application. Figure 9 As shown, the display device transmits human voice data through an external microphone 410. After the human voice data is processed by the first sound effect of the system-level chip corresponding to the karaoke application, it is transmitted to the audio chip through the middleware and kernel. Based on the acquisition frequency of the human voice data and the preset sound effect delay conditions corresponding to the karaoke application, the function permissions for executing the human voice data are determined. The audio chip performs second sound effect processing on the human voice data based on the function permissions and plays it through devices such as power amplifiers and speakers.

[0157] This application provides a display device and a sound effect processing method. The display device includes a system-on-a-chip (SoC), a sound effect chip, and a controller. The SoC performs first sound effect processing on initial audio data to determine intermediate audio data. It further determines the acquisition frequency of the intermediate audio data. After the SoC transmits the intermediate audio data to the sound effect chip, it can determine the adjustment stage of the intermediate audio data based on the acquisition frequency and the preset audio delay conditions of the application corresponding to the initial audio data. Furthermore, based on the adjustment stage, it can determine the functional permission to perform second sound effect processing on the intermediate audio data and trigger the sound effect chip to perform corresponding second sound effect processing on the intermediate audio data according to the functional permission. This allows the sound effect chip to implement sound effect data processing according to the specific situation of the intermediate audio data, avoiding delays caused by sound effect processing and improving the user experience.

[0158] In some embodiments, the display device further includes a speaker, and the audio chip is communicatively connected to the speaker via a power amplifier. After the audio chip performs second audio processing on the intermediate audio data based on functional permissions, the controller determines the target audio data and outputs the target audio data through the speaker.

[0159] Figure 10 This application illustrates a flowchart of another sound effect processing method in a display device according to some embodiments, such as... Figure 10 As shown, after step 160, the following steps are also included:

[0160] S170, Output the target audio data through the speaker.

[0161] The target audio data is obtained by the sound effects chip performing second sound effects processing on the intermediate audio data based on functional permissions.

[0162] In some embodiments, the audio chip in the display device is communicatively connected to the speaker, and an amplifier module may be provided between the audio chip and the speaker to convert the digital signal output by the audio chip into an analog signal.

[0163] This application provides a display device and a sound effect processing method. The display device includes a system-on-a-chip (SoC), a sound effect chip, a controller, and a speaker. The SoC performs first sound effect processing on initial audio data to determine intermediate audio data. It further determines the acquisition frequency of the intermediate audio data. After the SoC transmits the intermediate audio data to the sound effect chip, it can determine the adjustment stage of the intermediate audio data based on the acquisition frequency and the preset audio delay conditions corresponding to the initial audio data. Furthermore, based on the adjustment stage, it can determine the function permission to perform second sound effect processing on the intermediate audio data and trigger the sound effect chip to perform corresponding second sound effect processing on the intermediate audio data according to the function permission. The target audio data obtained through the second sound effect processing is output through the speaker, so that the sound effect chip can implement sound effect data processing according to the specific situation of the intermediate audio data, avoiding delays caused by sound effect processing and improving the user experience.

[0164] Figure 11 This application illustrates a flowchart of another sound effect processing method in a display device according to some embodiments, such as... Figure 11 As shown, this sound effect processing method includes the following steps:

[0165] The following steps are performed via a system-on-a-chip:

[0166] S510 receives the initial audio data of the application in response to the sound processing command when the application starts.

[0167] S520: Perform first sound effect processing on the initial audio data to determine the intermediate audio data.

[0168] The following steps are performed via the audio chip:

[0169] S530, Receive intermediate audio data and determine the acquisition frequency of the intermediate audio data.

[0170] S540. Based on the acquisition frequency and the preset audio delay conditions of the application corresponding to the initial audio data, determine the adjustment stage of the intermediate audio data.

[0171] S550. Based on the adjustment phase, determine the function permission to perform the second sound effect processing on the intermediate audio data, and perform the corresponding second sound effect processing on the intermediate audio data according to the function permission.

[0172] The above steps and Figure 4 The implementation principles and technical effects of each step are similar, and will not be elaborated here.

[0173] This application provides a display device and a sound effect processing method. The display device includes a system-on-a-chip (SoC) and a sound effect chip. The SoC performs first sound effect processing on initial audio data to determine intermediate audio data. It further determines the acquisition frequency of the intermediate audio data. After the SoC transmits the intermediate audio data to the sound effect chip, it can determine the adjustment stage of the intermediate audio data based on the acquisition frequency and the preset audio delay conditions of the application corresponding to the initial audio data. Furthermore, based on the adjustment stage, it can determine the functional permission to perform second sound effect processing on the intermediate audio data, and perform corresponding second sound effect processing on the intermediate audio data according to the functional permission. This allows the sound effect chip to implement sound effect data processing according to the specific situation of the intermediate audio data, avoiding delays caused by sound effect processing and improving the user experience.

[0174] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0175] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

Claims

1. A display device, characterized in that, include: The system-on-a-chip is configured to perform first sound effect processing on the initial audio data to determine the intermediate audio data; The audio chip is configured to perform secondary audio processing on intermediate audio data; The controller, which is communicatively connected to the system-on-a-chip and the audio chip, is configured to: Determine the acquisition frequency of the intermediate audio data; After the system-on-a-chip transmits the intermediate audio data to the sound effects chip, the adjustment stage of the intermediate audio data is determined based on the sampling frequency and the preset audio delay conditions of the application corresponding to the initial audio data. Based on the adjustment phase, the function permission to perform the second sound effect processing on the intermediate audio data is determined, and the sound effect chip is triggered to perform the corresponding second sound effect processing on the intermediate audio data according to the function permission; The adjustment phase includes a first phase, a second phase, and a third phase. In the step of determining the function permission to perform the second sound effect processing on the intermediate audio data based on the adjustment phase, the controller is configured to: if the adjustment phase of the intermediate audio data is the second phase, determine that the function permission is to perform sound effect processing on the intermediate audio data where the function requirement weight of the application is greater than a preset threshold.

2. The display device according to claim 1, characterized in that, In the step of determining the adjustment phase of the intermediate audio data based on the acquisition frequency and the preset audio delay conditions of the application, the controller is further configured to: If the sampling period corresponding to the sampling frequency is less than the fourth duration in the preset audio delay condition, the adjustment stage is the first stage. If the sampling period corresponding to the sampling frequency is greater than or equal to the fourth duration, and the sampling period corresponding to the sampling frequency is less than the fifth duration in the preset audio delay condition, the adjustment stage is the second stage. If the acquisition period corresponding to the acquisition frequency is greater than or equal to the fifth duration, the adjustment phase is the third phase.

3. The display device according to claim 1, characterized in that, In the step of determining that the function permission is to perform sound effect processing functions with a weight greater than a preset threshold on the intermediate audio data if the adjustment stage of the intermediate audio data is the second stage, the controller is configured as follows: If the functional requirement weight in the application is greater than the preset threshold, which is the bass sound effect requirement, the sound effect processing function performs bass sound effect processing on the intermediate audio data. If the functional requirement weight in the application is greater than a preset threshold, and that is a noise suppression sound effect requirement, then the sound effect processing function performs noise suppression sound effect processing on the intermediate audio data.

4. The display device according to claim 1, characterized in that, The controller is also configured to: If the audio processing cache resources of the system-on-a-chip do not meet the audio processing requirements of the application, the initial audio data will be used as the intermediate audio data.

5. The display device according to claim 1, characterized in that, The determination of the acquisition frequency of the intermediate audio data includes one of the following: Obtain the acquisition frequency of the intermediate audio data; The acquisition frequency of the intermediate audio data is determined based on the difference between the acquisition frequency of the initial audio data and the application delay time; wherein the acquisition frequency of the intermediate audio data is greater than or equal to the acquisition frequency of the initial audio data.

6. A display device, characterized in that, include: The system-on-a-chip is configured to perform first sound effect processing on the initial audio data to determine the intermediate audio data; The audio chip is communicatively connected to the system-on-a-chip, and the audio chip is configured as follows: Receive the intermediate audio data and determine the acquisition frequency of the intermediate audio data; Based on the acquisition frequency and the preset audio delay conditions of the application corresponding to the initial audio data, the adjustment stage of the intermediate audio data is determined; Based on the adjustment phase, the function permission to perform second sound effect processing on the intermediate audio data is determined, and the corresponding second sound effect processing is performed on the intermediate audio data according to the function permission; The adjustment phase includes a first phase, a second phase, and a third phase. In the step of determining the function permission to perform the second sound effect processing on the intermediate audio data based on the adjustment phase, the sound effect chip is configured to: if the adjustment phase of the intermediate audio data is the second phase, determine that the function permission is to perform sound effect processing on the intermediate audio data where the function requirement weight of the application is greater than a preset threshold.

7. A sound effect processing method, characterized in that, include: The acquisition frequency of the intermediate audio data is determined, wherein the intermediate audio data is determined by the system-on-a-chip performing first sound effect processing on the initial audio data; After the system-on-a-chip transmits the intermediate audio data to the sound effects chip, the adjustment stage of the intermediate audio data is determined based on the sampling frequency and the preset audio delay conditions of the application corresponding to the initial audio data. Based on the adjustment phase, the function permission to perform the second sound effect processing on the intermediate audio data is determined, and the sound effect chip is triggered to perform the corresponding second sound effect processing on the intermediate audio data according to the function permission; The adjustment phase includes a first phase, a second phase, and a third phase. The step of determining the function permission to perform the second sound effect processing on the intermediate audio data based on the adjustment phase includes: if the adjustment phase of the intermediate audio data is the second phase, determining that the function permission is to perform sound effect processing functions on the intermediate audio data with a function requirement weight of the application greater than a preset threshold.

8. The sound effect processing method according to claim 7, wherein the step of determining the adjustment stage of the intermediate audio data based on the acquisition frequency and the preset audio delay conditions of the application includes: If the sampling period corresponding to the sampling frequency is less than the fourth duration in the preset audio delay condition, the adjustment stage is the first stage. If the sampling period corresponding to the sampling frequency is greater than or equal to the fourth duration, and the sampling period corresponding to the sampling frequency is less than the fifth duration in the preset audio delay condition, the adjustment stage is the second stage. If the acquisition period corresponding to the acquisition frequency is greater than or equal to the fifth duration, the adjustment phase is the third phase.

9. The sound effect processing method according to claim 7, characterized in that, In the step of determining that the function permission is to perform sound effect processing functions with a weight greater than a preset threshold on the intermediate audio data if the adjustment stage of the intermediate audio data is the second stage, the method includes: If the functional requirement weight in the application is greater than the preset threshold, which is the bass sound effect requirement, the sound effect processing function performs bass sound effect processing on the intermediate audio data. If the functional requirement weight in the application is greater than a preset threshold, and that is a noise suppression sound effect requirement, then the sound effect processing function performs noise suppression sound effect processing on the intermediate audio data.

Images