Audio processing method, sound box equipment and display equipment

Abstract

The embodiment of the invention provides an audio processing method, sound box equipment and display equipment, and the method comprises the steps: receiving original audio data sent by the display equipment in response to a received path switching trigger prompt of the display equipment; decoding and rendering the audio data to obtain first pulse code modulation data; playing the first pulse code modulation data; intercepting second pulse code modulation data from the first pulse code modulation data; and transmitting the second pulse code modulation data to the display equipment so as to synchronously play the second pulse code modulation data through a built-in loudspeaker of the display equipment when the sound box equipment plays the first pulse code modulation data. According to the method, decoding and rendering can be carried out through the sound box device instead of the television end, and the defect of the loudness of the television end is made up from the loudness. The sound box device transmits the data to the built-in loudspeaker of the television end through the data line hardware, the transmission time can be ignored, the sound transmission is basically synchronous, and the problems that the sound is not synchronous and the loudness is not matched when the audio data is played are solved.

Description

Technical Field

[0001] This application relates to the field of display device technology, and in particular to an audio processing method, a speaker device, and a display device. Background Technology

[0002] To provide users with a better audio-visual experience, combined sound solutions can be used in home or other usage scenarios. In some embodiments, by integrating the TV's built-in speakers with external audio devices (such as soundbars, subwoofers, etc.), a multi-channel surround sound field can be constructed to achieve a wider sound field and a more immersive audio experience. Taking a soundbar as an example, in the above application scenario, the TV is responsible for playing local or online streaming video signals, while the external soundbar can be connected to the TV via an audio return channel, fiber optic, or wireless means. The two work together to achieve synchronous output of video images and multi-channel audio.

[0003] In existing integrated sound systems, the TV SoC (System-on-a-Chip) is typically the core. All audio data must first be decoded by the TV's internal decoder (e.g., Dolby Digital, DTS-X), then processed by the TV's built-in renderer for virtual surround sound, loudness equalization, and other functions, finally outputting Pulse-Code Modulation Data (PCMData), which drives the TV speakers. Simultaneously, the TV transmits the audio data to a soundbar, where it decodes, renders, and drives its own speakers for playback. In this chain, key processing steps such as decoding, rendering, resampling, and gain control are all concentrated on the TV end.

[0004] However, because televisions are comprehensive display terminals, their focus is primarily on image processing and display modules, resulting in audio links often only equipped with basic decoders and simplified rendering algorithms, making it difficult to support advanced sound effects such as DTS-X and Dolby Atmos. Simultaneously, the limited DSP computing power of the television introduces additional latency, and the direct playback of uncorrected audio signals by the soundbar further exacerbates the audio desynchronization between the television speakers and the soundbar. Furthermore, the lack of unified calibration between the television and the soundbar in terms of maximum loudness, dynamic range, and gain strategies leads to inconsistent loudness perception of the same audio track on the two sets of speakers. Therefore, current combined sound solutions suffer from audio data playback issues of audio desynchronization and loudness mismatch. Summary of the Invention

[0005] This application provides an audio processing method, a speaker device, and a display device to solve the problems of audio data playback asynchrony and loudness mismatch in current combined sound generation solutions.

[0006] In a first aspect, some embodiments of this application provide an audio processing method applied to a speaker device, including...

[0007] In response to receiving a path switching trigger prompt from the display device, the device receives raw audio data sent by the display device, wherein the raw audio data is audio data that has not been decoded and rendered by the display device;

[0008] The audio data is decoded and rendered to obtain the first pulse code modulation data;

[0009] Play the first pulse-code modulation data;

[0010] Extract the second pulse code modulation data from the first pulse code modulation data;

[0011] The second pulse code modulation data is transmitted to the display device via a data line so that when the speaker device plays the first pulse code modulation data, the second pulse code modulation data is played synchronously through the built-in speaker of the display device.

[0012] The above technical solution has the following advantages or beneficial effects: The method can replace the TV end in performing decoding and rendering with a speaker device, reducing processing links and compensating for the lack of loudness on the TV end, achieving loudness consistency. Simultaneously, the speaker device transmits data to the TV's built-in speaker via a data cable, and the transmission time is negligible, so the sound transmission between the two is basically synchronized, thus solving the problems of audio data asynchrony and loudness mismatch in current combined sound generation solutions.

[0013] In some embodiments, the second pulse code modulation data is transmitted to the display device via a data line so that when the speaker device plays the first pulse code modulation data, the second pulse code modulation data is synchronously played through the built-in speaker of the display device, including:

[0014] The second pulse code modulation data is transmitted to the digital-to-analog converter module in the display device via the data line, so that after receiving the second pulse code modulation data, the digital-to-analog converter module converts the format of the second pulse code modulation data into a format that matches the built-in speaker, and transmits the converted second pulse code modulation data to the built-in speaker, so that when the speaker device plays the first pulse code modulation data, the second pulse code modulation data is played synchronously through the built-in speaker.

[0015] The above technical solution has the following advantages or beneficial effects: By uniformly controlling loudness parameters (such as volume gain, dynamic range compression, etc.) through the SoundBar and synchronously applying them to the built-in speakers, it ensures that the audio output from both is consistent in loudness, avoiding volume imbalance caused by device differences and improving the sound field uniformity when using multi-channel combined sound. Simultaneously, the data received by the built-in speakers comes directly from the SoundBar's conversion module, rather than going through the decoding and rendering process on the TV. The TV only acts as a data pass-through channel and does not participate in audio signal processing, thus eliminating the additional latency caused by TV processing. The audio data transmission path from the SoundBar to the built-in speakers is shorter, with fewer processing steps, significantly reducing signal transmission latency and improving the synchronization between audio and video.

[0016] In some embodiments, transmitting the second pulse code modulation data to the digital-to-analog converter module in the display device via the data line includes:

[0017] The second pulse code modulation data is encapsulated into a transmission frame according to the protocol format of the integrated circuit built-in audio bus in the speaker device;

[0018] The transmission frame is transmitted to the digital-to-analog converter module of the display device via a data line connected to the terminal, so that the digital-to-analog converter module performs format conversion on the pulse code modulation data and sends it to the built-in speaker, so that the first pulse code modulation data can be played through the built-in speaker.

[0019] The above technical solution has the following advantages or beneficial effects: by encapsulating PCM data into transmission frames that conform to the integrated circuit built-in audio bus protocol format, the stability and integrity of the data during transmission can be ensured, and the compatibility of audio data transmission between devices can be improved.

[0020] In some embodiments, after transmitting the transmission frame to the digital-to-analog converter module of the display device via a data line connected to the terminal, the method further includes:

[0021] The performance parameters of the digital-to-analog converter module are detected; the performance parameters include the sampling rate support range, quantization accuracy parameters, and clock jitter characteristics of the pulse code modulation data.

[0022] The transmission rate of the pulse code modulation data transmitted to the digital-to-analog conversion module is adjusted according to the performance parameters.

[0023] The above technical solution has the following advantages or beneficial effects: based on these detected performance parameters, the speaker device can dynamically adjust the rate at which it transmits PCM data to the digital-to-analog converter module, ensuring that the data transmission matches the processing capability of the digital-to-analog converter module.

[0024] In some embodiments, decoding and rendering are performed on the audio data to obtain first pulse code modulation data, including:

[0025] Obtain the encoding format of the audio data;

[0026] The decoding algorithm corresponding to the encoding format is invoked to decode the audio data to obtain a multi-channel audio stream;

[0027] The multi-channel audio stream is mixed to generate a rendered audio stream;

[0028] The rendered audio stream is encoded into pulse code modulation data.

[0029] The above technical solution has the following advantages or beneficial effects: The speaker device can acquire and identify the encoding formats of audio data from different sources, and invoke the corresponding decoding algorithm for decoding, making it compatible with multiple audio encoding formats and improving the device's versatility. By mixing multi-channel audio streams, the speaker device can finely adjust the audio signal according to its own hardware characteristics and the user's personalized settings. For example, through volume balancing processing, it ensures that the sound proportions of each channel are coordinated, avoiding any channel being too loud or too soft and affecting the overall listening effect, thereby optimizing the audio effect and improving the user's listening experience.

[0030] In some embodiments, before playing the first pulse-code modulation data, the method further includes:

[0031] Gain adjustment is performed on the first pulse code modulation data so that the amplitude of the first pulse code modulation data matches the rated power of the built-in speaker of the display device;

[0032] And / or, perform delay compensation on the first pulse code modulation data to eliminate the audio output time difference between the speaker device and the display device.

[0033] The above technical solution has the following advantages or beneficial effects: By matching the rated power of the display device's speakers, speaker damage (such as overload burnout) or sound distortion (such as clipping) caused by signal amplitude mismatch can be avoided. Simultaneously, it ensures that speakers of different power ratings can output clear, distortion-free audio, improving the reliability of the combined sound system. Furthermore, it eliminates the audio output time difference between the speaker equipment and the display device, ensuring accurate sound image positioning for multi-channel audio (such as left and right channels, center channel), and avoiding sound field chaos or sound lagging behind the picture caused by inconsistent time delays.

[0034] In some embodiments, extracting second pulse code modulation data from the first pulse code modulation data includes:

[0035] Based on a preset ratio, second pulse code modulation data corresponding to each channel of the built-in speaker of the display device is extracted from the first pulse code modulation data; the preset ratio is the proportion of the second pulse code modulation data corresponding to each channel of the built-in speaker in the first pulse code modulation data.

[0036] The above technical solution has the following advantages or beneficial effects: By extracting data at a preset ratio, it can be ensured that the audio data transmitted to the built-in speaker of the display device matches the audio played by the speaker device in terms of loudness, avoiding a decline in the listening experience caused by inconsistencies in loudness between devices. Simultaneously, since the audio data is extracted from the same set of first pulse code modulation data and transmitted to the display device via a hardware data line (with virtually no delay), the synchronization of the built-in speaker of the speaker device and the display device when playing audio can be ensured.

[0037] In some embodiments, extracting a preset proportion of the pulse code modulation data includes:

[0038] A configuration query request is sent to the display device via the terminal;

[0039] Receive the configuration response packet returned by the display device according to the configuration query request;

[0040] Parse the configuration response packet to obtain the channel configuration parameters of the display device; the channel configuration parameters include the number of channels supported by the built-in speaker, the frequency response range of each channel, the coordinate information of the built-in speaker, and the interface protocol type;

[0041] Determine the preset ratio based on the aforementioned channel configuration parameters;

[0042] According to the preset ratio, a preset ratio of the second pulse code modulation data is extracted from the first pulse code modulation data.

[0043] The above technical solution has the following advantages or beneficial effects: By acquiring the channel configuration parameters of the display device 200, the speaker device can detect the channel support capability of the TV end, flexibly adjust the data allocation strategy, ensure that the intercepted PCM data is completely matched with the hardware capabilities, and guarantee that the combined sound effect matches the TV end hardware. For example, it avoids transmitting center channel data to display devices without a center speaker, preventing sound loss or noise interference. At the same time, the TV end does not need to participate in complex decoding, but only needs to receive channel data adapted to its own characteristics, which can reduce processing load and reduce latency.

[0044] Secondly, some embodiments of this application provide a speaker device, including:

[0045] In response to receiving a path switching trigger prompt from the display device, the device receives raw audio data sent by the display device, wherein the raw audio data is audio data that has not been decoded and rendered by the display device;

[0046] The audio data is decoded and rendered to obtain the first pulse code modulation data;

[0047] Play the first pulse-code modulation data;

[0048] Extract the second pulse code modulation data from the first pulse code modulation data;

[0049] The second pulse code modulation data is transmitted to the display device via a data line so that when the speaker device plays the first pulse code modulation data, the second pulse code modulation data is played synchronously through the built-in speaker of the display device.

[0050] The above technical solution has the following advantages or beneficial effects: The speaker device can replace the TV end in performing decoding and rendering, reducing processing links and compensating for the lack of loudness on the TV end, achieving loudness consistency. Simultaneously, the speaker device transmits data to the TV end's built-in speaker via a data cable; the transmission time is negligible, so the sound transmission between the two is essentially synchronized, thus solving the problems of audio asynchrony and loudness mismatch during audio data playback in current combined sound generation solutions.

[0051] Thirdly, some embodiments of this application provide a display device, including:

[0052] The monitor is configured to display the user interface;

[0053] The controller is configured as follows:

[0054] Monitor the level of input and output pins to determine the insertion status of terminals in speaker and display devices;

[0055] When the voltage level changes, it is determined that the speaker device has been inserted into the terminal, and a path switching trigger prompt is generated;

[0056] In response to the path switching trigger prompt, the data path between the main chip and the integrated circuit's built-in audio bus is disconnected;

[0057] Connect the data path between the digital-to-analog converter module and the built-in audio bus of the integrated circuit;

[0058] The audio data is sent to the speaker device.

[0059] The above technical solution has the following advantages or beneficial effects: the display device, through path switching, data pass-through, and status feedback, collaborates with the speaker device to transfer audio processing tasks. When solving the loudness matching problem, the display device provides channel configuration information and acts as a pass-through channel, ensuring that the speaker device uniformly controls the loudness. When solving the synchronization problem, the display device bypasses its own processing steps, shortens the transmission path, and, in conjunction with a clock synchronization mechanism, achieves a high degree of synchronization between sound and image, effectively overcoming the loudness inconsistency and sound asynchrony defects in traditional combined sound generation solutions.

[0060] As can be seen from the above technical solutions, some embodiments of this application provide an audio processing method, a speaker device, and a display device. The method includes: responding to a channel switching trigger prompt received from the display device, receiving raw audio data sent by the display device, wherein the raw audio data is audio data that has not been decoded and rendered by the display device; performing decoding and rendering on the audio data to obtain first pulse code modulation data; playing the first pulse code modulation data; extracting second pulse code modulation data from the first pulse code modulation data; and transmitting the extracted pulse code modulation data to the display device through a data cable connected to a terminal of the display device, so that when the speaker device plays the first pulse code modulation data, the second pulse code modulation data is simultaneously played through the built-in speaker of the display device, realizing combined sound output from the built-in speaker and the speaker device. This method can replace the TV end in performing decoding and rendering with the speaker device, reducing processing links and compensating for the lack of loudness on the TV end, achieving loudness consistency. Meanwhile, the speaker device transmits data to the built-in speaker on the TV via a data cable. The transmission time is negligible, so the sound transmission between the two is basically synchronized. This solves the problem of audio asynchrony and loudness mismatch in current combined sound solutions. Attached Figure Description

[0061] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0062] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application;

[0063] Figure 2 This is a schematic diagram of the hardware configuration of a display device provided in some embodiments of this application;

[0064] Figure 3 This is a schematic diagram of the software configuration of a display device provided in some embodiments of this application;

[0065] Figure 4 The following are schematic diagrams illustrating existing combined sound generation solutions in some embodiments of this application;

[0066] Figure 5 This is a flowchart illustrating the audio processing method performed by a speaker according to some embodiments of this application;

[0067] Figure 6 A timing diagram illustrating the path switching process performed on the display device side, provided for some embodiments of this application;

[0068] Figure 7 A timing diagram illustrating the audio processing method performed by a speaker device according to some embodiments of this application;

[0069] Figure 8 A schematic diagram illustrating the architecture of a combined sound generation scheme provided in some embodiments of this application;

[0070] Figure 9 This is a schematic diagram illustrating the process of decoding and rendering audio data by a speaker device provided in some embodiments of this application;

[0071] Figure 10 This is a schematic diagram illustrating the process of extracting second pulse code modulation data from first pulse code modulation data in a speaker device provided in some embodiments of this application. Detailed Implementation

[0072] The embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described below do not represent all embodiments consistent with this application. They are merely examples of systems and methods consistent with some aspects of this application as detailed in the claims.

[0073] It should be noted that the brief descriptions of terms in this application are only for the convenience of understanding the embodiments described below, and are not intended to limit the embodiments of this application. Unless otherwise stated, these terms should be understood in their ordinary and common meaning.

[0074] The terms "first," "second," "third," etc., used in the specification, claims, and accompanying drawings of this application are used to distinguish similar or related objects or entities, and do not necessarily imply a specific order or sequence, unless otherwise specified. It should be understood that such terms are interchangeable where appropriate.

[0075] The terms “comprising” and “having”, and any variations thereof, are intended to cover but not exclude inclusion, for example, a product or device that includes a range of components is not necessarily limited to all of the components that are clearly listed, but may include other components that are not clearly listed or that are inherent to such product or device.

[0076] The term "module" refers to any known or subsequently developed hardware, software, firmware, artificial intelligence, fuzzy logic, or combination of hardware and / or software code that is capable of performing the functions associated with that element.

[0077] In this embodiment, the display device 200 generally refers to a device with screen display and data processing capabilities. For example, the display device 200 includes, but is not limited to, smart TVs, mobile terminals, computers, monitors, advertising screens, wearable devices, virtual reality devices, augmented reality devices, etc.

[0078] Figure 1 This is a schematic diagram illustrating an operational scenario between a display device and a control device provided in some embodiments of this application. For example... Figure 1 As shown, a user can operate the display device 200 via touch operation, a mobile terminal 300, and a control device 100. The control device 100 receives user input commands and converts them into control commands that the display device 200 can recognize and respond to. For example, the control device 100 can be a remote control, a stylus, a gamepad, etc.

[0079] The mobile terminal 300 can function as a control device for human-computer interaction between the user and the display device 200. It can also function as a communication device for establishing a communication connection with the display device 200 and exchanging data. In some embodiments, the mobile terminal 300 can have software applications installed on it and communicate with the display device 200 via network communication protocols to achieve one-to-one control and data communication. Furthermore, it can transmit audio and video content displayed on the mobile terminal 300 to the display device 200 for synchronized display.

[0080] In some embodiments, the mobile terminal 300 or other electronic devices may also simulate the functions of the control device 100 by running an application that controls the display device 200.

[0081] like Figure 1 The diagram also shows that the display device 200 communicates with the server 400 via various communication methods. This allows the display device 200 to communicate via a local area network (LAN), a wireless local area network (WLAN), and other networks.

[0082] Display device 200 can provide broadcast television reception function, and can also be equipped with intelligent network television function that provides computer support, including but not limited to network television, smart television, Internet Protocol television (IPTV), etc.

[0083] Figure 2 Provided for some embodiments of this application Figure 1 Hardware configuration block diagram of display device 200.

[0084] In some embodiments, the display device 200 may include at least one of a tuner 210, a communication device 220, a detector 230, a device interface 240, a controller 250, a display 260, an audio output device 270, a memory, a power supply, and a user input interface.

[0085] In some embodiments, detector 230 is used to acquire signals from the external environment or to interact with the outside world. For example, detector 230 includes a light receiver, a sensor for acquiring ambient light intensity; or, detector 230 includes an image acquisition device, such as a camera, which can be used to acquire external environmental scenes, user attributes, or user interaction gestures; or, detector 230 includes a sound acquisition device, such as a microphone, for receiving external sounds.

[0086] In some embodiments, the display 260 includes display function components for presenting images and driving components for driving image display. The display 260 is used to receive and display image signals output from the controller 250. For example, the display 260 can be used to display video content, image content, menu control interface components, and user control UI interfaces, etc.

[0087] In some embodiments, the communication device 220 is a component used to communicate with external devices or the server 400 according to various communication protocol types. The display device 200 may have multiple communication devices 220 depending on the supported communication methods. For example, when the display device 200 supports wireless network communication, it may have a communication device 220 with WiFi functionality. When the display device 200 supports Bluetooth connectivity, it needs to have a communication device 220 with Bluetooth functionality.

[0088] The communication device 220 enables the display device 200 to communicate with external devices or the server 400 via wireless or wired connections. Wired connections utilize data cables, interfaces, or other components to connect the display device 200 to external devices. Wireless connections utilize wireless signals or wireless networks. The display device 200 can directly establish a connection with external devices or indirectly through gateways, routers, or other connection devices.

[0089] In some embodiments, the controller 250 may include at least one of a central processing unit, a video processor, an audio processor, a graphics processor, and a power processor, and a first to an nth interface for input / output. The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in memory. The controller 250 controls the overall operation of the display device 200.

[0090] In some embodiments, the controller 250 and the tuner 210 may be located in different separate devices, that is, the tuner 210 may also be located in an external device of the main device where the controller 250 is located, such as an external set-top box.

[0091] In some embodiments, a user can input user commands through a graphical user interface (GUI) displayed on a display 260, and the user input interface receives user input commands through the graphical user interface (GUI).

[0092] In some embodiments, the audio output device 270 can be a built-in speaker of the display device 200 or an external audio output device connected to the display device 200. For the external audio output device connected to the display device 200, the display device 200 may also be provided with an external audio output terminal, through which the audio output device can be connected to the display device 200 to output sound from the display device 200.

[0093] In some embodiments, the user input interface 280 can be used to receive instructions from user input.

[0094] To enable user interaction, in some embodiments, the display device 200 may run an operating system. The operating system is a computer program used to manage and control the hardware and software resources of the display device 200. The operating system can control the display device to provide a user interface; for example, the operating system can directly control the display device to provide a user interface, or it can provide a user interface by running an application. The operating system also allows users to interact with the display device 200.

[0095] It should be noted that the operating system can be a native operating system based on a specific operating platform, a third-party operating system that is deeply customized based on a specific operating platform, or an independent operating system specifically developed for display devices.

[0096] An operating system can be divided into different modules or levels based on the functions it implements, for example... Figure 3As shown, in some embodiments, the system is divided into four layers, from top to bottom: the Applications layer (referred to as the "application layer"), the middleware layer, and the hardware layer.

[0097] In some embodiments, the application layer provides services and interfaces for applications, enabling the display device 200 to run applications and interact with the user based on the applications. The application layer may contain at least one application, which may be a built-in Windows program, system settings program, or clock program of the operating system; or it may be an application developed by a third-party developer. In specific implementations, the application packages in the application layer are not limited to the examples above.

[0098] The framework layer provides application programming interfaces (APIs) and a programming framework for applications. The application framework layer includes predefined functions. It acts as a central processing unit, determining the actions taken by applications within the application layer. Through the API, applications can access system resources and obtain system services during execution.

[0099] like Figure 3 As shown, Figure 3 Provided for some embodiments of this application Figure 1 The diagram shows the software configuration of the display device. In some embodiments, the system of the display device 200 can be divided into three layers, from top to bottom: the application layer, the middleware layer, and the hardware layer.

[0100] The application layer mainly includes commonly used applications on TVs, as well as the application framework. The commonly used applications are mainly browser-based applications, such as HTML5 apps, and native apps.

[0101] An application framework is a complete program model that has all the basic functions required by standard application software, such as file access, data exchange, and the user interface for these functions (toolbar, status bar, menu, dialog box).

[0102] Native apps can support online or offline access, push notifications, or access to local resources.

[0103] The middleware layer includes various television protocols, multimedia protocols, and system components. Middleware can use the basic services (functions) provided by system software to connect different parts of application systems or different applications on the network, achieving resource sharing and function sharing.

[0104] The hardware layer mainly includes the Hardware Abstraction Layer (HAL) interface, hardware, and drivers. The HAL interface serves as a unified interface for all TV chips, with the specific logic implemented by each individual chip. Drivers primarily include: audio drivers, display drivers, Bluetooth drivers, camera drivers, Wi-Fi drivers, USB drivers, HDMI drivers, sensor drivers (such as fingerprint sensors, temperature sensors, and pressure sensors), and power drivers.

[0105] It should be noted that the above examples are merely a simple division of operating system functions and do not limit the specific form of the operating system of the display device 200 in this application embodiment. Depending on the function of the display device, the type of operating system, and other factors, the number of levels and the specific level type of the operating system may be expressed in other forms.

[0106] Display device 200 is an intelligent device capable of presenting a user interface and supporting user interaction. Taking a smart TV as an example, a smart TV is a television product based on Internet application technology, possessing an open operating system and chip, an open application platform, and enabling two-way human-computer interaction. It integrates multiple functions such as audio-visual, entertainment, and data to meet diverse and personalized user needs. The display device can present different media asset recommendation interfaces, which may include multiple media asset options for users to select and play.

[0107] To provide users with a better audio-visual experience, combined sound solutions can be used in home or other usage scenarios. In some embodiments, by integrating the TV's built-in speakers with external audio devices (such as soundbars, subwoofers, etc.), a multi-channel surround sound field can be constructed to achieve a wider sound field and a more immersive audio experience. Taking a soundbar as an example, in the above application scenario, the TV is responsible for playing local or online streaming video signals, while the external soundbar can be connected to the TV via an audio return channel, fiber optic, or wireless means. The two work together to achieve synchronous output of video images and multi-channel audio.

[0108] Figure 4 The following are schematic diagrams illustrating existing combined sound generation solutions in some embodiments of this application, such as... Figure 4As shown, in some embodiments, external audio sources (such as HDMI, digital broadcasting, etc.) are input into the TV's decoder in the form of audio data. The system-on-a-chip (SoC) includes a decoder, a renderer, and an output module. The audio data can be directly output to the decoder. The decoder decompresses the compressed audio data into lossless pulse code modulation (PCM) data, i.e., the raw digital audio stream. Then, the renderer performs post-processing on the PCM data (such as sample rate conversion, channel mixing, volume equalization, etc.) to generate the final playable PCM signal. The processed audio data can then be output through the TV's built-in speakers. Simultaneously, the TV can transmit unprocessed audio data to a soundbar via the output module, where the soundbar decodes, renders, and drives its own speakers for playback. This enables synchronized playback across multiple devices.

[0109] In other words, in existing combined sound generation solutions, the TV SoC is typically the core. All audio data must first be decoded by the TV's internal decoder (e.g., Dolby Digital, DTS-X), then processed by the TV's built-in renderer for virtual surround sound, loudness equalization, etc., before finally outputting a PCM stream to drive the TV speakers. Simultaneously, the TV transmits the audio data to a soundbar, where it is decoded, rendered, and used to drive its own speakers for playback. In this chain, key processing steps such as decoding, rendering, resampling, and gain control are all concentrated on the TV end.

[0110] However, because televisions are comprehensive display terminals, their focus is primarily on image processing and display modules, resulting in audio links often only equipped with basic decoders and simplified rendering algorithms, making it difficult to support advanced sound effects such as DTS-X and Dolby Atmos. Simultaneously, the limited DSP computing power of the television introduces additional latency, and the soundbar directly plays back uncorrected audio signals, further causing audio desynchronization between the television speakers and the soundbar. Furthermore, the lack of unified calibration between the television and the soundbar in terms of maximum loudness, dynamic range, and gain strategies leads to inconsistent loudness perception of the same audio track on the two sets of speakers. Therefore, current combined sound solutions suffer from audio data playback issues of audio desynchronization and loudness mismatch.

[0111] Based on the above scenario, some embodiments of this application provide an audio processing method that changes the TV's audio data processing mode to a SoundBar-based audio data processing mode. The SoundBar has powerful decoding and rendering capabilities. After entering the SoundBar-based sound output mode, the data path between the SoC and the speaker is disconnected. Figure 4 After rendering, the path from the PCM data output to the speaker is broken (this is achieved by the physical interface connecting the TV and external devices; inserting the terminal triggers some hardware movement, which will be explained in detail later). With the data path broken, the speaker no longer receives audio data from the TV's main chip; instead, the soundbar directly receives the audio data. This process requires no TV involvement in decoding or rendering. After decoding and rendering, the soundbar plays some channel data directly. In a combined sound system, the soundbar needs to transmit the decoded channel data to the TV's speaker. Since the soundbar outputs digital signals, while the speaker hardware may only support analog input, direct transmission can lead to data incompatibility. Therefore, depending on the speaker's hardware characteristics (such as supported signal types and interface protocols), a digital-to-analog converter (e.g., on the TV) needs to be added to the data path to convert the soundbar's digital signal into an analog signal that the speaker can recognize. The SoundBar extracts a portion of the audio data and transmits it to the TV via a data cable, thus compensating for the TV's inherent loudness limitations. Furthermore, since the audio data no longer passes through the TV's processing loop but is processed by the SoundBar, which then transmits it to the speaker via hardware via the data cable, the transmission time is negligible, resulting in near-synchronous audio transmission between the two. By using the SoundBar to decode and render the audio data, it eliminates reliance on the TV, enabling combined sound output from both the SoundBar and the TV's speakers. This addresses the issues of audio asynchrony and loudness mismatch that exist in current combined sound systems.

[0112] To facilitate understanding of the technical solutions in some embodiments of this application, the steps are described in detail below with reference to some specific embodiments and accompanying drawings. Figure 5 This is a flowchart illustrating the audio processing method performed by a speaker according to some embodiments of this application, such as... Figure 5 As shown, in some embodiments, the audio processing method may include the following steps:

[0113] Step S1: In response to receiving a path switching trigger prompt from the display device, receive the raw audio data sent by the display device, wherein the raw audio data is audio data that has not been decoded and rendered by the display device.

[0114] In existing combined sound generation methods, decoding and rendering are performed by the television terminal. Figure 4 The audio data processing path is shown. In this embodiment, decoding and rendering are performed by a speaker device such as a soundbar. Upon receiving a path switching trigger prompt from the display device, the speaker device can receive audio data sent by the display device 200 that has not been decoded and rendered by the television. It should be noted that this path switching trigger prompt signifies a change from the original audio data processing path on the television to the audio data processing path on the speaker device. The above process will be described in detail below with reference to the accompanying drawings and actual operation flow.

[0115] In some embodiments, the display device 200 can monitor the level of the input / output pins to determine the insertion status of the terminals in the speaker device and the display device 200; if the level changes, it is determined that the speaker device has been inserted into the terminal, and a path switching trigger prompt is generated; in response to the path switching trigger prompt, the TV disconnects the data path between the main chip and the integrated circuit built-in audio bus; connects the data path between the digital-to-analog converter module and the integrated circuit built-in audio bus; and sends audio data to the speaker device.

[0116] For example, Figure 6 This is a timing diagram illustrating the path switching process performed on the display device side in some embodiments of this application, such as... Figure 6 As shown, in some embodiments, the display device 200 may include a TV terminal, a detection input / output pin DetectGPIO, an integrated circuit built-in audio bus IIS GPIO, a main chip SoC, and a digital-to-analog converter module, such as an analog-to-digital converter (ADC) or a digital-to-analog converter (DAC). That is, the DAC module can be an ADC / DAC, hereinafter referred to as ADC / DAC. The TV terminal refers to the physical interface connecting the SoundBar to the television, which can be an HDMI (ARC / EARC) or optical fiber (SPDIF) interface, used for transmitting audio data. The input / output pin (General-Purpose Input / Output, GPIO) is a general-purpose pin on the system's main chip. DetectGPIO is used to detect the insertion status of the TV terminal by changes in its level. IIS GPIO can be used to control the switching of processing paths.

[0117] For example, a System-on-a-Chip (SoC) can monitor the level (high / low) of the Detect GPIO (General Purpose Input / Output pin) in real time through software to determine whether a TV terminal is plugged in. The digital audio interface clock of the built-in speaker on the TV is directly synchronized with the main chip's clock source to ensure the timing stability of data transmission. Thus, changes in the level can indicate whether the speaker is plugged in. Simultaneously, a unified clock source eliminates clock discrepancies between multiple devices, preventing data misalignment or frame drops caused by clock asynchrony. When the TV is plugged in via Audio Return Channel (ARC), Enhanced Audio Return Channel (EARC), or Sony / Philips Digital Interface Format (SPDIF) interface, the hardware circuitry can automatically pull the Detect GPIO level from low to high, such as from 0V to 3.3V. Upon detecting the increased GPIO level, the SoC software immediately controls the IIS GPIO pin to output a high level. The IIS GPIO acts as a control signal, instructing the hardware to perform a data path switching operation. Pulling the IIS GPIO high disconnects the SoC from the IIS path and connects the ADC / DAC to the IIS path. At this time, the audio data source switches from the TV (SoC decoding) to the SoundBar (direct decoding), avoiding latency and loudness issues caused by insufficient TV decoding capabilities. When the TV terminal is unplugged, the Detect GPIO is pulled low by hardware. That is, when the TV terminal is disconnected, the hardware circuit automatically pulls the Detect GPIO level from high to low, providing an initial low-level state for the next insertion detection, ensuring a closed-loop logic. When the Detect GPIO is detected to be low, the TV pulls the IIS GPIO low. If the SoC software detects the GPIO level drop, it controls the IIS GPIO pin to output a low level, triggering a reverse switch and instructing the hardware to restore the connection between the SoC and the IIS interface, disconnecting the ADC / DAC path, and reverting to the TV's processing path mode. The above describes the control logic for data path switching. The above example uses a high level for insertion; in actual use cases, a low level state may also occur, and this application does not specifically limit this.

[0118] Figure 7 This is a timing diagram illustrating the audio processing method performed by a speaker device according to some embodiments of this application. Figure 8 This is a schematic diagram of the architecture of the combined sound generation scheme provided in some embodiments of this application, combined with Figure 7 and Figure 8In this embodiment, the combined sound generation method is played through two paths. One path involves the audio data being directly sent from the output module to a speaker device, such as a soundbar, where decoding and rendering are performed. After decoding and rendering, the soundbar directly plays the decoded and rendered audio data. The other path involves the soundbar generating pulse code modulation (PCM) data through decoding and rendering. A portion of this PCM data is then extracted and transmitted via a data cable to a digital-to-analog converter (ADC / DAC) module. The PCM data is then converted by the ADC / DAC and transmitted to the TV's built-in speaker. These two paths together form the combined sound generation method of this application.

[0119] It should be noted that, in Figure 8 In this system, the data received by the speaker on the TV is obtained by the output of the SoundBar and processed by the digital-to-analog converter module, rather than by the audio data transmitted after being processed by the decoder and renderer in the SoC. Therefore, the audio data output by the speaker does not undergo the decoding and rendering process on the TV, and there will be no sound desynchronization problem as in the existing method.

[0120] Continue to combine Figure 7In some embodiments, the TV can connect to the SoundBar via ARC, EARC, or SPDIF, providing a physical channel for audio data transmission between the TV and the SoundBar. The SoundBar can connect to the TV's digital-to-analog converter module via a data cable (such as an IIS interface cable), allowing direct interaction between the SoundBar and the TV's audio hardware, bypassing the TV's SoC decoding process. The TV can directly transmit undecoded raw audio data to the SoundBar, skipping the TV's own decoding and rendering steps. The audio data can be in Bypass format or other formats. Taking Bypass format as an example, the TV only acts as a data relay station and does not modify the audio content to ensure lossless signal transmission. The decoding task is handled by the SoundBar, utilizing its dedicated audio processing chip to enhance the effect. The SoundBar decodes and renders the received audio data, then plays multi-channel audio through its built-in speakers. A portion of the audio data can also be extracted and transmitted to the digital-to-analog converter module. For example, the SoundBar extracts the center channel data from the audio and transmits it back to the TV's ADC / DAC module via a data cable (such as an IIS cable). Next, the ADC / DAC module on the TV converts the digital center channel signal returned by the SoundBar into an analog signal, adapting it to the input requirements of the TV's built-in speakers. If the TV speaker requires a specific format (such as adjusting the sampling rate or bit depth), the digital-to-analog converter module can further process the signal. In this way, professional decoding by the SoundBar replaces decoding on the TV, supporting advanced audio formats (such as Dolby Atmos). The TV skips the decoding stage and directly transmits the raw data, reducing processing steps. The SoundBar can compensate for the TV's inherent loudness deficiency. The SoundBar transmits data to the speaker via hardware via a data cable, and the transmission time is negligible, so the sound transmission between the two is essentially synchronized, thus solving the problems of audio asynchrony and loudness mismatch in current combined sound systems. After step S1 is completed, step S2 can be executed.

[0121] Step S2: The speaker device decodes and renders the audio data to obtain the first pulse code modulation data.

[0122] Figure 9 This is a schematic diagram illustrating the process of decoding and rendering audio data by a speaker device provided in some embodiments of this application, such as... Figure 9As shown, in some embodiments, the speaker device can perform decoding and rendering on audio data to obtain first pulse code modulation data in the following manner. First, the speaker device can obtain the encoding format of the audio data, then call the decoding algorithm corresponding to the encoding format to decode the audio data to obtain a multi-channel audio stream, then perform rendering processing on the multi-channel audio stream to generate a rendered audio stream, and finally encode the rendered audio stream into pulse code modulation data.

[0123] For example, a speaker device can parse the header information or other relevant metadata of the audio data to extract the encoding format used by the audio data, such as MP3, AAC, WAV, FLAC, etc. The speaker device can internally store an algorithm library containing various decoding algorithms. After obtaining the encoding format of the audio data, the speaker device's control system can retrieve the matching decoding algorithm from the algorithm library according to the encoding format. For example, if the audio data is encoded in MP3, the MP3 decoding algorithm is retrieved; if it is encoded in Dolby Digital, the corresponding Dolby Digital decoding algorithm is retrieved. The retrieved decoding algorithm is used to reverse process the compressed audio data, restoring it to the original, uncompressed multi-channel audio stream. A multi-channel audio stream typically contains multiple independent sound channel information, such as the common 5.1 channels (front left, front right, center, rear left, rear right, and subwoofer) or 7.1 channels, which can create a more three-dimensional and realistic audio environment. After obtaining the multi-channel audio stream, mixing processing is performed on the multi-channel audio stream to generate the rendered audio stream. Audio mixing is the process of blending and adjusting audio signals from multiple channels according to certain rules and parameters. Speaker devices can perform operations such as volume balancing, frequency adjustment, and spatial positioning on the individual channel signals in a multi-channel audio stream based on their own audio characteristics (e.g., speaker layout, frequency response range) and user-preset audio effect parameters (e.g., equalizer settings, sound effect mode selection). For example, appropriately enhancing the vocals in the center channel makes dialogue clearer; adjusting the volume and delay of the surround channels creates a wider sound field. Through these processes, a rendered audio stream that meets the output requirements of the speaker device and the user's expectations is generated. Finally, the rendered audio stream is encoded into Pulse Code Modulation (PCM) data. PCM is a coding method that converts analog audio signals into digital audio signals. By sampling, quantizing, and encoding the analog signal, a continuous analog audio waveform is discretized into a series of digital pulse sequences. The speaker device converts the rendered audio stream (still a digital signal, but possibly processed into a specific format) according to PCM encoding rules to obtain PCM data. PCM data has a standardized format, making it easy for speaker devices to further process via digital-to-analog converters (DACs), ultimately converting it into analog audio signals to drive the speakers. This allows speaker devices to acquire and identify the encoding formats of audio data from different sources and invoke the corresponding decoding algorithms, ensuring compatibility with multiple audio encoding formats and improving device versatility. By mixing multi-channel audio streams, speaker devices can finely adjust the audio signal based on their hardware characteristics and user-defined settings. For example, volume balancing ensures a harmonious sound ratio across all channels, preventing any single channel from being too loud or too soft and affecting the overall listening experience, thereby optimizing the audio quality and enhancing the user's listening experience.After step S2 is completed, step S3 can be executed.

[0124] Step S3: The speaker device plays the first pulse code modulation data.

[0125] In some embodiments, after obtaining the pulse code modulation data and before playing the first pulse code modulation data, the speaker device may perform gain adjustment on the first pulse code modulation data so that the amplitude of the first pulse code modulation data matches the rated power of the built-in speaker of the display device, and / or perform time delay compensation on the first pulse code modulation data to eliminate the audio output time difference between the speaker device and the display device.

[0126] For example, based on the rated power of the built-in speakers in the display device (such as the maximum sound pressure level or power capacity of the speakers), the speaker device can modify the amplitude (i.e., signal strength) of the PCM data through digital volume control algorithms (such as dynamic range compression or volume gain adjustment). For instance, if the rated power of the 200 speakers in the display device is low, the speaker device can reduce the amplitude of the PCM data to avoid speaker overload or distortion due to excessive signal strength; conversely, if the speaker power is high, the amplitude can be appropriately increased to fully utilize its performance. By monitoring the audio output time difference between the speaker device's own speakers and the display device's speakers in real time (which may be caused by differences in data transmission paths, processing delays, or hardware performance), the speaker device can insert precise delay samples into the PCM data (such as adjusting the data output time through a buffer) to ensure that the audio signals of both reach the user's ears synchronously. For example, if the audio processing path of the speaker device is 5ms longer than that of the display device, the speakers on the TV end will delay their own output by 5ms or advance the output time of the display device to eliminate the time difference. In this way, by matching the rated power of the display device's speakers, speaker damage (such as overload burnout) or sound distortion (such as clipping) caused by signal amplitude mismatch can be avoided. At the same time, it ensures that speakers of different power ratings can output clear, distortion-free audio, improving the reliability of the combined sound system. Furthermore, it eliminates the audio output time difference between the speaker equipment and the display device, ensuring accurate sound image positioning for multi-channel audio (such as left and right channels, center channel), and avoiding sound field chaos or sound lagging behind the picture caused by inconsistent time delays.

[0127] After gain adjustment and delay compensation of the pulse code modulation data, the speaker device can play the adjusted first pulse code modulation data. This not only protects the display device speaker and improves sound quality, but also achieves seamless collaboration between multiple devices, thereby improving the reliability of the combined sound output. After step S3 is completed, step S4 can be executed.

[0128] Step S4: The speaker device extracts the second pulse code modulation data from the first pulse code modulation data.

[0129] In some embodiments, when the speaker device extracts second pulse code modulation data from the first pulse code modulation data, it can be done in the following manner: Based on a preset ratio, the second pulse code modulation data corresponding to each channel of the built-in speaker of the display device is extracted from the first pulse code modulation data, wherein the preset ratio is the proportion of the second pulse code modulation data corresponding to each channel of the built-in speaker in the first pulse code modulation data.

[0130] For example, the preset ratio can be set according to the channel configuration and capabilities of the built-in speakers on the TV. This ratio can take into account factors such as the number of channels supported by the built-in speakers and the frequency response range of each channel. In this way, by extracting data through the preset ratio, it can be ensured that the audio data transmitted to the built-in speakers of the display device matches the audio played by the speaker device in terms of loudness, avoiding a decline in the listening experience caused by loudness inconsistencies between devices. At the same time, since the audio data is extracted from the same set of first pulse code modulation data and transmitted to the display device through a hardware data line (with almost no delay), the synchronization of the built-in speakers of the speaker device and the display device when playing audio can be ensured.

[0131] Figure 10 This application provides a schematic diagram illustrating the process of extracting second pulse code modulation data from first pulse code modulation data in a speaker device according to some embodiments, such as... Figure 10 As shown, in some embodiments, the speaker device can extract the second pulse code modulation data from the first pulse code modulation data in the following manner. First, the speaker device sends a configuration query request to the display device through a terminal and receives a configuration response packet returned by the display device according to the configuration query request; then, the speaker device parses the configuration response packet to obtain the channel configuration parameters of the display device; the channel configuration parameters include the number of channels supported by the built-in speaker, the frequency response range of each channel, the coordinate information of the built-in speaker, and the interface protocol type; then, a preset ratio is determined according to the channel configuration parameters; finally, the second pulse code modulation data of the preset ratio is extracted from the first pulse code modulation data according to the preset ratio.

[0132] For example, the speaker device can send a configuration query request to the TV via a data cable connected to the TV. This request can carry a device identifier (such as the speaker's model code) to trigger the display device to return its audio-related configuration information. After receiving the request, the display device 200 can generate a response packet containing channel configuration parameters based on its built-in audio hardware parameters (such as the speaker driver chip model) and software settings (such as sound effect modes), and return it to the speaker device through the same terminal. The response packet can be in a structured format (such as JSON or binary fields) and can contain the following key information: the number of channels, i.e., the number of physical channels supported by the built-in speakers of the display device (e.g., 2.0 stereo, 5.1 surround sound, or 7.1 immersive sound); the frequency response range, i.e., the frequency response characteristics of each channel (e.g., 20Hz-200Hz for woofers, 200Hz-5kHz for midrange speakers, and 5kHz-20kHz for tweeters), used to determine whether the speakers can reproduce specific frequency band signals; speaker coordinate information, i.e., based on the physical coordinate system of the display device screen (e.g., with the center of the screen as the origin), marking the spatial position of each speaker (e.g., the coordinates of the front left speaker are (-0.5, 0.3, 0.2), in meters), used for sound field positioning calculations; and the interface protocol type, i.e., the audio transmission protocol supported by the display device (e.g., HDMI EARC, Dolby Digital Plus, or LPCM), ensuring that the captured data can be transmitted correctly. The speaker device dynamically calculates the proportion of PCM data to be captured based on the parsed channel configuration parameters using a preset algorithm. For example, if the display device is 2.0 channel and has a center speaker (determined by coordinate information that the speaker is located directly below the screen), the speaker device can capture the PCM data of the center channel (approximately 25%), while retaining the left and right channel data for its own playback. If the display device is 5.1 channel but only the front left and right speakers are connected (determined by the interface protocol type that surround sound is not enabled), the speaker device only captures the PCM data of the front left and right channels (approximately 50%), avoiding the transmission of invalid data that consumes bandwidth. The calculation of the capture ratio can also consider the frequency response range. If the display device's 200 speakers cannot reproduce high-frequency signals (e.g., the tweeter's frequency response limit is 15kHz), the speaker device can reduce the capture ratio of high-frequency channels (e.g., the right surround channel) to prevent signal distortion.

[0133] In other words, in this combined sound system, after the SoundBar decodes and renders the audio data, it doesn't directly play all the audio data from all channels. Instead, based on the TV's hardware characteristics (such as the number of built-in speakers, supported channel configurations, and interface protocols), it dynamically extracts a portion of the channel data (e.g., the center channel) and transmits it to the TV's speaker via a data cable. For example, if the TV only supports 2.0 channels but has a center speaker, the SoundBar will extract the center channel data separately and hand it over to the TV for processing, while the remaining channels are played by the SoundBar itself. In this way, by obtaining the channel configuration parameters of the display device 200, the speaker device can detect the TV's channel support capabilities and flexibly adjust the data allocation strategy to ensure that the extracted PCM data is completely matched with the hardware capabilities and that the combined sound effect matches the TV's hardware. This avoids transmitting center channel data to display devices without a center speaker, preventing sound loss or noise interference. Simultaneously, the TV doesn't need to participate in complex decoding; it only needs to receive channel data adapted to its own characteristics, reducing processing load and latency. After step S4 is completed, step S5 can be executed.

[0134] Step S5: The speaker device transmits the second pulse code modulation data to the display device via a data cable, so that when the speaker device plays the first pulse code modulation data, the second pulse code modulation data is played synchronously through the built-in speaker of the display device.

[0135] In some embodiments, the speaker device transmits the captured PCM data to the corresponding input port of the display device 200 via a data cable. After receiving the data, the display device 200 directly converts it into an analog signal through its built-in digital-to-analog converter module to drive its speakers to produce sound. At this time, the loudness control parameters (such as volume gain) of the speaker device are synchronously applied to the speaker output of the display device 200 to ensure that the volume of the two is balanced. Meanwhile, since the display device 200 only acts as a data pass-through channel and does not participate in decoding or rendering, additional latency and loudness deviation are avoided. In this way, by synchronizing the loudness control parameters, the output volume of the speaker device and the display device 200 can be kept consistent, achieving loudness matching. At the same time, the audio data processing path bypasses the decoding and rendering stages of the display device 200, the transmission path of audio data from the speaker device to the speaker of the display device 200 is shorter, the processing steps are fewer, and the latency is negligible, ensuring sound synchronization and thus solving the problems of audio data asynchrony and loudness mismatch in current combined sound generation solutions.

[0136] In some embodiments, when the speaker device plays the first pulse-code modulation data, the second pulse-code modulation data is simultaneously played through the built-in speaker of the display device. This can be achieved as follows: The second pulse-code modulation data is transmitted to the digital-to-analog converter module in the display device via a data cable. Upon receiving the second pulse-code modulation data, the digital-to-analog converter module converts the format of the second pulse-code modulation data to a format compatible with the built-in speaker, and then transmits the converted second pulse-code modulation data to the built-in speaker. This allows the second pulse-code modulation data to be played synchronously through the built-in speaker when the speaker device plays the first pulse-code modulation data.

[0137] For example, in the combined sound generation method of this application, after the speaker device, such as a soundbar, extracts the PCM data, the extracted data can be transmitted to the digital-to-analog converter module (such as ADC / DAC) on the TV end via a data cable connected to the terminal. After receiving the PCM data, the digital-to-analog converter module converts its format to a format compatible with the built-in speakers. The converted PCM data is then transmitted to the built-in speakers, which ultimately output the data to form sound playback. In this way, by uniformly controlling loudness parameters (such as volume gain, dynamic range compression, etc.) through the soundbar and synchronously acting on the built-in speakers, it is ensured that the audio output by both is consistent in loudness, avoiding volume imbalance problems caused by device differences and improving the sound field uniformity when using multi-channel combined sound generation. At the same time, the data received by the built-in speakers comes directly from the soundbar's conversion module, rather than going through the decoding and rendering process on the TV end. The TV end only acts as a data pass-through channel and does not participate in audio signal processing, thereby eliminating the additional latency caused by TV end processing. The audio data transmission path from the SoundBar to the built-in speaker is shorter, with fewer processing steps, significantly reducing signal transmission latency and improving the synchronization of audio and video.

[0138] In some embodiments, when the speaker device transmits the intercepted pulse code modulation data to the digital-to-analog converter module in the display device via a data line connected to the terminal, the second pulse code modulation data can first be encapsulated into a transmission frame according to the protocol format of the integrated circuit built-in audio bus in the speaker device. Then, the transmission frame is transmitted to the digital-to-analog converter module in the display device via a data line connected to the terminal, so that the digital-to-analog converter module performs format conversion on the pulse code modulation data and sends it to the built-in speaker to play the first pulse code modulation data through the built-in speaker.

[0139] For example, the captured PCM data can be encapsulated into transmission frames according to the protocol format of the integrated circuit's built-in audio bus, ensuring stable and efficient transmission of audio data under a specific bus protocol. The encapsulated transmission frames are then transmitted to the digital-to-analog converter (DAC) on the TV end via a data line connected to the terminal. Upon receiving the transmission frame, the DAC converts the PCM data to a format compatible with the built-in speaker. Finally, the converted PCM data is transmitted to the built-in speaker, which outputs sound. In this way, by encapsulating PCM data into transmission frames conforming to the integrated circuit's built-in audio bus protocol format, the stability and integrity of the data during transmission are ensured, and the compatibility of audio data transmission between devices is improved.

[0140] In some embodiments, after transmitting the transmission frame to the digital-to-analog converter module of the display device via a data line connected to the terminal, the speaker device can also detect the performance parameters of the digital-to-analog converter module. These performance parameters include the sampling rate support range for pulse-code modulation data, quantization accuracy parameters, and clock jitter characteristics. The speaker device then adjusts the transmission rate of the pulse-code modulation data transmitted to the digital-to-analog converter module based on these performance parameters.

[0141] For example, the speaker device can detect the performance parameters of the digital-to-analog converter (DAC) for PCM data. These parameters may include key performance parameters such as: sampling rate support range (the range of PCM data sampling frequencies that the DAC can process, such as 44.1kHz, 48kHz, etc.); quantization accuracy (the number of bits used for quantization of PCM data by the DAC, such as 16-bit, 24-bit, which directly affects the audio dynamic range and signal-to-noise ratio); and clock jitter characteristics (the stability of the internal clock of the DAC; excessive jitter can lead to audio distortion). Based on these detected performance parameters, the speaker device can dynamically adjust the rate at which it transmits PCM data to the DAC, ensuring that the data transmission rate matches the processing capabilities of the DAC.

[0142] In some embodiments, in addition to performing format conversion on PCM data through the digital-to-analog conversion module, format conversion can also be performed through the SoundBar. Alternatively, the SoundBar and the digital-to-analog conversion module may not perform format conversion, and the filtering may be performed directly through the speaker function on the TV. This application does not make any specific limitations on this.

[0143] As can be seen from the above technical solutions, the above embodiments provide an audio processing method, the method comprising: responding to receiving a path switching trigger prompt from a display device, receiving raw audio data sent by the display device, wherein the raw audio data is audio data that has not been decoded and rendered by the display device; performing decoding and rendering on the audio data to obtain first pulse code modulation data; playing the first pulse code modulation data; extracting second pulse code modulation data from the first pulse code modulation data; transmitting the extracted pulse code modulation data to the display device through a data cable connected to a terminal of the display device, so that when the speaker device plays the first pulse code modulation data, the second pulse code modulation data is played synchronously through the built-in speaker of the display device, thereby realizing the combined sound output of the built-in speaker and the speaker device. The method can replace the TV end in performing decoding and rendering by the speaker device, reducing the processing links and compensating for the lack of loudness on the TV end in terms of loudness, achieving loudness consistency. At the same time, the speaker device transmits to the built-in speaker of the TV end through a data cable hardware, and the transmission time is negligible, so the sound transmission of the two is basically synchronous, thereby solving the problems of audio data playback asynchrony and loudness mismatch in the current combined sound output scheme.

[0144] Based on the above audio processing method, some embodiments of this application also provide a speaker device:

[0145] In response to receiving a path switching trigger prompt from the display device, the device receives raw audio data sent by the display device, wherein the raw audio data is audio data that has not been decoded and rendered by the display device;

[0146] Decoding and rendering are performed on the audio data to obtain the first pulse code modulation data;

[0147] Play the first pulse code modulation data;

[0148] Extract the second pulse code modulation data from the first pulse code modulation data;

[0149] The intercepted pulse code modulation data is transmitted to the display device via a data cable connected to the terminal of the display device. When the speaker device plays the first pulse code modulation data, the second pulse code modulation data is played synchronously through the built-in speaker of the display device, so as to realize the combined sound production of the built-in speaker and the speaker device.

[0150] The speaker device can replace the TV in decoding and rendering audio data, reducing processing steps and compensating for the lack of loudness on the TV, achieving loudness consistency. Simultaneously, the speaker device transmits data to the TV's built-in speaker via a data cable; the transmission time is negligible, so the sound transmission between the two is essentially synchronized. This solves the problems of audio asynchrony and loudness mismatch in current combined sound systems.

[0151] This application also provides a display device 200, which includes a display 260 and a controller 250. The display 260 is configured to display a user interface, and the controller 250 executes instructions via a specific computer to cause the display device 200 to perform an audio processing method.

[0152] The monitor is configured to display the user interface;

[0153] The controller is configured as follows:

[0154] Monitor the level of input and output pins to determine the insertion status of terminals in speaker and display devices;

[0155] If the voltage level changes, it is determined that the speaker device has been plugged into the terminal, and a path switching trigger prompt is generated;

[0156] In response to a path switching trigger prompt, disconnect the data path between the main chip and the integrated circuit's built-in audio bus;

[0157] Connect the data path between the digital-to-analog converter module and the integrated audio bus of the integrated circuit;

[0158] Send audio data to the speaker device.

[0159] In conjunction with the aforementioned embodiments, the display device determines whether the speaker device is plugged into a terminal by monitoring the level changes of its input and output pins. When an insertion event is detected or the user actively selects the speaker device to output audio, the display device generates a path switching trigger prompt. In response to the path switching trigger prompt, the display device disconnects the data path between the main chip and the integrated circuit's built-in audio bus (IIS), while simultaneously connecting the data path between the digital-to-analog converter module and the IIS. This operation redirects the audio data stream from the TV to the speaker device. The display device directly transmits the raw audio data to the speaker device via an audio return channel (such as HDMI ARC / EARC) or a digital audio interface (such as optical fiber), bypassing the decoding and rendering stages on the TV side. The display device only acts as a pass-through channel for audio data and does not participate in decoding and rendering, thus avoiding loudness deviations caused by TV processing. The display device and the speaker device eliminate clock deviations by using a unified clock source (such as the main chip clock) to prevent data misalignment or frame loss caused by clock asynchrony, further improving synchronization. In this way, the display device, through path switching, data pass-through, and status feedback, collaborates with the speaker device to transfer audio processing tasks. When solving the loudness matching problem, the display device provides channel configuration information and acts as a pass-through channel to ensure that the speaker devices control the loudness uniformly. When solving the synchronization problem, the display device bypasses its own processing steps, shortens the transmission path, and works with a clock synchronization mechanism to achieve a high degree of synchronization between sound and picture, effectively overcoming the defects of loudness inconsistency and sound asynchrony in traditional combined sound solutions.

[0160] The same or similar parts among the various embodiments in this specification can be referred to mutually, and will not be repeated here.

[0161] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods of various embodiments or certain parts of the embodiments of the present invention.

[0162] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of this application, and are not intended to limit them. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features therein. Such modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of this application.

[0163] For ease of explanation, the above description has been provided in conjunction with specific embodiments. However, the above exemplary discussion is not intended to be exhaustive or to limit the embodiments to the specific forms disclosed above. Various modifications and variations can be obtained based on the above teachings. The selection and description of the above embodiments are for the purpose of better explaining the principles and practical applications, thereby enabling those skilled in the art to better utilize the described embodiments and various different variations of embodiments suitable for specific use considerations.

Claims

1. An audio processing method applied to a speaker device, characterized in that, include: In response to receiving a path switching trigger prompt from the display device, the device receives raw audio data sent by the display device, wherein the raw audio data is audio data that has not been decoded and rendered by the display device; The audio data is decoded and rendered to obtain the first pulse code modulation data; Play the first pulse-code modulation data; Extract the second pulse code modulation data from the first pulse code modulation data; The second pulse code modulation data is transmitted to the display device via a data line so that when the speaker device plays the first pulse code modulation data, the second pulse code modulation data is played synchronously through the built-in speaker of the display device.

2. The audio processing method according to claim 1, characterized in that, The second pulse code modulation data is transmitted to the display device via a data line so that when the speaker device plays the first pulse code modulation data, the second pulse code modulation data is played synchronously through the built-in speaker of the display device, including: The second pulse code modulation data is transmitted to the digital-to-analog converter module in the display device via the data line, so that after receiving the second pulse code modulation data, the digital-to-analog converter module converts the format of the second pulse code modulation data into a format that matches the built-in speaker, and transmits the converted second pulse code modulation data to the built-in speaker, so that when the speaker device plays the first pulse code modulation data, the second pulse code modulation data is played synchronously through the built-in speaker.

3. The audio processing method according to claim 2, characterized in that, Transmitting the second pulse code modulation data to the digital-to-analog converter module in the display device via the data line includes: The second pulse code modulation data is encapsulated into a transmission frame according to the protocol format of the integrated circuit built-in audio bus in the speaker device; The transmission frame is transmitted to the digital-to-analog converter module of the display device via a data line connected to the terminal of the display device.

4. The audio processing method according to claim 3, characterized in that, After transmitting the transmission frame to the digital-to-analog converter module of the display device via a data line connected to a terminal of the display device, the method further includes: The display device is requested to provide the performance parameters of the digital-to-analog conversion module; the performance parameters include the sampling rate support range, quantization accuracy parameters, and clock jitter characteristics of the pulse code modulation data. Receive the performance parameters fed back by the display device; The transmission rate of the pulse code modulation data transmitted to the digital-to-analog conversion module is adjusted according to the performance parameters.

5. The audio processing method according to claim 1, characterized in that, Decoding and rendering are performed on the audio data to obtain first pulse code modulation data, including: Obtain the encoding format of the audio data; The decoding algorithm corresponding to the encoding format is invoked to decode the audio data to obtain a multi-channel audio stream; Rendering processing is performed on the multi-channel audio stream to obtain a rendered audio stream; The rendered audio stream is encoded into pulse code modulation data.

6. The audio processing method according to claim 1, characterized in that, Before playing the first pulse code modulation data, the following is also included: Gain adjustment is performed on the first pulse code modulation data so that the amplitude of the first pulse code modulation data matches the rated power of the built-in speaker of the display device; And / or, perform delay compensation on the first pulse code modulation data to eliminate the audio output time difference between the speaker device and the display device.

7. The audio processing method according to claim 3, characterized in that, Extracting second pulse code modulation data from the first pulse code modulation data includes: Based on a preset ratio, second pulse code modulation data corresponding to each channel of the built-in speaker of the display device is extracted from the first pulse code modulation data; the preset ratio is the proportion of the second pulse code modulation data corresponding to each channel of the built-in speaker in the first pulse code modulation data.

8. The audio processing method according to claim 7, characterized in that, Extracting a preset proportion of second pulse code modulation data from the first pulse code modulation data includes: A configuration query request is sent to the display device via the terminal; Receive the configuration response packet returned by the display device according to the configuration query request; Parse the configuration response packet to obtain the channel configuration parameters of the display device; the channel configuration parameters include the number of channels supported by the built-in speaker, the frequency response range of each channel, the coordinate information of the built-in speaker, and the interface protocol type; Determine the preset ratio based on the aforementioned channel configuration parameters; According to the preset ratio, a preset ratio of the second pulse code modulation data is extracted from the first pulse code modulation data.

9. A speaker device, characterized in that, include: In response to receiving a path switching trigger prompt from the display device, the system receives audio data sent by the display device that has not been decoded or rendered by the display device. The audio data is decoded and rendered to obtain the first pulse code modulation data; Play the first pulse-code modulation data; Extract the second pulse code modulation data from the first pulse code modulation data; The second pulse code modulation data is transmitted to the display device via a data line so that when the speaker device plays the first pulse code modulation data, the second pulse code modulation data is played synchronously through the built-in speaker of the display device.

10. A display device, characterized in that, include: The monitor is configured to display the user interface; The controller is configured as follows: Monitor the level of input and output pins to determine the insertion status of terminals in speaker and display devices; When the voltage level changes, it is determined that the speaker device has been inserted into the terminal, and a path switching trigger prompt is generated; In response to the path switching trigger prompt, the data path between the main chip and the integrated circuit's built-in audio bus is disconnected; Connect the data path between the digital-to-analog converter module and the built-in audio bus of the integrated circuit; The audio data is sent to the speaker device.

Images