Display device and audio control method

Abstract

The invention discloses a display device and an audio control method, and relates to the field of display device.Under the scene of multi-window display, target audio devices corresponding to all windows and output of audio data of the corresponding windows, the method comprises the steps that a target window in a multi-window interface is determined, and when the volume adjustment type is media asset volume adjustment, the audio data of the corresponding windows are output; in response to the volume adjustment operation, a first volume value of a target media asset played in a target window is obtained, and the target window is a window in multi-window display; and adjusting the first volume value of the target media asset to obtain a second volume value, and adjusting the volume of a target audio device corresponding to the target window according to the second volume value to obtain a target volume value. That is to say, the target audio devices corresponding to the different windows are adjusted respectively, the auditory experience of the user is improved, and the requirements of the user are met.

Description

Technical Field

[0001] This application relates to the technical field of display devices, and in particular, to a display device and an audio control method. Background Art

[0002] With the continuous development of terminal technologies, display devices are more and more widely used. For example, a display device supports simultaneous presentation of multiple windows, that is, multiple windows are displayed in a display interface, and the content presented in each window is different, so as to meet the viewing needs of different users at the same time.

[0003] Taking a smart TV as an example, a smart TV usually mixes different audio data, and the mixed audio data is output through an audio device. For example, the mixed audio data is output through the built-in speaker of the smart TV or an external audio device. Therefore, under multi-window display, the audio data of different windows will also be mixed, resulting in poor audio-visual effects under multi-window display. Summary of the Invention

[0004] This application provides a display device and an audio control method, which improve the audio-visual effects under multi-window display.

[0005] In a first aspect, a display device is provided, including a display, an audio output interface, and a controller, where the controller is communicatively connected to the display and the audio output interface; wherein, the audio output interface is configured to be communicatively connected to a plurality of target audio devices, and the target audio devices are used for audio data output.

[0006] The controller is configured to perform the following processes: under multi-window display, based on the target audio devices corresponding to each window, output the audio data of the corresponding window; when the volume adjustment type is the adjustment of the media volume, in response to a volume adjustment operation, obtain a first volume value of a target media played in a target window, where the target window is one of the windows in the multi-window display; adjust the first volume value of the target media to obtain a second volume value, and adjust the volume of the target audio device corresponding to the target window according to the second volume value to obtain a target volume value.

[0007] In the scenario of multi-window display, where audio data corresponding to each window is output based on the target audio device corresponding to the window, determine the target window in the multi-window interface. When the volume adjustment type is the adjustment of the media volume, in response to a volume adjustment operation, obtain the first volume value of the target media played in the target window, where the target window is one of the windows in the multi-window display; adjust the first volume value of the target media to obtain a second volume value, and adjust the volume of the target audio device corresponding to the target window according to the second volume value to obtain a target volume value. That is to say, it realizes the separate adjustment of the target audio devices corresponding to different windows, improves the user's auditory experience, and meets the user's needs.

[0008] In a possible implementation, after the controller adjusts the first volume value of the target media to obtain a second volume value, it is further configured to: in the target window, display a first volume control bar corresponding to the second volume value, where the first volume control bar displays the icon of the target audio device corresponding to the target window and the second volume value.

[0009] The icon of the target audio device corresponding to the target window displayed in the first volume control bar corresponds to the icon of the third playback control button under the target window. Users can understand which target audio device the volume is adjusted for and what the volume value of the sound played by the target audio device is through the displayed first volume control bar, enhancing the visual experience of the smart TV and improving the user's understanding of the relevant operations of volume adjustment.

[0010] In a possible implementation, after the controller outputs the audio data corresponding to each window based on the target audio device corresponding to the window in the multi-window display and before obtaining the first volume value of the target media played in the target window when the volume adjustment type is the adjustment of the media volume, it is further configured to: in response to a focus positioning operation on the multi-window interface, determine the target window in the multi-window interface.

[0011] Determining the target window in response to a focus positioning operation on the multi-window interface enables users to quickly and accurately find and operate the required window in the complex interface with multi-windows displayed simultaneously; at the same time, this method also optimizes the user operation experience and makes the operation more smooth and convenient.

[0012] In a possible implementation, if the target media is the media projected from the terminal device to the display device, after the controller outputs the audio data corresponding to each window based on the target audio device corresponding to the window in the multi-window display, it is further configured to: obtain the projection volume of the target media and use the projection volume of the target media as a preset threshold.

[0013] The preset threshold is used to establish the relationship between the volume of the target audio device and the volume of the media asset, improving the conversion efficiency between the volume of the media asset and the volume of the target audio device.

[0014] In a possible implementation, the controller adjusts the first volume value of the target media asset to obtain a second volume value, and is further configured to: when the volume adjustment operation is a volume increase operation and the first volume value is not less than the preset threshold, determine the first volume value of the target media asset as the second volume value.

[0015] When the volume adjustment operation is a volume increase operation and the first volume value is not less than the preset threshold, keep the first volume value of the target media asset unchanged and determine the first volume value as the second volume value, so that the solution can effectively run in the display device, improving the security and rationality of the audio control solution.

[0016] In a possible implementation, when multiple windows are displayed and presented by the target application, the controller adjusts the first volume value of the target media asset to obtain a second volume value, and is configured to: obtain the volume change value corresponding to the volume adjustment operation; determine the second volume value based on the first volume value, the volume change value, and the preset volume grading; and the target application calls the preset first interface to adjust the volume of the target media asset to the second volume value.

[0017] In a second aspect, a display device is provided, including a display, an audio output interface, and a controller, where the controller is communicatively connected to the display and the audio output interface; wherein, the audio output interface is configured to be communicatively connected to multiple target audio devices, and the target audio devices are used for audio data output.

[0018] The controller is configured to perform the following process: under multi-window display, output the audio data of the corresponding window based on the target audio device corresponding to each window; when the volume adjustment type is the adjustment of the system volume, in response to the volume adjustment operation, obtain the initial volume value of the target audio device corresponding to the target window, where the target window is one of the windows in the multi-window display; and adjust the initial volume value of the target audio device corresponding to the target window to obtain the target volume value.

[0019] In the scenario of outputting the audio data of the corresponding window based on the target audio device corresponding to each window under multi-window display, determine the target window in the multi-window interface. When the volume adjustment type is the adjustment of the system volume, in response to the volume adjustment operation, obtain the initial volume value of the target audio device corresponding to the target window; and adjust the volume of the target audio device corresponding to the target window to obtain the target volume value. That is, realizing the separate adjustment of the target audio devices corresponding to different windows, improving the user's auditory experience and meeting the user's needs.

[0020] In a possible implementation, after the controller adjusts the initial volume value of the target audio device corresponding to the target window to obtain the target volume value, it is further configured to: in a multi-window interface, display a second volume control bar corresponding to the target volume value, where the second volume control bar displays the icon of the target audio device corresponding to the target window and the target volume value.

[0021] When the volume adjustment type is the adjustment of the system volume, the second volume control bar is displayed in the multi-window interface, generally located at the top or bottom of the multi-window interface, and the icon of the target audio device corresponding to the target window is displayed in the second volume control bar, used to distinguish whether the volume value of the first preset audio device or the second preset audio device is adjusted.

[0022] In a possible implementation, the controller adjusts the initial volume value of the target audio device corresponding to the target window to obtain the target volume value, and is configured to: obtain the volume change value corresponding to the volume adjustment operation; determine the target volume value based on the initial volume value and the volume change value; call a preset second interface to adjust the volume of the target audio device corresponding to the target window to the target volume value.

[0023] In a third aspect, an audio control method is provided, which is applied to the display device in any one of the first aspects, and includes the following steps: under multi-window display, output the audio data of the corresponding window based on the target audio device corresponding to each window; when the volume adjustment type is the adjustment of the media volume, in response to the volume adjustment operation, obtain the first volume value of the target media played in the target window, where the target window is one of the windows in the multi-window display; adjust the first volume value of the target media to obtain the second volume value, and adjust the volume of the target audio device corresponding to the target window according to the second volume value to obtain the target volume value.

[0024] In a fourth aspect, an audio control method is provided, which is applied to the display device in any one of the first aspects, and includes the following steps: under multi-window display, output the audio data of the corresponding window based on the target audio device corresponding to each window; when the volume adjustment type is the adjustment of the system volume, in response to the volume adjustment operation, obtain the initial volume value of the target audio device corresponding to the target window, where the target window is one of the windows in the multi-window display; adjust the initial volume value of the target audio device corresponding to the target window to obtain the target volume value.

[0025] In a fifth aspect, an audio control device is provided, including a unit for executing any one of the audio control methods in the third aspect or the fourth aspect. The device can be a display device or a chip inside the display device.

[0026] In a sixth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is run by an audio control device, the audio control device is caused to execute any one of the audio control methods in the third aspect or the fourth aspect.

[0027] In a seventh aspect, a computer program product is provided. The computer program product includes: a computer program, and when the computer program is run by an audio control device, the audio control device is caused to execute any one of the audio control methods in the third aspect or the fourth aspect.

[0028] It can be understood that for the beneficial effects of the above second aspect to the seventh aspect, reference can be made to the relevant descriptions in the first aspect above, and details will not be repeated here. Description of the Drawings

[0029] Figure 1 A schematic diagram showing an operation scenario between a display device and a control device;

[0030] Figure 2 A schematic diagram showing mixing control in a smart TV;

[0031] Figure 3 A configuration block diagram of a display device 200 provided by an embodiment of the present application is shown;

[0032] Figure 4 A system architecture diagram of a display device 200 provided by an embodiment of the present application is shown;

[0033] Figure 5 A schematic diagram showing a multi-window display application scenario provided by an embodiment of the present application;

[0034] Figure 6 A flowchart showing an audio control method provided by an embodiment of the present application;

[0035] Figure 7 A schematic diagram showing a multi-window display application scenario provided by an embodiment of the present application;

[0036] Figure 8 An example diagram showing audio control in a smart TV provided by an embodiment of the present application;

[0037] Figure 9 A flowchart showing another audio control method provided by an embodiment of the present application;

[0038] Figure 10 A schematic diagram showing another multi-window display application scenario provided by an embodiment of the present application;

[0039] Figure 11Shows the system architecture diagram of another display device provided by an embodiment of the present application;

[0040] Figure 12 Shows the schematic flowchart of another audio control method provided by an embodiment of the present application;

[0041] Figure 13 Shows an example diagram of the channel between a window and a preset audio device provided by an embodiment of the present application;

[0042] Figure 14 Shows the schematic diagram of another multi-window display application scenario provided by an embodiment of the present application;

[0043] Figure 15 Shows the schematic flowchart of another audio control method provided by an embodiment of the present application;

[0044] Figure 16 Shows the schematic diagram of another application scenario of multi-window display provided by an embodiment of the present application;

[0045] Figure 17 Shows the schematic diagram of another application scenario of multi-window display provided by an embodiment of the present application;

[0046] Figure 18 Shows the system architecture diagram of another display device provided by an embodiment of the present application;

[0047] Figure 19 Shows the schematic diagram of another application scenario of multi-window display provided by an embodiment of the present application;

[0048] Figure 20 Shows the schematic diagram of another application scenario of multi-window display provided by an embodiment of the present application;

[0049] Figure 21 Shows the schematic flowchart of another audio control method provided by an embodiment of the present application;

[0050] Figure 22 Shows the system architecture diagram of another display device provided by an embodiment of the present application;

[0051] Figure 23 Shows the schematic diagram of another application scenario of multi-window display provided by an embodiment of the present application. Detailed implementation manners

[0052] The technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application. In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0053] In the following, the terms "first", "second", and "third" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first", "second", and "third" may explicitly or implicitly include one or more of the features.

[0054] References to "one embodiment" or "some embodiments" etc. described in the specification of the present application mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in the specification of the present application do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. And, the terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0055] In order to better understand the embodiments of the present application, the terms or concepts that may be involved in the embodiments are explained below.

[0056] 1. System prompt tone

[0057] System prompt sounds refer to the sound signals emitted by the operating system or application of the display device to convey specific system events, operating status or information to the user. System prompt sounds include operation prompt sounds, connection prompt sounds, and notification prompt sounds.

[0058] Operation prompt sound: When the user uses the remote control or body buttons to turn on or off the machine, switch channels, adjust the volume, switch input signals, etc., the operation prompt sound will sound. It is usually a short sound effect, such as "beep", "dumb", "click", etc., or a simple melody, such as startup music.

[0059] Connection-related prompt sound: When the TV is connected to external devices such as speakers and game consoles through interfaces such as HDMI and USB, a prompt sound will be emitted to indicate that the device has been successfully connected or recognized. It may be a relatively loud sound, such as "ding dong", to prompt the user that they can start using the external device.

[0060] Notification prompt sound: When there is a software update available for the TV system, a prompt sound will be emitted to remind the user. It may be a relatively special sound effect, such as "dingling", to attract the user's attention and let the user know that there is a new system version available for update to obtain better functions and performance.

[0061] 2. Built-in speaker

[0062] A speaker, also known as a "loudspeaker", is a transducer that converts electrical signals into sound signals and is commonly used in electronic devices to play sound.

[0063] In the embodiments of the present application, the display device integrates a speaker internally, so this speaker is also called a built-in speaker.

[0064] In common traditional application scenarios, the display device outputs audio data through a single audio device (such as a built-in speaker or an external audio output device). For example, when watching TV programs in the living room or conducting video conferencing demonstrations in the office, a single audio device is often used to output audio data.

[0065] The display device can have various implementation forms. For example, a TV (which can also be called a smart TV, intelligent screen, or large-screen device), a tablet computer, a projection device, an electronic bulletin board, a mobile phone, a laptop computer, a netbook, a personal digital assistant (PDA), a vehicle-mounted device, etc. The display device processes the audio data, converts it into an audio signal, and transmits it to the built-in speaker, or transmits it to a corresponding external audio output device such as a speaker or a headphone through different connection methods such as Bluetooth, 3.5 mm audio cable, HDMI cable, coaxial cable, USB cable, or optical fiber, so as to realize the playback output of audio and meet people's needs for receiving sound in various scenarios.

[0066] Exemplarily, as a common display device, a smart TV often outputs audio data through a single audio device and meets the diverse audio-visual needs of users through different audio devices.

[0067] For example, an audio device can use a built-in speaker, which is convenient and fast, and has good compatibility with smart TVs. That is to say, there is no need for additional and complex pairing, connection, etc. operations. As long as the TV is turned on, users can quickly enter the state of watching programs or videos, which is very convenient to use. Moreover, the built-in speaker is part of the overall TV system, highly integrated with the TV's hardware and software systems, and can perfectly cooperate with the TV's screen display, operation interface, etc. in terms of audio output, without compatibility problems, ensuring the synchronization and stability of sound and picture.

[0068] Also, for example, the audio device can also choose a Bluetooth speaker and establish a connection between the Bluetooth speaker and the smart TV through Bluetooth wireless technology. When the smart TV plays video resources, the Bluetooth speaker creates an immersive surround sound effect. When the smart TV plays music resources, the Bluetooth speaker can provide richer and fuller sound, such as a more mellow bass and a more powerful shock effect.

[0069] Figure 1 A schematic diagram showing an operation scenario between a display device and a control device is shown. As Figure 1 shown, the user can operate the display device 200 through a smart device 300 (such as a mobile terminal, a tablet computer, a computer, a laptop, etc.) or a control device 100 (such as a remote control, etc.). For example, the user can input user instructions through buttons on the remote control, voice input, control panel input, etc. to control the display device 200.

[0070] In some embodiments, the display device 200 can be controlled by obtaining the user's touch instructions or gesture instructions, receiving the user's voice instructions, etc.

[0071] In some embodiments, the display device 200 also communicates with the server 400 for data. The display device 200 is communicatively connected through a local area network (LAN), a wireless local area network (WLAN), and other networks. The server 400 can provide various contents and interactions to the display device 200. The server 400 can be a cluster or multiple clusters, and can include one or more types of servers.

[0072] In various application scenarios, the display device will receive audio data from different sources. Taking a smart TV as an example, while playing a TV program, it will also synchronously receive system prompt sounds such as operation prompt sounds. When the user switches channels, adjusts the volume, etc., the operation prompt sound will sound and coexist with the audio of the TV program. When the user is playing a game, the background music of the game itself and various sound effects in the game are constantly played. At the same time, the sound of the user's voice chat through the network will also be added, and various sound effects are intertwined.

[0073] To meet the demand for simultaneous playback of multiple audio sources and adapt to different application scenarios such as watching movies, daily operations, gaming, and voice chatting, display devices usually perform mixing processing internally, mixing different audio data, and the mixed audio data is output through a single audio device.

[0074] The target chip in a smart TV is a chip that can be used for audio data processing. Mixing processing of different audio data is achieved through the target chip; the target chip can refer to the main control chip in the display device, which is usually responsible for functions such as processing video and audio data, image rendering, data reception, and decoding; the target chip can also be an audio processing chip that assists the main control chip.

[0075] Figure 2 FIG. shows a schematic diagram of mixing control in a smart TV. It is a schematic diagram of mixing control of the target chip when the smart TV outputs audio data through a single audio device. For the convenience of explanation and display, only the part for audio mixing processing is shown in the target chip 10. Among them, the pins for receiving audio data are CH1, CH2, CH3, CH4, CH5, etc. Among them, the audio data received by each pin is in the form of stereo.

[0076] In the embodiments of the present application, CH1-CH4 are used to receive multi-channel (Multi-Channel, mch) audio data. For example, 5.1-channel audio data and 7.1-channel audio data. CH5 is used to receive stereo (2-Channel, 2ch) audio data.

[0077] It should be understood that the pins in different target chips may be different. Therefore, the interface for audio input in the target chip can also be different from the pin distribution in this embodiment. The embodiments of the present application do not limit the pins in the target chip.

[0078] Therefore, in common traditional application scenarios, to meet the demand for simultaneous playback of multiple audio sources in various application scenarios, as Figure 2 shown in (a) of, the target chip 10 is controlled to achieve mixing processing of the audio data A received by CH1-CH4 and the audio data B received by CH5, mixing different audio data, and the mixed audio data is output through the audio device 20. Here, the audio device 20 refers to a single audio device.

[0079] It should be understood that in the target chip 10, there may be more pins as interfaces for audio input.

[0080] Specifically, as Figure 2As shown in (b) in [description], the audio data A received by CH1-CH4 and the audio data B received by CH5 are mixed, and the mixed audio data is output through the built-in speaker 21. As Figure 2 As shown in (c) in [description], the audio data A received by CH1-CH4 and the audio data B received by CH5 are mixed, and the mixed audio data is output through the external audio output device, the Bluetooth speaker 22.

[0081] In some embodiments, the mixing control in the above target chip 10 can be executed and implemented by an audio mixer (Mixer) in the target chip. Among them, Mixer is a device or software tool for mixing, adjusting, and processing multiple audio signals. It can combine audio signals from different sound sources (such as microphones, musical instruments, audio players, etc.), and adjust parameters such as the volume, timbre, and phase of each signal, and finally output a mixed audio signal.

[0082] With the continuous development of terminal technology, the application of display devices is becoming more and more extensive. For example, display devices support the simultaneous presentation of multiple windows, that is, multiple windows are displayed in a display interface, and the content presented in each window is different, so as to meet the viewing needs of different users at the same time. However, in the case of multi-window display, the audio data of different windows is still mixed through the mixing process in the above technology. Specifically, the audio data corresponding to one window in the multi-windows and the audio data corresponding to other windows are mixed, and the audio data after mixing for multiple windows is played simultaneously, resulting in confusion for users in terms of hearing and reducing the audio-visual experience of users.

[0083] Therefore, in the related art, only the audio data corresponding to one window in the multi-windows is retained for output, and the audio data corresponding to this window is played through the built-in speaker or the external audio output device, while the audio data corresponding to other windows is muted. However, the above method of only playing the audio data of a single window, although effectively reducing the mutual interference of audio data between windows, makes users unable to hear the audio data of other windows, reducing the audio-visual experience of users under multi-window display.

[0084] In view of this, embodiments of the present application provide a display device and an audio control method. The audio control method is applied to the display device, which includes a display, an audio output interface, and a controller. The controller is communicatively connected to the display and the audio output interface. The audio output interface is configured to be communicatively connected to multiple target audio devices for audio data output. The method further includes: under multi-window display, first stop the mixing process of each audio data; then, according to the determined target relationship, establish channels for audio data transmission between the windows and preset audio devices (i.e., the first preset audio device and the second preset audio device) to enable the transmission of audio data of different windows to different audio devices, thereby improving the audio-visual experience of users under multi-window display.

[0085] The following describes the display device to which the audio control method provided by the embodiments of the present application is applied with reference to the accompanying drawings. Figure 3 The configuration block diagram of a display device 200 provided by an embodiment of the present application is shown; as Figure 3 shown, the display device 200 includes a tuner-demodulator 210, a communicator 220, a detector 230, an external device interface 240, a controller 250, a display 260, an audio output interface 270, a memory, a power supply, and a user interface.

[0086] It should be noted that Figure 3 the structure shown does not constitute a specific limitation on the display device 200. In other embodiments of the present application, the display device 200 may include more or fewer components than Figure 3 those shown, or the display device 200 may include Figure 3 a combination of some of the components shown, or the display device 200 may include Figure 3 sub-components of some of the components shown. Figure 3 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0087] The display 260 includes a display screen component for presenting a picture and a driving component for driving image display, which is used to receive an image signal output from the controller and display video content, image content, a menu control interface, and a user control UI interface.

[0088] The communicator 220 is a component for communicating with external devices or servers according to various communication protocol types. For example, the communicator may include a Wifi module, a Bluetooth module, a wired Ethernet module, and other network communication protocol chips or near-field communication protocol chips, as well as an infrared receiver.

[0089] For example, communicate with external audio output devices such as Bluetooth speakers and Bluetooth headsets through the Bluetooth module.

[0090] The controller 250 controls the operation of the display device and responds to user operations through various software control programs stored in the memory. The controller 250 controls the overall operation of the display device 200. For example, in response to receiving a user command for selecting a UI object to be displayed on the display 260, the controller 250 can perform operations related to the object selected by the user command.

[0091] In some embodiments, the controller may include some or all of the following: a central processing unit (CPU), a video processor, an audio processor, a graphics processing unit (GPU), a random access memory (RAM), a read-only memory (ROM), a first interface to an nth interface for input / output, and a communication bus. Among them, different processors may be independent devices or integrated devices.

[0092] The audio processor is a device for processing and optimizing audio data, and can perform functions such as mixing, equalizing, compressing, delaying, and noise reduction.

[0093] Before multi-window display, the audio processor can be used for mixing audio data. Multiple different audio data are mixed or superimposed through a mixer to form a new mixed audio data.

[0094] The audio output interface 270 can be a built-in speaker in the display device 200 or an external audio output device connected to the display device 200. Among them, the external audio output device is a device connected to the display device through different connection methods such as Bluetooth, USB cable, coaxial cable, etc., such as a Bluetooth speaker, a Bluetooth headset, a digital audio player, etc.

[0095] In some embodiments, the display device 200 may also be provided with external audio output terminals, such as a 3.5 mm audio jack, an optical fiber audio interface, a coaxial audio interface, a USB interface, etc. The external audio output device can be connected to the display device 200 through the external audio output terminal to output the sound of the display device 200.

[0096] An operating system runs on the display device 200, such as a Linux operating system, a Unix operating system, an Android operating system, an iOS operating system, or a Windows operating system, etc. Application programs can be installed and run on the operating system.

[0097] The operating system of the display device 200 may adopt a layered architecture, an event-driven architecture, a micro-core architecture, a micro-service architecture, or a cloud architecture. The embodiment of the present application takes the Android operating system as an example to exemplify the system architecture of the display device 200.

[0098] It should be noted that although the embodiment of the present application is described using the Android operating system as an example, its basic principles are also applicable to display devices 200 of other operating systems.

[0099] Figure 4 A system architecture diagram of a display device 200 provided in an embodiment of the present application is shown. Figure 4 As shown, the system architecture of the display device 200 can be divided into several layers, each layer has a clear role and division of labor. The layers communicate with each other through software interfaces. In some embodiments, the system is divided into five layers, from top to bottom, respectively, the application layer (Applications) layer (referred to as "application layer"), the application framework (Application Framework) layer (referred to as "application framework layer"), the Android runtime (Android runtime) and the system library layer (referred to as "system runtime library layer"), the hardware abstraction layer (HAL) and the Linux kernel layer.

[0100] In some embodiments, at least one application is running in the application layer, and these applications can be window programs, system settings programs, clock programs, etc. provided by the operating system, or applications developed by third-party developers. In specific implementations, the application packages in the application layer are not limited to the above examples.

[0101] Among them, in the embodiment of the present application, the multi-window application running in the application layer can be used as a functional module in the operating system, integrated into the bottom layer of the operating system, and can also exist as an independent application; the multi-window application has its own independent running logic and interface.

[0102] The application framework layer provides application programming interfaces (APIs) and programming frameworks for applications. The application framework layer includes some predefined functions. The application framework layer is equivalent to a processing center that determines the actions that applications in the application layer take. Applications can access system resources and obtain system services during execution through the API interface.

[0103] like Figure 4As shown in the figure, in the embodiment of the present application, the application framework layer includes managers, etc. Among them, the managers include an Audio Manager for managing and controlling audio functions, mainly for volume control (such as volume type, volume size, etc.), audio mode (such as the output path of audio data), and audio device management (such as the status of audio devices, such as connection).

[0104] The managers also include some or all of the following modules: an Activity Manager for interacting with all the activities running in the system; a Location Manager for providing access to the system location service for system services or applications; a Package Manager for retrieving various information related to the application program packages currently installed on the device; a Notification Manager for controlling the display and clearing of notification messages; and a Window Manager for managing icons, windows, toolbars, wallpapers, and desktop widgets on the user interface.

[0105] In some embodiments, the application framework layer further includes server components responsible for processing and managing screen mirroring related functions. For example, a CastServer for performing screen mirroring monitoring, interruption, and management, etc.

[0106] After the application program involving screen mirroring is started or the smart TV is powered on, the CastServer will perform initialization preparations, such as starting to monitor the mirror or push the corresponding protocol service.

[0107] During the screen mirroring process, a session between the client (such as a mobile phone, tablet, etc., terminal devices) and the server (smart TV) will be established first. The session contains various key information related to screen mirroring, such as the screen mirroring identifier, screen mirroring type (such as mirroring, URL, etc.), information about the screen mirroring media assets (such as video stream width, video stream height, video name, the URL address for pushing the video, metadata in the address, etc.).

[0108] After the session is established, the CastServer starts and registers a series of screen mirroring related monitoring functions. Specifically, it will monitor the connection and disconnection status of the session between the client and the server in real time, and also closely monitor changes in playback information and status, etc.

[0109] Thereafter, the encoded data is sent to the CastServer through the network. After receiving this data, the CastServer will perform decoding operations and transmit the decoded audio and video data to the display screen of the display device, audio device, etc., to achieve playback.

[0110] In some embodiments, the system runtime layer provides support for the upper layer, i.e., the application framework layer. When the application framework layer is used, the Android operating system runs the C / C++ libraries included in the system runtime layer to implement the functions to be achieved by the application framework layer.

[0111] The hardware abstraction layer is located above the Linux kernel layer, and its main function is to provide a unified and abstract hardware access interface for the upper-layer software; different hardware devices of the Android operating system (such as speakers, headphones, etc. produced by different manufacturers) can transmit audio data to the corresponding audio devices for playback through their respective corresponding HAL interfaces.

[0112] Among them, the primary HAL is responsible for managing and controlling key hardware such as cameras, sensors, audio devices, and display devices. For example, in terms of audio processing, the audio part in the primary HAL provides a unified audio device access interface for the system's audio services.

[0113] In the hardware abstraction layer, the audio path (AudioPatch) is a concept used to describe the transmission path of audio data in the system. It involves the complete connection link from the audio source (such as the audio output of an application) to the audio destination (such as speakers, headphones, etc.).

[0114] In some embodiments, the Linux kernel layer is the layer between hardware and software. As Figure 4 shown, the kernel layer includes audio drivers, display drivers, Bluetooth drivers, and power drivers, and may also include some or all of the following: camera drivers, WIFI drivers, USB drivers, HDMI drivers, sensor drivers (such as fingerprint sensors, temperature sensors, pressure sensors, etc.).

[0115] The execution subject of the audio control method provided in the embodiments of the present application can be the above-mentioned display device (such as a smart TV, etc.), or a functional module or functional entity in the display device that can implement the functions of this method, and this solution can be implemented through hardware and / or software, and in a combined manner, and can be specifically determined according to actual usage requirements, and the embodiments of the present application do not limit this.

[0116] Next, the embodiments of the present application will be illustrated by the following multiple exemplary embodiments in conjunction with the accompanying drawings. The methods in the following embodiments can all be implemented in the display device with the above-mentioned hardware structure and system architecture. The hardware structure diagram of the display device can be as Figure 3 shown, and the system architecture diagram of the display device can be as Figure 4 shown, but the embodiments of the present application are not limited thereto.

[0117] For ease of explanation, in the embodiments of the present application, the display device is taken as an intelligent TV as an example. The intelligent TV can display multiple windows simultaneously (i.e., multi-windows), which provides convenience for users to handle multiple tasks simultaneously. The multi-window display can display two windows, three windows, four windows, or more windows at the same time. And the multi-windows can be presented in one or a combination of display modes such as tiling, cascading, side-by-side, maximizing, and thumbnails.

[0118] Figure 5 FIG. shows a schematic diagram of a multi-window display application scenario provided by an embodiment of the present application. When the display 260 in the intelligent TV performs multi-window display, there are various display forms. It can be as Figure 5 shown in (a) of FIG., where window 31 and window 32 are tiled and evenly distributed on the display, and the window sizes are the same and do not overlap; it can be as Figure 5 shown in (b) of FIG., where window 33 and window 34 are side-by-side and the window sizes are different and do not overlap; it can also be as Figure 5 shown in (c) of FIG., where window 35 is full-screen and window 36 is displayed on window 35 in the form of a small window or a floating window, and there is an overlap between window 35 and window 36.

[0119] It should be understood that generally, the content presented in each window is different, and the form of the content can be multimedia resources (i.e., media assets) such as videos, pictures, music, files, games, etc. In the embodiments of the present application, when the media assets presented in at least two windows involve the output of audio data, the corresponding audio data output provides a more intuitive and immersive audio-visual experience for users. For example, Figure 5 as shown in (a) of FIG., window 31 plays video A and window 32 plays video B, and the intelligent TV can play the audio data corresponding to video A and the audio data corresponding to video B; for another example, Figure 5 as shown in (c) of FIG., window 35 plays video C and window 36 plays music A, and the intelligent TV can play the audio data corresponding to video C and the audio data corresponding to music A.

[0120] For a better understanding of the technical solutions in some embodiments of the present application, the technical solutions of the display device and the audio control method, and how the technical solutions solve the above technical problems will be described in detail below with reference to some specific embodiments and the accompanying drawings. The embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. Obviously, the described embodiments are some but not all of the embodiments of the present application.

[0121] The following will Figures 6 to 14 describe in detail the audio control method provided by the embodiments of the present application. Figure 6The flowchart of an audio control method provided by an embodiment of the present application is shown. This method is applied to a display device. Taking a smart TV as an example, as Figure 6 shown, the audio control method provided by the embodiment of the present application includes the following steps:

[0122] S510. In response to a first operation, enter an interface for multi-window display.

[0123] Among them, the first operation is used to trigger multi-window display and can be a start instruction for multi-windows.

[0124] In some embodiments, multi-window presentation is implemented in a single application. With the help of an application that supports multi-window display, multiple windows are displayed. For example, multiple windows play TV programs on different channels, or different media resources presented by screen mirroring or other means; in another embodiment, multi-window presentation is jointly implemented by multiple applications. For example, one window plays a movie with a video software, and another window plays a short video with a short video software.

[0125] It should be understood that during the use of a smart TV, there may be various forms of presenting multi-windows, and the ways to trigger multi-window display are also rich and diverse, involving multiple channels and conditions. Therefore, the embodiment of the present application does not limit the first operation for triggering multi-window presentation.

[0126] Figure 7 The schematic diagram of an application scenario of multi-window display provided by the embodiment of the present application is shown. The application scenario of the embodiment of the present application is described. Taking the case where multi-windows are presented by a single application as an example, for the sake of convenience of description, in the embodiment of the present application, this single application is a split-screen application, and this is not limited.

[0127] Exemplarily, in response to a trigger instruction for long pressing the "main menu" key on the remote control, enter the display interface of the service center, as Figure 7 shown in (a); in the display interface of this service center, when the "split-screen movie viewing" or "split-screen screen mirroring" control is triggered, the multi-window display corresponding to the split-screen application can be entered; for example, when the "split-screen screen mirroring" control is triggered, enter the interface for multi-window display, as Figure 7 shown in (b), and this interface is the first interface of the split-screen application; Figure 7 The multi-windows in (b) are used to play media resources screen mirrored from different terminal devices (such as mobile phones, tablets, etc.).

[0128] S520. Under multi-window display, stop the mixing process of the first audio data and the second audio data.

[0129] The first audio data and the second audio data are different audio data. For example, the number of channels of the first audio data is multi-channel, and the number of channels of the second audio data is stereo.

[0130] Exemplarily, for Figure 2 the target chip 10 in, CH1-CH4 are used to receive multi-channel audio data, that is, the first audio data; CH5 is used to receive stereo audio data, that is, the second audio data. Stop the mixing process of the first audio data and the second audio data, that is to say, control the target chip 10 to stop the mixing process of the first audio data received by CH1-CH4 and the second audio data received by CH5.

[0131] It should be noted that in the embodiments of the present application, under multi-window display, stopping the mixing process of the first audio data and the second audio data indicates that the smart TV is in a state of supporting multiple audio data to be output through different audio devices.

[0132] It should be understood that in order to output the audio data corresponding to different windows in the embodiments of the present application through different audio devices, the smart TV needs to support multiple audio devices at the same time. The multiple audio devices may refer to a built-in speaker and at least one external audio output device, or multiple external audio output devices. Among them, the audio output device may be a Bluetooth speaker, a Bluetooth headset, etc.

[0133] It should also be understood that in the case where the multiple audio devices are implemented by multiple external audio output devices, the built-in speaker in the smart TV can still maintain the ability to output normal audio data.

[0134] Figure 8 Fig. shows an example diagram of audio control in the smart TV provided by the embodiments of the present application. The audio devices include a built-in speaker and a Bluetooth headset. Among them, CH1-CH4 are used to receive the first audio data; CH5 is used to receive the second audio data.

[0135] The smart TV is in a state of supporting multiple audio data to be output through different audio devices. In this example, the audio devices currently supported by the smart TV are a built-in speaker and a Bluetooth headset.

[0136] It should be understood that in some embodiments, before entering multi-window display, the smart TV already supports multiple audio devices, such as a built-in speaker and a Bluetooth headset, and only outputs audio data through a single audio device. For example, if the audio data is output through the built-in speaker, the Bluetooth headset is muted. If the audio data is output through the Bluetooth headset, the built-in speaker is muted.

[0137] Therefore, under multi-window display, it is necessary to stop the mixing process of the first audio data and the second audio data, and it is also necessary to unmute some of the audio devices.

[0138] In one case, such as Figure 8As shown in (a) of [the figure], for the first audio data received by CH1 - CH4, it is output through the Bluetooth headset; for the second audio data received by CH5, it is output through the built - in speaker. Among them, the mixing process of the first audio data received by CH1 - CH4 and the second audio data received by CH5 is stopped.

[0139] In another case, as Figure 8 shown in (b) of [the figure], for the first audio data received by CH1 - CH4, it is output through the built - in speaker; for the second audio data received by CH5, it is output through the Bluetooth headset. Among them, the mixing process of the first audio data received by CH1 - CH4 and the second audio data received by CH5 is stopped.

[0140] That is to say, under multi - window display, there are various situations where the audio data corresponding to each window is output through different audio devices. Therefore, it is necessary to further determine whether the audio data (such as the first audio data) corresponding to a window that is being played or to be played is output from the built - in speaker or from the Bluetooth headset.

[0141] S530. Determine the corresponding relationship between each window and the target audio device to obtain the target relationship.

[0142] It should be understood that the audio data corresponding to a window refers to the audio data that the window is playing or will play. Different window states result in different references to the corresponding audio data.

[0143] Under multi - window display, each window includes the first window and the second window.

[0144] It should be understood that the number of the first windows and the number of the second windows can be one or more. For example, the number of the first windows is 1 and the number of the second windows is 1; the number of the first windows is 1 and the number of the second windows is 3; the number of the first windows is 2 and the number of the second windows is 2.

[0145] In some embodiments, the window further includes a third window and even more.

[0146] As Figure 7 shown in (b) of [the figure], the first interface of the split - screen application is the main interface for two - way screen mirroring. Among them, the first window is the left window 41 and the second window is the right window 42. In this example, the media resources played by the left window 41 and the right window 42 are presented through the screen mirroring technology. Among them, both the left window and the right window display information such as the guiding picture, the Wi - Fi name accessed by the smart TV, and the name of the smart TV, which is convenient for users to perform screen mirroring.

[0147] It should be understood that the screen mirroring technology includes wired screen mirroring and wireless screen mirroring. Among them, wireless screen mirroring includes Wi-Fi screen mirroring. Wi-Fi screen mirroring is based on the Wi-Fi network to achieve the transmission of pictures and sounds between devices. For example, mirror screen mirroring and DLNA screen mirroring.

[0148] Among them, DLNA screen mirroring is a function implemented according to the Digital Living Network Alliance (DLNA) technology, which pushes the media resources on the terminal device to the smart TV in the form of a Uniform Resource Locator (URL) for playback.

[0149] The target audio device refers to an audio device that is communicatively connected to the smart TV through an audio output interface. The target audio device is used for audio data output. There are multiple target audio devices, including a first target audio device and a second target audio device. For example, Figure 8 the audio device shown in (a) in includes a built-in speaker and a Bluetooth headset.

[0150] In some embodiments, the target audio device further includes a third target audio device, and even more.

[0151] The target relationship is one of the following corresponding relationships: First, the corresponding relationship between the first window and the first target audio device, and between the second window and the second target audio device; Second, the corresponding relationship between the first window and the second target audio device, and between the second window and the first target audio device.

[0152] When the target audio device includes a built-in speaker and a Bluetooth headset, Figure 7 in the example shown in (b) in, the target relationship is one of the following corresponding relationships: First, the corresponding relationship between the left window and the built-in speaker, and between the right window and the Bluetooth headset; Second, the corresponding relationship between the left window and the Bluetooth headset, and between the right window and the built-in speaker.

[0153] It should be noted that the determination of the corresponding relationship (i.e., the target relationship) between the window and the target audio device depends on the analysis of the number of channels of the audio data corresponding to the window.

[0154] For example, when the number of channels of the audio data corresponding to the left window is multi-channel, the audio data corresponding to the left window belongs to the first audio data. In the case where the audio data (i.e., the first audio data) is output through the built-in speaker by the left window, combined with Figure 8 as shown in (b) in, it is determined that the target relationship is the corresponding relationship between the left window and the built-in speaker, and between the right window and the Bluetooth headset; In the case where the audio data (i.e., the first audio data) is output through the Bluetooth headset by the left window, combined with Figure 8As shown in (a) therein, determine the target relationship as the corresponding relationship between the left window and the Bluetooth headset, and between the right window and the built-in speaker.

[0155] For another example, when the number of channels of the audio data corresponding to the left window is stereo, the audio data corresponding to the left window belongs to the second audio data. In the case where the audio data (i.e., the second audio data) is output through the built-in speaker in the left window, in combination with Figure 8 As shown in (a) therein, determine the target relationship as the corresponding relationship between the left window and the built-in speaker, and between the right window and the Bluetooth headset; in the case where the audio data (i.e., the second audio data) is output through the Bluetooth headset in the left window, in combination with Figure 8 As shown in (b) therein, determine the target relationship as the corresponding relationship between the left window and the Bluetooth headset, and between the right window and the built-in speaker.

[0156] S540. According to the target relationship, respectively establish audio output channels between each window and the target audio device.

[0157] It should be understood that the determination of the target relationship depends on the analysis of the number of channels of the audio data corresponding to the window. Therefore, the number of channels of the audio data corresponding to the first window and the number of channels of the audio data corresponding to the second window are implied in the determined target relationship.

[0158] That is to say, according to the target relationship, and the number of channels of the audio data corresponding to the first window and the number of channels of the audio data corresponding to the second window in the target relationship, establish audio output channels between each window and the target audio device.

[0159] For example, establish an audio output channel between the first window and the first target audio device, and an audio output channel between the second window and the second target audio device, or establish an audio output channel between the first window and the second target audio device, and an audio output channel between the second window and the first target audio device.

[0160] In some embodiments, the first preset audio device is the first target audio data or the second target audio data, and the second preset audio device is a target audio device different from the first preset audio device. That is to say, if the first preset audio device is the first target audio data, then the second preset audio device is the second target audio data; if the first preset audio device is the second target audio data, then the second preset audio device is the first target audio data.

[0161] That is to say, according to the target relationship, and the number of channels of the audio data corresponding to the first window and the number of channels of the audio data corresponding to the second window in the target relationship, establish a first audio output channel between the first window and the first preset audio device, and a second audio output channel between the second window and the second preset audio device.

[0162] For example, when the number of channels of the audio data corresponding to the first window is multi-channel (i.e., the audio data corresponding to the first window belongs to the first audio data), the audio data corresponding to the first window is transmitted to the first pin for receiving the first audio data, and the audio data corresponding to the second window is transmitted to the second pin for receiving the second audio data (the number of channels is stereo). An audio output channel of the first preset audio device is established for the audio data of the first window to be transmitted through the first pin, and an audio output channel of the second preset audio device is established for the audio data of the second window to be transmitted through the second pin.

[0163] For another example, when the number of channels of the audio data corresponding to the first window is stereo (i.e., the audio data corresponding to the first window belongs to the second audio data), the audio data corresponding to the first window is transmitted to the second pin for receiving the second audio data, and the audio data corresponding to the second window is transmitted to the first pin for receiving the first audio data (the number of channels is multi-channel). An audio output channel of the first preset audio device is established for the audio data of the first window to be transmitted through the second pin, and an audio output channel of the second preset audio device is established for the audio data of the second window to be transmitted through the first pin.

[0164] For each window, a channel between the window and the corresponding audio device is established so that the audio data corresponding to the window can be output through the corresponding audio device; for example, in the example of Figure 7 (b), the left window 41 is the first window and the right window 42 is the second window; when the number of channels of the audio data corresponding to the first window (i.e., the left window 41) is multi-channel, the audio data corresponding to the left window 41 (i.e., the first audio data) is transmitted to the CH1-CH4 pins shown in Figure 8 (a) and shown in Figure 8 (b), and the audio data corresponding to the right window 42 (i.e., the second audio data) is transmitted to the CH5 pin shown in Figure 8 (a) and shown in Figure 8 (b).

[0165] When the first preset audio device is a built-in speaker and the second preset audio device is a Bluetooth speaker, referring to Figure 8 (b), control the first audio data received by the CH1-CH4 pins to be transmitted to the built-in speaker, and the second audio data received by the CH5 pin to be transmitted to the Bluetooth headset.

[0166] That is to say, establishing a first audio output channel between the left window 41 and the built-in speaker and a second audio output channel between the right window 42 and the Bluetooth headset can bring a more flexible and personalized audio experience to the user.

[0167] In one scenario, different usage requirements of users are met. For example, user A can simultaneously enjoy external audio through the built-in speaker in the left window to meet the sharing or convenient listening needs in an open environment; in the right window, use a Bluetooth headset to achieve private listening, avoiding disturbing others, which greatly improves the freedom of audio usage and scene adaptability.

[0168] In another scenario, the usage requirements of different users are met. For example, when user B watches a video in the left window, the video sound will be played through the built-in speaker for convenient direct listening; when user C watches a video in the right window, the video sound will be played through a Bluetooth headset to achieve private audio playback; and user B and user C do not interfere with each other and watch the video as needed.

[0169] In the embodiments of the present application, in multi-window display, first stop the mixing process of each audio data; then, according to the determined target relationship, establish a channel for audio data transmission between the window and the preset audio devices (i.e., the first preset audio device and the second preset audio device) to realize the transmission of audio data of different windows to different audio devices, thereby improving the audio-visual experience of users in multi-window display.

[0170] It should be understood that before the first operation, the smart TV usually outputs the mixed audio data through a single audio device. That is to say, before the first operation, the smart TV will mix the first audio data and the second audio data to obtain mixed data, and output the mixed data to a single audio device to play the sound.

[0171] In order to provide users with a coherent and comfortable audio-visual experience, after entering multi-window display, the mixed data will be output to the corresponding audio device as the first preset audio device corresponding to the first window.

[0172] Generally, the first window is the main window that is mainly displayed in multi-window display.

[0173] It should be understood that in the scenario of two windows side by side, such as Figure 5 shown in (a) of Figure 5 shown in (b) of

[0174] It should be understood that the first preset audio device is the audio device corresponding to the output of the mixing data before the smart TV responds to the first operation, and in the embodiments of the present application, the smart TV supports multiple audio devices at the same time, that is, the target audio device. Therefore, the first preset audio device is the first target audio device, or the first preset audio device is the second target audio device.

[0175] For example, before the first operation, the smart TV usually outputs the mixing data through the built-in speaker, and after entering the multi-window display, the built-in speaker is used as the first preset audio device corresponding to the first window.

[0176] It should be noted that the smart TV can send query instructions to the target audio device adapted to it through different connection methods such as Bluetooth, 3.5mm audio interface, HDMI interface, etc. With these interaction instructions, the smart TV can accurately determine the connection status of the target audio device and obtain relevant information of the target audio device, including the name, volume, channel mode, etc. of the audio device, which is convenient for the management of the target audio device. In the embodiments of the present application, the query or monitoring methods corresponding to different connection methods are not limited.

[0177] In some embodiments, in order to improve the accuracy of the determined first preset audio device, when entering the multi-window display, a query or monitoring of information such as the connection status of the target audio device will be triggered. If the result of the query or monitoring indicates that the audio device corresponding to the mixing data is still in the connected state, the audio device corresponding to the mixing data will be used as the first preset audio device.

[0178] For example, before entering the multi-window display, the audio device is a Bluetooth speaker, and after entering the multi-window display, the Bluetooth speaker is in the connected state, then the first preset audio device corresponding to the first window is the Bluetooth speaker.

[0179] For another example, before entering the multi-window display, the audio device is the built-in speaker, and after entering the multi-window display, the first preset audio device corresponding to the first window is the built-in speaker. It should be understood that in the normal operation state of the smart TV, the built-in speaker can continuously and stably output normal audio data. Therefore, in this scenario, after entering the multi-window display, the query or monitoring of information such as the connection status of the target audio device may not be triggered.

[0180] In some embodiments, the connection status, device information, etc. of the target audio device can be stored in the audio device list. When it is queried or monitored that the target audio device adapted to the smart TV has changed (such as the connection status changes, the audio device goes into standby, etc.), the audio device list is updated to facilitate the query and management of the target audio device.

[0181] It should be understood, see Figure 8 in (a), Figure 8As shown in (b) in [description], during the process of controlling the audio data transmitted to each audio device, CH1-CH4 are used to receive the first audio data; CH5 is used to receive the second audio data. Therefore, in order to determine the target relationship between each window in the multi-window and the corresponding audio device, it is also necessary to determine the number of channels corresponding to each window. Figure 9 FIG. [figure number] shows a schematic flow chart of another audio control method provided by an embodiment of the present application. For the above step 530, to determine the correspondence between each window and the target audio device and obtain the target relationship, as Figure 9 shown, the following steps are further included:

[0182] S531. Determine that the audio device corresponding to the first window is the first preset audio device, and the number of channels of the audio data corresponding to the first window is the first number of channels.

[0183] Among them, the first preset audio device is determined by the audio device output from the mixing data before entering the multi-window display.

[0184] It should be understood that the smart TV supports multi-channel and dual-channel audio data. In order to output the audio data of the first window and the second window through different audio devices respectively, it is necessary to make the number of channels of the audio data of the first window and the number of channels of the audio data of the second window different.

[0185] That is to say, when the first number of channels is multi-channel, adjust the number of channels of the audio data corresponding to the second window to dual-channel to obtain the second number of channels. When the first number of channels is dual-channel, adjust the number of channels of the audio data corresponding to the second window to multi-channel to obtain the second number of channels. Among them, the second number of channels is the number of channels of the audio data corresponding to the second window.

[0186] The number of channels of the audio data corresponding to the first window can be multi-channel or dual-channel. In order to improve the efficiency of audio data processing, the determination of the first number of channels can be divided into the following three cases:

[0187] (1) Before entering the multi-window display, the smart TV is playing the first media asset, and after entering the multi-window interface, continue to play the first media asset in the first window to keep the continuous playback of the first media asset.

[0188] In this case, determine the first number of channels according to the number of channels of the audio stream corresponding to the first media asset.

[0189] It should be understood that before entering the multi-window display, the audio device that plays the first media asset on the smart TV is used as the first preset audio device. In this scenario, the first window maintains the number of channels of the audio device that plays the first media asset and the audio data in the first media asset, reducing the data processing process. At the same time, it makes the first media asset displayed full-screen before entering the multi-window continuous with the first media asset displayed in the first window, and the played sound effects remain unchanged, providing users with a coherent and comfortable audio-visual experience.

[0190] (2) Before and after entering the multi-window display, when no media asset is being played, determine the first number of channels according to the preset number of channels.

[0191] Before entering the multi-window display, the smart TV does not play any media asset. After entering the multi-window display, it still does not play any media asset, but instead displays the interface shown in (b) of Figure 7 , the first interface of the split-screen application, also known as the guiding interface, where the left and right windows in the guiding page are displayed in equal proportion.

[0192] Therefore, when there is no media asset being played and no media asset to be played in the first window, the first number of channels can be determined through the preset number of channels.

[0193] It should be understood that the smart TV supports playing audio data with multiple different numbers of channels simultaneously, and the played sound. The preset number of channels supported by the smart TV includes multi-channel and stereo. The first number of channels can be flexibly determined to be stereo or multi-channel through random allocation, or based on historical data, user preferences, etc.

[0194] It should be understood that after determining the first number of channels, if the window receives a media asset to be played, the audio stream of the media asset needs to be adjusted to the corresponding first number of channels.

[0195] For example, if the determined first number of channels is stereo, and the number of channels in the audio stream of the media asset to be played in the first window is multi-channel, the audio stream of the media asset to be played needs to be adjusted to stereo.

[0196] (3) Before entering the multi-window display, when no media asset is being played, and after entering the multi-window display, when a second media asset is to be played in the first window, determine the first number of channels according to the number of channels of the audio stream corresponding to the second media asset.

[0197] When the number of channels of the audio stream corresponding to the second media asset is clear and unique, determine this number of channels as the first number of channels.

[0198] In some embodiments, multiple numbers of channels of the audio stream corresponding to the second media asset may coexist. The first number of channels can be determined according to the screen mirroring method corresponding to the second media asset. For example, if the second media asset for screen mirroring is pushed in the form of a URL, the number of channels of the audio data corresponding to the first window can be set to multi-channel; if the second media asset for screen mirroring is through mirror screen mirroring, the number of channels of the audio data corresponding to the first window can be set to stereo.

[0199] It should be understood that after screen mirroring, in the initially sent audio data (such as MP4 format or AAC format), the first few frames of this audio data will contain source format data in a pre-agreed format. Decoding these frames of data can obtain information such as the encoding format, sampling rate, and number of channels of the pushed source; therefore, for the second media asset to be played, the number of channels of the audio data of the second media asset can be obtained through its corresponding audio data, and the number of channels of the audio data corresponding to the first window can be determined from the number of channels of the audio data of the second media asset.

[0200] S532. Determine the first correspondence between the first window and the first preset audio device according to the first number of channels.

[0201] When the first number of channels of the audio data corresponding to the first window is multi-channel, the audio data corresponding to the first window belongs to the first audio data. When the first audio data is output through the first preset audio device in the first window, determine the first correspondence between the first window and the first preset audio.

[0202] When the first number of channels of the audio data corresponding to the first window is stereo, the audio data corresponding to the first window belongs to the second audio data. When the second audio data is output through the first preset audio device in the first window, determine the first correspondence between the first window and the first preset audio.

[0203] S533. Determine the second correspondence between the second window and the second preset audio device.

[0204] When the first number of channels of the audio data corresponding to the first window is multi-channel, the number of channels of the audio data corresponding to the second window is stereo, and the audio data corresponding to the second window belongs to the second audio data; when the second audio data is output through the second preset audio device in the second window, determine the second correspondence between the first window and the first preset audio.

[0205] When the first number of channels of the audio data corresponding to the first window is stereo, the number of channels of the audio data corresponding to the second window is multi-channel, and the audio data corresponding to the second window belongs to the first audio data; when the first audio data is output through the second preset audio device in the second window, determine the second correspondence between the first window and the first preset audio.

[0206] It should be understood that the target relationship includes a first corresponding relationship and a second corresponding relationship.

[0207] Figure 10 FIG. shows a schematic diagram of another application scenario of multi-window display provided by an embodiment of the present application. Based on (b) shown in Figure 7 it is a schematic diagram after further receiving the media assets for screen mirroring. The process presented from (b) in Figure 7 to (a) in Figure 10 corresponds to the situation described in the above case (3). After entering multi-window display, when the media asset A (i.e., the second media asset) to be played is in the left window 41, the number of channels of the audio data corresponding to the left window 41 is determined to be the first number of channels according to the number of channels of the audio stream corresponding to the media asset A; furthermore, the number of channels of the audio data corresponding to the second window is determined to be the second number of channels. Then, based on the fact that the number of channels of the audio data corresponding to the left window 41 is the first number of channels, the first corresponding relationship between the left window and the first preset audio device is determined, and based on the fact that the number of channels of the audio data corresponding to the right window is the second number of channels, the second corresponding relationship between the right window and the second preset audio device is determined.

[0208] Furthermore, through the target relationship, and the first number of channels and the second number of channels involved in the target relationship, a first audio output channel between the left window and the first preset audio device and a second audio output channel between the right window and the second preset audio device are established. Then, the media asset A is first played in the left window 41 (i.e., the main window), as shown in (a) in Figure 10 ; furthermore, after receiving the screen mirroring of the new media asset B, the media asset B is played in the right window 42, as shown in (b) in Figure 10 .

[0209] In the embodiment of the present application, multi-window display can be implemented in various scenarios. In each scenario, first, the first preset audio device corresponding to the first window and the first number of channels of the audio data corresponding to the first window are determined; since the audio data of the first window and the second window are output through different audio devices, the second preset audio device corresponding to the second window and the second number of channels of the audio data corresponding to the second window can be determined; furthermore, the determined target relationship; to achieve transmitting the audio data of different windows to different audio devices, this process improves the data processing efficiency in the smart TV and provides a coherent and comfortable audio-visual experience for the user.

[0210] In a possible case, Figure 11 FIG. shows a system architecture diagram of another display device provided by an embodiment of the present application, as shown in Figure 11As shown in the figure, the system architecture diagram of the display device includes a service module and an audio processing module. The service module includes a first application for multi-window display, such as a split-screen application. The audio processing module is used to control the target chip to process audio data, such as mixing processing, etc.

[0211] Among them, the service module is in the application layer of the system architecture, and the audio processing module is in the kernel layer of the system architecture.

[0212] Among them, AudioTrack is the main API (Application Programming Interface) for audio playback in the Android operating system. It can send audio data into the audio processing process of the system and finally play it through the audio device. For example, in a video playback application, AudioTrack can be used to play the decoded audio data (such as decoding MP3 to PCM audio data).

[0213] AudioTrack supports multiple audio formats, such as Pulse Code Modulation (PCM) format. PCM is a basic encoding method for converting analog audio signals into digital signals. It can handle different sampling rates (such as common 44.1kHz, 48kHz, etc.), channel numbers (stereo, multi-channel, etc.) and sample formats (such as 8bit, 16bit, etc.).

[0214] It should be understood that AudioTrack can also control the volume of the audio stream. It sends the audio data in the application to the audio device for playback, and during this process, it can adjust the volume of the audio data. For example, the volume control can be achieved by changing the sampling value of the audio data. In digital audio, the volume is proportional to the amplitude of the audio sampling value. By amplifying or reducing the sampling value, the volume of the audio can be changed.

[0215] Among them, AudioFlinger is the audio processing service in the Android operating system. It is mainly responsible for managing and processing operations such as routing and playback of audio data.

[0216] The application sends the audio data to AudioFlinger through the audio APIs (such as AudioTrack, MediaPlayer, etc.) of the Android operating system. After AudioFlinger processes these data, it passes the data to the audio hardware through the audio driver for playback.

[0217] It should be understood that audio data is output through Bluetooth connection, and corresponding audio data is obtained by controlling AudioPatch, and then this path of audio data is output to a Bluetooth headset or the like.

[0218] For the sake of convenience of description, in Figure 11 it also reflects the hardware layer that interacts with the above system structure. The hardware layer provides physical hardware support for the operating system. All software functions in the operating system rely on the support of the hardware layer to be effectively implemented, and the hardware layer interacts with the kernel layer through a driver to ensure smooth information interaction and precise control of physical hardware.

[0219] In the embodiments of the present application, it mainly involves built-in speakers in the hardware layer, audio output devices (such as Bluetooth speakers, Bluetooth headsets, etc.).

[0220] When the display device runs a split-screen application, according to the scenarios before and after entering multi-window display, determine the audio device corresponding to the main window and the number of channels for outputting audio data, and then determine the target relationship, and control the audio processing module to establish channels according to the target relationship. For example, realize the transmission of audio data corresponding to the first window to the built-in speaker, and the transmission of audio data corresponding to the second window to the Bluetooth headset, so as to realize the output of audio data corresponding to different windows through different audio devices.

[0221] Next, it will be described in detail through Figure 12 the embodiments shown below.

[0222] Figure 12 shows a schematic flowchart of another audio control method provided by the embodiments of the present application. As Figure 12 shown, the method includes:

[0223] S610. In response to the first operation, the service module generates a trigger message.

[0224] Among them, the trigger message is used to instruct the audio processing module to stop the mixing process of the first audio data and the second audio data.

[0225] In some embodiments, the trigger message is used to instruct the audio processing module to switch the control of the target chip from the mixing mode to the target mode; among them, in the mixing mode, the first audio data and the second audio data are mixed, and the mixed data after mixing is transmitted to a single audio device, which can be the first preset audio device or the second preset audio device; in the target mode, the mixing process of the first audio data and the second audio data is stopped, but the first audio data is transmitted to the first preset audio device, and the second audio data is transmitted to the second preset audio device.

[0226] Specifically, the trigger message is used to instruct the audio processing module to control the target chip to switch fromFigure 2 the pattern shown in (a) in Figure 8 either (a) in Figure 8 or the pattern shown in (b) in

[0227] It should also be understood that in response to the first operation, the interface for out-of-window display will also be entered.

[0228] S620. According to the preset message passing policy, the service module will trigger the message transmission to the audio processing module.

[0229] Among them, the preset message passing policy includes broadcasting, cross-process communication or encapsulated interface.

[0230] Broadcasting is a global message notification mechanism that allows applications to send and receive messages within the system. In some embodiments, in response to the first operation, the trigger message generated by the service module can be transmitted to the audio processing module in the form of broadcasting.

[0231] It should be understood that cross-process communication can be implemented through Android Interface Definition Language (AIDL), and AIDL is based on Binder. Through AIDL, communication between different applications can be achieved, such as the interaction between system services and applications, and between components of different applications.

[0232] In some embodiments, the interaction between the service module and the audio processing module can be implemented through AIDL, so that the service module transmits the trigger message to the audio processing module.

[0233] The encapsulated interface refers to the encapsulated native private interface. Among them, the native private interface is the interface used inside the Android operating system, and these native private interfaces can be packaged in some ways to make them safer and more controllable in the application. In some embodiments, the service module can transmit the trigger message to the audio processing module through the encapsulated interface. Through the encapsulated interface, for the application scenarios of the embodiments of the present application, better adaptation and control can be achieved through customization, and the functions and interactions that meet its requirements can be realized.

[0234] In the application scenario, the preset message passing policy can be selected according to compatibility requirements, development and maintenance costs, etc., so that the service module transmits the trigger message to the audio processing module.

[0235] In some embodiments, the trigger message may include a first audio configuration parameter and a second audio configuration parameter. Among them, the first audio configuration parameter is used to indicate the parameter for transmitting the audio data corresponding to the first window to the audio processing module, and the second audio configuration parameter is used to indicate the parameter for transmitting the audio data corresponding to the second window to the audio processing module.

[0236] Among them, the first audio configuration parameter includes the identifier of the first preset audio device and the number of the first channels; the second audio configuration parameter includes the identifier of the second preset audio device and the number of the second channels.

[0237] For example, the first audio configuration parameter and the second audio configuration parameter are transmitted to the audio processing module in the kernel layer of the system architecture through the AudioManager.setParameters interface; for example, the first audio configuration parameter and the second audio configuration parameter are transmitted to the audio processing module by implementing AudioManager.setParameters(speaker+mch) and AudioManager.setParameters(bt_e+2ch), where speaker represents that the identifier of the first preset audio device is the identifier of the built-in speaker, mch represents that the number of the first channels is multi-channel, bt_e represents that the identifier of the second preset audio device is the identifier of the Bluetooth headset, and 2ch represents that the number of the first channels is two-channel.

[0238] It should be understood that AudioManager is a class in the Android operating system for managing audio-related functions, which can be used for setting audio devices, querying the status of the current audio device, and controlling the volume of the audio device, etc.

[0239] In some embodiments, in an application scenario of multi-screen projection to achieve multi-window display, the number of channels in the audio stream corresponding to the first screen projection can be used to determine the number of the first channels of the audio data output by the first window; furthermore, according to the identifier of the first preset audio device corresponding to the first window and the number of the first channels, the first audio configuration parameter corresponding to the first window is determined, and according to the identifier of the second preset audio device corresponding to the second window and the number of the second channels, the second audio configuration parameter corresponding to the second window is determined.

[0240] In some embodiments, the trigger message may not include the first audio configuration parameter and the second audio configuration parameter, and these parameters will be transmitted separately after the trigger message is transmitted.

[0241] In some embodiments, in the audio processing module, the first audio configuration parameter or the second audio configuration parameter will be further parsed and passed. Among them, the first parameter is used to determine the preset audio device for outputting audio data, and the second parameter is used to determine the number of channels for playing the audio data corresponding to the preset audio device.

[0242] It should be understood that when there is a first parameter, it can be determined that in the application scenario of multi-window display, audio data corresponding to different numbers of channels is supported to be output through different audio devices.

[0243] For example, the first parameter is output_device and the second parameter is acap_source; when output_device = bt and acap_source = 2ch, it means that the audio data of two channels is output through the Bluetooth audio device; when output_device = bt and acap_source = mch, it means that the audio data of multiple channels is output through the Bluetooth audio device.

[0244] S630. The audio processing module stops mixing the first audio data and the second audio data.

[0245] S640. The audio processing module determines the corresponding relationship between each window and the target audio device according to the number of channels of the audio data corresponding to the window in the multi-window, and obtains the target relationship.

[0246] Among them, the window corresponding audio device and the number of channels of the audio data are sent to the audio processing module by the service module. For example, the first audio configuration parameter corresponding to the first window and the first audio configuration parameter corresponding to the second window are sent to the audio processing module through AudioManager.setParameters(speaker + mch) and AudioManager.setParameters(bt_e + 2ch).

[0247] After receiving the first audio configuration parameter and the second audio configuration parameter, the audio processing module determines the first corresponding relationship according to the first audio configuration parameter; determines the second corresponding relationship according to the second audio configuration parameter, and further obtains the target corresponding parameter according to the first corresponding parameter and the second corresponding parameter.

[0248] For example, when the audio processing module receives speaker + mch, it determines that the audio data corresponding to the first window belongs to the first audio data (the number of channels of the first audio data is multiple channels). When the first audio data is output through the built-in speaker in the first window, it determines the first corresponding relationship between the first window and the built-in speaker. When the audio processing module receives bt_e + 2ch, it determines that the audio data corresponding to the second window belongs to the second audio data (the number of channels of the second audio data is two channels). When the first audio data is output through the Bluetooth headset in the second window, it determines the second corresponding relationship between the second window and the Bluetooth headset.

[0249] S650. The audio processing module establishes a first audio output channel between the first window and the first preset audio device, and a second audio output channel between the second window and the second preset audio device according to the target relationship.

[0250] It should be understood that the audio processing module includes a first audio input interface and a second audio input interface. The first audio input interface is used to receive multi-channel audio data, and the second audio input interface is used to receive stereo audio data.

[0251] When the number of channels of the first channel is multi-channel, the audio data corresponding to the first window is transmitted to the first audio input interface, and the audio data corresponding to the second window is transmitted to the second audio input interface. Furthermore, according to the first correspondence relationship, the audio data of the first window is transmitted through the first audio input interface to the first audio output channel of the first preset audio device; according to the second correspondence relationship, the audio data of the second window is transmitted through the second audio input interface to the second audio output channel of the second preset audio device, as Figure 13 shown in (a) of

[0252] In addition, when the number of channels of the first channel is stereo, the audio data corresponding to the first window is transmitted to the second audio input interface, and the audio data corresponding to the second window is transmitted to the first audio input interface. Furthermore, according to the first correspondence relationship, the audio data of the first window is transmitted through the second audio input interface to the first audio output channel of the first preset audio device; according to the second correspondence relationship, the audio data of the second window is transmitted through the first audio input interface to the second audio output channel of the second preset audio device, as Figure 13 shown in (a) of

[0253] S660. Transmit the audio data corresponding to the first window to the first preset audio device through the first audio output channel, and transmit the audio data corresponding to the second window to the second preset audio device through the second audio output channel.

[0254] Through the established first audio output channel and second audio output channel, it is possible to play the audio data corresponding to the first window through the first preset audio device and play the audio data corresponding to the second window through the second preset audio device.

[0255] In the embodiment of the present application under multi-window display, in response to the first operation, the service module generates a trigger message and transmits the trigger message to the audio processing module, so that the audio processing module can stop the mixing process of each audio data and determine the target relationship; then, according to the determined target relationship, establish a channel for audio data transmission between the window and the preset audio device (i.e., the first preset audio device, the second preset audio device), so as to realize the transmission of audio data of different windows to different audio devices, and further improve the audio-visual experience of the user under multi-window display.

[0256] In Figure 10 such as (a) of Figure 10In the embodiment corresponding to the application scenario presented in (b) thereof, after step 650 and before executing step 660, the following steps (1)-(2) are further included:

[0257] (1) Under multi-window display, in response to a first screen mirroring instruction, play a first target media asset in a first window, and play the audio data of the first target media asset through a first audio output channel; and establish a session list.

[0258] Among them, the first screen mirroring instruction is used to indicate, under a multi-window display interface, when receiving an instruction for the first media asset to be played corresponding to the first path of screen mirroring (i.e., the first target media asset); that is to say, the first target media asset is to be played in the first window (main window).

[0259] It should be understood that the first window is the main window in the multi-window display interface. Therefore, when receiving the screen mirroring instruction for the first time, play the audio data of the first target media asset played in the first window through the first audio output channel, and a session list will be established.

[0260] It should be understood that the session list is used to record the attribute information of each terminal device mirroring the screen to the smart TV. Among them, the session list includes the attribute information of the first window, and the attribute information includes the terminal device information mirroring the screen to the display device and the display position information, where the display position information is used to indicate the layout of the window in the multi-window display interface.

[0261] In some embodiments, the attribute information further includes sound information.

[0262] For example, the terminal device information is the identifier of the terminal device, and the display position information includes the display position corresponding to the upper left corner of the window, the width and height of the window, etc.

[0263] (2) In response to a second screen mirroring instruction, play a second target media asset in a second window, and play the audio data of the second target media asset through a second audio output channel; and update the session list.

[0264] Among them, the second screen mirroring instruction is used to indicate, under a multi-window display interface, when receiving an instruction for the second media asset to be played corresponding to the second path of screen mirroring (i.e., the second target media asset); that is to say, the second target media asset is to be played in the second window.

[0265] It should be understood that when receiving the screen mirroring instruction for the second time, play the audio data of the second target media asset played in the second window through the second audio output channel, and update the session list. Among them, the updated session list includes the attribute information of the first window and the attribute information of the second window.

[0266] Under multi-window display, in response to a first screen mirroring instruction, a first target media asset is played in a first window, and the audio data of the first target media asset is played through a first audio output channel. And in response to a second screen mirroring instruction, a second target media asset is played in a second window, and the audio data of the second target media asset is played through a second audio output channel. During this period, a session list is established and updated. In the application scenario of screen mirroring and displaying multiple windows, the audio data of different windows is transmitted through corresponding channels to an audio device for playing, achieving the goal of outputting multiple windows from multiple audio devices. Furthermore, the audio-visual experience of the user under multi-window display is improved.

[0267] During the process of presenting multi-window content, the audio devices corresponding to each window can be dynamically adjusted. The user can perform modification operations such as switching and replacing audio devices. In this scenario, it can be achieved through methods such as a remote control, body buttons, voice input, and playback control buttons. For the sake of easy understanding, the switching of the audio device of the first window is taken as an example for description.

[0268] In response to an audio switching instruction for the first window, a list of audio devices currently supported by the display device is presented; in response to a selection instruction for the audio device list, the first preset audio device corresponding to the first window is switched to the third preset audio device corresponding to the selection instruction.

[0269] Among them, the third preset audio device can be the same audio device as the second preset audio device or a different audio device from the second preset audio device.

[0270] It should be understood that if the third preset audio device is the same audio device as the second preset audio device, it is necessary to switch the second preset audio device corresponding to the second window, or remind the user and guide the user to make the switch.

[0271] Figure 14 Shows a schematic diagram of another application scenario provided by an embodiment of the present application. It is in Figure 10 the scenario shown in (b) of Figure 10 A schematic diagram of further switching the audio devices corresponding to each window by means of a remote control, body buttons, voice input, playback control buttons, etc. In the scenario presented in (b) of

[0272] As Figure 14 shown in (a) of A playback control button group corresponding to the window is displayed below the left window 41. For example, the playback control button group sequentially includes volume down (the first playback control button), volume up (the second playback control button), audio device (the third playback control button), play / pause (the fourth playback control button), full screen display (the fifth playback control button), hide window (the sixth playback control button), close window (the seventh playback control button).

[0273] As shown Figure 14 in (a) of Figure 14 , the third playback control button in the playback control button group is the built-in speaker button 51, indicating that the audio device corresponding to this window is the built-in speaker; by selecting the built-in speaker button 51, a list 61 of audio devices currently supported by the smart TV is displayed; as shown Figure 14 in (b) of Figure 14 , the user can select and switch the audio device in the audio device list 61, such as switching the audio device corresponding to the left window from the built-in speaker to a Bluetooth speaker; as shown Figure 14 in (c) of Figure 14 , the third playback control button in the playback control button group below the left window 41 is the Bluetooth speaker button 52, indicating that the audio device corresponding to this window is the Bluetooth speaker.

[0274] It should be understood that if the audio device after switching in the left window 41 is the same as the audio device in the right window 42, a prompt message can be generated to inform the user that the audio devices of the two windows conflict, or the audio device in the right window 42 can be switched to a different audio device from that in the left window 41.

[0275] It should be understood that the switching operation of the audio device can also be directly performed on the second window. For the implementation process, reference can be made to the detailed description of the operation on the first window above, which will not be elaborated here.

[0276] It should also be understood that in some embodiments, if a connection interruption occurs in the audio device in use, resulting in no sound in the window corresponding to the audio data output through this audio device, within a preset time period, wait for the audio device to reconnect, or in the case of no reconnection, remind the user whether to automatically switch to other playable audio devices, or display a list of audio devices for the user to select independently.

[0277] The above examples mainly describe how to dynamically switch the control of audio devices. However, in the scenario of multi-window display, there are also cases where the window content or the number of channels changes. For example, when two video contents are played simultaneously or the screen mirroring content changes, how to control the audio devices of the screen mirroring contents on both sides. For example, if the left window presents media asset A and the right window presents media asset B, it can be replaced by media asset D, etc.

[0278] That is to say, there are cases where the user replaces or closes the first target media asset or the second target media asset being played. In some embodiments, it also includes updating the attribute information of the first window in the session list when it is monitored that the first target media asset played in the first window is replaced or closed; and updating the attribute information of the second window in the session list when it is monitored that the second target media asset played in the second window is replaced or closed.

[0279] It should be understood that in some embodiments, when it is monitored that the first target media asset played in the first window is replaced or closed, or when it is monitored that the second target media asset played in the second window is replaced or closed, the attribute information of the first window and the attribute information of the second window in the session list are updated.

[0280] For example, in the scenario presented in (b) of Figure 10 , if the user switches the screen mirroring of the second window to the mirror of the mobile phone, at this time, if the interface presented in (b) of Figure 10 is still used for display, the user experience will be poor. At this time, the interface presented in (b) of Figure 10 can be switched to the interface presented in (b) of Figure 5 , which is achieved by updating the attribute information of the first window and the attribute information of the second window in the session list.

[0281] It should be understood that in the process of implementing the above audio control, there may be more modules involved. The embodiments of the present application do not limit the types and uses of the involved modules.

[0282] In the process of the above modules executing audio control, the establishment of each window and the corresponding audio device in the multi-window display of the display device is realized, and the independent output of the audio data corresponding to each window is realized, without interference with each other, improving the user's audio-visual experience.

[0283] The above embodiments describe that in multi-window display, the audio data of different windows are output through different audio devices (such as speakers, external audio output devices); in the process of using a smart TV, there is a user's need to adjust the volume.

[0284] Usually, the audio device connected to the smart TV presents the volume size to the user in a quantified digital form, such as displaying a volume control bar containing digital scales from 0 to 100; however, in the scenario corresponding to the above embodiments, the audio data of different windows are output through different audio devices, and there are differences between different audio devices; these differences are caused by factors such as the volume characteristics, sound generation principles, and power of the audio devices; furthermore, when different audio devices adjust the same amount of gain, the auditory feelings brought to the user are different, that is, the corresponding volume increases are different.

[0285] In an application scenario where audio data is usually output through a single audio device, the volume value of the audio data played by a smart TV through an audio device (such as a built-in speaker or a Bluetooth speaker) can be adjusted by means of a remote control, body buttons, voice, etc. For example, currently, the smart TV plays audio data through the built-in speaker, and the volume value is 20. By pressing the volume button on the remote control, the volume of the smart TV is increased by 5 digital scales, and the adjusted volume value is 25. Another example is that currently, the smart TV plays audio data through a Bluetooth speaker, and the volume value is 20. By pressing the volume button on the remote control, the volume of the smart TV is increased by 5 digital scales, and the adjusted volume value is 25.

[0286] In the above two examples, due to the differences between audio devices, after adjustment, the volume of the speaker can meet the clear volume requirements for the user to watch videos in the current quiet environment, while the volume of the Bluetooth speaker fails to reach the clear volume requirements for the user to watch videos in this environment.

[0287] Due to the differences between audio devices, therefore, for the Bluetooth speaker, it is necessary to increase the volume value by 30 digital scales from 20 until the volume value reaches 50 to meet the clear volume requirements for the user to watch videos in the current quiet environment and achieve the same sound playback effect as the built-in speaker.

[0288] It should be understood that for different Bluetooth-connected audio devices, when the same volume value is adjusted, there may also be differences in the volume levels of the corresponding output sounds. For example, under the same conditions, the smart TV is respectively communicatively connected to Bluetooth speaker A and Bluetooth speaker B, and when playing audio data with the set volume value of 60, the sound played by Bluetooth speaker A is higher than the sound played by Bluetooth speaker B.

[0289] In an application scenario where in a multi-window display, the audio data of different windows is output through different audio devices, if the volume of the audio data played by the corresponding audio devices (such as built-in speakers or Bluetooth speakers) of the smart TV is adjusted simultaneously by means of a remote control, body buttons, voice, etc., it may cause the sound output by one of the audio devices to be too high or too low, resulting in failure to meet the user's needs, and even due to the too high sound, it may bring an uncomfortable auditory experience to the user.

[0290] For example, in a multi-window display scenario, the audio data corresponding to the first window is played through the built-in speaker, and the audio data corresponding to the second window is played through a Bluetooth speaker. The current volume value of the built-in speaker is 20, and the current volume value of the Bluetooth speaker is also 20. If the second user watching the second window feels that the sound played by the Bluetooth speaker is too low and increases the volume value to 50 by pressing the volume button on the remote control, the sound played by the Bluetooth speaker meets the auditory needs of the second user; correspondingly, the volume value of the built-in speaker is also adjusted to 50. For the first user watching the first window, when the volume value of the sound played by the built-in speaker is 20, it already meets the auditory needs of the first user. After adjustment, the sound played by the built-in speaker is too loud, resulting in poor experiences for both the first user and the second user.

[0291] In view of this, in a scenario where a display device involves multiple audio devices for outputting audio data, in order to meet the user's volume requirements for the played sound, the audio control method provided in the embodiments of the present application outputs the audio data corresponding to each window based on the target audio device corresponding to each window in a multi-window display, and further includes: when the volume adjustment type is the adjustment of the system volume, in response to a volume adjustment operation, obtaining the initial volume value of the target audio device corresponding to the target window in the multi-window; adjusting the volume of the target audio device corresponding to the target window to obtain a target volume value, thereby realizing separate adjustment of the target audio devices corresponding to different windows, improving the user's auditory experience, and meeting the user's needs.

[0292] The following combines Figures 15 to 23 In a scenario where, in a multi-window display, audio data corresponding to each window is output based on the target audio device corresponding to each window, a detailed description is given of the audio control method involving volume adjustment. Figure 15 FIG. shows a flowchart of another audio control method provided in the embodiments of the present application. This method is applied to a display device. Taking a smart TV as an example, as Figure 15 shown, the audio control method provided in the embodiments of the present application includes the following steps:

[0293] S710. In a multi-window display, output the audio data corresponding to each window based on the target audio device corresponding to each window.

[0294] For the implementation process of step 710, reference can be made to the detailed description of steps 510 to 540 and steps 610 to 660 above, and details are not repeated here.

[0295] S720. Determine the target window in the multi-window interface.

[0296] It should be understood that the multi-window interface refers to the interface corresponding to a multi-window display. In this multi-window interface, at least two windows will be presented, such as Figure 5 in (a), Figure 5 in (b) orFigure 5 As shown in (c) in [reference], two windows are presented in the window interface; for another example Figure 16 As shown in (a) in [reference], three windows, window 37, window 38, and window 39, are presented in the window interface; for another example Figure 16 As shown in (b) in [reference], four windows, window 81, window 82, window 83, and window 84, are presented in the window interface.

[0297] It should be understood that the target window is one of the multiple windows in the multi-window interface. The target window can be a window selected by the user or a default window. Among them, the default window can be the main window in the multi-window display.

[0298] In the scenario of multi-window display for screen mirroring, the default window can be the window corresponding to the first screen mirroring or the window corresponding to the last screen mirroring.

[0299] If the target window is a window selected by the user, the focus positioning operation on the multi-window interface in the smart TV can be performed by means of a control device (such as a remote control, a smartphone, etc.), body buttons, voice, touch, etc. The window in the multi-window interface that obtains the focus positioning operation becomes the currently operable window, that is, the target window.

[0300] Taking Figure 10 the scenario shown in (b) in [reference] as an example, if the focus positioning operation corresponds to the left window 41, it is determined that the target window is the left window 41, as Figure 17 shown in (a) in [reference]; if the focus positioning operation corresponds to the right window 42, it is determined that the target window is the right window 42, as Figure 17 shown in (b) in [reference].

[0301] Under the multi-window interface, the window selected by the user (i.e., the target window) is determined by analyzing the focus positioning operation. In Figure 17 (a) in [reference] is the multi-window interface of the split-screen application. In the split-screen application, the first marker can be used to record the window corresponding to the focus positioning selected by the user under the current multi-window interface.

[0302] For example, the multi-window interface includes a first window and a second window. When using FocusIndex to represent the first marker, when the position corresponding to the focus positioning operation belongs to the first window, FocusIndex is 0, and when the position corresponding to the focus positioning operation belongs to the second window, FocusIndex is 1. Furthermore, through FocusIndex, it can be determined whether the target window is the first window or the second window.

[0303] It should be understood that if the focus positioning operation changes and the window corresponding to the focus positioning involves switching, the corresponding target window will change. For example, the target window switches from the first window to the second window, and the corresponding first marker changes. If the focus positioning operation changes, but the window corresponding to the focus positioning does not switch, the corresponding first marker remains unchanged, and the target window remains unchanged.

[0304] S730. When the volume adjustment type is the adjustment of the system volume, in response to the volume adjustment operation, obtain the initial volume value of the target audio device corresponding to the target window.

[0305] It should be understood that for the volume adjustment of a smart TV to play sound through an audio device, the volume adjustment type mainly involves two types of volume adjustment, namely, the adjustment of the system volume and the adjustment of the media volume.

[0306] Among them, the adjustment of the system volume is to adjust the volume of the audio device and is realized through the system. The adjustment of the media volume is to adjust the volume of the played media and is realized through the application program.

[0307] It should be understood that for the user, regardless of whether the volume adjustment type is the adjustment of the system volume or the adjustment of the media volume, it is presented to the user through the audio device for playback; therefore, what the user is concerned about is the result of the volume adjustment, and the attention to the volume adjustment type is low. Therefore, the volume adjustment type can be determined by the system or the application program.

[0308] In some embodiments, the selection corresponding to the volume adjustment type can be presented in the multi-window interface. Furthermore, whether it is the adjustment of the system volume or the adjustment of the media volume can be determined by the user's selection of the audio adjustment type.

[0309] The volume adjustment operation includes a volume increase operation and a volume decrease operation, and can be realized by means of a control device (such as a remote control, a smart phone, etc.), a body button, voice, touch, a broadcast control button, etc. For example, the "Volume +" and "Volume -" on the remote control.

[0310] It should be understood that when the volume adjustment type is the adjustment of the system volume, the volume adjustment operation is to adjust the volume of the target audio device corresponding to the target window.

[0311] For example, Figure 17 As shown in (a) below, when the left window 41 is the target window, a group of broadcast control buttons corresponding to this window is displayed below the left window 41. Among them, the group of broadcast control buttons includes a first broadcast control button for volume decrease and a second broadcast control button for volume increase. The first broadcast control button is used to decrease the volume played by the first preset audio device corresponding to the left window 41, and the second broadcast control button is used to increase the volume played by the first preset audio device corresponding to the left window 41.

[0312] Figure 17 As shown in (b) in [reference], when the right window 42 is the target window, a set of broadcast control buttons corresponding to this window is displayed below the right window 42. Among them, the set of broadcast control buttons includes a first broadcast control button for volume reduction and a second broadcast control button for volume increase. The first broadcast control button is used to reduce the volume played by the second preset audio device corresponding to the right window 42, and the second broadcast control button is used to increase the volume played by the second preset audio device corresponding to the right window 42.

[0313] Referring to the above embodiments, it can be seen that the first preset audio device is one of the audio devices in the target audio device, and the second preset audio device is the other audio device in the target audio device.

[0314] It should be understood that the set of broadcast control buttons may also include other broadcast control buttons. For example, a third broadcast control button for the audio device, a fourth broadcast control button for play / pause, a fifth broadcast control button for full-screen display, a sixth broadcast control button for hiding the window, a seventh broadcast control button for closing the window, etc. It should be understood that in the embodiments of the present application, there are no restrictions on other broadcast control buttons.

[0315] It should be noted that before adjusting the volume of the target audio device in response to the volume adjustment operation, it is necessary to first obtain the initial volume value of the target audio device corresponding to the target window.

[0316] It should be understood that the initial volume value of the target audio device corresponding to the target window is obtained through a preset second interface. For different target audio devices, the corresponding preset second interfaces are different.

[0317] Figure 18 shows a system architecture diagram of another display device provided by the embodiments of the present application; as Figure 18 shown, the preset second interface refers to the speaker volume adjustment interface and the Bluetooth volume adjustment interface. Among them, the initial volume value of the built-in speaker corresponding to the first window is adjusted through the speaker volume adjustment interface, such as obtaining and setting; the initial volume value of the Bluetooth headset corresponding to the second window is adjusted through the Bluetooth volume adjustment interface, such as obtaining and setting.

[0318] For example, if the target audio device is a built-in speaker, the corresponding initial volume value is obtained through a private interface, such as: the getVolume interface; if the target audio device is a Bluetooth headset, the corresponding initial volume value is obtained through an Android native interface, for example: the getStreamVolume interface in AudioManager.

[0319] S740. Adjust the initial volume value of the target audio device corresponding to the target window to obtain the target volume value.

[0320] Among them, step 740 is also a process in response to a volume adjustment operation, including the following contents (1)-(3):

[0321] (1) Obtain the volume change value corresponding to the volume adjustment operation.

[0322] Among them, the volume change value refers to the number corresponding to the volume adjustment operation, generally N or -N, where N is a positive integer. It should be understood that usually, the volume change value is +1 or -1, that is, each volume adjustment operation can only increase the initial volume value by 1 or decrease it by 1. By continuously triggering the volume adjustment operation, the volume of the played sound gradually meets the user's needs.

[0323] (2) Determine the target volume value based on the initial volume value and the volume change value.

[0324] Among them, the target volume value is the sum of the initial volume value and the volume change value. For example, if the initial volume value is 20 and the changed volume value is 1, then the target volume value is 21. Another example, if the initial volume value is 20 and the changed volume value is -3, then the target volume value is 17.

[0325] (3) Call the preset second interface to adjust the volume of the target audio device corresponding to the target window to the target volume value.

[0326] As Figure 18 shown, through the preset second interface, set the volume value of the target audio device corresponding to the target window to the target volume value. For example, if the target audio device is the built-in speaker, the corresponding target volume value is adjusted through a private interface, such as: the setVolume interface; if the target audio device is a Bluetooth headset, the corresponding target volume value is adjusted through an Android native interface, such as: the setStreamVolume interface in AudioManager.

[0327] It should be understood that the digital scales of the volume division levels corresponding to the initial volume value, the volume change value, and the target volume value are the same. For example, the range of the digital scale corresponding to the volume is 0-100, where 0 is also mute, and the larger the digital scale, the higher the corresponding volume.

[0328] An embodiment of the present application provides an audio control method. In a scenario where multi-window display is performed and audio data corresponding to each window is output based on a target audio device corresponding to each window, a target window in the multi-window interface is determined. When the volume adjustment type is the adjustment of the system volume, in response to a volume adjustment operation, an initial volume value of the target audio device corresponding to the target window is obtained; the volume of the target audio device corresponding to the target window is adjusted to obtain a target volume value. That is to say, separate adjustment of the target audio devices corresponding to different windows is realized, improving the user's auditory experience and meeting the user's needs.

[0329] In some embodiments, after step 740, it further includes displaying a second volume control bar corresponding to the target volume value in the multi-window interface, where the second volume control bar displays an icon of the target audio device corresponding to the target window and the target volume value.

[0330] It should be understood that when the volume adjustment type is the adjustment of the system volume, the second volume control bar is displayed in the multi-window interface, generally located at the top or bottom of the multi-window interface, and an icon of the target audio device corresponding to the target window is displayed in the second volume control bar, used to distinguish whether the adjusted volume value is that of the first preset audio device or the second preset audio device.

[0331] Among them, the target volume value displayed in the second volume control bar can be presented by a volume bar, presented by a number, or presented in a combination of a volume bar and a number.

[0332] Figure 19 FIG. shows a schematic diagram of another application scenario of multi-window display provided by an embodiment of the present application; as Figure 19 shown in (a) therein, at this time the target window is the left window 41, and the second volume control bar displays an icon of the audio device corresponding to the left window 41, that is, the icon corresponding to the built-in speaker, as well as the volume bar and the number "20" corresponding to the target volume value. As Figure 19 described in (b) therein, through the second playback control button, a volume increase operation in the volume adjustment operation is performed. When the volume change value is 5, the second volume control bar displays the icon corresponding to the built-in speaker corresponding to the left window 41, as well as the volume bar and the number "25" corresponding to the target volume value.

[0333] Figure 20 FIG. shows a schematic diagram of another application scenario of multi-window display provided by an embodiment of the present application, as Figure 20 shown, at this time the target window is the right window 42, and the second volume control bar displays an icon of the audio device with Bluetooth communication corresponding to the right window 42, as well as the volume bar and the number "50" corresponding to the target volume value.

[0334] It should be understood that the icon of the target audio device corresponding to the target window displayed in the second volume control bar corresponds to the icon corresponding to the third playback control button under the target window. The user can, through the displayed second volume control bar, understand which target audio device the volume is being adjusted for and what the volume value of the sound played by the target audio device is, enhancing the visualization experience of the smart TV and improving the user's understanding of the relevant operations for volume adjustment.

[0335] In some embodiments, the volume adjustment type is the adjustment of the media volume. Figure 21 The flowchart of another audio control method provided by the embodiments of the present application is shown. As Figure 21 shown, the audio control method provided by the embodiments of the present application includes the following steps:

[0336] S810. Under multi-window display, based on the target audio device corresponding to each window, output the audio data of the corresponding window.

[0337] S820. Determine the target window in the multi-window interface.

[0338] For the implementation processes of step 410 and step 420, reference can be made to the detailed descriptions of the above steps 710 to 720, which will not be elaborated here.

[0339] S830. When the volume adjustment type is the adjustment of the media volume, in response to a volume adjustment operation, obtain the first volume value of the target media played in the target window.

[0340] Among them, the adjustment of the media volume is to adjust the volume of the played media, which is achieved through an application program.

[0341] It should also be understood that during the adjustment process of the media volume, the adjustment of the media volume will be converted into the volume corresponding to the audio device, and then played through the audio device. Therefore, in some embodiments, the common adjustment of the media volume and the system volume is involved, and the volume adjustment of the sound played by the audio device is jointly achieved through the two.

[0342] In some embodiments, the selection corresponding to the volume adjustment type can be presented in the multi-window interface. Furthermore, through the user's selection of the audio adjustment type, it can be determined whether it is the adjustment of the system volume or the adjustment of the media volume.

[0343] It should be understood that regardless of whether the volume adjustment type is the adjustment of the system volume or the adjustment of the media volume, the volume adjustment operation includes a volume increase operation and a volume decrease operation, and both can be achieved through a control device (such as a remote control, a smart phone, etc.), body buttons, voice, touch, playback control buttons, etc., such as "Volume +" and "Volume -" on the remote control.

[0344] It should be understood that when the volume adjustment type is the adjustment of the media asset volume, the volume adjustment operation is to adjust the volume of the target media asset played in the target window.

[0345] For example, Figure 17 As shown in (a) of, when the left window 41 is the target window, the playback control button group corresponding to this window is displayed below the left window 41. Among them, the playback control button group includes a first playback control button for reducing the volume and a second playback control button for increasing the volume. The first playback control button is used to reduce the volume of media asset A played in the left window 41, and the second playback control button is used to increase the volume of media asset A played in the left window 41.

[0346] Figure 17 As shown in (b) of, when the right window 42 is the target window, the playback control button group corresponding to this window is displayed below the right window 42. Among them, the playback control button group includes a first playback control button for reducing the volume and a second playback control button for increasing the volume. The first playback control button is used to reduce the volume of media asset B played in the right window 42, and the second playback control button is used to increase the volume of media asset B played in the right window 42.

[0347] It should be noted that before adjusting the volume of the target media asset in the target window in response to the volume adjustment operation, it is necessary to first obtain the first volume value of the target media asset. It should be understood that the first volume value of the target media asset is obtained through a preset first interface.

[0348] Figure 22 shows the system architecture diagram of another display device provided by the embodiment of the present application; as Figure 22 shown, the preset first interface refers to the media asset volume adjustment interface, and this media asset volume adjustment interface refers to the interface in the split-screen application program in the service module. Among them, the first volume value of the target media asset corresponding to the first window is adjusted through the media asset volume adjustment interface, such as obtaining and setting; the first volume value of the target media asset corresponding to the second window is adjusted through the media asset volume adjustment interface, such as obtaining and setting.

[0349] For example, the first volume value of the target media asset is obtained through the SessionActionInvoker.getPlayerVolume() interface in the screen mirroring corresponding session in the split-screen application program.

[0350] S840. Adjust the first volume value of the target media asset to obtain a second volume value.

[0351] It should be understood that adjusting the first volume value of the target media asset to obtain a second volume value is also a step corresponding to the volume adjustment operation. Among them, the volume adjustment operation includes a volume increase operation and a volume decrease operation.

[0352] The target media asset played in the target window is the media asset cast from the terminal device to the display device. After receiving the target media asset to be played, obtain the cast volume of the target media asset, and use the cast volume of the target media asset as the preset threshold.

[0353] Among them, the cast volume is obtained by parsing the audio-video stream corresponding to the target media asset. At this time, the obtained cast volume is the default value 1, that is to say, the preset threshold is 1. It should be understood that the range of the volume value corresponding to the target media asset is 0-1, where 0 is also mute, and the larger the number, the higher the corresponding volume. In order to correspond to the volume of the target audio device, the range of the volume value corresponding to the target media asset can be divided based on the preset volume grading. For example, the preset volume grading is 100 parts, and the range of the volume value corresponding to the target media asset is divided into 0, 0.01, 0.02,..., 0.99, 1, etc.

[0354] It should be noted that when obtaining the cast volume, there is a corresponding relationship, that is, a conversion coefficient, between the system volume of the target audio device corresponding to the target window and the cast volume. The conversion coefficient is the ratio of the system volume to the preset threshold.

[0355] Figure 23 The figure shows a schematic diagram of another application scenario of multi-window display provided by the embodiments of the present application. As Figure 23 shown in (a) therein, the cast volume corresponding to the left window 41 is 1. At this time, the volume value of the corresponding first preset audio device is 20, then the conversion coefficient = 20 / 1 = 20.

[0356] It should be understood that the target media asset for casting will ultimately process the audio through a player (such as IJK-player, mediacodec), and the corresponding sound object is AudioTrack. In AudioTrack, the split-screen application will adjust the volume value of the target media asset through the interface in the player.

[0357] It should be noted that when analyzing the first volume value, if the first volume value is not less than the preset threshold, it indicates that the volume of the target media asset can be reduced, but increasing is ineffective; if the first volume value is less than the preset threshold, it indicates that the volume of the target media asset can be reduced or increased.

[0358] It should be understood that when the first volume value is equal to the preset threshold, at this time the first volume value is 1, which is the maximum value in the range of the volume value corresponding to the target media asset. At this time, the volume corresponding to the target media asset cannot be increased, but can be reduced.

[0359] That is to say, when the first volume value is equal to the preset threshold, in response to the volume increase operation, maintain the first volume value of the target media asset; in response to the volume decrease operation, reduce the first volume value of the target media asset to the second volume value.

[0360] When the first volume value is less than the preset threshold, in response to a volume increase operation, increase the first volume value of the target media asset to a second volume value; in response to a volume decrease operation, decrease the first volume value of the target media asset to a second volume value.

[0361] Among them, step 840 is also a process in response to a volume adjustment operation, including the following contents (1)-(3):

[0362] (1) Obtain the volume change value corresponding to the volume adjustment operation.

[0363] Among them, the volume change value refers to the number corresponding to the volume adjustment operation, generally N or -N, and N is a positive integer. It should be understood that usually, the volume change value is +1 or -1, that is, each volume adjustment operation can only increase the initial volume value by 1 or decrease it by 1. By continuously triggering the volume adjustment operation, the volume of the played sound gradually meets the user's needs.

[0364] It should be understood that the volume change value is determined by the volume adjustment operation. Therefore, regardless of whether the volume adjustment type is the adjustment of the system volume or the adjustment of the media asset volume, the volume change value is the same number.

[0365] (2) Based on the first volume value, the volume change value, and the preset volume grading, determine the second volume value.

[0366] Among them, the target change value is obtained by converting the volume change value through the preset volume grading to adapt to the first volume value. The volume change value can be divided by the preset volume grading to obtain the target change value. For example, when the volume change value is 5 and the preset volume grading is 100, the target change value = 5 / 100 = 0.05.

[0367] The second volume value is the sum of the first volume value and the target change value. For example, when the first volume value is 0.2 and the changed volume value is 5, the second volume value is 0.25. Another example, when the first volume value is 0.2 and the changed volume value is -3, the second volume value is 0.17.

[0368] It should be understood that when the volume adjustment operation is a volume increase operation and the first volume value is not less than the preset threshold, the volume value of the target media asset remains unchanged, that is, the first volume value of the target media asset is determined as the second volume value.

[0369] (3) The target application calls the preset first interface to adjust the volume of the target media asset to the second volume value.

[0370] Such as Figure 22As shown, the target application sets the volume of the target media asset to a second volume value through a preset first interface. This refers to the media asset volume adjustment interface, which is an interface in the split-screen application in the service module. For example, the second volume value of the target media asset is set through the SessionActionInvoker.setPlayerVolume() interface in the screen mirroring corresponding session in the split-screen application.

[0371] S850. Adjust the volume of the target audio device corresponding to the target window according to the second volume value to obtain a target volume value.

[0372] It should be understood that the volume value corresponding to the target media asset is displayed through the target audio device corresponding to the target window. Therefore, it is necessary to convert the volume value corresponding to the target media asset into the volume value of the target audio device corresponding to the target window. Step 850 includes the following contents (1)-(3):

[0373] (1) Obtain the system volume of the target audio device corresponding to the target window.

[0374] It should be understood that in the embodiment of the present application, the system volume of the target audio device corresponding to the target window is the volume when receiving the screen-mirrored media asset. Therefore, there is a corresponding relationship between the system volume of the target audio device corresponding to the target window and the screen-mirroring volume.

[0375] (2) Determine a conversion coefficient based on the ratio of the system volume to a preset threshold.

[0376] (3) Calculate the product of the second volume value and the conversion coefficient to obtain the target volume value, and adjust the volume of the target audio device corresponding to the target window to the target volume value.

[0377] Among them, the target volume value = the second volume value × the conversion coefficient = the second volume value × (system volume / preset threshold).

[0378] For example, when the second volume value is 0.25 and the conversion coefficient is 20, the corresponding target volume value is 5. Another example, when the second volume value is 1 and the conversion coefficient is 20, the corresponding target volume value is 20.

[0379] An embodiment of the present application provides an audio control method. In a scenario where multiple windows are displayed and audio data corresponding to each window is output based on a target audio device corresponding to each window, a target window in the multi-window interface is determined. When the volume adjustment type is the adjustment of the media volume, in response to a volume adjustment operation, a first volume value of a target media played in the target window is obtained, where the target window is one of the windows in the multi-window display; the first volume value of the target media is adjusted to obtain a second volume value, and the volume of the target audio device corresponding to the target window is adjusted according to the second volume value to obtain a target volume value. That is to say, separate adjustment of the target audio devices corresponding to different windows is realized, improving the user's auditory experience and meeting the user's needs.

[0380] In some embodiments, after step 840 of adjusting the first volume value of the target media to obtain a second volume value, it further includes displaying a first volume control bar corresponding to the second volume value in the target window, where the first volume control bar displays an icon of the target audio device corresponding to the target window and the second volume value.

[0381] It should be understood that when the volume adjustment type is the adjustment of the media volume, the first volume control bar is displayed in the target window, generally at the top or side of the target window, and an icon corresponding to the second volume value of the target media is displayed in the first volume control bar.

[0382] Among them, the second volume value displayed in the first volume control bar can be presented by a volume bar, presented by a number, or presented in a combination of a volume bar and a number. Among them, the number can be a percentage corresponding to the second volume value.

[0383] As Figure 23 shown in (a) of, at this time the target window is the left window 41, and a first volume control bar is displayed at the top in the left window 41. The first volume control bar displays an icon of the audio device corresponding to the left window 41, that is, the icon corresponding to the built-in speaker, and a volume bar and the number "20%" corresponding to the second volume value. As Figure 23 described in (b) of, a first volume control bar is displayed on the side in the left window 41. The first volume control bar displays an icon of the audio device corresponding to the left window 41, that is, the icon corresponding to the built-in speaker, and a volume bar corresponding to the second volume value.

[0384] In some embodiments, after step 850 of adjusting the volume of the target audio device corresponding to the target window according to the second volume value to obtain a target volume value, it further includes displaying a first volume control bar corresponding to the target volume value in the target window, where the first volume control bar displays an icon of the target audio device corresponding to the target window and the target volume value.

[0385] It should be understood that the icon of the target audio device corresponding to the target window displayed in the first volume control bar corresponds to the icon of the third playback control button under the target window. Through the displayed first volume control bar, the user can understand which target audio device the volume is adjusted for and what the volume value of the sound played by the target audio device is, enhancing the visualization experience of the smart TV and improving the user's understanding of the relevant operations of volume adjustment.

[0386] For the purpose of illustration rather than limitation, specific details such as specific system architectures, technologies, etc. are presented to provide a thorough understanding of the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from hindering the description of the present application.

[0387] In the present application, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or its similar expressions refer to any combination of these items, including any combination of single item or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, c can be single or multiple.

[0388] It should be understood that in various embodiments of the present application, the magnitudes of the sequence numbers of the above processes do not mean the order of execution is prior or posterior. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0389] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0390] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated herein.

[0391] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative; for example, the division of units is only a logical function division, and there can be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces, and the indirect coupling or communication connection of devices or units can be in electrical, mechanical or other forms.

[0392] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0393] In addition, in each embodiment of the present application, the functional units can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0394] The above is only the specific implementation manner of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art within the technical scope disclosed in the present application can easily think of changes or substitutions, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope of the claims.

Claims

1. A display device, characterized in that, Comprising: A display; An audio output interface configured to communicatively connect with a plurality of target audio devices for outputting audio data; A controller configured to: Under multi-window display, output audio data of a corresponding window based on the target audio device corresponding to each window; When the volume adjustment type is for adjusting the media volume, in response to a volume adjustment operation, obtain a first volume value of a target media played in a target window, where the target window is one of the windows in the multi-window display; Adjust the first volume value of the target media to obtain a second volume value, and adjust the volume of the target audio device corresponding to the target window according to the second volume value to obtain a target volume value.

2. The display device according to claim 1, wherein After the controller adjusts the first volume value of the target media to obtain a second volume value, it is further configured to: In the target window, display a first volume control bar corresponding to the second volume value, where the first volume control bar displays an icon of the target audio device corresponding to the target window and the second volume value.

3. The display device according to claim 1, characterized in that, After the controller outputs the audio data of the corresponding window based on the target audio device corresponding to each window, and before obtaining the first volume value of the target media played in the target window in response to a volume adjustment operation when the volume adjustment type is for adjusting the media volume, it is further configured to: In response to a focus positioning operation in the multi-window interface, determine the target window in the multi-window interface.

4. The display device according to any one of claims 1, characterized in that, If the target media is a media cast from a terminal device to the display device, after the controller outputs the audio data of the corresponding window based on the target audio device corresponding to each window under multi-window display, it is further configured to: Obtain the cast volume of the target media, and use the cast volume of the target media as a preset threshold.

5. The display device according to any one of claims 1 to 4, characterized in that, After the controller adjusts the first volume value of the target media to obtain a second volume value, it is further configured to: When the volume adjustment operation is a volume increase operation and the first volume value is not less than the preset threshold, determine the first volume value of the target media as the second volume value.

6. The display device according to any one of claims 1 to 4, characterized in that, When the multi-window display is presented by a target application program, after the controller adjusts the first volume value of the target media to obtain a second volume value, it is configured to: Obtain a volume change value corresponding to the volume adjustment operation; Based on the first volume value, the volume change value, and a preset volume grading, determine the second volume value; The target application program calls a preset first interface to adjust the volume of the target media to the second volume value.

7. A display device, characterized in that, Comprising: A display; An audio output interface configured to communicatively connect with a plurality of target audio devices for outputting audio data; A controller configured to: Under multi-window display, output audio data of a corresponding window based on the target audio device corresponding to each window; When the volume adjustment type is for adjusting the system volume, in response to a volume adjustment operation, obtain an initial volume value of the target audio device corresponding to the target window, where the target window is one of the windows in the multi-window display; Adjust the initial volume value of the target audio device corresponding to the target window to obtain a target volume value.

8. The display device according to claim 7, wherein After the controller adjusts the initial volume value of the target audio device corresponding to the target window to obtain a target volume value, it is further configured to: In a multi-window interface, display a second volume control bar corresponding to the target volume value, where the second volume control bar displays an icon of the target audio device corresponding to the target window and the target volume value.

9. An audio control method, characterized in that, Applied to the display device according to any one of claims 1 to 6, the method includes: Under multi-window display, output audio data of the corresponding window based on the target audio device corresponding to each window; When the volume adjustment type is the adjustment of media volume, in response to a volume adjustment operation, obtain a first volume value of the target media played in the target window, where the target window is one of the windows in the multi-window display; Adjust the first volume value of the target media to obtain a second volume value, and adjust the volume of the target audio device corresponding to the target window according to the second volume value to obtain a target volume value.

10. An audio control method, characterized in that, Applied to the display device according to any one of claims 7 to 8, the method includes: Under multi-window display, output audio data of the corresponding window based on the target audio device corresponding to each window; When the volume adjustment type is the adjustment of system volume, in response to a volume adjustment operation, obtain the initial volume value of the target audio device corresponding to the target window, where the target window is one of the windows in the multi-window display; Adjust the initial volume value of the target audio device corresponding to the target window to obtain a target volume value.

Images