Audio playing method, display device and storage medium

Abstract

The invention relates to the technical field of smart home, particularly provides an audio playing method, display equipment and a storage medium, and aims to solve the technical problem that the audio playing effect of existing display equipment is poor. In order to achieve the purpose, the audio playing method comprises the steps of obtaining original audio data in response to an audio playing instruction; determining an audio format based on the original audio data; determining target audio data based on the audio format and the original audio data; and distributing the target audio data to the first audio device and the at least one second audio device so as to play the target audio data through the first audio device and the at least one second audio device. Therefore, the audio playing effect is improved.

Description

Technical Field

[0001] The present invention relates to the field of smart home technology, and specifically provides an audio playback method, a display device and a storage medium. Background Art

[0002] With the extensive application of Bluetooth technology, Bluetooth functions have been added to some display devices such as smart phones, speakers, and televisions.

[0003] At present, TVs and smart speakers communicate and connect via Bluetooth. The primary disadvantage of Bluetooth connection is that the sound and picture are not synchronized. In the movie-watching scenario, users have extremely high requirements for the consistency of sound and picture. The inherent delay of Bluetooth connection and its susceptibility to the wireless environment of the TV may result in a delay of 350-500ms. In addition, since the Bluetooth protocol can only transmit two channels, it is naturally unable to meet the wireless transmission of 5.1 channels or 7.1 channels, resulting in poor audio playback. Summary of the invention

[0004] In order to overcome the above defects, the present application is proposed to provide a solution or at least partially solve the technical problem of poor audio playback effect of existing display devices. The present application provides an audio playback method, a display device and a storage medium.

[0005] In a first aspect, the present application provides an audio playback method, the method comprising:

[0006] Responding to the audio playback instruction, acquiring original audio data;

[0007] determining an audio format based on the original audio data;

[0008] determining target audio data based on the audio format and the original audio data;

[0009] The target audio data is distributed to a first audio device and at least one second audio device, so as to play the target audio data through the first audio device and the at least one second audio device.

[0010] In one embodiment of the present application, determining the audio format based on the original audio data includes:

[0011] Decoding the original audio data to obtain multi-channel data;

[0012] Acquire channel information of the original audio data based on the multi-channel data;

[0013] The audio format is determined based on the channel information.

[0014] In one embodiment of the present application, determining the target audio data based on the audio format and the original audio data includes:

[0015] Decoding the original audio data to obtain multi-channel data;

[0016] Expanding the multi-channel data based on the audio format;

[0017] The expanded audio data is distributed to different channels to obtain the target audio data.

[0018] In one embodiment of the present application, distributing the target audio data to the first audio device and at least one second audio device includes:

[0019] Acquire location information of at least one of the second audio devices;

[0020] The target audio data is distributed to the first audio device and at least one of the second audio devices based on the position information.

[0021] In one embodiment of the present application, the display device includes an application program interface and an audio service module; the target audio data includes first audio data and second audio data, wherein the first audio data is audio data corresponding to a center channel and a bass channel, and the second audio data is audio data corresponding to other channels;

[0022] The distributing the target audio data to the first audio device and at least one second audio device comprises:

[0023] The audio service module sends the first audio data to the first audio device through the application program interface;

[0024] The second audio data is sent by the audio service module to at least one of the second audio devices.

[0025] In one embodiment of the present application, before obtaining the original audio data in response to the audio playback instruction, the method further includes:

[0026] Get device configuration information;

[0027] The device configuration information is sent to at least one of the second audio devices, so that the second audio device is networked based on the device configuration information.

[0028] In one embodiment of the present application, the method further includes:

[0029] Obtaining first volume information of a display device;

[0030] The first volume information is sent to at least one of the second audio devices, so that the at least one of the second audio devices adjusts the volume of the second audio device according to the first volume information.

[0031] In one embodiment of the present application, the method further includes:

[0032] Acquire second volume information of at least one of the second audio devices;

[0033] adjusting the volume of the display device based on the second volume information;

[0034] The second volume information is sent to other second audio devices, so that the other second audio devices adjust the volume of the second audio devices according to the second volume information.

[0035] In a second aspect, a display device is provided, comprising:

[0036] at least one processor;

[0037] and, a memory communicatively coupled to the at least one processor;

[0038] Wherein, a computer program is stored in the memory, and the computer program is the aforementioned audio playing method when executed by the at least one processor.

[0039] In a third aspect, a computer-readable storage medium is provided, wherein a plurality of program codes are stored in the computer-readable storage medium, wherein the program codes are suitable for being loaded and run by a processor to execute any of the aforementioned audio playback methods.

[0040] The above one or more technical solutions of this application have at least one or more of the following Beneficial effects:

[0041] The audio playback method in the present application includes: obtaining original audio data in response to an audio playback instruction; determining an audio format based on the original audio data; determining target audio data based on the audio format and the original audio data; distributing the target audio data to a first audio device and at least one second audio device, so as to play the target audio data through the first audio device and at least one second audio device. In this way, the audio playback effect is improved, and a multi-channel format can be realized, thereby providing users with a real immersive audio experience and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The disclosure of the present application will become more easily understood with reference to the accompanying drawings. It is easy for those skilled in the art to understand that these drawings are only for illustrative purposes and are not intended to limit the scope of protection of the present application. In addition, similar numbers in the drawings are used to represent similar components, among which:

[0043] Figure 1 This is a schematic diagram of the hardware environment of the audio playback method in one embodiment of the present application;

[0044] Figure 2 This is a schematic diagram of the main process of the audio playback method in one embodiment of the present application;

[0045] Figure 3 This is a schematic diagram of the Android system software structure in one embodiment of the present application;

[0046] Figure 4 is a schematic diagram of the structure of an audio playback system in one embodiment of the present application;

[0047] Figure 5 This is a schematic diagram of the process of networking a television and a speaker in one embodiment of the present application;

[0048] Figure 6 is a schematic diagram of the structure of an audio playback device in one embodiment of the present application;

[0049] Figure 7 It is a schematic diagram of the structure of a display device in one embodiment of the present application. DETAILED DESCRIPTION

[0050] Some embodiments of the present application are described below with reference to the accompanying drawings. Those skilled in the art should understand that these embodiments are only used to explain the technical principles of the present application and are not intended to limit the protection scope of the present application.

[0051] In the description of the present application, "module" and "processor" may include hardware, software or a combination of the two. A module may include hardware circuits, various suitable sensors, communication ports, memory, and may also include software parts, such as program code, or a combination of software and hardware. The processor may be a central processing unit, a microprocessor, an image processor, a digital signal processor or any other suitable processor. The processor has data and / or signal processing functions. The processor may be implemented in software, hardware or a combination of the two. Non-temporary computer-readable storage media include any suitable medium that can store program code, such as a disk, a hard disk, an optical disk, a flash memory, a read-only memory, a random access memory, etc. The term "A and / or B" means all possible combinations of A and B, such as only A, only B or A and B. The term "at least one A or B" or "at least one of A and B" has a similar meaning to "A and / or B" and may include only A, only B or A and B. The singular terms "one" and "the" may also include plural forms.

[0052] Currently, smart TVs and smart speakers communicate via Bluetooth, which leads to technical problems such as poor audio playback and a poor user experience.

[0053] To this end, the present application proposes an audio playback method, a display device and a storage medium.

[0054] According to one aspect of an embodiment of the present application, an audio playback method, a display device, and a storage medium are provided. The audio playback method, display device, and storage medium are widely used in smart home (Smart Home), smart home, smart home device ecology, smart residential (Intelligence House) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the above-mentioned audio playback method, display device, and storage medium can be applied to Figure 1 In the hardware environment composed of the terminal device 10 and the server 20 shown in FIG. Figure 1 As shown, the server 20 is connected to the terminal device 10 via a network, and can be used to provide services (such as application services, etc.) for the terminal or a client installed on the terminal. A database can be set on the server or independently of the server to provide data storage services for the server 20. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 20.

[0055] The network may include but is not limited to at least one of the following: wired network, wireless network. The wired network may include but is not limited to at least one of the following: wide area network, metropolitan area network, local area network, and the wireless network may include but is not limited to at least one of the following: WIFI (Wireless Fidelity), Bluetooth. The terminal device 10 may be but is not limited to a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing equipment, a smart dishwasher, a smart projection equipment, a smart TV, a smart clothes drying rack, a smart curtain, a smart audio and video, a smart socket, a smart speaker, a smart speaker, a smart fresh air equipment, a smart kitchen and bathroom equipment, a smart bathroom equipment, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purification equipment, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.

[0056] See attached Figure 2 , Figure 2 It is a flowchart of the main steps of an audio playback method according to an embodiment of the present application.

[0057] The audio playback method in the present application can be applied to a display device, wherein the display device includes a first audio device, which may include a first speaker. The display device is communicatively connected to at least one external second audio device (for example, via a WIFI connection), and the second audio device may include a second speaker.

[0058] The display device includes a display screen, which is a physical screen used for input and display of the display device. It allows the user to control the first display device through icons, buttons and menus on the display screen.

[0059] The display device described in the embodiments of the present application may be a smart TV, a mobile phone, a tablet computer, a personal computer (PC), a personal digital assistant (PDA), a smart watch, a netbook, a wearable display device, an augmented reality (AR) device, a virtual reality (VR) device, a vehicle-mounted device, a smart car, a robot, smart glasses, and other devices with a display screen and capable of realizing a display function. The present application does not impose any special restrictions on the specific form of the display device.

[0060] In the embodiment of the present application, the external second audio device can be a smart speaker or other audio device that can realize the audio playback function. The present application does not impose any special restrictions on the specific form of the audio device.

[0061] like Figure 2 As shown, the audio playback method in the embodiment of the present application mainly includes the following steps S10 to S40.

[0062] Step S10: acquiring original audio data in response to the audio playing instruction.

[0063] An audio playback instruction is a command or signal issued by a user or system to instruct a display device to start playing audio. The audio playback instruction may be triggered by a user interface (such as a remote control button, a touch screen, or a voice command), or may be the result of automatic execution of a program.

[0064] Audio playback instructions usually include playback source, playback mode, and playback options. The playback source specifies the location of the audio file or stream to be played. The playback mode includes single loop, random playback, and other modes. The playback options may include volume settings, equalizer settings, and so on.

[0065] Raw audio data refers to the audio information in an audio file or audio stream that has not been processed in any way. Raw audio data contains the original sample values ​​of the audio signal, usually represented in digital form. Raw audio data can be uncompressed (such as PCM data) or compressed (such as MP3, AAC and other formats).

[0066] Step S20: Determine the audio format based on the original audio data.

[0067] Audio format refers to a specific standard or protocol for storing or transmitting audio data. Audio formats can include dual-channel, 5.1-channel, 7.1-channel, etc.

[0068] Step S30: Determine target audio data based on the audio format and the original audio data.

[0069] The target audio data refers to the audio data prepared for playback after audio processing.

[0070] Step S40: Distribute the target audio data to the first audio device and the at least one second audio device, so as to play the target audio data through the first audio device and the at least one second audio device.

[0071] Based on the above steps S10 to S40, the original audio data is obtained by responding to the audio playback instruction; the audio format is determined based on the original audio data; the target audio data is determined based on the audio format and the original audio data; the target audio data is distributed to the first audio device and at least one second audio device, so as to play the target audio data through the first audio device and at least one second audio device. In this way, it is conducive to achieving synchronization between video and audio, and improving the audio playback effect. Surround sound formats such as 5.1 channels and 7.1 channels can be realized, thereby providing users with a real immersive audio experience and improving the user experience.

[0072] The above steps S10 to S40 are further explained below.

[0073] Before step S10, the method further includes: acquiring device configuration information; and sending the device configuration information to at least one second audio device, so that the second audio device is networked based on the device configuration information.

[0074] The device configuration information may be configuration information obtained by the user after the smart home application combines the display device and at least one second audio device, such as the display device ID, the enable switch of the second audio device, and the channel of the audio device.

[0075] The enable switch of the second audio device refers to the switch state of whether the second audio device is networked. When the enable switch is turned on, the second audio device can be networked, and when the enable switch is closed, the second audio device cannot be networked.

[0076] The channels of an audio device refer to independent audio signals collected or played back at different spatial locations when the sound is played, such as the left channel, right channel, left front channel, right rear channel, center channel, and bass channel.

[0077] Specifically, the display device can send the device configuration information to the cloud, which stores the device configuration information and sends the device configuration information to the corresponding second audio device for networking.

[0078] Before responding to audio playback instructions and obtaining raw audio data, first perform appropriate networking based on device configuration information to optimize the audio playback experience, simplify user operations, enhance system compatibility, and improve system stability.

[0079] The above is a further description of step S10 , and the following is a further description of step S20 .

[0080] Specifically, the above step S20 can be implemented through the following steps S201 to S203.

[0081] Step S201: Decode original audio data to obtain multi-channel data.

[0082] Decoding is the process of converting compressed and encoded raw audio data into uncompressed audio data. The decoded audio data contains data of multiple channels, which are the output of the decoding process. For audio data in 2-channel, 5.1-channel or 7.1-channel format, the corresponding number of channel data will be obtained after decoding.

[0083] Decoding usually involves using a corresponding audio decoder to process the audio data. For example, if the original audio data is AAC encoded, then an AAC decoder is needed to decode the data into PCM format.

[0084] Step S202: Acquire channel information of original audio data based on multi-channel data.

[0085] Channel information refers to the number of channels included in the audio data.

[0086] Specifically, the channel information of the original audio data can be determined according to the multi-channel data. For example, if the audio data obtained after parsing includes a left front channel, a center channel, a right front channel, a left surround channel, a right surround channel and a bass channel, then the number of channels is 6.

[0087] Step S203: Determine the audio format based on the channel information.

[0088] Specifically, the specific audio format can be determined according to the number of channels. For example, when the number of channels is 6, the corresponding audio format is a 5.1 channel format.

[0089] By decoding the original audio data, obtaining channel information and determining the audio format, the flexibility and efficiency of audio playback can be improved, the user's operation process can be simplified, the audio playback experience can be optimized, the system's compatibility and stability can be enhanced, and the user experience can be ultimately improved.

[0090] The above is a further description of step S20 , and the following is a further description of step S30 .

[0091] Specifically, the above step S30 can be implemented through the following steps S301 to S303.

[0092] Step S301: Decode the original audio data to obtain multi-channel data.

[0093] Decoding is the process of converting compressed and encoded raw audio data into uncompressed audio data. The decoded audio data contains data of multiple channels, which are the output of the decoding process. For audio data in 2-channel, 5.1-channel or 7.1-channel format, the corresponding number of channel data will be obtained after decoding.

[0094] Step S302: Expand the multi-channel data based on the audio format.

[0095] Specifically, taking the audio format as an example, the current TV and speaker devices have been networked in 5.1 channel format, but the decoded data is in dual channel format. At this time, the virtual roll-up technology can be used to expand the multi-channel data and convert the multi-channel data into 5.1 channel format data. The virtual roll-up technology is a common method for audio data processing and will not be described here.

[0096] Step S303: distribute the expanded audio data to different channels to obtain target audio data.

[0097] Specifically, the expanded audio data is allocated to the corresponding channels, for example, the audio data corresponding to the left front channel is allocated to the left front channel, and the audio data corresponding to the right front channel is allocated to the right front channel.

[0098] The above is a further description of step S30 , and the following is a further description of step S40 .

[0099] Specifically, the above step S40 can be implemented through the following steps S401 to S402.

[0100] Step S401: Acquire location information of at least one second audio device.

[0101] The position information refers to the specific position of the second audio device in space. The position information may include the position coordinates, distance, angle, etc. of the second audio device relative to the display device.

[0102] Specifically, the user may manually input the location information of each second audio device in the settings; alternatively, the display device or the second audio device may automatically detect the location of the second audio device through a sensor or wireless communication technology.

[0103] Step S402: Distribute the target audio data to the first audio device and at least one second audio device based on the location information.

[0104] Specifically, according to the position information of the second device, it is determined to which second audio device the data of each channel should be sent. For example, the data of the left surround channel should be sent to the second audio device located on the left, the data of the right surround channel should be sent to the second audio device located on the right, and the audio data of the center channel and the bass channel should be sent to the first audio device in the display device.

[0105] The above is a further description of step S40.

[0106] like Figure 3 The Android system software structure shown in the figure is composed of the application layer, the application framework layer and the hardware abstraction layer (HAL) from top to bottom. In addition, between the program framework layer and the HAL layer, there are also the Android runtime (Android Runtime) and the system library ( Figure 3 not shown).

[0107] The application layer can include a series of application packages, such as gallery, photo album, music player and video player.

[0108] The application framework layer is referred to as the framework layer for short. The framework layer provides a programming framework for the applications in the application layer. The framework layer includes some predefined functions.

[0109] like Figure 3 As shown, the framework layer in this embodiment includes an audio service module (TplayTV Service).

[0110] Other managers may also be included in the framework layer, such as window managers, content providers, view systems, phone managers, and notification managers.

[0111] In addition, Android Runtime includes core libraries and virtual machines. Android Runtime is responsible for scheduling and management of the Android system. Specifically, the core library contains two parts: one part is the function functions that need to be called by the Java language, and the other part is the Android core library.

[0112] The system library may include multiple functional modules, such as surface manager, media library, 3D graphics processing library (such as OpenGL ES), 2D graphics engine (such as SGL), etc.

[0113] The hardware abstraction layer (HAL) is an interface layer between the operating system kernel and the hardware circuit, and its purpose is to abstract the hardware. It hides the hardware interface details of a specific platform and provides a virtual hardware platform for the operating system, making it hardware-independent and portable on multiple platforms. From the perspective of software and hardware testing, the testing of software and hardware can be completed based on the hardware abstraction layer, making it possible to conduct software and hardware testing in parallel.

[0114] In this embodiment, the hardware abstraction layer HAL includes a virtual audio module, namely, a virtual audio (VirtualAudio) HAL. Among them, the VirtualAudio HAL can be used to connect to distributed virtual audio devices. The hardware abstraction layer HAL also includes a target application program interface (TplayTV API) for implementing the forwarding function of related audio data.

[0115] In a specific embodiment of the present application, the display device includes a target application interface and an audio service module; the target audio data includes first audio data and second audio data, wherein the first audio data is audio data corresponding to a center channel and a bass channel, and the second audio data is audio data corresponding to other channels; distributing the target audio data to the first audio device and at least one of the second audio devices includes: the audio service module sends the first audio data to the first audio device through the application interface; and the audio service module sends the second audio data to at least one of the second audio devices.

[0116] The first audio data is audio data corresponding to a center channel and a bass channel.

[0117] The second audio data is audio data corresponding to other channels (taking a 5.1 channel format as an example, such as a left front channel, a right front channel, a left rear channel, and a right rear channel, etc.).

[0118] Specifically, in this embodiment, the first audio device in the display device (for example, the local speaker of the display device) is used to play the audio data corresponding to the center channel and the bass channel, and the second audio device plays the audio data corresponding to other channels. Specifically, the audio service module sends the first audio data to the first audio device on the display device via the target application interface, so that the first audio device of the display device plays the audio data corresponding to the center channel and the bass channel. And the audio service module sends the second audio data to the corresponding second audio device through WIFI or other means, so that the second audio device plays the audio data corresponding to other channels.

[0119] By sending different channel data to corresponding speakers, the audio playback experience can be optimized. The audio data of the center channel and bass channel are played through the local speakers of the display device (the first audio device), which can ensure that the key information of these channels is clearly expressed. The audio data of other channels are played through the second audio device, which can provide richer surround sound effects, so that users can obtain rich audio playback effects.

[0120] In a specific embodiment of the present application, the method further includes: obtaining first volume information of a display device; sending the first volume information to at least one of the second audio devices, so that at least one of the second audio devices adjusts the volume of the second audio device according to the first volume information.

[0121] The first volume information refers to the volume of the audio data played on the display device.

[0122] Specifically, the audio service module can obtain the volume of the display device and send the volume of the display device to the second audio device that is networked with the display device. The second audio device can synchronize the volume according to the volume of the display device, so that the volume adjustment between the display device and the second audio device remains coordinated and consistent, ensuring a smooth transition of the volume change of the entire audio system and avoiding volume mismatches between different devices. Through unified volume control, the volume level of all speakers can be ensured to be consistent, thereby providing a better audio experience.

[0123] In a specific embodiment of the present application, the method also includes: obtaining second volume information of at least one second audio device; adjusting the volume of the display device based on the second volume information; and sending the second volume information to other second audio devices so that the other second audio devices adjust the volume of the second audio devices according to the second volume information.

[0124] The second volume information is the volume level of a certain audio device.

[0125] Specifically, when the user directly changes the volume of the second audio device, the audio service module receives the second volume information and changes the volume of the display device according to the second volume information, and the audio service module sends the second volume information to other external audio devices to synchronize the volume of other external audio devices according to the second volume information.

[0126] By keeping the volume adjustment between different second audio devices consistent, the volume change of the entire audio system is ensured to be smooth, avoiding the volume mismatch between different devices, helping to maintain the volume balance of the entire audio system and providing a better audio experience. Through unified volume control, the volume level of all speakers can be ensured to be consistent, providing a better audio experience, helping to avoid sound distortion or disharmony caused by volume differences, and improving audio playback effects.

[0127] In one embodiment of the present application, the audio playback method of the present application is described in detail by taking a smart TV as an example of the display device and a smart speaker as an example of the second audio device. It should be understood that the smart TV should not be limited to the display device, and the smart speaker should not be limited to the second audio device.

[0128] Specific as Figure 4 As shown, for multi-channel immersive viewing scenarios, a local wireless network can be used to build an audio playback system including a TV system, smart speakers and cloud servers. Specifically, an audio synchronization control module TplayTV (including TPlayTV API and TplayTV Service) is added to the original smart TV to achieve a multi-channel, 360-degree sound field sound experience. The entire system is distributed across two terminals, TV and smart speakers, and control between terminal devices is achieved through cloud services. The audio data can be transmitted directly over the Wi-Fi wireless network.

[0129] The smart speaker includes an audio playback module (Tplay) to meet the combined playback between speakers. The speaker's Tplay is equivalent to a virtual sound card based on the Ethernet network. The Tplays of different speakers are combined into stereo or multi-room playback mode according to the instructions of the cloud. The audio data is distributed to the corresponding speakers according to the specified flow mechanism for playback.

[0130] The audio synchronization control module TplayTV in the smart TV includes an audio service module TplayTV Service and a target application program interface TplayTV API.

[0131] TplayTV Service is a background service program in the application framework layer and has independent process management. When receiving a combination command, it will be activated by the TV application (such as Haier Smart Home Application). The TplayTV background process after activation is similar to the Tplay background process in the speaker.

[0132] At the same time, TplayTV Service can only have one active process. Before the combination ends, it is necessary to ensure that the process is valid and can obtain sufficient and smooth system resources. The TplayTV Service process requires a maximum of 500K of running memory. Due to the real-time processing of audio streams, it is necessary to obtain a stable CPU resource of about 15%. That is, in the combination, other application processes of the TV system cannot monopolize CPU resources (100%) for more than 20ms.

[0133] TplayTV Service can also receive the audio data stream of TV playback forwarded by Tplay API, and forward it to the Tplay of the combined speakers according to the corresponding rules.

[0134] TPlayTV API runs in the underlying environment of the TV system, such as the Android Audio system. It provides an interface for the Android system to forward audio data streams to the Tplay Service by calling the API interface. According to the needs of the application scenario, TPlayTV API will convert and customize the audio data stream that the TV itself needs to play while forwarding the audio stream. When playing native stereo audio and video content, the TV's own audio stream data will not be changed. When playing native 5.1 surround sound audio and video content, the TV's own center channel audio data is converted through the data of the TplayTV API.

[0135] In order to reduce system overhead, TPlayTV API will forward and convert only after the speaker combination is successful. TPlayTV API and TPlay Service belong to different processes, and the communication between them is carried out through the system named channel (name pipe). The named channel exists in the form of a file and needs to meet the permission requirements of TPlay Service (TPlay Service belongs to the application level permission, and TPlayTV API belongs to the system level permission). If TPlay Service cannot open or establish a named channel due to insufficient permissions, it will affect the subsequent audio forwarding.

[0136] In addition to audio data, the volume of the TV will also be synchronized to the TPlay Service through the TPlayTV API, and then forwarded to the smart speaker. If the volume of the smart speaker is manually changed, the new volume value also needs to be synchronized to the TV through the TPlay Service.

[0137] The audio formats supported by TPlayTV include stereo (2-channel), 5.1-channel, and 7.1-channel. The audio data bit number supported by TPlayTV is 16 bits. The audio data sampling rate supported by TPlayTV is 48000 Hz. The audio data format supported by TPlayTV is PCM interleaved.

[0138] The Smart Home application on the TV side can control the combination and stop of the TV and speaker devices, and can view the current combination status. The basic combination process includes the following steps: bind the speakers of the same user in the same LAN; the user operates the TV to select the speaker and the corresponding channel and combination mode, and clicks on the combination; the cloud accepts the combination application and initiates the combination command to the TV and the relevant speakers respectively; after receiving the combination command, the TPlay Service is started and loaded, waiting for the speaker combination; after receiving the combination command, the speaker TPlay is started and loaded, and the combination operation begins; the speaker TPlay and the TV TPlayService report the combination results respectively.

[0139] After the smart TV and smart speaker are successfully combined, TPlayAPI forwards the TV audio data to TPlayService, which processes the audio data to obtain the target audio data. TPlayAPI forwards the audio data played by the local speaker (first audio device) to the local speaker of the TV, and the audio data played by the smart speaker (second audio device) is distributed to the corresponding smart speaker through TPlay Service.

[0140] When the TV starts forwarding audio data, it will automatically synchronize the volume to keep the volume of the TV and the speakers relatively consistent. During audio playback, if the user changes the volume of the TV, the new volume data will be forwarded to the TPlay Service through the TPlayAPI and synchronized to all the combined speakers in real time. When the speaker TPlay receives the new volume data, it will set it to the current volume.

[0141] In addition, during the combined playback process, if the user changes the volume of a certain speaker, the new volume number will be synchronized to the TV's TPlay Service, and the TPlay Service will adjust the TV's volume accordingly. The TPlay Service will synchronize the volume to all the combined speakers in real time. When the speaker TPlay receives the new volume number, it will be set to the current volume.

[0142] In the smart speaker and TV set, the original smart home network construction system is used to expand the ability of TVs and speakers to distribute wireless streaming media based on WiFi; the multi-channel (including 5.1 and 7.1 channels) format surround experience and sound and picture synchronization consistency brought by WiFi wireless distribution are far superior to the two-channel audio distribution experience built by Bluetooth.

[0143] Figure 5 1 is a flow chart of networking of a smart TV and a smart speaker in one embodiment of the present application. Figure 5 As shown, when the user starts the TV and speaker combination function, the TV APP requests the speaker combination list from the cloud, and the cloud returns the LAN speaker combination list (including only online speakers) to the TV APP.

[0144] The user selects the speaker device and channel from the list and clicks to create a combination. The TV APP sends configuration information to the TV, including the TV's unique ID and the speaker enable switch status (for example, turn on the enable switch); the TV receives the configuration information and the enable switch status, and returns the status setting result. The TV APP sends configuration information to the speaker, including the unique ID, the speaker enable switch status, and the channel information; the speaker receives the configuration information, turns on the enable switch, and returns the status setting result. The TV APP determines whether the speaker combination is successful. If successful, the speaker DeviceID name and channel information are saved; if it fails, the timeout mechanism is triggered and the configuration timeout fails. Exemplarily, the TV APP determines whether the combination status of the four speakers and the TV is all on. If so, proceed to the next step; if not, save the speaker DeviceID name and channel information and end the process.

[0145] The user can also send a command to turn off the speaker enable switch through the TV; the TV APP sends a command to turn off the enable switch to the TV; the TV receives the command, turns off the enable switch, and returns the status setting result; the TV APP notifies the user that the enable switch has been turned off.

[0146] It should be pointed out that although the various steps in the above embodiments are described in a specific order, those skilled in the art can understand that in order to achieve the effect of the present application, different steps do not have to be executed in such an order. They can be executed simultaneously (in parallel) or in other orders. These changes are within the scope of protection of the present application.

[0147] Furthermore, the present application also provides an audio playback device.

[0148] See attached Figure 6 , Figure 6 FIG. 1 is a main structural block diagram of an audio playback device according to an embodiment of the present application. Figure 6 As shown, the audio playback device in the embodiment of the present application mainly includes an acquisition module 11, a first determination module 12, a second determination module 13 and a distribution module 14. In some embodiments, one or more of the acquisition module 11, the first determination module 12, the second determination module 13 and the distribution module 14 can be combined into one module.

[0149] In some embodiments, the acquisition module 11 may be configured to acquire original audio data in response to an audio playback instruction.

[0150] The first determination module 12 may be configured to determine an audio format based on the original audio data.

[0151] The second determination module 13 may be configured to determine the target audio data based on the audio format and the original audio data.

[0152] The distribution module 14 may be configured to distribute the target audio data to the first audio device and the at least one second audio device, so as to play the target audio data through the first audio device and the at least one second audio device.

[0153] In one implementation, the description of the specific implementation functions can refer to steps S10 to S40.

[0154] The above-mentioned audio playback device is used to perform Figure 2 The audio playback method embodiment shown in the figure has similar technical principles, technical problems solved and technical effects produced. Technical personnel in this technical field can clearly understand that for the convenience and conciseness of description, the specific working process and related instructions of the audio playback device can refer to the contents described in the embodiment of the audio playback method, which will not be repeated here.

[0155] Further, it should be understood that since the setting of each module is only for illustrating the functional units of the device of the present application, the physical devices corresponding to these modules may be the processor itself, or a part of the software in the processor, a part of the hardware, or a part of the combination of software and hardware. Therefore, the number of each module in the figure is only schematic.

[0156] It is understood by those skilled in the art that each module in the device can be adaptively split or merged. Such splitting or merging of specific modules will not cause the technical solution to deviate from the principle of the present application, and therefore, the technical solutions after splitting or merging will fall within the protection scope of the present application.

[0157] It is understood by those skilled in the art that all or part of the processes in the method for implementing the above-mentioned embodiment of the present application can also be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable storage medium may include: any entity or device, medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory, random access memory, electrical carrier signal, telecommunication signal and software distribution medium, etc. that can carry the computer program code.

[0158] Furthermore, the present application also provides a display device, which may include at least one processor; and a memory connected to the at least one processor; wherein a computer program is stored in the memory, and when the computer program is executed by the at least one processor, the audio playback method described in any of the above embodiments is implemented. Figure 7 As shown, Figure 7 , the structure of a display device is exemplarily shown in FIG. , which includes a processor 100 , a memory 200 and a display screen 300 .

[0159] Furthermore, the present application also provides a computer-readable storage medium. In a computer-readable storage medium embodiment according to the present application, the computer-readable storage medium can be configured to store a program for executing the audio playback method of the above-mentioned method embodiment, and the program can be loaded and run by the processor to implement the above-mentioned audio playback method. For ease of explanation, only the parts related to the embodiment of the present application are shown. For specific technical details not disclosed, please refer to the method part of the embodiment of the present application. The computer-readable storage medium can be a memory device formed by various display devices. Optionally, the computer-readable storage medium in the embodiment of the present application is a non-temporary computer-readable storage medium.

[0160] So far, the technical solutions of the present application have been described in conjunction with the specific implementations shown in the accompanying drawings, but it is easy for those skilled in the art to understand that the protection scope of the present application is obviously not limited to these specific implementations. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the protection scope of the present application.

Claims

1. An audio playback method, characterized in that: The method comprises: Responding to the audio playback instruction, acquiring original audio data; determining an audio format based on the original audio data; determining target audio data based on the audio format and the original audio data; The target audio data is distributed to a first audio device and at least one second audio device, so as to play the target audio data through the first audio device and the at least one second audio device.

2. The audio playback method according to claim 1, characterized in that: The determining of the audio format based on the original audio data comprises: Decoding the original audio data to obtain multi-channel data; Acquire channel information of the original audio data based on the multi-channel data; The audio format is determined based on the channel information.

3. The audio playback method according to claim 1, characterized in that: Determining target audio data based on the audio format and the original audio data includes: Decoding the original audio data to obtain multi-channel data; Expanding the multi-channel data based on the audio format; The expanded audio data is distributed to different channels to obtain the target audio data.

4. The audio playback method according to claim 1, characterized in that: The distributing the target audio data to the first audio device and at least one second audio device comprises: Acquire location information of at least one of the second audio devices; The target audio data is distributed to the first audio device and at least one of the second audio devices based on the position information.

5. The audio playback method according to claim 4, characterized in that: The display device includes a target application program interface and an audio service module; the target audio data includes first audio data and second audio data, wherein the first audio data is audio data corresponding to a center channel and a bass channel, and the second audio data is audio data corresponding to other channels; The distributing the target audio data to the first audio device and at least one second audio device comprises: The audio service module sends the first audio data to the first audio device through the target application program interface; The second audio data is sent by the audio service module to at least one of the second audio devices.

6. The audio playback method according to claim 1, characterized in that: Before acquiring the original audio data in response to the audio playing instruction, the method further includes: Get device configuration information; The device configuration information is sent to at least one of the second audio devices, so that the second audio device is networked based on the device configuration information.

7. The audio playback method according to claim 1, characterized in that: The method further comprises: Obtaining first volume information of a display device; The first volume information is sent to at least one of the second audio devices, so that the at least one of the second audio devices adjusts the volume of the second audio device according to the first volume information.

8. The audio playback method according to claim 1, characterized in that: The method further comprises: Acquire second volume information of at least one of the second audio devices; adjusting the volume of the display device based on the second volume information; The second volume information is sent to other second audio devices, so that the other second audio devices adjust the volume of the second audio devices according to the second volume information.

9. A display device, characterized in that: include: Display screen; at least one processor; and, a memory communicatively coupled to the at least one processor; The memory stores a computer program, and when the computer program is executed by the at least one processor, the audio playback method according to any one of claims 1 to 8 is implemented.

10. A computer-readable storage medium storing a plurality of program codes, characterized in that: The program code is suitable for being loaded and run by a processor to execute the audio playing method according to any one of claims 1 to 8.

Images