Audio playback control method and system, and device and medium

Abstract

The present application relates to the technical field of computers. Provided are an audio playback control method and system, and a device and a medium. The method is applied to a playback control device in an audio playback system, wherein the system further comprises a first device, and the first device is in communication connection with a second device by means of the playback control device. The method comprises: acquiring a playing instruction from a second device, wherein the playing instruction carries an audio stream to be played, and the playing instruction is used for instructing the playing of the audio stream; and in response to the playing instruction, outputting to a first device an audio signal corresponding to the audio stream, wherein the first device is used for playing a sound on the basis of the audio signal. By means of the method, the compatibility of audio playback devices is improved, and the audio playback effect is enhanced.

Description

An audio playback control method, system, device and medium

[0001] This application claims priority to Chinese Patent Application No. 202411596972.1, filed on November 8, 2024, entitled "An Audio Playback Control Method, System, Device and Medium", the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of computer technology, and in particular to an audio playback control method, system, device and medium. Background Technology

[0003] In smart home environments, as people increasingly pursue a higher quality of life, audio playback has become widely used in our daily lives. Taking home entertainment systems as an example, people are paying more and more attention to the playback quality of these systems. Therefore, how to create a high-quality playback experience in smart homes has become a current focus of audio playback technology. Summary of the Invention

[0004] This application provides an audio playback control method, system, device, and medium to improve the compatibility of audio playback devices and enhance the audio playback effect.

[0005] To achieve the above objectives, the embodiments of this application adopt the following technical solutions:

[0006] In a first aspect, embodiments of this application provide an audio playback control method, which is applied to a broadcast control device in an audio playback system. The system also includes a first device, such as an audio device. The first device is communicatively connected to a second device (such as a device in a smart home, a smart TV, a smart assistant, etc.) through the broadcast control device. The method includes: First, the broadcast control device obtains a playback instruction from the second device. The playback instruction carries an audio stream to be played, such as sound from a video, sound from a voice call, etc., and is used to instruct the playback of the audio stream. Then, in response to the playback instruction, the broadcast control device outputs an audio signal corresponding to the audio stream to the first device. The first device is used to play sound according to the audio signal.

[0007] As can be seen from the above embodiments, on the one hand, compared to the second device being directly connected to the first device via a high-definition multimedia interface cable (HDMI), in this application, the broadcast control device, acting as an intermediary between the second and first devices, can obtain playback instructions from the second device and respond to those instructions by sending audio signals corresponding to the audio stream to the first device. Even if the first and second devices have differences in audio transmission protocols, ports, etc., the broadcast control device can still achieve compatibility between the first and second devices, thereby enhancing the compatibility between them. On the other hand, by using the broadcast control device as an intermediary, the flexible communication connection between the first and second devices allows the audio playback system to flexibly add or remove either the first or second device without being limited by the number of interfaces on the device, improving the scalability and flexibility of the audio playback system, thereby further enhancing the user experience of the audio playback system.

[0008] In one possible implementation, the broadcast control equipment and the second equipment are connected via a network cable.

[0009] In this implementation, on the one hand, the second device communicates with the broadcast control device via a network cable, meaning that audio stream transmission between the second device and the broadcast control device is based on a network transmission protocol. Since most second devices are equipped with network cable interfaces such as Ethernet interfaces, and network transmission protocols are more universal than audio transmission protocols, the connection method between the second device and the broadcast control device provided in this application is more universal, compatible, and practical than the connection method based on HDMI cables. On the other hand, compared to a wireless connection between the broadcast control device and the second device, a network cable offers higher stability and reliability. Transmitting the audio stream to be played via a network cable is unaffected by wireless signal interference, ensuring data integrity and accuracy, and reducing the risk of playback interruption or sound quality degradation.

[0010] In one possible implementation, the broadcast control equipment is connected to the first equipment via a power line.

[0011] In this implementation, on the one hand, since power lines can transmit data across different devices, there is no need to worry about incompatibility issues between the interface or communication protocol between the broadcast control equipment and the first device. On the other hand, since power lines are existing infrastructure within buildings, there is no need to lay additional communication lines. Therefore, connecting the broadcast control equipment and the first device using power lines can greatly simplify wiring work and has a wide range of application scenarios, such as homes and offices.

[0012] In one possible implementation, the audio stream includes at least one channel signal, the audio signal being a channel signal; the system includes at least one first device; in response to a playback command, outputting an audio signal corresponding to the audio stream to the first device includes: in response to the playback command, sending at least one channel signal to the first device corresponding to the at least one channel signal.

[0013] In this implementation, the audio stream contains at least one channel signal. Therefore, the audio playback system can provide users with richer audio playback scenarios, such as stereo scenarios, 5.1 surround sound scenarios, etc., thereby improving the user's audio experience.

[0014] In one possible implementation, the method further includes: obtaining the playback delay of the audio stream; the playback delay is the time difference between the first device playing the audio stream and the second device indicating that the audio stream is to be played; if the playback delay does not meet a preset duration, controlling the first device to play the sound at a first playback rate according to the playback delay; if the playback delay meets the preset duration, controlling the first device to play the sound at a second playback rate; the first playback rate and the second playback rate are different.

[0015] In this implementation, by acquiring the playback delay of the audio stream between the first and second devices, the audio playback system can identify the time difference between the first and second devices playing the same audio stream. Then, based on the magnitude of the playback delay, it dynamically adjusts the playback rate of the first device to reduce or eliminate the time difference between the audio played by the first and second devices (i.e., using dynamic adjustment of the playback rate of the first device to compensate for the playback delay between the first and second devices). This allows the audio stream played by the first device to be synchronized with the audio stream played by the second device. This helps reduce the discomfort experienced by listeners when listening to the same audio stream played by the first and second devices simultaneously due to the playback delay of multiple devices, thus providing users with a high-quality audio playback experience.

[0016] In one possible implementation, controlling the first device to play sound at a first playback rate based on the playback delay includes: determining the playback sampling rate of the first device based on the playback delay and the original sampling rate of the audio stream, and sending the playback sampling rate to the first device; the playback sampling rate is used by the first device to play sound based on the audio signal.

[0017] In this implementation, the sampling rate of the audio stream is controlled to adjust the speed at which the first device plays the audio stream, thereby reducing the playback latency between the first and second devices and achieving playback synchronization between the first and second devices.

[0018] In one possible implementation, the second device synchronizes its clock with the first device via a broadcast control device.

[0019] In this implementation, clock synchronization among the first device, the second device, and the broadcast control device is fundamental to ensuring collaborative operation between audio devices. If the clocks of the various devices are not synchronized, the transmission time of the audio stream will deviate, affecting its accurate transmission. Clock synchronization ensures that the audio stream maintains a consistent clock reference throughout its transmission across all devices, significantly reducing audio playback asynchrony caused by clock differences. This is particularly important for applications involving multiple devices collaboratively playing audio.

[0020] In one possible implementation, the playback instruction includes a first moment, which is the time when the second device sends the playback instruction; obtaining the playback delay of the audio stream includes: determining a second moment, which is the time when the broadcast control device receives the playback instruction; and determining the playback delay based on the first moment and the second moment.

[0021] In this implementation, considering the transmission delay in the second device transmitting the audio stream to the broadcast control device, which affects the playback synchronization between the second device and the first device, the transmission delay is calculated by the time the second device sends the audio stream and the time the broadcast control device receives the audio stream. The transmission delay between the second device and the broadcast control device is then used as the playback delay, thereby overcoming the playback asynchrony between the second device and the first device caused by the data transmission delay between the second device and the broadcast control device.

[0022] Secondly, embodiments of this application provide an audio playback control system. The system includes a playback control device and a first device; the first device is communicatively connected to a second device via the playback control device; the playback control device is used to obtain a playback instruction from the second device, wherein the playback instruction carries an audio stream to be played, and the playback instruction is used to instruct the audio stream to be played; in response to the playback instruction, it outputs an audio signal corresponding to the audio stream to the first device; the first device is used to receive the audio signal corresponding to the audio stream and play sound according to the audio signal.

[0023] As can be seen, in the audio playback control system, on the one hand, compared to the second device being directly connected to the first device via a high-definition multimedia interface cable (HDMI), in this application, the playback control device, acting as an intermediary between the second and first devices, can obtain playback instructions from the second device and respond to those instructions by sending the audio signal corresponding to the audio stream to the first device. Even if the first and second devices have differences in audio transmission protocols, ports, etc., the playback control device can still achieve compatibility between the first and second devices, thereby enhancing their compatibility. On the other hand, by using the playback control device as an intermediary, the flexible communication connection between the first and second devices allows the audio playback system to flexibly add or remove either the first or second device without being limited by the number of interfaces on the device, improving the scalability and flexibility of the audio playback system, thereby further enhancing the user experience of the audio playback system.

[0024] Thirdly, embodiments of this application provide an audio playback system. The system includes a broadcast control device, a first device, and a second device; the second device is connected to the first device via the broadcast control device; the second device is used to acquire an audio stream to be played and send a playback command to the broadcast control device; the playback command carries the audio stream and is used to instruct the audio stream to be played; the broadcast control device is used to receive the playback command from the second device; in response to the playback command, it outputs an audio signal corresponding to the audio stream to the first device; the first device is used to receive the audio signal corresponding to the audio stream and play sound according to the audio signal.

[0025] In one possible implementation, the second device is also used to identify the playback scenario of the audio stream, and if the playback scenario of the audio stream is a preset scenario, no playback command is issued.

[0026] In this implementation, in playback scenarios with high latency requirements, such as video call scenarios and game scenarios, to ensure the user experience and avoid potential playback latency issues caused by the second device transmitting the audio stream to the broadcast control device and then the broadcast control device transmitting the corresponding audio signal to the first device, the second device will not issue playback commands to the broadcast control device. Instead, the audio playback in this scenario will be completed by the second device's built-in speakers, thereby ensuring the user's audio playback experience and improving the flexibility of the audio playback system.

[0027] In one possible implementation, the second device is used to acquire the audio stream to be played, including: the second device is used to acquire the original audio file and extract at least one channel signal from the original audio file as the audio stream.

[0028] In this implementation, the second device can acquire one or more channel signals from the original audio stream, providing users with a variety of channels to choose from. Users can select mono, stereo, or specific channel signals as audio streams for playback according to their personal preferences or playback needs, in order to meet specific auditory requirements.

[0029] Fourthly, a broadcast control device is provided, comprising: a processor and a memory, wherein the processor is connected to the memory. The memory is used to store computer-executable instructions, and the processor executes the computer-executable instructions stored in the memory, thereby implementing any of the methods provided in the first aspect.

[0030] Fifthly, a chip is provided, comprising: a processor and an interface circuit; the interface circuit is used to receive code instructions and transmit them to the processor; the processor is used to execute the code instructions to perform any of the methods provided in the first aspect above.

[0031] In a sixth aspect, a computer-readable storage medium is provided, storing computer-executable instructions that, when executed on a computer, cause the computer to perform any of the methods provided in the first aspect above.

[0032] In a seventh aspect, a computer program product is provided, including computer execution instructions that, when executed on a computer, cause the computer to perform any of the methods provided in the first aspect above.

[0033] The technical effects of any of the implementation methods in aspects two through seven can be found in the technical effects of different implementation methods in aspect one, and will not be repeated here. Attached Figure Description

[0034] Figure 1 is a schematic diagram of a home theater system provided in an embodiment of this application;

[0035] Figure 2 is a schematic diagram of an audio playback system provided in an embodiment of this application;

[0036] Figure 3 is a schematic diagram of a broadcast control device provided in an embodiment of this application;

[0037] Figure 4 is a schematic diagram of the structure of a second device provided in an embodiment of this application;

[0038] Figure 5 is a software structure block diagram of the second device 100 provided in an embodiment of this application;

[0039] Figure 6 is a schematic diagram of a scenario for determining an audio stream to be played according to an embodiment of this application;

[0040] Figure 7 is a schematic diagram of an application scenario of an audio playback system provided in an embodiment of this application;

[0041] Figure 8 is a flowchart of an audio playback control method provided in an embodiment of this application;

[0042] Figure 9 is a schematic diagram of a 5.1 audio networking method consisting of left and right channel devices and surround channel devices provided in an embodiment of this application;

[0043] Figure 10 is a schematic diagram of a 5.1.2 audio networking method consisting of a sky sound channel device and a surround sound channel device provided in an embodiment of this application;

[0044] Figure 11 is a schematic diagram of a 5.1 audio networking method composed of surround sound channel devices provided in an embodiment of this application;

[0045] Figure 12 is an information interaction diagram of a clock deviation calculation method provided in an embodiment of this application;

[0046] Figure 13 is a flowchart of a playback synchronization method provided in an embodiment of this application. Detailed Implementation

[0047] The technical solutions in the embodiments of this application will now be described with reference to the accompanying drawings.

[0048] In the description of this application, unless otherwise stated, " / " indicates that the objects before and after are in an "or" relationship. For example, A / B can mean A or B. "And / or" in this application is merely a description of the relationship between the related objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone. A and B can be singular or plural.

[0049] Furthermore, in the description of this application, unless otherwise stated, "multiple" means two or more. "At least one of the following" or similar expressions refer to any combination of these items, including any combination of a single item or a plurality of items. For example, at least one of a, b, or c can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0050] Furthermore, to facilitate a clear description of the technical solutions in the embodiments of this application, the terms "first" and "second" are used in the embodiments of this application to distinguish identical or similar items with substantially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and that "first" and "second" are not necessarily different. Meanwhile, in the embodiments of this application, the terms "exemplary" or "for example" are used to indicate that something is being used as an example, illustration, or description. Any embodiment or design scheme described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or advantageous than other embodiments or design schemes. Specifically, the use of terms such as "exemplary" or "for example" is intended to present related concepts in a concrete manner for ease of understanding.

[0051] First, the application scenarios of the embodiments of this application will be introduced by way of example.

[0052] With the rapid development and popularization of smart home technology, audio playback systems, as an important component of smart homes, have been widely used in people's daily lives. Taking home theaters as an example, users connect their user terminals (such as smart TVs, laptops, etc.) to speakers, and use the speakers to play the audio stream output by the user terminals, thereby providing a three-dimensional and layered sound effect, allowing users to have an immersive auditory experience when watching videos (such as movies) or listening to music.

[0053] Figure 1 is a schematic diagram of a home theater system. In Figure 1, the home theater system includes a smart TV and speakers. The smart TV connects to the speakers via audio transmission protocols, such as Audio Return Channel (ARC) and Enhanced Audio Return Channel (eARC). Smart TVs supporting the same audio transmission protocol connect to speakers via a High Definition Multimedia Interface (HDMI) cable to transmit audio streams. However, due to differences in the audio transmission protocols supported by different smart TVs and speakers, or the lack of an HDMI port on the smart TV or speakers, incompatibility issues can easily arise, affecting the transmission of audio streams between them. Of course, smart TVs and speakers can also connect wirelessly (such as via Bluetooth), but wireless connections suffer from instability, susceptibility to interference, and latency.

[0054] In view of this, embodiments of this application provide an audio playback control method applied to a broadcast control device in an audio playback system. The system further includes a first device, which is communicatively connected to a second device via the broadcast control device. The first device is an audio device, and the second device may optionally be a user terminal, such as a smart TV, tablet, or computer. The method specifically includes: First, the broadcast control device obtains a playback instruction from the second device. The playback instruction carries the audio stream to be played and is used to instruct the playback of the audio stream. Then, in response to the playback instruction, the broadcast control device outputs an audio signal corresponding to the audio stream to the first device. The first device plays sound according to the audio signal, thereby enabling the system to play the audio stream.

[0055] The audio playback control method provided in this application has two advantages. First, the playback control device acts as an intermediary between the second device and the first device. It can obtain playback instructions from the second device and respond to these instructions by sending audio signals corresponding to the audio stream to the first device. Even if the first and second devices have differences in audio transmission protocols, ports, etc., the playback control device can still achieve compatibility between them, thereby enhancing their compatibility. Second, by using the playback control device as an intermediary, the flexible communication connection between the first and second devices allows the audio playback system to flexibly add or remove either the first or second device without being limited by the number of interfaces on the device. This improves the scalability and flexibility of the audio playback system, further enhancing the user experience.

[0056] The system architecture of the embodiments of this application will be described below by way of example.

[0057] This application provides an audio playback system, as shown in Figure 2. The audio playback system includes a first device, a playback control device, and a second device. The second device is communicatively connected to the first device through the playback control device.

[0058] The second device acquires the audio stream to be played and sends a playback command to the broadcast control device. The playback command carries the audio stream and instructs the playback of the audio stream. The broadcast control device receives the playback command from the second device and, in response to the playback command, outputs the audio signal corresponding to the audio stream to the first device. The first device receives the audio signal corresponding to the audio stream and plays sound based on the audio signal.

[0059] The first device can be an audio device, such as a ceiling speaker, loudspeaker, portable audio player (such as an MP3 player), etc. This application does not limit the specific form of the first device, and it can be selected according to actual needs.

[0060] Figure 3 is a schematic diagram of a broadcast control device. In Figure 3, the broadcast control device 200 includes: a bus 202, a processor 204, a memory 206, and a communication interface 208. The processor 204, memory 206, and communication interface 208 communicate via the bus 202. The broadcast control device 200 can be a server or a terminal device. It should be understood that this application does not limit the number of processors and memories in the broadcast control device 200.

[0061] Bus 202 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, only one line is used in Figure 3, but this does not imply that there is only one bus or one type of bus. Bus 202 can include pathways for transmitting information between various components of the broadcast control device 200 (e.g., memory 206, processor 204, communication interface 208).

[0062] Processor 204 may include any one or more processors such as a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0063] Memory 206 may include volatile memory, such as random access memory (RAM). Processor 204 may also include non-volatile memory, such as read-only memory (ROM), flash memory, hard disk drive (HDD), or solid state drive (SSD).

[0064] The memory 206 stores executable program code, which the processor 204 executes to implement the aforementioned audio playback control methods. That is, the memory 206 stores instructions for executing the audio playback control methods.

[0065] The communication interface 208 uses transceiver modules such as, but not limited to, network interface cards and transceivers to enable communication between the broadcast control device 200 and other devices or communication networks.

[0066] The second device is the user terminal, which can be a television, mobile phone, tablet computer, desktop computer, laptop computer, handheld computer, notebook computer, ultra-mobile personal computer (UMPC), netbook, as well as cellular phone, personal digital assistant (PDA), augmented reality (AR) / virtual reality (VR) device, speaker and other devices that can acquire audio files.

[0067] Figure 4 shows a schematic diagram of the structure of a second device.

[0068] The second device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a touch sensor 180K, etc.

[0069] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the second device 100. In other embodiments of this application, the second device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0070] In some embodiments, the processor 110 may include one or more interfaces. These interfaces may include an Ethernet interface, an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0071] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the second device 100. In other embodiments of this application, the second device 100 may also employ different interface connection methods or a combination of multiple interface connection methods as described in the above embodiments.

[0072] The charging management module 140 receives charging input from the charger. The power management module 141 connects to the battery 142, and the charging management module 140 connects to the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140 to power the processor 110, internal memory 121, external memory, display 194, camera 193, and wireless communication module 160, etc.

[0073] The wireless communication function of the second device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0074] In some embodiments, antenna 1 of the second device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, so that the second device 100 can communicate with networks and other devices through wireless communication technology.

[0075] The second device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0076] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a Miniled LED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the second device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0077] The second device 100 can perform shooting functions through a camera 193, video codec, GPU, display 194, and application processor.

[0078] Video codecs are used to compress or decompress digital video. The second device 100 may support one or more video codecs. Thus, the second device 100 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG 2, MPEG 3, MPEG 4, etc.

[0079] The external storage interface 120 can be used to connect an external storage card, such as a Micro SD card, to expand the storage capacity of the second device 100. The external storage card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external storage card.

[0080] Internal memory 121 can be used to store computer executable program code, including instructions. Processor 110 executes various functional applications and data processing of the second device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of the second device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0081] The second device 100 can implement audio functions, such as music playback and recording, through an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, and an application processor.

[0082] The audio module 170 is used to convert digital audio information into analog audio signals for output, and also to convert analog audio input into digital audio signals. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 may be located in the processor 110, or some functional modules of the audio module 170 may be located in the processor 110.

[0083] The speaker 170A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. The second device 100 can listen to music or make hands-free calls through the speaker 170A.

[0084] The receiver 170B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the second device 100 receives a telephone call or voice message, it can listen to the voice by bringing the receiver 170B close to the listener's ear.

[0085] Microphone 170C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 170C, inputting the sound signal into microphone 170C. Second device 100 may include at least one microphone 170C. In some embodiments, second device 100 may include two microphones 170C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, second device 100 may include three, four, or more microphones 170C, enabling sound signal collection, noise reduction, sound source identification, and directional recording, among other functions.

[0086] The 170D headphone jack is used to connect wired headphones. The 170D headphone jack can be a USB 130 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0087] Touch sensor 180K, also known as a "touch panel," can be located on display screen 194. The touch sensor 180K and display screen 194 together form a touchscreen, also known as a "touchscreen." Touch sensor 180K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 194. In other embodiments, touch sensor 180K may also be located on the surface of the second device 100, in a different position than display screen 194.

[0088] Buttons 190 include a power button, volume buttons, etc. Buttons 190 can be mechanical buttons or touch-sensitive buttons. The second device 100 can receive button input and generate key signal inputs related to user settings and function control of the second device 100.

[0089] Motor 191 can generate vibration alerts. Motor 191 can be used for incoming call vibration alerts or for touch vibration feedback.

[0090] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0091] The software system of the second device 100 can adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. This embodiment of the invention uses the layered architecture Android system as an example to exemplify the software structure of the second device 100.

[0092] Figure 5 is a software structure block diagram of the second device 100 according to an embodiment of the present invention.

[0093] A layered architecture divides software into several layers, each with a clear role and function. Layers communicate with each other through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom: the application layer, the application framework layer, the Android runtime and system libraries, and the kernel layer.

[0094] The application layer can include a series of application packages.

[0095] As shown in Figure 5, the application package may include applications such as camera, gallery, calendar, call, map, navigation, WLAN, Bluetooth, music, video, and SMS.

[0096] The application framework layer provides application programming interfaces (APIs) and a programming framework for applications in the application layer. The application framework layer includes some predefined functions.

[0097] As shown in Figure 5, the application framework layer may include a window manager, content provider, view system, phone manager, resource manager, notification manager, etc.

[0098] The window manager is used to manage windowed applications. It can retrieve screen size, determine the presence of a status bar, lock the screen, and capture screenshots, among other things.

[0099] Content providers store and retrieve data, making that data accessible to applications. This data can include videos, images, audio, phone calls made and received, browsing history and bookmarks, phone books, and more.

[0100] A view system includes visual controls, such as controls for displaying text and controls for displaying images. View systems can be used to build applications. A display interface can consist of one or more views. For example, a display interface including a text notification icon could include views for displaying text and views for displaying images.

[0101] The phone manager is used to provide communication functions for the second device 100. For example, it manages call status (including connection, hang-up, etc.).

[0102] The file explorer provides applications with various resources, such as localized strings, icons, images, layout files, video files, and more.

[0103] The notification manager allows applications to display notifications in the status bar. These notifications can be used to deliver informational messages and can disappear automatically after a short pause, requiring no user interaction. For example, the notification manager can be used to notify users of completed downloads or message alerts. The notification manager can also display notifications as icons or scrolling text in the top status bar, such as notifications from background applications, or as dialog boxes on the screen. Examples include displaying text messages in the status bar, emitting sounds, vibrating electronic devices, and flashing indicator lights.

[0104] System libraries can include multiple functional modules. For example: surface manager, media libraries, 3D graphics processing libraries (e.g., OpenGL ES), 2D graphics engines (e.g., SGL), etc.

[0105] The Surface Manager is used to manage the display subsystem and provides the blending of 2D and 3D layers for multiple applications.

[0106] The media library supports playback and recording of various common audio and video formats, as well as still image files. It supports multiple audio and video encoding formats, such as MPEG4, H.264, MP3, AAC, AMR, JPG, and PNG.

[0107] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, compositing, and layer processing.

[0108] A 2D graphics engine is a graphics engine for 2D drawing.

[0109] The kernel layer is the layer between hardware and software. The kernel layer contains at least the display driver, camera driver, audio driver, and sensor driver.

[0110] The following example, using a scene of capturing a photograph, illustrates the workflow of the software and hardware of the second device 100.

[0111] When touch sensor 180K receives a touch operation, a corresponding hardware interrupt is sent to the kernel layer. The kernel layer processes the touch operation into a raw input event (including touch coordinates, timestamp of the touch operation, etc.). The raw input event is stored in the kernel layer. The application framework layer retrieves the raw input event from the kernel layer and identifies the control corresponding to the input event. Taking a touch click as an example, where the corresponding control is the camera application icon, the camera application calls the application framework layer's interface to launch the camera application, and then calls the kernel layer to launch the camera driver, capturing still images or videos through camera 193.

[0112] This application also provides an audio playback control system. The audio playback control system includes a broadcast control device and a first device. The broadcast control device performs the same function as the broadcast control device in the aforementioned audio playback system, and the first device performs the same function as the first device in the aforementioned audio playback system; therefore, further details are omitted here.

[0113] It should be noted that the system architecture and application scenarios described in the embodiments of this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided in the embodiments of this application. As those skilled in the art will know, with the evolution of system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of this application are also applicable to similar technical problems.

[0114] In some embodiments, the broadcast control device includes a network cable interface, such as an Ethernet interface, and the broadcast control device and the second device are connected via a network cable. Thus, the second device communicates with the broadcast control device via the network cable, meaning that audio stream transmission between the second device and the broadcast control device is based on a network transmission protocol. Since most second devices are equipped with network cable interfaces such as Ethernet interfaces, and network transmission protocols are more universal than audio transmission protocols, the connection method between the second device and the broadcast control device provided in this application is more universal, compatible, and practical than the connection method based on HDMI cables. Furthermore, compared to a wireless connection between the broadcast control device and the second device, a network cable offers higher stability, reliability, and transmission speed. Transmitting the audio stream to be played via a network cable is unaffected by wireless signal interference, ensuring data integrity and accuracy while guaranteeing fast data transmission, and reducing the risk of playback interruption or sound quality degradation.

[0115] Of course, the broadcast control equipment and the second equipment can also be interconnected and communicate wirelessly, such as via Bluetooth, wireless LAN, Zigbee protocol, etc.

[0116] In some embodiments, the broadcast control equipment and the first device are connected via power line communication (PLC). This allows for data transmission across different devices via power lines, eliminating concerns about interface or communication protocol incompatibility between the broadcast control equipment and the first device. Furthermore, since power lines are existing infrastructure within buildings, there is no need to lay additional communication lines. Connecting the broadcast control equipment and the first device via power lines significantly simplifies wiring and has wide applicability in various settings, such as homes and offices.

[0117] In some embodiments, the second device responds to the user's operation, determines a playback instruction, and sends the playback instruction to the playback control device.

[0118] Optionally, if the second device includes an input device (such as a touchscreen, keyboard, remote control, etc.), the second device obtains the user's operation through the input device and determines the playback command. For example, taking a smart TV as the second device, the user determines the playback command by clicking on the audio stream to be played in the smart TV's display interface (as shown in Figure 6, clicking on the audio stream to be played in the display interface using the smart TV's remote control), and sends the playback command to the second device to instruct it to play the audio stream. Taking a smartphone as an example, the user determines the audio stream to be played by swiping their finger on the smartphone's screen.

[0119] In some embodiments, after receiving a playback instruction from the second device, the broadcast control device, in response to the playback instruction, outputs an audio signal corresponding to the audio stream to the first device so that the first device can play sound based on the received audio signal.

[0120] In one possible implementation, the audio stream includes at least one channel signal. The broadcast control device sends at least one channel signal from the audio stream to a first device corresponding to each channel signal.

[0121] Optionally, the audio stream's channel configuration can be mono, stereo, or multi-channel. Specifically, when the audio stream is configured as mono, it contains only one channel signal. When the audio stream is configured as stereo, it contains two channel signals, such as a left channel signal and a right channel signal. When the audio stream is configured as multi-channel, it contains multiple channel signals, such as a left front channel signal, a right front channel signal, a center channel signal, a low-frequency channel signal, a sky surround channel signal, and rear surround channel signals.

[0122] Channel signals represent information about sound in a specific direction or channel. Different types of channel signals represent different information. The left and right front channel signals provide sound information from the left and right sides in front of the listener, enhancing the width of the sound field and the stereo effect. The center channel is located in front of the listener and is mainly responsible for transmitting sound information such as dialogue and human voices. Taking the center channel signal in movies as an example, the center channel signal is often used to convey dialogue between characters, making it easier for the audience to hear the dialogue clearly, which helps to enhance the immersion and realism of the movie plot. The low-frequency channel signal is mainly responsible for transmitting low-frequency sound information, usually including low-frequency sound effects from instruments such as bass drums and bass guitars, as well as special effects sounds such as explosions and earthquakes. The frequency range of the low-frequency channel signal is usually between 20Hz and 120Hz. The sky surround channel signal is mainly responsible for transmitting sound information from above or diagonally above, such as birdsong, airplanes flying by, and raindrops falling on the roof. Sky surround sound creates a more immersive and realistic sound experience, making the audience feel that the sound is coming from all directions, enhancing immersion and realism. Rear surround sound primarily transmits sound information from behind or to the sides of the audience. Rear surround sound typically works together with the left front, right front, and center channels to form a multi-channel surround sound system. Rear surround sound enhances the surround effect in films or music, making the audience feel that the sound is coming from all directions, playing a crucial role in creating tension, enhancing the realism of action scenes, and improving the overall sound experience.

[0123] In some embodiments, the audio playback system includes multiple first devices (as shown in Figure 2, first device 1, first device 2, etc.), with different first devices used to play different channel signals. Based on their sound effect requirements, users can determine the number of first devices in the audio playback system and their spatial installation positions, thereby selecting mono, stereo, or specific channel signals as the audio stream for playback. For example, if a strong surround sound and stereo effect is desired, more first devices are needed to meet the user's audio needs. Conversely, if the user only requires a simple stereo effect from the speaker playback system, the number of first devices can be reduced.

[0124] In some embodiments, the second device responds to the user's operation, identifies the playback scenario of the audio stream, and if the playback scenario of the audio stream is a preset scenario, it does not send a playback command to the broadcast control device, and the audio stream is played by the speaker built into the second device.

[0125] The preset scenarios are those where the playback latency is less than the latency threshold, i.e., playback scenarios with high requirements for playback latency, such as video call scenarios and game scenarios.

[0126] In playback scenarios with high latency requirements, to ensure the user experience and avoid potential playback delays caused by the second device transmitting the audio stream to the playback control device and then the playback control device transmitting the corresponding audio signal to the first device, the second device will not issue playback commands to the playback control device. Instead, the audio playback in this scenario will be completed by the second device's built-in speakers (such as built-in speakers), thereby ensuring the user's audio playback experience and improving the flexibility of the audio playback system.

[0127] Figure 7 is a schematic diagram of an application scenario for an audio playback system provided in an embodiment of this application. The playback control device is connected to the smart TV via a network cable. The playback control device is connected to each of the first devices via power lines. The audio playback system includes five first devices: a left sky surround channel device, a right sky surround channel device, a left rear surround channel device, a right rear surround channel device, and a subwoofer channel device. The smart TV has a built-in left channel device, a right channel device, and a center channel device.

[0128] As shown in Figure 7, the left channel device, used to play the left channel signal of the audio stream, is located on the left side of the smart TV. The right channel device, used to play the right channel signal of the audio stream, is located on the right side of the smart TV. The center channel device, used to play the center channel signal of the audio stream, is located in the center of the smart TV. The sky surround channel devices, used to play the sky surround channel signal of the audio stream, are installed on the ceiling of the space. Specifically, the left sky surround channel device is installed on the left side of the ceiling; the right sky surround channel device is installed on the right side of the ceiling. The rear surround channel devices, used to play the rear surround channel signal of the audio stream, are installed. The left rear surround channel device is installed behind the left side of the space; the right rear surround channel device is installed behind the right side of the space.

[0129] The following embodiments of this application will be divided into two parts to exemplarily describe the audio playback control method.

[0130] The first part, in conjunction with Figures 8, 9, 10, and 11, introduces the audio playback control method provided in the embodiments of this application, aiming to illustrate the overall process of the audio playback system playing an audio stream.

[0131] The second part, in conjunction with Figures 12 and 13, introduces the audio playback control method provided in the embodiments of this application, aiming to introduce the implementation method of controlling playback delay by the broadcast control device.

[0132] In some embodiments, a first device (such as a ceiling speaker) communicates with a second device (also known as a user terminal, such as a device in a smart home, a smart TV, or a smart assistant) through a broadcast control device. The broadcast control device, acting as an intermediary between the first and second devices, can obtain playback instructions from the second device and respond to these instructions by sending the audio signal corresponding to the audio stream to the first device. Even if the first and second devices have differences in audio transmission protocols, ports, etc., the broadcast control device can still achieve compatibility between the two devices, thereby enhancing their interoperability.

[0133] The following describes the audio playback control method in detail through an interactive approach. Figure 8 is a schematic diagram of an audio playback method according to an exemplary embodiment. Exemplarily, this method is implemented through the interaction of a second device, a playback control device, and a first device in the audio playback system, as shown in Figure 8. The method includes the following steps S801-S803.

[0134] S801: The second device determines the playback command and sends the playback command to the broadcast control device.

[0135] The playback command carries the audio stream to be played. The playback command is used to instruct the audio stream to be played.

[0136] In one possible implementation, the second device, in response to a user's operation, acquires the original audio file to be played and extracts at least one channel signal from the original audio file as an audio stream, thereby determining the playback instruction. The original audio file can be digital audio data of any format, such as the audio portion of a movie or video, or the audio file of a song. For example, the format of the original audio file can be Waveform Audio File (WAV), Windows Media Audio (WMA), MPEG Audio Layer III (MP3), Pulse-Code Modulation (PCM), or Free Lossless Audio Codec (FLAC), etc.

[0137] For example, if the original audio file is configured as mono, the second device extracts that channel signal from the original audio file. In this case, the audio stream includes that channel signal. If the original audio file is configured as stereo, the second device extracts two channel signals from the original audio file: the left channel signal and the right channel signal. The second device then sends the audio stream containing these two channel signals to the playback control device. If the original audio file is configured as multi-channel, the second device extracts multiple channel signals from the original audio file, such as the left front channel signal, right front channel signal, center channel signal, low-frequency channel signal, sky surround channel signal, and rear surround channel signal. The second device then sends the audio stream containing these multiple channel signals to the playback control device.

[0138] In one possible implementation, the audio stream carried in the playback command can be in PCM format. Since PCM audio streams are obtained by directly converting analog audio signals to digital signals without compression, they are lossless and high-quality. This preserves all the audio details and richer timbre of the original audio file, thus improving the playback experience.

[0139] In one possible implementation, the second device can communicate with the broadcast control device using a fast and reliable arq protocol (KCP) to transmit playback instructions. During transmission, the playback instructions can be encrypted to ensure secure delivery to the broadcast control device. For example, Datagram Transport Layer Security (DTLS) can be used to encrypt the playback instructions, ensuring their security.

[0140] In one possible implementation, the second device can also send connection commands, pairing commands, pause commands, volume adjustment commands, etc., to the broadcast control device, thereby controlling the first device.

[0141] The connection command is used to indicate whether the second device is connected to the network of the audio playback control system. For example, the connection command instructs the second device to connect to the audio playback control system via a network cable. The pairing command is used by the second device to instruct it to connect to the first device. The pairing command includes a device identifier, which identifies the first device. The device identifier can be the name, code, etc., of the first device, and is not specifically limited here. Of course, in addition to pairing with the first device, the connection command can also be used by the second device to instruct it to pair with the playback control device. The pause command is used to instruct the audio playback system to pause sound playback. For example, the pause command can specifically instruct which first device in the audio playback system to pause sound playback. The volume adjustment command is used to instruct the adjustment of the volume of sound played by the first devices. For example, the second device can use the volume adjustment command to uniformly adjust the volume of each first device in the audio playback system. Of course, the second device can also send volume adjustment commands for one or more first devices to control the volume of one or more first devices.

[0142] For example, the aforementioned connection commands, pairing commands, pause commands, volume adjustment commands, and other commands can be transmitted to the broadcast control equipment using the Transmission Control Protocol (TCP). Alternatively, DTLS encryption can be used during the transmission of these commands to ensure their secure delivery to the broadcast control equipment.

[0143] S802: The broadcast control device responds to the playback command and outputs the audio signal corresponding to the audio stream to the first device.

[0144] In one possible implementation, the audio stream includes at least one channel signal, i.e., an audio signal. Upon receiving a playback command from the second device, the broadcast control device, in response to the playback command, sends at least one channel signal to the first device corresponding to the at least one channel signal, i.e., each channel signal is sent to the first device corresponding to each channel signal.

[0145] For example, when the audio stream includes a left channel signal, a right channel signal, a left surround channel signal, a right surround channel signal, and a center channel signal, the broadcast control device sends the left channel signal to the first device corresponding to the left channel signal (also called the left channel device), sends the right channel signal to the first device corresponding to the right channel signal (also called the right channel device), sends the left surround channel signal to the first device corresponding to the left surround channel signal (also called the left surround channel device), sends the right surround channel signal to the first device corresponding to the right surround channel signal (also called the right surround channel device), and sends the center channel signal to the first device corresponding to the center channel signal (also called the center channel device).

[0146] For example, when the audio stream includes not only the left channel signal, right channel signal, center channel signal, left surround channel signal, and right surround channel signal, but also the left sky channel signal and right sky channel signal, in addition to sending the left channel signal, right channel signal, center channel signal, left surround channel signal, and right surround channel signal to the corresponding first device, the left sky channel signal is also sent to the first device corresponding to the left sky channel signal (also called the left sky channel device), and the right sky channel signal is sent to the first device corresponding to the right sky channel signal (also called the right sky channel device).

[0147] S803: The first device plays sound based on the audio signal.

[0148] In one possible implementation, the number of first devices included in the audio playback system can be set according to the audio networking method determined by the user's sound effect requirements, and this application embodiment does not specifically limit this.

[0149] For example, the audio networking method can be a 5.1 audio networking method consisting of left and right channel devices + surround channel devices. In this networking method, there are 5 main full-range channel devices, namely left channel device, right channel device, center channel device, left surround channel device, right surround channel device, and 1 subwoofer channel device (also known as subwoofer channel signal).

[0150] Figure 9 is a schematic diagram of a 5.1 audio networking scheme consisting of left and right channel devices and surround channel devices according to an embodiment of this application. In the scenario of Figure 9, in addition to a smart TV and broadcast control equipment, it also includes left channel devices, right channel devices, left surround channel devices, and right surround channel devices. The smart TV has a built-in center channel device and a subwoofer channel device. The center channel device in the smart TV plays the center channel signal corresponding to the audio stream, and the subwoofer channel devices in the smart TV play the low-frequency channel signals corresponding to the audio stream. In Figure 9, the left and right channel devices are ceiling-mounted speakers, and the two left and right channel devices are far apart, resulting in a wider sound field and thus achieving a better 5.1 home theater effect.

[0151] For example, the audio networking method can be the audio networking method 5.1.2 consisting of overhead sound channel devices and surround sound channel devices. In this networking method, in addition to including 5 main full-screen sound channel devices, namely the left channel device, the right channel device, the center channel device, the left surround sound channel device, the right surround sound channel device, and 1 subwoofer channel device, it also includes 2 overhead sound channel devices, namely the left overhead sound channel device and the right overhead sound channel device.

[0152] Figure 10 is a schematic diagram of a 5.1.2 audio networking method consisting of a ceiling-mounted sound channel device and a surround sound channel device, as provided in an embodiment of this application. In the scenario of Figure 10, in addition to a smart TV and a broadcast control device, it also includes a left ceiling-mounted sound channel device, a right ceiling-mounted sound channel device, a left surround sound channel device, and a right surround sound channel device. The smart TV has built-in left channel device, right channel device, center channel device, and subwoofer channel device, which respectively play the corresponding channel signals of the audio stream. In Figure 10, the left ceiling-mounted sound channel device, right ceiling-mounted sound channel device, left surround sound channel device, and right surround sound channel device are ceiling-mounted speakers, which can achieve a 5.1.2 channel effect. Compared to the 5.1 audio networking method, the 5.1.2 audio networking method provides users with a stronger surround sound playback effect. In Figure 10, since the left and right channel signals use the built-in left and right channel devices of the smart TV, the sound field effect is weaker compared to the sound field effect in Figure 9.

[0153] In cases where the audio stream does not contain a sky channel signal, such as when the original audio file is configured as a stereo audio file, an upmixing algorithm can be used to convert the original audio file from a stereo configuration to a multi-channel surround stereo file, thereby obtaining the sky channel signal.

[0154] For example, the audio networking method can be a 5.1 audio network composed of surround channel devices. In this networking method, there are 5 main full-screen channel devices, namely the left channel device, the right channel device, the center channel device, the left surround channel device, the right surround channel device, and 1 subwoofer channel device.

[0155] Figure 11 is a schematic diagram of a 5.1 audio networking scheme composed of surround sound devices according to an embodiment of this application. In the scenario of Figure 11, in addition to a smart TV and broadcast control equipment, it also includes left surround sound devices and right surround sound devices. The smart TV has built-in left, right, center, and subwoofer channels, which play the channel signals of the audio stream. In Figure 11, the left and right surround sound devices are ceiling-mounted speakers. The left and right surround sound devices need to be installed on both sides of the sofa to achieve a good surround sound effect. Furthermore, in Figure 11, because the left and right channel signals are played using the smart TV's built-in left and right channels, the sound field effect is weaker compared to that in Figure 9.

[0156] In the first part of this application's embodiments, compared to the second device being directly connected to the first device via a high-definition multimedia interface cable (HDMI), in this application, the broadcast control device acts as an intermediary between the second and first devices. It can obtain playback instructions from the second device and respond to these instructions by sending audio signals corresponding to the audio stream to the first device. Even if the first and second devices have differences in audio transmission protocols, ports, etc., the broadcast control device can still achieve compatibility between the two devices, thereby enhancing their compatibility. Furthermore, by using the broadcast control device as an intermediary, the flexible communication connection between the first and second devices allows the audio playback system to flexibly add or remove devices without being limited by the number of interfaces on the devices. This improves the scalability and flexibility of the audio playback system, further enhancing the user experience.

[0157] The above is the first part of the embodiments of this application. The following, with reference to Figures 12 and 13, describes the implementation method of controlling playback delay using a broadcast control device.

[0158] In some embodiments, considering the playback delay between the first device and the second device, the playback rate of the first device is dynamically adjusted by the playback control device based on the playback delay to compensate for the playback delay between the first device and the second device, so that the first device and the second device play in sync. This helps to reduce the discomfort caused by the playback delay of multiple devices when the first device and the second device play the same audio file together, and provides users with a high-quality audio playback effect.

[0159] In this embodiment, the clocks of the broadcast control device, the first device, and the second device are synchronized. This is because clock synchronization among the first device, the second device, and the broadcast control device is fundamental to ensuring collaborative operation between audio devices. If the clocks of the various devices are not synchronized, the transmission time of the audio stream will deviate, thus affecting the accurate transmission of the audio stream. Clock synchronization among the devices ensures that the audio stream maintains a consistent clock reference during transmission across all devices, significantly reducing the problem of audio playback asynchrony caused by clock differences. This is particularly important for applications where multiple devices collaboratively play audio (such as when an audio playback system plays video, where the synchronization requirements between the video and audio are high).

[0160] In one possible implementation, the second device synchronizes its clock with the first device via the broadcast control device. Optionally, the clock of the broadcast control device is used as the reference clock, and clock synchronization between the broadcast control device, the second device, and the first device is achieved based on the precision time protocol synchronization algorithm (PTP clock synchronization algorithm).

[0161] Figure 12 is an information interaction diagram illustrating a clock deviation calculation method according to an exemplary embodiment. This clock deviation calculation method is implemented through interaction between a broadcast control device and a second device. In this embodiment, the clock of the broadcast control device is used as the reference clock, and since the clock deviation between the first device and the broadcast control device is small and negligible, only the clock deviation between the second device and the broadcast control device is considered. That is, the clock deviation between the second device and the first device is the same as the clock deviation between the second device and the broadcast control device. The clock deviation calculation method specifically includes the following steps S1201-S1205.

[0162] S1201: The second device sends a synchronization request (sync) message to the broadcast control device and records the time t1 when the synchronization request message is sent.

[0163] For example, a synchronization request message includes a request header and a request body. The request header contains metadata about the request, such as the type of request (e.g., query, modification, etc.), and the request body contains the specific content of the request (e.g., which parameter's data to query, which parameter's data to modify, etc.).

[0164] S1202: After receiving the synchronization request message, the broadcast control device records the time t2 of receiving the synchronization request message and returns the synchronization response (sync rsp) message corresponding to the synchronization request message to the second device.

[0165] The synchronous response message carries t2.

[0166] In addition, the synchronization response message may also include the processing result of the synchronization request (such as success, failure, etc.), response header (the type of request being responded to, such as query, modification, etc.), and response body (such as the data of a certain parameter obtained from the query, etc.).

[0167] S1203: When the second device receives the synchronization response message, it records the time t4 of receiving the synchronization response message and then sends an additional (follow-up) message.

[0168] The additional message contains t1 and t4.

[0169] S1204: After receiving the additional message, the broadcast control device records the time t3 of receiving the additional message and returns the corresponding response message to the second device.

[0170] The response message corresponding to the additional message carries t3.

[0171] S1205: The second device receives the response message corresponding to the follow-up message and calculates the time offset between the second device and the broadcast control device based on t1, t2, t3, and t4. The specific calculation formula is as follows: Calculate the time offset = (t2-t1)-[(t2-t1)+(t4-t3)] / 2 = [(t2-t1)-(t4-t3)] / 2 (Formula 1)

[0172] In this way, the second device adjusts its own clock based on the calculated clock deviation, thereby synchronizing the clocks of the second device and the broadcast control device. For example, the second device periodically acquires the clock deviation between itself and the broadcast control device, and adjusts its own clock accordingly to achieve clock synchronization.

[0173] In this embodiment, the clock deviation between the first device and the broadcast control device is small. Therefore, the clock deviation between the first device and the broadcast control device is ignored, and only the clock deviation between the second device and the broadcast control device is considered. Of course, if there is a large clock deviation between the first device and the broadcast control device, the above-described method for clock synchronization can also be used to achieve clock synchronization between the first device and the broadcast control device, which will not be elaborated here.

[0174] Of course, the clock of the first device can also be used as the reference clock, and the clock of the broadcast control device and the clock of the second device can be synchronized according to the clock of the first device to achieve clock synchronization of the audio playback system. Alternatively, the clock of the second device can be used as the reference clock, and the clock of the broadcast control device and the clock of the first device can be synchronized according to the clock of the second device to achieve clock synchronization of the audio playback system. The specific implementation methods of these two clock synchronization are similar to the clock deviation calculation method in Figure 12 above, and will not be elaborated here.

[0175] In addition, to further achieve clock synchronization between the broadcast control device, the first device, and the second device in the audio playback system, the frequency deviation between the clocks of each device can be calculated to improve the clock synchronization accuracy between the devices in the audio playback system.

[0176] Figure 13 is a flowchart illustrating a playback synchronization method according to an exemplary embodiment. Exemplarily, the method is executed by a broadcast control device and specifically includes the following steps S1301-S1302.

[0177] S1301: Playback latency of the audio stream acquired by the broadcast control equipment.

[0178] The playback latency is the time difference between the first device playing the audio stream and the second device instructing the second device to play the audio stream.

[0179] In one possible implementation, the playback instruction includes a first moment, where the first moment is the time when the second device sends the playback instruction. Specifically, S1301 includes the following a1-a2.

[0180] a1: The broadcast control equipment determines the second moment.

[0181] The second moment refers to the time when the broadcast control equipment receives the playback instruction.

[0182] a2: The broadcast control equipment determines the playback delay based on the first and second moments.

[0183] For example, the difference between the first moment and the second moment is used as the playback delay when the second device and the first device play the same audio stream. In this embodiment of the invention, it is assumed that there is no transmission delay between the broadcast control device and the first device, or that the transmission delay between the broadcast control device and the first device is small enough to be negligible. That is, the duration of transmitting the audio stream between the broadcast control device and the first device is used as the playback delay when the second device and the first device play the same audio stream.

[0184] Since there is a transmission delay when the second device transmits the audio stream to the broadcast control device, this transmission delay will affect the playback synchronization between the second device and the first device. Therefore, the transmission delay is calculated by the time when the second device sends the audio stream and the time when the broadcast control device receives the audio stream, and the transmission delay between the second device and the broadcast control device is used as the playback delay to overcome the playback asynchrony between the second device and the first device caused by the data transmission delay between the second device and the broadcast control device.

[0185] In another possible implementation, the playback delay can be determined based on the position where the second device indicates the playback of sound and the position where the first device plays sound at the same time.

[0186] The position where the first device plays sound refers to the time position of the audio signal played by the first device within the entire audio stream. For example, if the first device is playing the 10th second (s) of the audio stream at the same moment, then the position where the first device plays sound is 10 seconds (s).

[0187] The position where the second device indicates the playback of sound refers to the time position of the audio signal that the second device indicates to be played within the entire audio stream.

[0188] For example, when a first device and a second device collaboratively play an audio stream, and the first device is used to play the audio stream while the second device is used to obtain a playback command, the position indicated by the second device for playing the sound refers to the duration from the moment the playback command is obtained to that same moment. For instance, if the second device has received the playback command for 10 seconds at that same moment, that is, it has already instructed to play the audio stream for 10 seconds (s), then the position indicated by the second device for playing the sound is 10 seconds (s).

[0189] For example, when a first device and a second device are playing the same video in concert, with the first device playing the audio and the second device playing the video frame, the position indicated by the second device for playing the audio can be the time position of the second device playing the video frame within the entire video. For instance, if the second device is playing the 10th second (s) of the video at that same moment, then the second device indicates the position for playing the audio as 10 seconds (s).

[0190] When the first device and the second device play the same video together, with the first device playing the audio and the second device playing the video frame, the audio and video frame of the video are out of sync due to the different audio processing capabilities of the first device and the video frame processing capabilities of the second device. Therefore, the playback delay of the audio stream can be determined based on the position of the first device playing the video and the position of the second device playing the video at the same moment, so as to overcome the problem of synchronization between the first and second devices in the audio playback system.

[0191] Optionally, the playback delay of the audio stream can be obtained by subtracting the position where the second device indicates the playback of sound from the position where the first device plays sound at the same time.

[0192] For example, if the second device indicates the position for playing the sound at 10 seconds (s) and the first device indicates the position for playing the sound at 8 seconds (s), then the playback delay is 10 - 8 = 2 seconds (s).

[0193] In this way, the playback delay of the audio stream can be obtained by directly obtaining the difference between the position where the video is played on the first device and the position where the video is played on the second device. This makes the determined playback delay more accurate. No matter what causes the first device and the second device to be out of sync, the playback delay between the first device and the second device can be found, which improves the accuracy of the playback delay and provides a precise basis for further adjusting the playback rate of the sound.

[0194] S1302: The broadcast control equipment controls the playback rate of the sound played by the first device according to the broadcast control delay.

[0195] In one possible implementation, if the playback delay does not meet the preset duration, the first device is controlled to play sound at a first playback rate based on the playback delay; if the playback delay meets the preset duration, the playback control device controls the first device to play sound at a second playback rate; the first playback rate and the second playback rate are different.

[0196] The preset duration can be set according to actual needs, and no specific limit is set here.

[0197] For example, if the playback latency indicates that the first device is playing sound later than the second device indicates, the playback rate of the first device can be increased to catch up with the playback rate indicated by the second device. If the playback latency indicates that the first device is playing sound earlier than the playback rate indicated by the second device, the playback rate of the first device can be decreased.

[0198] Optionally, in S1302 above, the broadcast control device controls the playback rate of the sound played by the first device based on a resampling algorithm. For example, the broadcast control device specifically controls the playback rate of the sound played by the first device in the following way:

[0199] The broadcast control device determines the playback sampling rate of the first device based on the playback delay and the original sampling rate of the audio stream, and sends the playback sampling rate to the first device.

[0200] The playback sampling rate is used by the first device to play sound based on the audio signal; that is, the playback sampling rate is the sampling rate at which the first device plays sound.

[0201] For example, the broadcast control equipment determines the playback sampling rate of the first device using the following formula:

[0202] Playback sampling rate = Original sampling rate + Playback latency (Formula 2)

[0203] In this embodiment, the playback sampling rate and playback rate are positively correlated. That is, the higher the playback sampling rate, the faster the playback rate of the first device, and the shorter the time required to play the audio stream. For example, if the original sampling rate of the audio stream is 16000 Hz (16 kHz), the playback time is 3 seconds (s). If the playback sampling rate of the audio stream is adjusted to 48000 Hz (48 kHz), the playback time is 1 second (s). Based on the relationship between the playback sampling rate and the playback rate, the playback control device adjusts the playback rate of the first device by controlling its playback sampling rate, thereby achieving playback synchronization between the first device and the second device.

[0204] Of course, in addition to the playback synchronization method shown in Figure 13, the playback control device can also dynamically insert and drop frames on the audio signal output to the first device according to the playback delay of the audio stream, thereby overcoming the playback delay between the first device and the second device and achieving playback synchronization between the first device and the second device.

[0205] For example, if the playback delay indicates that the first device is playing sound later than the second device is indicating the playback progress, the playback control device can delete a portion of the audio signal (also called an audio frame) from the audio signal output to the first device, thereby shortening the duration of the audio stream played by the first device and keeping the playback progress of the first device consistent with that of the second device.

[0206] Conversely, when the playback delay indicates that the first device is playing sound earlier than the second device is indicating the playback progress, the playback control device can insert additional audio signals (also known as audio frames) into the audio signal output to the first device, thereby increasing the playback duration of the audio stream played by the first device and keeping the playback progress of the first device consistent with that of the second device.

[0207] In the second part of the embodiments of this application, considering the playback delay between the first device and the second device, the playback control device dynamically adjusts the playback rate of the first device based on the playback delay to compensate for the playback delay between the first device and the second device, so that the first device and the second device play synchronously. By implementing this method of dynamically adjusting the playback rate, the playback control device can significantly improve the audio playback synchronization effect in a multi-device environment and provide listeners with a higher quality and more consistent listening experience.

[0208] This application also provides a computer-readable storage medium. The computer-readable storage medium can be any available medium that a computing device can store, or a data storage device such as a data center containing one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state drive). The computer-readable storage medium includes instructions that instruct the computing device to perform an audio playback control method, or instruct the computing device to perform an audio playback control method.

[0209] This application also provides a chip. The chip integrates a control circuit for implementing the functions of the aforementioned computing device and one or more ports. Optionally, the functions supported by the chip can be referred to above, and will not be repeated here. Those skilled in the art will understand that all or part of the steps of the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium. The aforementioned storage medium can be a read-only memory, random access memory, etc. The aforementioned processing unit or processor can be a central processing unit, a general-purpose processor, an application-specific integrated circuit (ASIC), a microprocessor (digital signal processor, DSP), a field-programmable gate array (FPGA), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof.

[0210] This application also provides a computer program product containing instructions. The computer program product may be a software or program product containing instructions, capable of running on a computing device or stored on any usable medium. When the computer program product runs on at least one computing device, it causes the at least one computing device to execute an audio playback control method.

[0211] It should be noted that the devices for storing computer instructions or computer programs provided in the embodiments of this application, such as but not limited to the memory, computer-readable storage medium and communication chip, are all non-transitory.

[0212] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented using software programs, implementation can be, in whole or in part, in the form of a computer program product. This computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium accessible to a computer or a data storage device containing one or more servers, data centers, etc., that can be integrated with the medium. The available media can be magnetic media (e.g., floppy disks, hard disks, magnetic tapes), optical media (e.g., DVDs), or semiconductor media (e.g., solid-state disks, SSDs).

[0213] Although this application has been described herein in conjunction with various embodiments, those skilled in the art, by reviewing the accompanying drawings, the disclosure, and the appended claims, will understand and implement other variations of the disclosed embodiments in carrying out the claimed application. In the claims, the word "comprising" does not exclude other components or steps, and "a" or "an" does not exclude multiple instances. A single processor or other unit can implement several functions listed in the claims. While different dependent claims may recite certain measures, this does not mean that these measures cannot be combined to produce good results.

[0214] Although this application has been described in conjunction with specific features and embodiments, it is obvious that various modifications and combinations can be made thereto without departing from the spirit and scope of this application. Accordingly, this specification and drawings are merely exemplary illustrations of this application as defined by the appended claims, and are considered to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from the spirit and scope of this application. Thus, if such modifications and modifications of this application fall within the scope of the claims of this application and their equivalents, this application is also intended to include such modifications and modifications.

Claims

1. An audio playback control method, characterized in that, A broadcast control device for use in an audio playback system, the system further comprising a first device, the first device being communicatively connected to a second device via the broadcast control device; the method comprising: A playback instruction is obtained from the second device; the playback instruction carries an audio stream to be played, and the playback instruction is used to instruct the audio stream to be played; In response to the playback command, an audio signal corresponding to the audio stream is output to the first device; the first device is used to play sound according to the audio signal.

2. The audio playback control method according to claim 1, characterized in that, The broadcast control device and the second device are connected via a network cable.

3. The audio playback control method according to claim 1 or 2, characterized in that, The broadcast control equipment is connected to the first equipment via a power line.

4. The audio playback control method according to any one of claims 1-3, characterized in that, The audio stream includes at least one channel signal, and the audio signal is the channel signal; the system includes at least one first device; The step of responding to the playback command by outputting the audio signal corresponding to the audio stream to the first device includes: In response to the playback command, the at least one channel signal is sent to the first device corresponding to the at least one channel signal.

5. The audio playback control method according to any one of claims 1-4, characterized in that, The method further includes: The playback delay of the audio stream is obtained; the playback delay is the time difference between the first device playing the audio stream and the second device indicating that the audio stream is to be played. If the playback delay does not meet the preset duration, the first device is controlled to play sound at a first playback rate according to the playback delay. If the playback delay meets the preset duration, the first device is controlled to play sound at a second playback rate; the first playback rate is different from the second playback rate.

6. The audio playback control method according to claim 5, characterized in that, The step of controlling the first device to play sound at a first playback rate based on the playback delay includes: Based on the playback delay and the original sampling rate of the audio stream, the playback sampling rate of the first device is determined and sent to the first device; the playback sampling rate is used by the first device to play sound according to the audio signal.

7. The audio playback control method according to claim 5 or 6, characterized in that, The second device synchronizes its clock with that of the first device through the broadcast control device.

8. The audio playback control method according to claim 7, characterized in that, The playback instruction includes a first moment, which is the time when the second device sends the playback instruction; The step of obtaining the playback latency of the audio stream includes: The second moment is determined as the time when the broadcast control device receives the playback instruction; The playback delay is determined based on the first time point and the second time point.

9. An audio playback control system, characterized in that, The system includes a broadcast control device and a first device; the first device is communicatively connected to the second device through the broadcast control device. The playback control device is used to obtain a playback instruction from the second device, wherein the playback instruction carries an audio stream to be played and is used to instruct the audio stream to be played; in response to the playback instruction, the device outputs an audio signal corresponding to the audio stream to the first device; The first device is used to receive the audio signal corresponding to the audio stream and play the sound according to the audio signal.

10. An audio playback system, characterized in that, The system includes a broadcast control device, a first device, and a second device; the second device is connected to the first device through the broadcast control device. The second device is used to acquire the audio stream to be played and send a playback command to the broadcast control device; The playback instruction carries the audio stream, and the playback instruction is used to instruct the audio stream to be played; The broadcast control device is used to receive the playback command from the second device; In response to the playback command, the audio signal corresponding to the audio stream is output to the first device; The first device is used to receive the audio signal corresponding to the audio stream and play the sound according to the audio signal.

11. The audio playback system according to claim 10, characterized in that, The second device is also used to identify the playback scenario of the audio stream, and if the playback scenario of the audio stream is a preset scenario, it will not issue the playback command.

12. The audio playback system according to claim 10 or 11, characterized in that, The second device is used to acquire the audio stream to be played, including: The second device is used to acquire the original audio file and extract at least one channel signal from the original audio file as the audio stream.

13. A broadcast control device, characterized in that, include: Processor and memory; The processor is connected to a memory for storing computer execution instructions. The processor executes the computer execution instructions stored in the memory to enable the broadcast control device to implement the method as described in any one of claims 1-8.

14. A computer-readable medium, characterized in that, The computer-readable storage medium stores instructions that, when executed on an electronic device, cause the electronic device to perform the method as described in any one of claims 1-8.

15. A computer program product, characterized in that, The computer program product includes instructions that, when executed on a computer or processor, cause the computer or processor to perform the method as described in any one of claims 1-8.

Images