Play control method, controller and play device

Abstract

This invention provides a playback control method, controller, and playback device. The method includes: determining anchor point time information for transmitting data to be played and an initial margin duration corresponding to the data to be played, wherein the time after the initial margin duration of the anchor point time corresponding to the anchor point time information is taken as the target playback time; inserting invalid data with a preset margin duration before the data to be played; the preset margin duration is determined based on the difference between the initial margin duration and a relative delay duration, wherein the relative delay duration is the sum of the relative transmission delay of the data to be played and the relative playback delay of the data to be played in the receiving device; and playing the data to be played starting from the inserted invalid data, so that the data to be played is played at the target playback time. This playback control scheme enables the playback of the data to be played at the target playback time on multiple playback devices, achieving synchronous playback of the data to be played on multiple devices.

Description

Technical Field

[0001] This invention relates to the field of audio and video playback technology, specifically to a playback control method, a controller, and a playback device. Background Technology

[0002] In the field of wireless digital audio and video playback, it is often necessary to simultaneously play audio and video data on multiple wirelessly connected playback devices. Therefore, how to achieve simultaneous playback of audio and video data on multiple playback devices is a hot technical issue in this field. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a playback control method, a controller, and a playback device to achieve synchronous playback of audio and video data on multiple playback devices.

[0004] To address the above problems, embodiments of the present invention provide the following technical solutions: In a first aspect, embodiments of the present invention provide a playback control method, the method being applied to a playback device, comprising: Determine the anchor point time information for transmitting the data to be played and the initial remaining time duration corresponding to the data to be played, wherein the time after the initial remaining time duration of the anchor point time corresponding to the anchor point time information is taken as the target playback time. Invalid data with a preset margin duration is inserted before the data to be played to form target playback data; the preset margin duration is determined based on the difference between the initial margin duration and the relative delay duration, and the relative delay duration is the sum of the relative transmission delay of the data to be played and the relative playback delay of the data to be played within the playback device; The target playback data is played starting from the inserted invalid data, so that the data to be played is played at the target playback time.

[0005] Optionally, the anchor point time information is based on the connection module clock indication; Alternatively, the initial value of the initial margin duration is based on the controller clock indication, and the initial margin duration is calculated based on the initial value and the clock deviation between the controller clock and the connection module clock.

[0006] Optionally, the playback device is a source device, and determining the anchor point time information for transmitting the data to be played and the initial remaining time duration corresponding to the data to be played includes: The maximum relative delay of the data to be played is obtained in advance, wherein the maximum relative delay is the sum of the maximum relative transmission delay of the data to be played and the maximum relative playback delay of the data to be played within the receiving device; Select a value greater than or equal to the maximum relative delay as the initial margin duration.

[0007] Optionally, the playback device is a receiving device. The determination of the anchor point time information for transmitting the data to be played and the initial remaining time duration of the corresponding data to be played includes: Receive configuration information from the source device, the configuration information including the anchor point time information and the initial margin duration; or, Receive configuration information from the source device, the configuration information including the anchor point time information; The device receives the data to be played and its redundancy information from the source device, wherein the redundancy information includes the initial margin duration.

[0008] Optionally, the playback device is a source device, and the step of starting to play the target playback data from the inserted invalid data is triggered from the anchor point moment; Alternatively, the step of starting playback of the target playback data from the inserted invalid data is triggered based on a preset instruction. In the step of inserting invalid data with a preset margin of time before the data to be played, the preset margin of time also includes a pre-transmission delay, which is the delay between the start of playback of the source device from the inserted invalid data and the transmission of the data to be played by the connection module.

[0009] Optionally, before inserting invalid data of a preset remaining duration before the data to be played, the method further includes: Acquire first clock time information and second clock time information; wherein, the first clock time information is recorded when the connection module periodically generates a synchronization test signal, and the second clock time information is recorded when the clock synchronization module receives the synchronization test signal from the connection module, and the synchronization test signal is transmitted based on the general input / output interface between the connection module and the clock synchronization module; Based on the first clock time information and the second clock time information, the clock deviation between the connection module clock and the controller clock is calculated, and the clock deviation includes the initial deviation and the frequency deviation.

[0010] Secondly, embodiments of the present invention provide a controller, including a playback control module, the playback control module being used to execute the following logic: Determine the anchor point time information for transmitting the data to be played and the initial remaining time duration corresponding to the data to be played, wherein the time after the initial remaining time duration of the anchor point time corresponding to the anchor point time information is taken as the target playback time. Invalid data of a preset margin duration is inserted before the data to be played; the preset margin duration is determined based on the difference between the initial margin duration and the relative delay duration, and the relative delay duration is the sum of the relative transmission delay of the data to be played and the relative playback delay of the data to be played within the playback device; Playback operations are initiated from the inserted invalid data so that the data to be played is played at the target playback time.

[0011] Thirdly, embodiments of the present invention provide a playback device, including the controller described in the second aspect.

[0012] Optional components also include a speaker, a connectivity module, and a clock synchronization module; The speaker is used to play the data to be played, the connection module is used to realize wireless connection, and the clock synchronization module is used to realize clock synchronization between the connection module and the controller.

[0013] Optionally, the clock synchronization module includes a normally open domain processor and a timer; the controller is configured with a processor core, and a computing application and a playback control module run based on the processor core; The connection module is connected to the normally open domain processor based on a general-purpose input / output interface; The connection module is used to record the level of the general input / output interface between the control connection module and the normally open domain processor based on the interrupt function periodically, generate the first clock moment information corresponding to the synchronous test signal, and send the first clock moment information to the driver layer of the controller through the serial port; the computing application is used to obtain the first clock moment information through the host controller interface command. The normally open domain processor is used to record the value of the timer corresponding to the interrupt-received synchronization test signal as the second clock moment information, and send it to the controller's computing application via a hardware mailbox.

[0014] The playback control method, controller, and playback device provided in this invention include: determining anchor point time information for transmitting data to be played and an initial margin duration corresponding to the data to be played, wherein the time after the initial margin duration of the anchor point time corresponding to the anchor point time information is taken as the target playback time; inserting invalid data with a preset margin duration before the data to be played to form target playback data; the preset margin duration is determined based on the difference between the initial margin duration and a relative delay duration, wherein the relative delay duration is the sum of the relative transmission delay of the data to be played and the relative playback delay of the data to be played within the playback device; and playing the target playback data starting from the inserted invalid data so that the data to be played is played at the target playback time.

[0015] As can be seen, this embodiment of the invention determines the target playback time through corresponding anchor point time information and a preset initial margin duration. The initial margin duration is used to offset any potential delay after the anchor point time, i.e., the sum of the relative transmission delay and the relative playback delay. A preset margin duration is determined based on the difference between the initial margin duration and the relative delay duration. Invalid data of this preset margin duration is inserted before the data to be played, thereby ensuring that the data to be played is played at the target playback time. It is understood that by implementing this playback control scheme on multiple devices to achieve synchronous playback of the data to be played on multiple devices, the target playback time can be achieved. Attached Figure Description

[0016] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of this application. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort.

[0017] Figure 1 This is a diagram illustrating synchronized digital audio and video playback. Figure 2 This is a schematic diagram of an optional structure of a playback device provided in an embodiment of the present invention; Figure 3 This is a schematic diagram of an optional flow of the playback control method provided in an embodiment of the present invention; Figure 4 This is an example diagram of the playback control timeline provided in an embodiment of the present invention; Figure 5 This is a schematic diagram of another optional structure of the playback device provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of an optional process for coordinated synchronization testing provided in an embodiment of the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0019] Wireless connectivity technology refers to the technology of transmitting data in space using electromagnetic waves (such as radio waves, infrared rays, etc.) without physical wiring. Typical wireless connection methods between playback devices include Bluetooth, Wi-Fi, and Thread.

[0020] Synchronous playback of digital audio and video can be achieved using multiple wirelessly connected playback devices, providing users with a realistic and immersive user experience. For specific implementation details, refer to... Figure 1 The diagram shown illustrates synchronized digital audio and video playback. The playback devices include source device 10 (shown as Source in the diagram) and receiving devices 20 and 30 (shown as Sink1 and Sink2 in the diagram).

[0021] The source device can be connected to the receiving device (such as a mobile phone or TV) via CIS (Connected Isochronous Stream) or A2DP (Advanced Audio Distribution Profile).

[0022] The source device and the receiving device (including Sink1 and Sink2) can be Bluetooth devices, meaning that both the source device and the receiving device are equipped with a Bluetooth module (shown as a BT module in the figure), thereby playing digital audio and video based on Bluetooth connection.

[0023] As described in the background section, in the field of digital audio and video wireless playback, it is typically necessary to simultaneously play audio and video data on multiple wirelessly connected playback devices. How to achieve simultaneous playback of audio and video data on multiple playback devices is a hot technical issue in this field.

[0024] In some specific implementations, synchronized playback of audio and video data can be achieved based on network time. That is, the source device (Source) and the sink device (Sink) synchronize network time, and then the synchronized playback of multiple devices is completed based on a unified network timestamp. Alternatively, in other implementations, multiple electronic devices can be designed as a clock system with the same source to achieve synchronized playback.

[0025] However, the above implementations all rely heavily on hardware, thus limiting their application scenarios. The inventors believe it is necessary to devise a new solution that reduces hardware dependence. To achieve this, the inventors discovered that during data transmission, the data transmission modules of the source and receiving devices use the same reference clock for data transmission. The anchor time of the data to be played is the aligned data transmission node; that is, the anchor time is a fixed time jointly determined by the source and receiving devices during data transmission. Therefore, it is possible to consider using the anchor time of the data transmission process as a basis for synchronous playback of audio and video data.

[0026] In view of this, embodiments of the present invention provide a playback control method, a controller, and a playback device. The method includes: determining anchor point time information for transmitting data to be played and an initial margin duration corresponding to the data to be played, wherein the time after the initial margin duration of the anchor point time corresponding to the anchor point time information is taken as the target playback time; inserting invalid data with a preset margin duration before the data to be played to form target playback data; the preset margin duration is determined based on the difference between the initial margin duration and a relative delay duration, wherein the relative delay duration is the sum of the relative transmission delay of the data to be played and the relative playback delay of the data to be played within the playback device; and playing the target playback data starting from the inserted invalid data, so that the data to be played is played at the target playback time.

[0027] As can be seen, this embodiment of the invention determines the target playback time through corresponding anchor point time information and a preset initial margin duration. The initial margin duration is used to offset any potential delay after the anchor point time, i.e., the sum of the relative transmission delay and the relative playback delay. A preset margin duration is determined based on the difference between the initial margin duration and the relative delay duration. Invalid data of this preset margin duration is inserted before the data to be played, thereby ensuring that the data to be played is played at the target playback time. It is understood that by implementing this playback control scheme on multiple devices to achieve synchronous playback of the data to be played on multiple devices, the target playback time can be achieved.

[0028] It should be noted that the playback control scheme applies to both the source device and the receiving device. Specifically, at the source device transmitting the data to be played, the relative delay duration can be understood as 0, and the corresponding preset margin duration is the initial margin duration. This allows invalid data of the initial margin duration to be inserted before the data to be played, thereby ensuring that the data to be played is simultaneously played on both the source device and the receiving device at the target playback time.

[0029] For ease of understanding, the hardware device for implementing the playback control method provided in the embodiments of the present invention will be described below.

[0030] refer to Figure 2 The diagram illustrates an optional structure of a playback device, where the playback device, acting as both a source device and a sink device, can have the same structure. Specifically, the playback device may include a speaker, a connection module, a clock synchronization module, and a controller. The speaker is used to play the data to be played; the connection module is used to establish a wireless connection, and in a specific implementation, the connection module may be a Bluetooth module; the clock synchronization module is used to synchronize the clock within the device, for example, to synchronize the clock between the connection module and the controller; and the controller is used to control the playback of the data to be played and to control the clock within the device.

[0031] The controller, for example, can be an ARM (Advanced RISC Machine) controller used to deploy a System on a Chip (SOC). This SOC can be a non-real-time operating system such as Linux, thereby reducing the solution's dependence on hardware configuration and system environment. Specifically, the controller can be configured with a playback control module to control the playback of the data to be played.

[0032] In the specific implementation, a general purpose input / output (GPIO) interface can be configured between the connection module and the clock synchronization module. Based on this GPIO interface, the time information of the connection module and the clock synchronization module at different nodes can be recorded periodically (the clock synchronization module records the time information based on the controller clock, and the connection module records the time information based on the connection module clock). This allows the clock deviation between the connection module and the controller to be determined, thereby achieving clock synchronization between the connection module and the controller.

[0033] It is understandable that, given that clock synchronization of the connection modules between different playback devices is the foundation for data transmission, clock synchronization of the controllers between different playback devices can be indirectly achieved through clock conversion between the connection modules and controllers within the playback devices.

[0034] The playback control method provided in the embodiments of the present invention will now be described in detail. Specifically, this playback control method can be applied to the controller of a playback device, for example, to a playback control module of the controller. Specifically, the controller includes a SOC chip, which can be configured with the playback control module. Correspondingly, the playback control method can be applied to the SOC chip, or to the playback control module of the SOC chip. (Refer to...) Figure 3 The diagram shows an optional flow chart of a playback control method, which includes: Step S100: Determine the anchor point time information for transmitting the data to be played and the initial remaining time of the corresponding data to be played; The target playback time is the time after the initial remaining duration of the anchor point corresponding to the anchor point time information.

[0035] The data to be played can be a playback unit, which can be a single playback file or a combination of multiple playback files with a preset playback order. A playback file is a file with a preset duration and intended for continuous playback; for example, a playback file can correspond to a song. It should be noted that in a Bluetooth playback scenario, the corresponding data transmission unit is a data packet. One playback unit can correspond to multiple data packets, which are played sequentially. However, the playback control method described in this embodiment only controls the first data packet of the playback unit; subsequent data packets are played sequentially. In other words, playback control is applied to the starting position of the playback unit without affecting the overall playback of the unit.

[0036] The anchor point time information is used to indicate the anchor point time for transmitting the data to be played. It is understood that in a specific data transmission process, the connection module (e.g., Bluetooth module) of the source device and the connection module (e.g., Bluetooth module) of the receiving device can synchronize their clocks based on a preset mechanism and determine the corresponding anchor point time. Furthermore, both the source and receiving devices are aware of the anchor point time of the data to be played in advance. Therefore, during the data transmission process, both the source and receiving devices remain in a working state at the anchor point time (especially in scenarios where the receiving device enters a sleep state when idle; in such cases, the receiving device is woken up to a working state before the anchor point time).

[0037] In a specific implementation, taking Bluetooth data transmission as an example, clock synchronization between multiple Bluetooth devices can be achieved based on the anchor point mechanism in the LE Audio (Low Energy Audio) standard protocol. Under this mechanism, a corresponding anchor point time is configured for the preset data to be played, and both the source and receiving devices are aware of the anchor point time in advance. This ensures that the receiving device remains in an active state during Bluetooth data transmission. Conversely, if the receiving device enters a sleep state due to low power requirements, it will wake up and resume active status before the anchor point time. It should be noted that the anchor point time described in this embodiment is the anchor point time of the corresponding playback unit, specifically the anchor point time corresponding to the first data packet of the playback unit.

[0038] It should be noted that when the clocks of the connection module and the controller are different, the anchor point time is based on the clock indication of the connection module.

[0039] The initial margin duration is a time gap reserved by the source device for the data to be played, used to offset any delay that may occur after the anchor point. This initial margin duration can be fixed or adjusted in real time according to device parameters. In a specific implementation, the initial margin duration is greater than or equal to the maximum relative delay corresponding to the data to be played, where the maximum relative delay is the sum of the maximum relative transmission delay corresponding to the data to be played and the maximum relative playback delay of the data to be played within the receiving device.

[0040] The maximum relative transmission delay is the maximum relative transmission delay of the data to be played from the source device to the receiving device. It can be understood that in the data transmission process, the source device can have one or more receiving devices. When there is only one receiving device, the maximum relative transmission delay from the source device to the receiving device is the relative transmission delay itself. When there are multiple receiving devices, the maximum relative transmission delay from the source device to the receiving devices is the maximum of the multiple relative transmission delays from the source device to the multiple receiving devices.

[0041] Wherein, the maximum relative playback delay is the maximum value of the relative playback delay within the receiving device. Similarly, in the data transmission process, the source device can correspond to one or more receiving devices; when there is one receiving device, the maximum value of the relative playback delay within that receiving device is its own relative playback delay; when there are multiple receiving devices, the maximum value of the relative playback delay within that receiving device is the maximum value among the multiple relative playback delays corresponding to those multiple receiving devices.

[0042] It should be noted that when the clocks of the connection module and the controller are different, the initial margin duration is based on the clock indication of the connection module. Alternatively, in other implementations, if the initial value of the initial margin duration is based on the controller clock indication, then the initial margin duration is calculated based on the initial value and the clock offset between the controller clock and the connection module clock.

[0043] The target playback time is the time at which the data to be played is played. It can be understood that synchronous playback of the data is achieved by controlling both the source and receiving devices to play the data at the target playback time.

[0044] In a specific implementation, the source device can pre-obtain the maximum relative delay of the data to be played, that is, obtain the maximum relative transmission delay and the maximum relative playback delay of the receiving device, and then select a value greater than or equal to the maximum relative delay as the initial margin duration. To avoid excessively long waiting times, the selected initial margin duration can be less than 10 times the maximum relative delay, and preferably less than 2 times the maximum relative delay.

[0045] Alternatively, in other implementations, the source device can estimate the maximum relative delay of the data to be played and further select a value greater than (e.g., far exceeding) the maximum relative delay as the initial margin duration. This provides a margin to compensate for the relative delay of the data to be played at the receiving device, thereby achieving synchronous playback control of the data to be played. In other words, the initial margin duration can be an estimated empirical value, which can be used as a fixed value for the initial margin duration. In other implementations, the initial margin duration can also be a real-time adjusted value; this invention does not impose specific limitations on this.

[0046] In the specific implementation, the anchor point time is the time information known in advance by the source device and the receiving device. Accordingly, in this step, for the anchor point time, both the source device and the receiving device can obtain the anchor point time information of the corresponding data to be played in advance, thereby determining the anchor point time information.

[0047] In the specific implementation, the initial margin duration is determined by the source device. Accordingly, in this step, the source device can determine the initial margin duration based on the aforementioned scheme, such as based on the maximum relative transmission delay or based on the estimated information. In the specific data transmission process, the source device can send configuration information containing the anchor point time information and the initial margin duration to the receiving device, so that the receiving device receives the configuration information from the source device and then determines the anchor point time information and the initial margin duration.

[0048] It should be noted that, on the source device, this step can pre-determine the anchor point time information and initial remaining duration of the data to be played based on its own preset information before transmitting the data to be played; while on the receiving device, this step can obtain the anchor point time information and initial remaining duration before or after transmitting the data to be played. Specifically, the anchor point time information and initial remaining duration can be configured into configuration information and pre-sent to the receiving device before sending the data to be played, so that the receiving device determines the corresponding anchor point time information and initial remaining duration based on the received configuration information; or, the anchor point time information can be configured into configuration information and pre-sent to the receiving device before sending the data to be played, so that the receiving device receives the configuration information, determines the corresponding anchor point time information, and the initial remaining duration can be configured into the redundancy information of the data to be played, thus being sent to the receiving device simultaneously with the data to be played at the anchor point time. In this step, the receiving device can receive the data to be played and its redundancy information to determine the corresponding initial remaining duration.

[0049] Step S110: Insert invalid data with a preset remaining duration before the data to be played to form the target playback data; The preset margin duration is determined based on the difference between the initial margin duration and the relative delay duration, where the relative delay duration is the sum of the relative transmission delay of the data to be played and the relative playback delay of the data to be played within the playback device.

[0050] Invalid data of a preset margin duration, determined by inserting the difference between the initial margin duration and the relative delay duration, is used to ensure that the data to be played is played at the target playback time. In a specific implementation, after determining the anchor point time information and the initial margin duration for the data to be played, the corresponding preset margin duration can be determined based on the relative delay duration.

[0051] In the relative delay duration, the relative transmission delay is the relative delay corresponding to the process of the connection module of the source device transmitting the data to be played to the connection module of the receiving device (i.e., the process of the data to be played being transmitted over the air). It can be understood that, at the source device, this relative delay can be considered to be 0, so the preset margin duration obtained by the source device covers the transmission process of the data to be played. Accordingly, during the transmission process of the data to be played, the source device is already in the stage of playing invalid data.

[0052] In the relative delay duration, the relative playback delay is the relative delay within the receiving device's internal processing flow after receiving the data to be played. It can be understood that, at the source device, this relative delay can be considered zero, thus the preset margin duration obtained by the source device covers the processing flow of the data to be played within the receiving device. Consequently, during the processing of the data to be played, the source device is already in the stage of playing invalid data.

[0053] In particular, for the receiving device, the relative transmission delay of the data to be played and the relative playback delay of the data to be played within the receiving device are usually fixed values. Therefore, the preset margin duration can be determined based on the pre-obtained relative transmission delay and relative playback delay.

[0054] Specifically, the relative transmission delay can typically be determined in advance based on configuration information or test information in the data transmission process before the data to be played is transmitted; this invention does not impose specific limitations on this. The relative playback delay can be determined in advance based on test data in the playback process before the data to be played is played, thereby allowing the determination of the relative delay duration based on the test results, and subsequently calculating the preset margin duration.

[0055] After determining the preset margin duration, invalid data of the corresponding duration can be inserted before the data to be played to form target playback data that includes both invalid data and the data to be played, so that the data to be played can be played at the target playback time.

[0056] In this implementation, the inserted invalid data can be empty data. The data to be played is filled in the ALSA buffer. Accordingly, in this step, empty data is filled into the ALSA buffer as invalid data before the data to be played, thus forming the target playback data. This delays the playback time of the data to be played until the target playback time. In a specific implementation, the decoded data to be played is filled into the Advanced Linux Sound Architecture Buffer (ALSA buffer, also known as the Alsa Buffer). The audio hardware then obtains and plays the target playback data (also known as consuming data from the buffer) from this ALSA buffer. In this embodiment of the invention, when it is necessary to insert empty data, it can be achieved by filling the Alsa Buffer with empty data.

[0057] As can be seen, in this embodiment of the invention, invalid data with different preset margin durations are inserted into the source device and the receiving device to ensure that the source device and the receiving device play the data to be played at the same time at the target playback time, so as to achieve synchronous playback of the data to be played.

[0058] Step S120: Start playing the target playback data from the inserted invalid data, so that the data to be played is played at the target playback time; It is understandable that after inserting invalid data with a preset remaining duration before the data to be played, when the target playback data is played, playback starts from the inserted invalid data with the preset remaining duration, thereby delaying the playback of the data to be played until the target playback time.

[0059] Specifically, invalid data with a preset margin of time inserted by the source device causes the source device to actually start playing the target playback data from the invalid data when playing the target playback data, thus entering a waiting state until the data to be played is transmitted to the receiving device and the receiving device enters the playback state, at which time the data to be played is played synchronously with the receiving device at the target playback time.

[0060] It is understandable that the playback control scheme provided in this embodiment of the invention is more suitable for Bluetooth connection scenarios. This is because, in Bluetooth connection scenarios, Bluetooth data transmission has a unidirectional communication characteristic, which makes it impossible for the source device and the receiving device to determine a unified playback time through a self-negotiation mechanism. Obviously, this embodiment of the invention achieves synchronous playback of the data to be played by setting a target playback time based on the anchor point time, that is, starting the playback of the data to be played simultaneously after a fixed initial margin of time after the anchor point time.

[0061] It should be noted that in the playback control scheme provided in this embodiment of the invention, the source device triggers step S120 from the anchor point time, thereby achieving alignment of each time period. In a further optional implementation, if the source device triggers step S120 based on a preset instruction (i.e., triggers step S120 before the anchor point time), then for the source device, in step S110 provided in this embodiment of the invention, the preset margin duration further includes a pre-transmission delay duration. This pre-transmission delay duration is used to indicate the duration of the pre-transmission delay, which is the delay between the source device starting to play the data to be played from the insertion of invalid data and the time the data to be played is sent by the connection module. The preset instruction instructs the target playback data to perform a playback operation and simultaneously instructs the data to be played to start the data transmission process.

[0062] Accordingly, the preset margin duration is the difference between the initial margin duration and the relative delay duration, and the sum of the pre-transmission delay duration. Specifically, the step of determining the preset margin duration based on the difference between the initial margin duration and the relative delay duration includes calculating the difference between the initial margin duration and the relative delay duration, and further calculating the sum of the difference and the pre-transmission delay duration as the preset margin duration.

[0063] In a specific example, refer to Figure 4The illustrated playback control timeline example uses audio data as the data to be played, which includes multiple frames of audio data as valid data. Specifically, in the target playback data, one frame of audio data is treated as a data packet. For the first frame of audio data, the corresponding data packet may include inserted empty data and valid data. It should be noted that, for this audio data, subsequent frames of audio data, except for the first frame, are processed following the first frame, and this is not suitable for the playback control with inserted empty data provided in this embodiment of the invention.

[0064] refer to Figure 4 The diagram shows the playback timeline of the source speaker of the source device and the sink speaker of the receiver device, as well as the control timeline of the source controller of the source device and the sink controller of the receiver device.

[0065] For the source speaker of the source device and the sink speaker of the receiving device, the first frame of audio data includes empty data before the target playback time and valid data after the target playback time. The empty data duration (source delay) of the source speaker is T. source The empty data duration is greater than that of the Sink Speaker. Specifically, the empty data of the Source Speaker includes the audio-to-air delay T. to_air_delay The initial margin duration, Threshold, corresponds to the empty data duration of the source device (i.e., the preset margin duration).

[0066] In terms of actual transport delay The relative transmission delay of the receiving device is expressed as the actual playback delay (present delay). The relative playback delay of the receiving device, and correspondingly, the empty data duration of the receiving device (i.e., the preset margin duration):

[0067] After conversion, the empty data duration of the source device (i.e., the preset margin duration):

[0068] It is understandable that, regardless of whether it is a broadcast source device (Source) or a receiving device (Sink), the anchor point corresponds to the same physical time base. If both types of devices start playing the data to be played (i.e., the valid data) at the threshold time node after the anchor point, then the audio playback time of the two will be completely consistent.

[0069] In the subsequent playback process, the transmitted second and third frames of audio data are played sequentially. It's understood that each frame of audio data has a preset duration; correspondingly, during data transmission, it's sufficient to ensure that the next frame is transmitted while the previous frame is playing. (Reference) Figure 4 As long as the second frame of audio data is transmitted during the playback of the first frame of audio data, and the third frame of audio data is transmitted during the playback of the second frame of audio data, the second frame of audio data can be played sequentially after the first frame of audio data has finished playing, and the third frame of audio data can be played sequentially after the second frame of audio data has finished playing.

[0070] In a further optional implementation, the relative playback delay can be determined based on the connection module clock during the specific determination process. Specifically, the relative playback delay can be calculated based on the clock deviation between the connection module clock and the controller clock, thereby determining the expression of the relative playback delay based on the connection module clock. The clock deviation includes initial deviation and frequency deviation.

[0071] Specifically, the initial deviation and frequency deviation can be obtained based on the coordinated synchronization test between the connection module and the clock synchronization module. In the specific implementation, refer to... Figure 5 The diagram illustrates another optional structure of the playback device. The controller contains a processor core, upon which computing applications and a playback control module run. The computing application calculates clock skew. The clock synchronization module may include a timer and an Always-On CPU (AO CPU). The AO CPU can run a Real-Time Operating System (RTOS) to handle low-power tasks such as wake-up and voice monitoring when the controller's main system is in sleep or standby mode. The timer can be understood as the controller clock.

[0072] The connection module can be connected to the normally open domain processor via a general purpose input / output (GPIO) interface. During the coordinated synchronization test between the connection module and the clock synchronization module, the connection module can periodically send synchronization test signals via GPIO and notify the normally open domain processor to receive and record the received synchronization test signals via an interrupt function. The connection module records the sending time of the synchronization test signal based on its own clock, while the normally open domain processor records the receiving time based on a timer. Since the transmission of the synchronization test signal from the connection module to the normally open domain processor is a short-distance transmission within the device, it can be understood as instantaneous transmission. Therefore, the initial deviation and frequency deviation between the connection module clock and the timer (controller clock) can be determined based on the sending time recorded by the connection module and the receiving time recorded by the normally open domain processor.

[0073] The coordination and synchronization test can be performed before step S110. In the specific implementation, refer to... Figure 6 The diagram illustrates an optional process for coordination and synchronization testing. The coordination and synchronization test between the connection module and the clock synchronization module may include the following steps: Step S200: The connection module periodically generates a synchronization test signal and records the corresponding first clock time information Bn, which is obtained based on the connection module clock. Specifically, the general-purpose input / output interface is configured between the normally open domain processors in the connection module and the clock synchronization module. The first clock signal can control the level generation of the general-purpose input / output interface between the connection module and the clock synchronization module based on an interrupt function. The interrupt function is used to prompt the clock synchronization module to receive the synchronization test signal. In specific implementation, the normally open domain processor in the clock synchronization module can be prompted to receive the synchronization test signal based on an interrupt. The priority of this interrupt function can be the highest level to ensure that the actions of controlling the GPIO level and recording the timestamp are atomic operations, so as not to be interrupted by system scheduling and other interrupts.

[0074] It should be noted that there are multiple first clock time information, that is, the value of n in Bn can be 1, 2, 3, ..., n.

[0075] Step S210: The clock synchronization module receives the synchronization test signal from the connection module and records the second clock time information En corresponding to the time. The second clock time information is obtained based on the clock of the synchronization module. The clock synchronization module can receive synchronization test signals from the general-purpose input / output interface via interrupts. In a specific implementation, the normally open domain processor in the clock synchronization module receives the synchronization test signal based on an interrupt and records the value of the synchronization module clock (i.e., the timer) at that moment as the second clock time information En. In this specific example, the synchronization module clock is a timer.

[0076] Given that the priority of this interrupt function can be the highest, it can be guaranteed that the actions of controlling the GPIO level and recording the timestamp are atomic operations, and thus will not be interrupted by system scheduling or other interrupts.

[0077] It should be noted that there are multiple first clock time information, that is, the value of n in En can be 1, 2, 3, ..., n.

[0078] Step S220: The controller acquires the first clock time information Bn and the second clock time information En; The first clock time information is recorded by the connection module, and correspondingly, the first clock time information can be sent to the controller by the connection module through the serial port.

[0079] In an optional implementation, the controller may be configured with a computing application that can be used to calculate clock skew. In a specific example, the computing application may be a Linux application. Accordingly, the first clock timing information can be sent to the controller's driver layer by the connection module via a serial port, and the computing application obtains the first clock timing information through HCI (Host Controller Interface) commands.

[0080] The second clock timing information is recorded by the clock synchronization module, for example, by the normally open domain processor of the clock synchronization module. Correspondingly, the second clock timing information can be sent to the controller by the normally open domain processor of the clock synchronization module.

[0081] In an optional implementation, the normally open domain processor can send the second clock information to the controller's computing application via a hardware mailbox, so that the computing application can obtain the second clock information.

[0082] Step S230: The controller calculates the clock deviation between the connection module clock and the controller clock based on the first clock time information Bn and the second clock time information En. The clock deviation includes the initial deviation and the frequency deviation.

[0083] The initial deviation can be determined based on the difference between the second clock time information En and the first clock time information Bn, that is, , where n takes the initial values ​​of the first clock time information and the second clock time information.

[0084] Furthermore, the frequency deviation ppm can be calculated based on two consecutive second clock time information En and first clock time information Bn, as shown in the following formula:

[0085] Where n can be the initial value of the first clock time information and the second clock time information, or it can be the intermediate value of the first clock time information and the second clock time information.

[0086] The hardware product for implementing the playback control method provided in the embodiments of the present invention will now be described in detail. In conjunction with the aforementioned playback control method, as an optional implementation of the hardware product, the embodiments of the present invention further provide a controller, which may include a playback control module, the playback control module being used to execute the following logic: Determine the anchor point time information for transmitting the data to be played and the initial remaining time duration corresponding to the data to be played, wherein the time after the initial remaining time duration of the anchor point time corresponding to the anchor point time information is taken as the target playback time. Invalid data with a preset margin duration is inserted before the data to be played to form target playback data; the preset margin duration is determined based on the difference between the initial margin duration and the relative delay duration, and the relative delay duration is the sum of the relative transmission delay of the data to be played and the relative playback delay of the data to be played within the playback device; The target playback data is played starting from the inserted invalid data, so that the data to be played is played at the target playback time.

[0087] It should be noted that the above logic can be implemented based on fixed hardware logic or by software calling a specific instruction set; this invention does not impose any specific limitations on it.

[0088] In an optional example, the anchor point timing information is based on the connection module clock indication; Alternatively, the initial value of the initial margin duration is based on the controller clock indication, and the initial margin duration is calculated based on the initial value and the clock deviation between the controller clock and the connection module clock.

[0089] In an optional example, the playback device is a source device, and determining the anchor point time information for transmitting the data to be played and the initial remaining duration corresponding to the data to be played includes: The maximum relative delay of the data to be played is obtained in advance, wherein the maximum relative delay is the sum of the maximum relative transmission delay of the data to be played and the maximum relative playback delay of the data to be played within the receiving device; Select a value greater than or equal to the maximum relative delay as the initial margin duration.

[0090] In an optional example, the playback device is a receiving device. The determination of the anchor point time information for transmitting the data to be played and the initial remaining time duration of the corresponding data to be played includes: Receive configuration information from the source device, the configuration information including the anchor point time information and the initial margin duration; or, Receive configuration information from the source device, the configuration information including the anchor point time information; The device receives the data to be played and its redundancy information from the source device, wherein the redundancy information includes the initial margin duration.

[0091] In an optional example, the playback device is a source device, and the step of starting to play the target playback data from the inserted invalid data is triggered from the anchor point time. Alternatively, the step of starting playback of the target playback data from the inserted invalid data is triggered based on a preset instruction. In the step of inserting invalid data with a preset margin of time before the data to be played, the preset margin of time also includes a pre-transmission delay, which is the delay between the start of playback of the source device from the inserted invalid data and the transmission of the data to be played by the connection module.

[0092] In an optional example, before inserting invalid data of a preset remaining duration before the data to be played, the method further includes: Acquire first clock time information and second clock time information; wherein, the first clock time information is recorded when the connection module periodically generates a synchronization test signal, and the second clock time information is recorded when the clock synchronization module receives the synchronization test signal from the connection module, and the synchronization test signal is transmitted based on the general input / output interface between the connection module and the clock synchronization module; Based on the first clock time information and the second clock time information, the clock deviation between the connection module clock and the controller clock is calculated, and the clock deviation includes the initial deviation and the frequency deviation.

[0093] It is understood that the specific implementation of the logic executed by the playback control module can be referred to the description of the aforementioned playback control method, and will not be repeated here.

[0094] In a further optional implementation, embodiments of the present invention also provide a playback device, referring to reference 2 and Figure 5 An optional structural diagram of the provided playback device is shown, which includes the controller provided in the foregoing embodiments.

[0095] In a further alternative example, the playback device further includes a speaker, a connection module, and a clock synchronization module; The speaker is used to play the data to be played, the connection module is used to realize wireless connection, and the clock synchronization module is used to realize clock synchronization between the connection module and the controller.

[0096] In a further optional example, the clock synchronization module includes a normally open domain processor and a timer; the controller is configured with a processor core, and a computing application and a playback control module run based on the processor core; The connection module is connected to the normally open domain processor based on a general-purpose input / output interface; The connection module is used to record the level of the general input / output interface between the periodic control connection module and the normally open domain processor, generate the first clock moment information corresponding to the synchronous test signal, and send the first clock moment information to the driver layer of the controller via the serial port; the computing application is used to obtain the first clock moment information through the host controller interface command. The normally open domain processor is used to record the value of the timer corresponding to the receipt of the synchronization test signal as the second clock time information, and send it to the computing application of the controller through the hardware mailbox.

[0097] The foregoing describes multiple embodiments of the present invention. The optional methods described in each embodiment can be combined and cross-referenced without conflict, thereby extending to a variety of possible embodiments. These can all be considered as embodiments disclosed or made public by the present invention.

[0098] While the embodiments of the present invention have been disclosed above, the present invention is not limited thereto. Any person skilled in the art can make various modifications and alterations without departing from the spirit and scope of the present invention; therefore, the scope of protection of the present invention should be determined by the scope defined in the claims.

Claims

1. A playback control method, characterized in that, The method is applied to a playback device and includes: Determine the anchor point time information for transmitting the data to be played and the initial remaining time duration corresponding to the data to be played, wherein the time after the initial remaining time duration of the anchor point time corresponding to the anchor point time information is taken as the target playback time. Invalid data with a preset margin duration is inserted before the data to be played to form target playback data; the preset margin duration is determined based on the difference between the initial margin duration and the relative delay duration, and the relative delay duration is the sum of the relative transmission delay of the data to be played and the relative playback delay of the data to be played within the playback device; The target playback data is played starting from the inserted invalid data, so that the data to be played is played at the target playback time.

2. The playback control method according to claim 1, characterized in that, The anchor point time information is based on the clock indication of the connection module; Alternatively, the initial value of the initial margin duration is based on the controller clock indication, and the initial margin duration is calculated based on the initial value and the clock deviation between the controller clock and the connection module clock.

3. The playback control method according to claim 1 or 2, characterized in that, The playback device is a source device, and determining the anchor point time information for transmitting the data to be played and the initial remaining time for the corresponding data to be played includes: The maximum relative delay of the data to be played is obtained in advance, wherein the maximum relative delay is the sum of the maximum relative transmission delay of the data to be played and the maximum relative playback delay of the data to be played within the receiving device; Select a value greater than or equal to the maximum relative delay as the initial margin duration.

4. The playback control method according to claim 1 or 2, characterized in that, The playback device is a receiving device. The determination of the anchor point time information for transmitting the data to be played and the initial remaining time duration of the corresponding data to be played includes: Receive configuration information from the source device, the configuration information including the anchor point time information and the initial margin duration; or, Receive configuration information from the source device, the configuration information including the anchor point time information; The device receives the data to be played and its redundancy information from the source device, wherein the redundancy information includes the initial margin duration.

5. The playback control method according to claim 1, characterized in that, The playback device is a source device, and the step of starting to play the target playback data from the inserted invalid data is triggered from the anchor point moment; Alternatively, the step of starting playback of the target playback data from the inserted invalid data is triggered based on a preset instruction. In the step of inserting invalid data with a preset margin of time before the data to be played, the preset margin of time also includes a pre-transmission delay, which is the delay between the start of playback of the source device from the inserted invalid data and the transmission of the data to be played by the connection module.

6. The playback control method according to claim 2, characterized in that, Before inserting invalid data of a preset remaining duration before the data to be played, the method further includes: Acquire first clock time information and second clock time information; wherein, the first clock time information is recorded when the connection module periodically generates a synchronization test signal, and the second clock time information is recorded when the clock synchronization module receives the synchronization test signal from the connection module, and the synchronization test signal is transmitted based on the general input / output interface between the connection module and the clock synchronization module; Based on the first clock time information and the second clock time information, the clock deviation between the connection module clock and the controller clock is calculated, and the clock deviation includes the initial deviation and the frequency deviation.

7. A controller, characterized in that, Includes a playback control module, which is used to execute the following logic: Determine the anchor point time information for transmitting the data to be played and the initial remaining time duration corresponding to the data to be played, wherein the time after the initial remaining time duration of the anchor point time corresponding to the anchor point time information is taken as the target playback time. Invalid data with a preset margin duration is inserted before the data to be played to form target playback data; the preset margin duration is determined based on the difference between the initial margin duration and the relative delay duration, and the relative delay duration is the sum of the relative transmission delay of the data to be played and the relative playback delay of the data to be played within the playback device; The target playback data is played starting from the inserted invalid data, so that the data to be played is played at the target playback time.

8. A playback device, characterized in that, Includes the controller as described in claim 7.

9. The playback device according to claim 8, characterized in that, It also includes a speaker, a connection module, and a clock synchronization module; The speaker is used to play the data to be played, the connection module is used to realize wireless connection, and the clock synchronization module is used to realize clock synchronization between the connection module and the controller.

10. The playback device according to claim 9, characterized in that, The clock synchronization module includes a normally open domain processor and a timer; the controller is equipped with a processor core, and a computing application and a playback control module run based on the processor core; The connection module is connected to the normally open domain processor based on a general-purpose input / output interface; The connection module is used to record the level of the general input / output interface between the control connection module and the normally open domain processor based on the interrupt function periodically, generate the first clock moment information corresponding to the synchronous test signal, and send the first clock moment information to the driver layer of the controller through the serial port. The computing application is used to obtain the first clock time information through host controller interface commands; The normally open domain processor is used to record the value of the timer corresponding to the interrupt-received synchronization test signal as the second clock moment information, and send it to the controller's computing application via a hardware mailbox.

Images