A bluetooth audio synchronization control method based on multi-terminal cooperation

Abstract

The application discloses a Bluetooth audio synchronous control method based on multi-terminal cooperation, relates to the technical field of audio control, and is used for solving the problem that differentiating synchronous adjustment cannot be implemented for terminal groups with different deviation levels. After the Bluetooth audio enters a multi-terminal synchronous playing state, a listening period is established, the audio buffer margin, link retransmission times and decoding processing data of each Bluetooth earphone are collected, the link receiving state is analyzed and the group migration trend of the terminal is evaluated, the interactive behavior data of the users corresponding to each earphone is collected and cluster analysis is performed, a plurality of earphone groups are formed, the main playing group and the deviation group are divided according to the playing advancing speed, the group distance between the deviation group and the main playing group is detected, the corresponding group convergence strategy is configured according to the deviation level, the deviation group is adjusted, and thus the group split formed in the multi-terminal playing process is suppressed, and synchronous control in the Bluetooth earphone cooperative playing state is realized.

Description

Technical Field

[0001] This invention relates to the field of audio control technology, and more specifically, to a Bluetooth audio synchronization control method based on multi-terminal collaboration. Background Technology

[0002] With the development of Bluetooth audio technology, there are increasingly more application scenarios where multiple Bluetooth headphones can simultaneously receive the same audio content and play it in coordination. These scenarios include multi-person audio sharing, group movie watching, team training, and immersive interactive entertainment. In such applications, the system needs to ensure that multiple Bluetooth headphones maintain a consistent playback progress during long-term playback to avoid significant misalignment of the audio content heard by different users.

[0003] The existing technology has the following shortcomings: Currently, in practical applications, there is a lack of a group migration recognition mechanism for the evolution of multi-terminal playback states and a hierarchical convergence control mechanism for the degree of deviation of terminal groups. This makes it difficult for the system to detect the trend of multiple Bluetooth headsets gradually splitting from a unified synchronization state to different playback groups in a timely manner. It is also impossible to implement differentiated synchronization adjustment for terminal groups with different levels of deviation, which leads to continuous deviation of playback progress between groups, repeated accumulation of synchronization errors, and decreased stability of multi-terminal collaborative playback. Therefore, a Bluetooth audio synchronization control method based on multi-terminal collaboration is proposed.

[0004] The information disclosed in the background section is only intended to enhance the understanding of the background of this disclosure, and therefore may include information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0005] To overcome the aforementioned deficiencies of the prior art, embodiments of the present invention provide a Bluetooth audio synchronization control method based on multi-terminal collaboration. This method addresses the problems mentioned in the background art by employing a group migration trend evaluation mechanism based on the fusion analysis of link reception status and decoding processing status, a terminal clustering analysis mechanism based on user interaction behavior, and a group convergence control mechanism oriented towards different deviation levels.

[0006] To achieve the above objectives, the present invention provides the following technical solution: a Bluetooth audio synchronization control method based on multi-terminal collaboration, comprising the following steps: Step S1: After Bluetooth audio playback, set the monitoring period, collect the audio buffer margin and link retransmission count of each Bluetooth headset within the monitoring period, calculate the link fluctuation coefficient based on the link retransmission count, and analyze the link reception status in combination with the audio buffer margin. Step S2: Detect the decoding processing data of each Bluetooth headset decoder, calculate the phase offset based on the decoding processing data, evaluate the group migration trend by combining the phase offset and the link reception status, and determine whether to enter the clustering convergence mechanism based on the group migration trend. Step S3: When entering the clustering convergence mechanism, collect the interaction behavior data of the corresponding users of each Bluetooth headset, perform clustering analysis based on the interaction behavior data and generate different headset groups, and count the playback progress speed of each headset group according to the clustering analysis results. Step S4: Count the number of Bluetooth headphones in the headphone group and divide the headphone group into the main playback group and the deviation group. Calculate the group distance between each deviation group and the main playback group based on the playback speed. Apply different group convergence strategies to the deviation groups based on the deviation level.

[0007] In a preferred embodiment, in step S1, after the Bluetooth audio enters the playback state, a preset monitoring period is started. The monitoring period is used to periodically sample and update the link reception characteristics of each Bluetooth headset during continuous playback. During the monitoring period, the audio buffer margin of each Bluetooth headset is collected. The audio buffer margin is the amount of audio data that has not yet been decoded and played in the local audio playback buffer queue of each Bluetooth headset. At the same time, the number of link retransmissions for each Bluetooth headset during the listening period is collected. The number of link retransmissions is the automatic retransmission count triggered during the transmission of audio data packets due to verification failure or failure to acknowledge receipt. Retrieve the preset standard retransmission baseline value, divide the number of link retransmissions of each Bluetooth headset in the current monitoring period by the standard retransmission baseline value, and obtain the link fluctuation coefficient. By combining the link fluctuation coefficient with the audio buffer margin of the corresponding Bluetooth headset using a weighted combination method, the link reception status is constructed.

[0008] In a preferred embodiment, in step S2, the operating status of the decoder of each Bluetooth headset is continuously monitored, and decoding processing data is obtained, including the decoding queue length and the number of audio frames dropped. The decoder is a functional module inside each Bluetooth headset used to decode and process received audio data; The decoding queue length is the current backlog length of the audio frame buffer queue to be processed inside each Bluetooth headset decoder; The number of audio frames dropped is the number of audio frames dropped during the decoding process due to decoding timeout, insufficient buffer, or frame verification failure.

[0009] In a preferred embodiment, in step S2, a standard decoding processing baseline queue length and a standard frame discard baseline value are introduced, and the normalized decoding length is obtained by dividing the decoding queue length by the standard decoding processing baseline queue length. The normalized frame drop count is obtained by dividing the number of audio frame drops by the standard frame drop baseline value. The phase offset is defined as the average of the normalized decoding length and the normalized frame drop count. The phase offset and link reception status are fused to construct the population migration trend.

[0010] In a preferred embodiment, in step S2, when the group migration trend is greater than or equal to a preset group convergence trigger threshold, the corresponding Bluetooth headset is determined to be in a high migration risk state; otherwise, the corresponding Bluetooth headset is determined not to be in a high migration risk state. The percentage of migrated terminals is obtained by dividing the number of Bluetooth headsets at high migration risk by the total number of Bluetooth headsets. When the proportion of migrating terminals exceeds the preset threshold for group structure instability, it is determined that the current multi-terminal playback has evolved from a local offset state to a group migration state, triggering the entry into the clustering convergence mechanism; Conversely, if the clustering convergence mechanism is not followed, the process will be continuously monitored.

[0011] In a preferred embodiment, in step S3, the interaction behavior data includes two items: volume adjustment event density and wearing posture adjustment frequency. Volume adjustment event density refers to the ratio of the number of times a user presses the volume up or volume down button on the navigation terminal paired with the Bluetooth headset to the duration of the monitoring period. Wearing posture adjustment frequency refers to the ratio of the number of times the Bluetooth headset body relative to the user's head coordinate system undergoes movements greater than the posture change threshold within the monitoring period to the duration of the monitoring period. The interaction perturbation index was calculated for each Bluetooth headset by combining the normalized results of the wearing posture adjustment frequency and the normalized results of the volume adjustment event density. Using the standardized phase offset of each Bluetooth headset as the horizontal axis and the interaction perturbation index as the vertical axis, a two-dimensional feature space is constructed, with each Bluetooth headset corresponding to a feature point in the two-dimensional feature space.

[0012] In a preferred embodiment, in step S3, the distance between all pairs of Bluetooth headsets in the two-dimensional feature space during the current monitoring period is calculated, the distances are sorted in ascending order, and the lower quartile is taken as the neighborhood radius. For each Bluetooth headset, count the number of feature points contained in its two-dimensional feature space neighborhood with the neighborhood radius as the radius, and use this as the local density of the Bluetooth headset. For each Bluetooth headset, among all Bluetooth headsets with a local density higher than its own, the distance to the closest one is taken as the local isolation distance of the Bluetooth headset; The median value of the local density of each Bluetooth headset is taken as the lower limit for density determination; the median value of the local isolation distance of each Bluetooth headset is taken as the lower limit for distance determination.

[0013] In a preferred embodiment, in step S3, Bluetooth earphones with local density greater than the density determination lower limit and local isolation distance greater than the distance determination lower limit are identified as cluster centers, and the remaining Bluetooth earphones are sequentially assigned to the cluster center closest to themselves, generating several earphone groups; For each headphone group, the inverse ratio of the interaction perturbation index of each Bluetooth headphone is used as the robustness weight, and the rate of change of the phase offset of each Bluetooth headphone within the statistical window is weighted and averaged to obtain the playback propagation speed of the headphone group.

[0014] In a preferred embodiment, in step S4, for each headphone group, the number of Bluetooth headphones it contains is counted; the headphone group with the largest number of Bluetooth headphones is identified as the main playback group, and the remaining headphone groups are identified as off-target groups; The median value of the phase offset of all Bluetooth headsets included in the main playback group is taken as the phase anchor point of the main playback group. The median value of the phase offset of each Bluetooth headset in the off-group is taken and subtracted from the phase anchor point of the main playback group to obtain the phase difference of the off-group. The speed difference of the off-group is obtained by subtracting the playback speed of the off-group from the playback speed of the main group. The group distance between the off-group and the main playback group is calculated by combining the phase difference and velocity difference.

[0015] In a preferred embodiment, in step S4, the group distance of each deviation group is compared with a first deviation level threshold and a second deviation level threshold, respectively: When the group distance of the offshoot is less than the first deviation level threshold, the buffer level fine-tuning strategy is executed, and the buffer level bias is sent to the decoder of each Bluetooth headset to which the offshoot belongs. When the group distance of the off-group is greater than or equal to the first deviation level threshold and less than the second deviation level threshold, the variable speed playback strategy is executed, and the playback rate coefficient is sent to the decoder of each Bluetooth headset to which the off-group belongs. When the group distance of the offshoot is greater than or equal to the second deviation level threshold, the anchor point alignment strategy is executed, and the current playback anchor point timestamp of the main playback group is sent to each Bluetooth headset belonging to the offshoot.

[0016] The technical effects and advantages of this invention are as follows: This invention establishes a monitoring cycle after Bluetooth audio enters a multi-terminal synchronous playback state, collects the audio buffer margin and link retransmission count of each Bluetooth headset, analyzes the current link reception state of each headset, obtains the decoding processing data of each Bluetooth headset decoder, calculates the phase offset based on the decoding processing data, and evaluates the group migration trend of the terminals in conjunction with the link reception state to initiate a clustering convergence mechanism. After entering the clustering convergence mechanism, it collects the interaction behavior data of the corresponding users of each headset, uses the interaction behavior data to perform cluster analysis on the headset terminals, forms multiple headset groups, and counts the playback progress speed to divide each headset group into a main playback group and a deviation group. It detects the group distance between the deviation group and the main playback group, evaluates the deviation level based on the group distance, configures corresponding group convergence strategies according to different deviation levels, and adjusts the deviation group to suppress the group split formed during multi-terminal playback, thereby achieving synchronous control in the Bluetooth headset collaborative playback state. Attached Figure Description

[0017] Figure 1 This is a flowchart illustrating the implementation of a Bluetooth audio synchronization control method based on multi-terminal collaboration according to the present invention.

[0018] Figure 2 This is a schematic diagram illustrating the steps of a Bluetooth audio synchronization control method based on multi-terminal collaboration according to the present invention. Detailed Implementation

[0019] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0020] This invention establishes a monitoring cycle after Bluetooth audio enters a multi-terminal synchronous playback state, collects the audio buffer margin and link retransmission count of each Bluetooth headset, analyzes the current link reception status of each headset, obtains the decoding processing data of each Bluetooth headset decoder, calculates the phase offset based on the decoding processing data, and evaluates the group migration trend of the terminals in conjunction with the link reception status to initiate a clustering convergence mechanism. After entering the clustering convergence mechanism, it collects the interaction behavior data of the corresponding users of each headset, uses the interaction behavior data to perform cluster analysis on the headset terminals, forms multiple headset groups, and counts the playback progress speed to divide each headset group into a main playback group and a deviation group. It detects the group distance between the deviation group and the main playback group, evaluates the deviation level based on the group distance, configures corresponding group convergence strategies according to different deviation levels, and adjusts the deviation group to suppress the group splitting formed during multi-terminal playback.

[0021] Example 1, as Figures 1 to 2 As shown, a Bluetooth audio synchronization control method based on multi-terminal collaboration includes the following steps: Step S1: After Bluetooth audio playback, set the monitoring period, collect the audio buffer margin and link retransmission count of each Bluetooth headset within the monitoring period, calculate the link fluctuation coefficient based on the link retransmission count, and analyze the link reception status in combination with the audio buffer margin. Step S2: Detect the decoding processing data of each Bluetooth headset decoder, calculate the phase offset based on the decoding processing data, evaluate the group migration trend by combining the phase offset and the link reception status, and determine whether to enter the clustering convergence mechanism based on the group migration trend. Step S3: When entering the clustering convergence mechanism, collect the interaction behavior data of the corresponding users of each Bluetooth headset, perform clustering analysis based on the interaction behavior data and generate different headset groups, and count the playback progress speed of each headset group according to the clustering analysis results. Step S4: Count the number of Bluetooth headphones in the headphone group and divide the headphone group into the main playback group and the deviation group. Calculate the group distance between each deviation group and the main playback group based on the playback speed. Apply different group convergence strategies to the deviation groups based on the deviation level.

[0022] The specific implementation is as follows: In step S1, after the Bluetooth audio enters the playback state, a preset monitoring period is started. The monitoring period is used to periodically sample and update the link reception characteristics of each Bluetooth headset during continuous playback.

[0023] During the monitoring period, the audio buffer margin of each Bluetooth headset is collected. The audio buffer margin is the amount of audio data that has not yet been decoded and played in the local audio playback buffer queue of each Bluetooth headset. It is represented by the duration of the current buffer data. The audio buffer margin reflects the redundancy reserve capacity of the current playback continuity of each Bluetooth headset. The larger the value, the stronger the resistance of the Bluetooth headset to link jitter.

[0024] Simultaneously, the number of link retransmissions for each Bluetooth headset within the monitoring period is collected. The link retransmission count is the automatic retransmission count triggered during the transmission of audio data packets due to verification failure or failure to acknowledge receipt. The link retransmission count is calculated as the cumulative number within a unit time window, reflecting the current wireless link transmission reliability level. The higher the link retransmission count, the higher the degree of interference to the current wireless link or the more unstable the link quality.

[0025] After obtaining the number of link retransmissions, the data is normalized and the link fluctuation coefficient is calculated. Specifically, a preset standard retransmission baseline value is retrieved, and the number of link retransmissions for each Bluetooth headset within the current monitoring period is divided by the standard retransmission baseline value to obtain the link fluctuation coefficient. The link fluctuation coefficient characterizes the relative degree of fluctuation in the link stability of each Bluetooth headset; a larger value indicates greater uncertainty in the link transmission process.

[0026] It should be noted that the standard retransmission baseline value reflects the basic retransmission level allowed by the link layer under normal and stable playback conditions. It is determined based on historical statistical data under a stable wireless environment during the preset calibration phase. Specifically, in a test scenario with no obvious external interference and the link quality is in a stable state, the number of link retransmissions within a unit time window of each Bluetooth headset is sampled multiple times, and the sampling results are averaged to obtain the standard retransmission baseline value.

[0027] After calculating the link fluctuation coefficient, the link fluctuation coefficient and the corresponding Bluetooth headset's audio buffer margin are jointly analyzed using a weighted combination method to construct the link reception state: ; in, This indicates the link reception status. This is for audio buffer margin. This is the link variability coefficient. This is the index value for the Bluetooth headset. and The preset weighting coefficient is used to balance the strength of the buffer redundancy capacity and the impact of link fluctuations.

[0028] The link reception status reflects the overall reception stability of each Bluetooth headset during the current listening period. The higher the value, the lower the link fluctuation and the more stable the reception status of the Bluetooth headset. Conversely, the lower the value, the more unstable the link factors are, and there is a risk of playback continuity.

[0029] It should be noted that during offline training or historical playback data playback analysis, multiple sets of samples with different combinations of link quality and playback stability are selected. The contribution of audio buffer margin and link fluctuation coefficient to the final playback interruption probability or phase offset error is calculated respectively. The ratio of the preset weight coefficients α and β is determined by the error minimization fitting method. The link reception status is non-negative. If it is negative in the calculation, it is assigned a value of 0.

[0030] In step S2, the operating status of the decoder of each Bluetooth headset is continuously monitored, and decoding processing data is obtained, including the decoding queue length and the number of audio frames dropped.

[0031] It should be noted that the decoder is a functional module inside each Bluetooth headset used to decode and process received audio data. It is used to restore the encoded audio data frames transmitted via the Bluetooth link into a playable audio signal sequence, and maintain a decoding queue to store audio frames to be processed during the decoding process. At the same time, it performs frame parsing, error checking and reconstruction output on the input audio data.

[0032] The decoding queue length is the current backlog length of the audio frame buffer queue inside each Bluetooth headset decoder. It is quantified by the number of frames within a unit time window, reflecting the processing pressure level of the decoder on the received audio data. The larger the decoding queue length, the higher the degree of decoding lag and the more obvious the trend of audio data backlog at the decoder.

[0033] The audio frame drop count refers to the number of audio frames dropped during the decoding process due to decoding timeouts, insufficient buffering, or frame verification failures. It is calculated based on the cumulative number of dropped frames within the monitoring period and reflects the level of decoding stability and data integrity. A higher audio frame drop count indicates poorer decoding continuity.

[0034] Based on the decoded data, the phase offset of each Bluetooth headset is calculated. The phase offset characterizes the degree of time deviation of each Bluetooth headset's playback progress relative to a standard playback reference. Specifically, a standard decoding processing reference queue length and a standard frame drop reference value are introduced. The normalized decoding length is obtained by dividing the decoding queue length by the standard decoding processing reference queue length, and the normalized frame drop count is obtained by dividing the number of audio frame drops by the standard frame drop reference value. The phase offset is defined as the average of the normalized decoding length and the normalized frame drop count. The phase offset reflects the degree of deviation of each Bluetooth headset's playback progress at the decoding level; the larger the value, the more lagging or unstable the actual playback progress of the Bluetooth headset is relative to the reference reference.

[0035] It should be noted that both the standard decoding processing baseline queue length and the standard frame drop baseline value are used to standardize the decoded processing data. The standard decoding processing baseline queue length is a reference length obtained by statistically analyzing the average or steady-state median of the decoding queue length within a unit time window in historical stable playback scenarios under normal decoder load conditions. It reflects the typical processing capability baseline of the decoder under no congestion or light load conditions. The standard frame drop baseline value is a baseline level of the number of audio frames dropped within a unit time window under the same stable playback conditions, reflecting the minimum data loss level allowed in the decoding process under normal conditions.

[0036] The phase offset and link reception status are fused to construct the group migration trend, as shown in the following expression: ; in, As a trend of group migration, This is the phase offset. This indicates the link reception status. To prevent small constants from being divided by zero, This is the index value for the Bluetooth headset.

[0037] The group migration trend characterizes the degree of phase offset amplification of each Bluetooth headset under the constraints of link reception status. The larger the value, the more significantly the playback phase offset of the Bluetooth headset is amplified under conditions of low link stability, and the easier it is to deviate from the synchronous playback trajectory of the main playback group; conversely, it indicates that its playback progress is more strongly constrained by the link status and still maintains a high degree of synchronization consistency.

[0038] The migration trend of each Bluetooth headset is compared with the preset group convergence trigger threshold. The group convergence trigger threshold represents the boundary of the maximum phase shift amplification allowed by the system, and its value is determined by the upper quantile value of the group migration trend distribution obtained from historical stable playback scenarios.

[0039] When the migration trend of the group is greater than or equal to the preset group convergence trigger threshold, the corresponding Bluetooth headset is determined to be in a high migration risk state. Under the constraints of the current link reception state, the phase offset has shown a nonlinear amplification effect relative to the link stability, indicating that the corresponding Bluetooth headset is no longer in a stable synchronous playback range and has begun to evolve continuously in a direction deviating from the main playback phase.

[0040] Conversely, it is determined that the corresponding Bluetooth headset is not in a high-risk migration state.

[0041] The percentage of Bluetooth headsets at high migration risk is calculated by dividing the number of Bluetooth headsets at high migration risk by the total number of Bluetooth headsets. This percentage represents the structural proportion of Bluetooth headsets at high migration risk within the overall population. A higher percentage indicates a greater degree of disruption to the population's synchronization structure and a more pronounced trend towards a fragmented evolutionary state.

[0042] When the proportion of migrating terminals exceeds a preset threshold for group structure instability, the current multi-terminal playback is determined to have evolved from a local offset state to a group migration state. That is, the phase offset between different terminals no longer exhibits isolated random fluctuations, but rather forms a group offset diffusion phenomenon with statistical clustering characteristics. At this point, the synchronization structure exhibits obvious precursory characteristics of group splitting, thus triggering the entry into a clustering convergence mechanism to further cluster and hierarchically converge to control the current group migration state of each Bluetooth headset, thereby suppressing further phase diffusion in group playback and restoring overall synchronization consistency.

[0043] Conversely, if the clustering convergence mechanism is not entered, the above monitoring process continues.

[0044] It should be noted that the preset group structure instability threshold is a judgment parameter used to determine whether a multi-terminal playback system has evolved from a local offset state to a group migration state. It is determined by the distribution data of group migration trend indicators under historical stable synchronous playback scenarios. Specifically, it is obtained by extracting the upper quantile statistical values ​​of the migration trend of each Bluetooth headset group during long-term stable playback. For example, the 95th percentile value is taken as the preset group structure instability threshold.

[0045] In step S3, when entering the clustering convergence mechanism, the interaction behavior data of the corresponding user of each Bluetooth headset is collected; Interactive behavior data refers to the data formed by user-initiated actions. In this embodiment, the interactive behavior data includes two items: volume adjustment event density and wearing posture adjustment frequency.

[0046] Volume adjustment event density refers to the ratio of the number of times a user presses the volume up or volume down button on the navigation terminal paired with each Bluetooth headset during the monitoring period to the duration of the monitoring period. It is obtained from the button logs of the navigation terminal.

[0047] The wearing posture adjustment frequency refers to the ratio of the number of times each Bluetooth headset body performs actions greater than the posture change threshold relative to the user's head coordinate system within the listening period to the duration of the listening period. It is obtained by the posture change detection module of the inertial measurement unit built into each Bluetooth headset.

[0048] It should be added that the attitude change detection module refers to a functional module integrated inside the Bluetooth headset that monitors the motion state of the headset body relative to the user's head coordinate system in real time based on the angular velocity output of the inertial measurement unit. The inertial measurement unit continuously outputs three-axis angular velocity data at a fixed sampling frequency. The attitude change detection module calculates the resultant vector of the angular velocities of each axis in real time. When the amplitude of the resultant vector exceeds the attitude change threshold, it determines that an attitude change event has occurred and counts it. At the end of each listening cycle, the cumulative number of attitude change events in the cycle is reported to the guide terminal. The guide terminal divides the number of events by the listening cycle duration to obtain the wearing posture adjustment frequency.

[0049] The posture change threshold is a critical value of angular velocity used to distinguish between minor daily shaking and posture changes caused by actively straightening or re-wearing the headphones due to abnormal hearing. The posture change threshold is set as follows: in the pre-deployment test, the statistical distribution of the angular velocity of the headphones relative to the head coordinate system of the subject in the normal listening state is recorded, and the upper quartile of the distribution is taken as the posture change threshold so that minor daily shaking is not counted.

[0050] The volume adjustment event density and the wearing posture adjustment frequency were respectively normalized by range, and the calculation formula is as follows: ; ; Where e represents the volume adjustment event density of the Bluetooth headset, and a represents the wearing posture adjustment frequency of the Bluetooth headset. This is the normalized result for the volume adjustment event density. This is a normalized result of the wearing posture adjustment frequency. , These represent the minimum and maximum values ​​of the volume adjustment event density for all Bluetooth headphones within this monitoring period, respectively. , These represent the minimum and maximum frequency of wearing posture adjustments for all Bluetooth headsets within the current monitoring period, respectively.

[0051] The interaction perturbation index is calculated for each Bluetooth headset based on the normalized results of the wearing posture adjustment frequency and the normalized results of the volume adjustment event density. The calculation formula is as follows: ; in, Let represent the interaction perturbation index of the i-th Bluetooth headset. This represents the normalized volume adjustment event density of the i-th Bluetooth headset. This represents the normalized wearing posture adjustment frequency of the i-th Bluetooth headset.

[0052] The Interaction Disturbance Index reflects the overall intensity of user intervention actions due to abnormal hearing. The higher the Interaction Disturbance Index value, the higher the intensity of active intervention by the user due to abnormal hearing in the Bluetooth headset, and the greater the degree to which the corresponding playback trajectory is disturbed by the user's operation.

[0053] Using the standardized phase offset of each Bluetooth headset as the x-axis and the interaction perturbation index as the y-axis, a two-dimensional feature space is constructed. Each Bluetooth headset corresponds to a feature point in the two-dimensional feature space during this monitoring period. Density peak clustering is then performed on all Bluetooth headsets in this feature space. The specific process is as follows: Calculate the distance between all pairs of Bluetooth earphones in the two-dimensional feature space during this monitoring period, sort the distances in ascending order, and take the lower quartile as the neighborhood radius.

[0054] For each Bluetooth headset, count the number of feature points it contains in a two-dimensional feature space with a radius equal to the neighborhood radius, and use this count as the local density of the Bluetooth headset.

[0055] For each Bluetooth headset, among all Bluetooth headsets with a local density greater than that of the Bluetooth headset, the one with the smallest distance from the Bluetooth headset is selected, and this smallest distance is taken as the local isolation distance of the Bluetooth headset; for the Bluetooth headset with the highest local density, the maximum value of its feature space distance from all other Bluetooth headsets is taken as its local isolation distance.

[0056] The median value of the local density of all Bluetooth headsets within this monitoring period is taken as the lower limit for density determination; the median value of the local isolation distance of all Bluetooth headsets within this monitoring period is taken as the lower limit for distance determination.

[0057] Bluetooth earphones with local density greater than the density threshold and local isolation distance greater than the distance threshold are identified as cluster centers. The remaining Bluetooth earphones are assigned to the cluster center closest to them, thus generating several earphone groups.

[0058] For each group of headphones, within the statistical window, the difference between the phase offset at the end of the statistical window and the beginning of the statistical window is taken, divided by the duration of the statistical window, to obtain the rate of change of the phase offset of the Bluetooth headphones within the statistical window. A robust weight is then applied to the rate contribution of each Bluetooth headphone using the interaction perturbation index to calculate the playback propagation speed. The calculation formula is as follows: ; ; in, This represents the playback speed of the g-th headphone group; This represents the change in phase offset of the i-th Bluetooth headset within the statistical window; For the duration of the statistical window; Let represent the robustness weight of the i-th Bluetooth headset; Let be the interaction perturbation index of the i-th Bluetooth headset; i∈g means summing over all Bluetooth headsets belonging to the g-th headset group.

[0059] It should be added that the statistical window refers to the time interval used to calculate the rate of change of phase offset, which is a fixed-length window composed of the most recent consecutive sampling moments within the current monitoring period.

[0060] The larger the interaction disturbance index, the more human intervention the user has in the playback process of the Bluetooth headset. Its phase change within the statistical window is more likely to be affected by user operation rather than link status. Therefore, it is given a smaller robustness weight in the statistics of playback progress speed. The larger the playback progress speed value, the faster the playback progress of the headset group drifts relative to the system master clock.

[0061] When the number of headphone groups identified in this monitoring cycle is greater than 1, it is determined that a group split has occurred, and all identified headphone groups and their corresponding playback speeds are passed to step S4. When the number of headphone groups identified in this monitoring cycle is equal to 1, it is determined to be a state of overall synchronization. This monitoring cycle will not proceed to step S4, and will wait for the next monitoring cycle to re-evaluate.

[0062] In step S4, the entire headphone group output in step S3 is divided into a main playback group and an off-target group, and the division rules are as follows: For each headphone group, count the number of Bluetooth headphones it contains; identify the headphone group with the largest number of Bluetooth headphones as the main playback group, and identify the other headphone groups as off-target groups.

[0063] The main playback group refers to the group of headphones that represents the actual listening progress of most listeners during the current monitoring period; the off-group refers to the group of headphones that deviates from the main playback group in terms of phase and playback speed.

[0064] When there are two or more groups of Bluetooth headphones with the same maximum number of headphones, the absolute value of the difference between the playback speed of each candidate group and the system master clock reference speed is further compared, and the group with the smallest absolute value of the difference is selected as the main playback group.

[0065] The system master clock reference advance speed is the nominal playback rate corresponding to the duration of audio content advanced per unit time by the playback source with reference to the system master clock. It is directly given by the clock module of the playback source.

[0066] The median value of the phase offset of all Bluetooth headsets included in the main playback group is taken as the phase anchor point of the main playback group.

[0067] For each deviation group, the group distance between the deviation group and the main playback group is calculated. The group distance is used to reflect the degree of deviation of the deviation group relative to the main playback group in the phase channel and playback speed channel.

[0068] The median value of the phase offset of all Bluetooth headsets included in the k-th deviation group is taken, and the difference is calculated with the phase anchor point of the main playback group to obtain the phase difference of the k-th deviation group: The speed difference of the kth deviation group is obtained by subtracting the playback speed of the kth deviation group from the playback speed of the headphone group in the main playback group: The phase difference and velocity difference of the deviation group are dimensionless by using phase-normalized and velocity-normalized references respectively: ; ; in, Indicates the phase normalization reference; The speed standardization reference is taken from the system master clock reference speed; This represents the dimensionless phase difference of the k-th deviation group; This represents the dimensionless velocity difference of the kth deviation group.

[0069] The phase standardization benchmark is determined by taking a reference value based on engineering experience (the perceptible threshold for audio synchronization deviation between different playback terminals is usually 10 to 20 milliseconds) and then correcting it based on the actual Bluetooth headset model and the sampling rate of the tour audio.

[0070] Based on different norm indices selected according to the deviation direction, the group distance between the k-th deviation group and the main playback group is calculated according to the following formula: ; in, β represents the group distance of the kth deviation group relative to the main playback group; β is the sensitivity coefficient of the velocity channel relative to the phase channel, and in this embodiment, β is taken as 1.2; p is the direction-sensitive norm exponent.

[0071] The sensitivity coefficient can be determined as follows: During the calibration phase before deployment, apply a known amplitude deviation to the phase channel and the velocity channel separately, and record the response amplitude of the buffer level of each Bluetooth headset decoder to the two types of deviations; take the ratio of the buffer level response amplitude caused by the phase channel deviation to the buffer level response amplitude caused by the velocity channel deviation, and obtain the sensitivity ratio of the two channels to the buffer level, and use this ratio as the value of β.

[0072] The rules for determining the norm exponent p are as follows: When the phase difference of the kth deviation group is greater than or equal to zero, the deviation group is determined to be ahead of the main playback group, and the norm exponent p is 3. When the phase difference of the kth deviation group is less than zero, the deviation group is determined to be lagging behind the main playback group, and the norm exponent p is 1.5.

[0073] Both the phase standardization benchmark and the velocity standardization benchmark were determined through pre-experimentation. The two values ​​of the norm index p were also obtained by comparing the fitting slopes of the distance curves of the two types of auditory scores, namely lead deviation and lag deviation, in the pre-experimentation.

[0074] The larger the group distance value, the more severe the deviation of the deviation group from the main playback group in terms of phase and playback speed; the smaller the group distance value, the closer the deviation group is to the main playback group.

[0075] Let there be two deviation level thresholds, denoted as the first deviation level threshold and the second deviation level threshold, with the first deviation level threshold being less than the second deviation level threshold.

[0076] The first deviation level threshold is determined based on the frame buffering mechanism of the Bluetooth audio link: the duration of a single frame of the Bluetooth audio codec is usually 7.5 milliseconds to 10 milliseconds. When the phase offset exceeds the duration of a complete codec frame, the playback content between adjacent terminals has become discontinuous at the frame level. The group distance calculation result corresponding to the duration of a complete codec frame is used as the first deviation level threshold, so that the discontinuity at the frame level becomes the basis for distinguishing between mild and moderate deviations.

[0077] The second deviation level threshold is determined based on the upper limit of the Bluetooth audio broadcast buffer capacity: the buffer capacity of the Bluetooth broadcast audio receiver is usually no more than 40 milliseconds. When the phase offset reaches the upper limit of the receiver buffer capacity, the buffer level fine-tuning and variable speed playback strategies can hardly completely eliminate the deviation within a single buffer cycle, and the anchor point alignment strategy is required to force alignment. The group distance calculation result corresponding to the upper limit of the buffer capacity is used as the second deviation level threshold, so that exceeding the adjustable range of the buffer becomes the basis for classifying moderate and severe deviation.

[0078] The group distance of each deviation group is compared with the first deviation level threshold and the second deviation level threshold, respectively: When the group distance of the deviation group is less than the first deviation level threshold, the deviation group is determined to be a mild deviation group. A buffer level fine-tuning strategy is implemented for the mild deviation group, that is, a buffer level offset is sent to the decoder of each Bluetooth headset to which the mild deviation group belongs. The sign of the buffer level offset is opposite to the sign of the phase difference of the deviation group. The amplitude of the buffer level offset is taken as a multiple of the preset buffer reduction coefficient of the phase difference amplitude of the deviation group. The preset buffer reduction coefficient is less than 1, so that each Bluetooth headset can gradually recover the phase difference through the small rise and fall of the buffer level without changing the audio content or changing the playback speed. When the group distance of an offshoot is greater than or equal to the first deviation level threshold and less than the second deviation level threshold, the offshoot is determined to be a moderate offshoot. A variable-speed playback strategy is implemented for moderate offshoots, i.e., playback rate coefficients are sent to the decoders of each Bluetooth headset belonging to the moderate offshoot. The formula for calculating the playback rate coefficient is: ; in, γ represents the playback rate coefficient corresponding to the kth deviation group, and γ is the speed variation amplitude limiting factor. The value of γ is greater than 1, which limits the deviation of the playback rate coefficient from 1. The decoders of each Bluetooth headset belonging to the moderate deviation group play at a speed slightly lower or slightly higher than the main playback group while keeping the tone unchanged, gradually smoothing out the speed difference and making the phase difference converge to zero. When the group distance of the deviation group is greater than or equal to the second deviation level threshold, the deviation group is determined to be a severe deviation group. Anchor point alignment strategy is executed for severe deviation groups, that is, the current playback anchor point timestamp of the main playback group is sent to each Bluetooth headset belonging to the severe deviation group, and each Bluetooth headset belonging to the severe deviation group inserts a silent segment with a preset silent bridging duration. Then, playback is resumed from the audio frame position corresponding to the current playback anchor point timestamp of the main playback group, thus forcibly achieving phase alignment with the main playback group.

[0079] The current playback anchor point timestamp of the main playback group refers to the timestamp of the audio frame being decoded and output by most Bluetooth headphones in the main playback group on the original audio stream timeline at the moment the anchor point alignment strategy is executed. It is obtained by taking the median value of the current frame index of the decoder of each Bluetooth headphone in the main playback group and mapping it to the original audio stream timeline.

[0080] The preset silence bridging duration is used to avoid obvious popping sounds when the audio frame position changes due to severe deviations. The value of the preset silence bridging duration is not greater than the lower limit of human hearing's ability to perceive short silences. In this embodiment, the value is determined based on pre-tests under the same deployment conditions.

[0081] The preset buffer reduction coefficient and the speed variation limit factor are engineering implementation parameters that are unrelated to the core algorithm of the invention. They can be set according to the upper limit of the buffer capacity of each Bluetooth headset decoder and the maximum allowable speed variation of the speed variation algorithm, and will not be elaborated here.

[0082] The greater the group distance, the stronger the corresponding group convergence strategy intervention: mild deviation groups use buffer level as a buffering means, which is imperceptible to users; moderate deviation groups use variable speed playback as a medium intensity means, which is basically imperceptible to users but the phase difference converges faster; severe deviation groups use anchor point alignment as a strong intervention means, which will make users perceive a brief silence, but the phase difference immediately returns to zero.

[0083] Finally, it should be noted that in this paper, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations.

[0084] Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0085] In this document, the singular forms “a,” “an,” and “the” may also include the plural forms unless the context clearly indicates otherwise. It should also be understood that terms such as “comprising / including” or “having” specify the presence of the stated features, integrals, steps, operations, components, parts, or combinations thereof, but do not preclude the possibility of the presence or addition of one or more other features, integrals, steps, operations, components, parts, or combinations thereof. Meanwhile, the term “and / or” as used in this specification includes any and all combinations of the associated listed items.

[0086] The various embodiments in this specification are described in a progressive manner. Each embodiment focuses on the differences from other embodiments. The various embodiments can be combined as needed, and the same or similar parts can be referred to each other.

[0087] The above description of the disclosed embodiments will enable those skilled in the art to make or use various modifications to these embodiments. It will be readily apparent to those skilled in the art that the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of this application. Therefore, this application is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A Bluetooth audio synchronization control method based on multi-terminal collaboration, characterized in that: Includes the following steps: Step S1: After Bluetooth audio playback, set the monitoring period, collect the audio buffer margin and link retransmission count of each Bluetooth headset within the monitoring period, calculate the link fluctuation coefficient based on the link retransmission count, and analyze the link reception status in combination with the audio buffer margin. Step S2: Detect the decoding processing data of each Bluetooth headset decoder, calculate the phase offset based on the decoding processing data, evaluate the group migration trend by combining the phase offset and the link reception status, and determine whether to enter the clustering convergence mechanism based on the group migration trend. Step S3: When entering the clustering convergence mechanism, collect the interaction behavior data of the corresponding users of each Bluetooth headset, perform clustering analysis based on the interaction behavior data and generate different headset groups, and count the playback progress speed of each headset group according to the clustering analysis results. Step S4: Count the number of Bluetooth headphones in the headphone group and divide the headphone group into the main playback group and the deviation group. Calculate the group distance between each deviation group and the main playback group based on the playback speed. Apply different group convergence strategies to the deviation groups based on the deviation level.

2. The Bluetooth audio synchronization control method based on multi-terminal collaboration according to claim 1, characterized in that: In step S1, after the Bluetooth audio enters the playback state, a preset monitoring period is started. The monitoring period is used to periodically sample and update the link reception characteristics of each Bluetooth headset during continuous playback. During the monitoring period, the audio buffer margin of each Bluetooth headset is collected. The audio buffer margin is the amount of audio data that has not yet been decoded and played in the local audio playback buffer queue of each Bluetooth headset. At the same time, the number of link retransmissions for each Bluetooth headset during the listening period is collected. The number of link retransmissions is the automatic retransmission count triggered during the transmission of audio data packets due to verification failure or failure to acknowledge receipt. Retrieve the preset standard retransmission baseline value, divide the number of link retransmissions of each Bluetooth headset in the current monitoring period by the standard retransmission baseline value, and obtain the link fluctuation coefficient. By combining the link fluctuation coefficient with the audio buffer margin of the corresponding Bluetooth headset using a weighted combination method, the link reception status is constructed.

3. The Bluetooth audio synchronization control method based on multi-terminal collaboration according to claim 1, characterized in that: In step S2, the operating status of the decoder of each Bluetooth headset is continuously monitored, and decoding processing data is obtained, including the decoding queue length and the number of audio frames dropped. The decoder is a functional module inside each Bluetooth headset used to decode and process received audio data; The decoding queue length is the current backlog length of the audio frame buffer queue to be processed inside each Bluetooth headset decoder; The number of audio frames dropped is the number of audio frames dropped during the decoding process due to decoding timeout, insufficient buffer, or frame verification failure.

4. The Bluetooth audio synchronization control method based on multi-terminal collaboration according to claim 3, characterized in that: In step S2, the standard decoding processing baseline queue length and the standard frame discard baseline value are introduced, and the normalized decoding length is obtained by dividing the decoding queue length by the standard decoding processing baseline queue length. The normalized frame drop count is obtained by dividing the number of audio frame drops by the standard frame drop baseline value. The phase offset is defined as the average of the normalized decoding length and the normalized frame drop count. The phase offset and link reception status are fused to construct the population migration trend.

5. The Bluetooth audio synchronization control method based on multi-terminal collaboration according to claim 4, characterized in that: In step S2, when the migration trend of the group is greater than or equal to the preset group convergence trigger threshold, the corresponding Bluetooth headset is determined to be in a high migration risk state; otherwise, the corresponding Bluetooth headset is determined not to be in a high migration risk state. The percentage of migrated terminals is obtained by dividing the number of Bluetooth headsets at high migration risk by the total number of Bluetooth headsets. When the proportion of migrating terminals exceeds the preset threshold for group structure instability, it is determined that the current multi-terminal playback has evolved from a local offset state to a group migration state, triggering the entry into the clustering convergence mechanism; Conversely, if the clustering convergence mechanism is not followed, the process will be continuously monitored.

6. The Bluetooth audio synchronization control method based on multi-terminal collaboration according to claim 1, characterized in that: In step S3, the interaction behavior data includes two items: volume adjustment event density and wearing posture adjustment frequency; Volume adjustment event density refers to the ratio of the number of times a user presses the volume up or volume down button on the navigation terminal paired with the Bluetooth headset to the duration of the monitoring period. Wearing posture adjustment frequency refers to the ratio of the number of times the Bluetooth headset body relative to the user's head coordinate system undergoes movements greater than the posture change threshold within the monitoring period to the duration of the monitoring period. The interaction perturbation index was calculated for each Bluetooth headset by combining the normalized results of the wearing posture adjustment frequency and the normalized results of the volume adjustment event density. Using the standardized phase offset of each Bluetooth headset as the horizontal axis and the interaction perturbation index as the vertical axis, a two-dimensional feature space is constructed, with each Bluetooth headset corresponding to a feature point in the two-dimensional feature space.

7. The Bluetooth audio synchronization control method based on multi-terminal collaboration according to claim 6, characterized in that: In step S3, the distances between all Bluetooth headsets in the two-dimensional feature space during the current monitoring period are counted, the distances are sorted in ascending order, and the lower quartile is taken as the neighborhood radius. For each Bluetooth headset, count the number of feature points contained in its two-dimensional feature space neighborhood with the neighborhood radius as the radius, and use this as the local density of the Bluetooth headset. For each Bluetooth headset, among all Bluetooth headsets with a local density higher than its own, the distance to the closest one is taken as the local isolation distance of the Bluetooth headset; The median value of the local density of each Bluetooth headset is taken as the lower limit for density determination; the median value of the local isolation distance of each Bluetooth headset is taken as the lower limit for distance determination.

8. The Bluetooth audio synchronization control method based on multi-terminal collaboration according to claim 7, characterized in that: In step S3, Bluetooth earphones with local density greater than the density judgment lower limit and local isolation distance greater than the distance judgment lower limit are identified as cluster centers, and the remaining Bluetooth earphones are assigned to the cluster center closest to them in turn, generating several earphone groups; For each headphone group, the inverse ratio of the interaction perturbation index of each Bluetooth headphone is used as the robustness weight, and the rate of change of the phase offset of each Bluetooth headphone within the statistical window is weighted and averaged to obtain the playback propagation speed of the headphone group.

9. A Bluetooth audio synchronization control method based on multi-terminal collaboration according to claim 8, characterized in that: In step S4, for each headphone group, the number of Bluetooth headphones it contains is counted; the headphone group with the largest number of Bluetooth headphones is identified as the main playback group, and the remaining headphone groups are identified as off-target groups. The median value of the phase offset of all Bluetooth headsets included in the main playback group is taken as the phase anchor point of the main playback group. The median value of the phase offset of each Bluetooth headset in the off-group is taken and subtracted from the phase anchor point of the main playback group to obtain the phase difference of the off-group. The speed difference of the off-group is obtained by subtracting the playback speed of the off-group from the playback speed of the main group. The group distance between the off-group and the main playback group is calculated by combining the phase difference and velocity difference.

10. A Bluetooth audio synchronization control method based on multi-terminal collaboration according to claim 9, characterized in that: In step S4, the group distance of each deviation group is compared with the first deviation level threshold and the second deviation level threshold, respectively: When the group distance of the offshoot is less than the first deviation level threshold, the buffer level fine-tuning strategy is executed, and the buffer level bias is sent to the decoder of each Bluetooth headset to which the offshoot belongs. When the group distance of the off-group is greater than or equal to the first deviation level threshold and less than the second deviation level threshold, the variable speed playback strategy is executed, and the playback rate coefficient is sent to the decoder of each Bluetooth headset to which the off-group belongs. When the group distance of the offshoot is greater than or equal to the second deviation level threshold, the anchor point alignment strategy is executed, and the current playback anchor point timestamp of the main playback group is sent to each Bluetooth headset belonging to the offshoot.

Images