Audio data synchronization method and data sending end

Abstract

The embodiment of the invention relates to an audio data synchronization method and a data sending end. The method comprises the following steps: acquiring first time information of a first sending end and second time information of a second sending end; determining the next sampling time of the first sending end based on the first time information and the sampling period by adopting a preset synchronization algorithm, and determining the next sampling time of the second sending end based on the second time information and the sampling period; and controlling the first sending end to collect audio data at the next sampling time of the first sending end, and controlling the second sending end to collect audio data at the next sampling time of the second sending end. Therefore, the next sampling time of the first transmitting end and the second transmitting end is synchronized through the preset synchronization algorithm, and then the first transmitting end and the second transmitting end are controlled to acquire the audio data at the synchronized sampling time, so that the audio acquisition synchronism of the transmitting ends can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an audio data synchronization method and a data sending end. Background Art

[0002] At present, audio synchronization technology is mainly used in two-channel or even multi-channel audio downlink, that is, synchronous playback based on timestamps.

[0003] For example, for wireless mic (microphone) products, the main thing is to synchronize audio collection between mics. Without synchronized collection, the audio will have a phase difference, affecting the audio experience. For low-latency wireless audio products, due to low-latency synchronization, the audio cache is limited, and it is more likely to cause problems such as disconnection and freezes due to wireless fluctuations.

[0004] It can be seen that how to improve the synchronization of audio collection at the sending end is a technical issue worthy of attention. Summary of the invention

[0005] In view of this, in order to solve the above technical problems, an embodiment of the present application provides an audio data synchronization method and a data sending end.

[0006] In a first aspect, an embodiment of the present application provides a method for synchronizing audio data, the method comprising:

[0007] Acquire first time information of the first transmitting end and second time information of the second transmitting end;

[0008] Adopting a preset synchronization algorithm, based on the first time information and the sampling period, determining the next sampling time of the first transmitting end, and based on the second time information and the sampling period, determining the next sampling time of the second transmitting end, wherein the sampling period represents the sampling period of the first transmitting end and the second transmitting end for collecting audio data, and the preset synchronization algorithm is used to synchronize the next sampling time of the first transmitting end and the next sampling time of the second transmitting end;

[0009] The first transmitting end is controlled to collect audio data at the next sampling time of the first transmitting end, and the second transmitting end is controlled to collect audio data at the next sampling time of the second transmitting end.

[0010] In a possible implementation manner, the first time information includes a first value representing time, and the second time information includes a second value representing time; and

[0011] The adopting a preset synchronization algorithm to determine the next sampling time of the first transmitting end based on the first time information and the sampling period, and determining the next sampling time of the second transmitting end based on the second time information and the sampling period, includes:

[0012] Determine a minimum value greater than the first value from the integer multiple values ​​of the sampling period as the next sampling time of the first transmitting end;

[0013] A minimum value greater than the second value is determined from integer multiple values ​​of the sampling period as the next sampling time of the second transmitting end.

[0014] In a possible implementation manner, before acquiring the first time information of the first transmitting end and the second time information of the second transmitting end, the method further includes:

[0015] receiving a first data packet, wherein the first data packet is used by a receiving end to establish a connection with the first transmitting end and / or the second transmitting end, and the first data packet includes time information of the receiving end;

[0016] Parsing the first data packet to obtain time information in the first data packet;

[0017] The local time of the first sending end and / or the second sending end is adjusted to the time indicated by the time information in the first data packet.

[0018] In a possible implementation manner, the sampling period is smaller than a value representing a time synchronization period of the receiving end; the time synchronization period represents a period for the receiving end to send synchronization time to the sending end.

[0019] In a possible implementation, the method further includes:

[0020] Receiving a second data packet sent by a receiving end according to a target sending period, wherein the second data packet includes time information of the receiving end;

[0021] Parsing the second data packet to obtain time information in the second data packet;

[0022] Based on the time information in the second data packet, local time synchronization of the target sending end is performed, wherein the target sending end is the first sending end or the second sending end.

[0023] In a possible implementation manner, the performing local time synchronization of the target sending end based on the time information in the second data packet includes:

[0024] Determine the time difference between the time information in the second data packet and the local time information of the target sending end to obtain a target time difference;

[0025] When the target time difference is greater than or equal to a preset time difference threshold, adjusting the sampling frequency of the target sending end;

[0026] Audio is collected according to the adjusted sampling frequency to perform local time synchronization of the target sending end.

[0027] In a possible implementation manner, adjusting the sampling frequency of the target transmitting end includes:

[0028] Determine the sampling times of the target transmitting end within the sampling period to obtain an initial sampling times;

[0029] Determine the difference between the initial sampling number and the value representing the target time difference to obtain a new sampling number;

[0030] In one of the sampling cycles, controlling the target transmitting end to collect audio at a number of times equal to the new number of sampling times;

[0031] After one of the sampling cycles, the target transmitting end is controlled to collect audio the number of times equal to the initial sampling number.

[0032] In a possible implementation manner, after controlling the first transmitting end to collect audio data at the next sampling time of the first transmitting end, and controlling the second transmitting end to collect audio data at the next sampling time of the second transmitting end, the method further includes:

[0033] The audio data is sent to a receiving end, wherein, in the case that audio packet loss occurs at a target sending end, the receiving end determines the audio data lost by the target sending end based on the audio data last sent by the target sending end, so as to synchronize the time of the first sending end and the second sending end, and the target sending end is the first sending end or the second sending end.

[0034] In a second aspect, an embodiment of the present application provides a data sending end, which is the first sending end and the second sending end in the method of any embodiment of the audio data synchronization method of the first aspect mentioned above, and the data sending end is used to execute the method of any embodiment of the audio data synchronization method of the first aspect mentioned above.

[0035] In a third aspect, an embodiment of the present application provides an audio transceiver system, the audio transceiver system comprising a data transmitting end and a data receiving end, wherein:

[0036] The data sending end includes the first sending end and the second sending end in the audio data synchronization method of the first aspect, and the data sending end is used to execute the audio data synchronization method of the first aspect;

[0037] The data receiving end is used to receive the audio data collected by the first sending end and the second sending end.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method of any embodiment of the audio data synchronization method of the first aspect described above is implemented.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program, which includes a computer-readable code. When the computer-readable code is executed on a device, a processor in the device implements a method as in any embodiment of the audio data synchronization method of the first aspect described above.

[0040] The audio data synchronization method provided in the embodiment of the present application can obtain the first time information of the first transmitter and the second time information of the second transmitter, and then, using a preset synchronization algorithm, based on the first time information and the sampling period, determine the next sampling time of the first transmitter, and based on the second time information and the sampling period, determine the next sampling time of the second transmitter, wherein the sampling period represents the sampling period for the first transmitter and the second transmitter to collect audio data, and the preset synchronization algorithm is used to synchronize the next sampling time of the first transmitter and the next sampling time of the second transmitter, and then control the first transmitter to collect audio data at the next sampling time of the first transmitter, and control the second transmitter to collect audio data at the next sampling time of the second transmitter. Thus, the next sampling time of the first transmitter and the second transmitter is synchronized by the preset synchronization algorithm, and then the first transmitter and the second transmitter are controlled to collect audio data at the synchronized sampling time, so that the synchronization of audio collection by the transmitter can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0042] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0043] One or more embodiments are exemplarily described by pictures in the corresponding drawings, and these exemplified descriptions do not constitute limitations on the embodiments. Elements with the same reference numerals in the drawings represent similar elements, and unless otherwise stated, the figures in the drawings do not constitute proportional limitations.

[0044] Figure 1 A flowchart of a method for synchronizing audio data provided in an embodiment of the present application;

[0045] Figure 2 A flowchart of another method for synchronizing audio data provided by an embodiment of the present application;

[0046] Figure 3A An architecture diagram of an audio data synchronization method provided in an embodiment of the present application;

[0047] Figure 3B A schematic diagram of the interaction between a data sending end and a data receiving end involved in a method for synchronizing audio data provided in an embodiment of the present application;

[0048] Figure 4 A schematic diagram of the structure of a data sending terminal provided in an embodiment of the present application;

[0049] Figure 5 A schematic diagram of the structure of an audio transceiver system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] Various exemplary embodiments of the present application will now be described in detail with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. It should be noted that unless otherwise specifically stated, the relative arrangement of the components and steps, the numerical expressions and the numerical values ​​described in these embodiments do not limit the scope of the present application.

[0051] Those skilled in the art will understand that the terms "first" and "second" in the embodiments of the present application are only used to distinguish between different steps, devices, modules and other objects, and do not represent any specific technical meanings, nor do they indicate the logical order between them.

[0052] It should also be understood that in this embodiment, “plurality” may refer to two or more than two, and “at least one” may refer to one, two or more than two.

[0053] It should also be understood that any component, data or structure mentioned in the embodiments of the present application can generally be understood as one or more, unless explicitly limited or otherwise indicated in the context.

[0054] In addition, the term "and / or" in this application is only a description of the association relationship of associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this application generally indicates that the associated objects before and after are in an "or" relationship.

[0055] It should also be understood that the description of the various embodiments in this application focuses on the differences between the various embodiments, and the same or similar aspects thereof can be referenced to each other, and for the sake of brevity, they will not be described one by one.

[0056] The following description of at least one exemplary embodiment is merely illustrative in nature and is in no way intended to limit the present application, its application, or uses.

[0057] Technologies, methods, and equipment known to ordinary technicians in the relevant art may not be discussed in detail, but where appropriate, the above-mentioned technologies, methods, and equipment should be considered as part of the specification.

[0058] It should be noted that like reference numerals and letters refer to similar items in the following figures, and therefore, once an item is defined in one figure, it need not be further discussed in subsequent figures.

[0059] It should be noted that, in the absence of conflict, the embodiments in this application and the features in the embodiments can be combined with each other. To facilitate the understanding of the embodiments of the present application, the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments. Obviously, the described embodiments are part of the embodiments of the present application, rather than all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0060] In order to solve the technical problem of low synchronization of audio collection by the sending end in the prior art, the present application provides a method for synchronizing audio data, which can improve the synchronization of audio collection by the sending end.

[0061] Figure 1A flowchart of a method for synchronizing audio data provided in an embodiment of the present application. The method can be applied to one or more electronic devices such as a transmitter (TX, transport), a smart phone, a laptop, a desktop computer, a portable computer, a server, etc. In addition, the execution subject of the method can be hardware or software. When the above-mentioned execution subject is hardware, the execution subject can be one or more of the above-mentioned electronic devices. For example, a single electronic device can execute the method, or a plurality of electronic devices can cooperate with each other to execute the method. When the above-mentioned execution subject is software, the method can be implemented as multiple software or software modules, or as a single software or software module. No specific limitation is made here.

[0062] like Figure 1 As shown, the method specifically includes:

[0063] Step 101: Acquire first time information of a first transmitting end and second time information of a second transmitting end.

[0064] In this embodiment, the transmitting end (TX, transport, including the first transmitting end and the second transmitting end) can be an end for sending audio data to the receiving end. For example, each transmitting end (including the first transmitting end and the second transmitting end) can collect audio data (i.e., sampling) according to a preset strategy, and then send the collected audio data to the receiving end (RX, receive). Each transmitting end can be used to transmit one channel of audio data to the above-mentioned receiving end. Each transmitting end can collect audio data according to the same sampling period (i.e., the sampling period represented by the sampling period described later).

[0065] In addition, the number of transmitting ends for transmitting audio data to the receiving end may be at least two. For example, the transmitting end may include a first transmitting end and a second transmitting end. The first transmitting end may be any transmitting end that transmits audio data to the receiving end.

[0066] Each transmitting end may include a timer, and the sampling of the transmitting end is controlled by the timer.

[0067] The first time information of the first transmitting end may be obtained by a timer included in the first transmitting end or the receiving end, or may be obtained by adjusting the time provided by the timer included in the first transmitting end or the receiving end.

[0068] The second transmitting end may be one or more transmitting ends different from the first transmitting end among the at least two transmitting ends for transmitting audio data to the receiving end.

[0069] The first time information of the second transmitting end may be obtained by a timer included in the second transmitting end or the receiving end, or may be obtained by adjusting the time provided by the timer included in the second transmitting end or the receiving end.

[0070] Step 102: Using a preset synchronization algorithm, determine the next sampling time of the first transmitting end based on the first time information and the sampling period, and determine the next sampling time of the second transmitting end based on the second time information and the sampling period, wherein the sampling period represents the sampling period for collecting audio data by the first transmitting end and the second transmitting end, and the preset synchronization algorithm is used to synchronize the next sampling time of the first transmitting end and the next sampling time of the second transmitting end.

[0071] In this embodiment, the next sampling time of the first transmitting end may be: the next sampling time of the first transmitting end obtained by predicting based on the first time information and the sampling period by using a preset synchronization algorithm.

[0072] The next sampling time of the second transmitting end may be: the next sampling time of the second transmitting end obtained by predicting based on the second time information and the sampling period by using a preset synchronization algorithm.

[0073] As an example, the first time information includes a first numerical value indicating time.

[0074] The first value may be a value included in the first time information, and the first value may represent the first time information of the first sending end.

[0075] The sampling period may include a numerical value, and the sampling period represents a sampling period for the first transmitting end and the second transmitting end to collect audio data.

[0076] On this basis, the following method can be used to determine the next sampling time of the first sending end:

[0077] From the integer multiple values ​​of the sampling period, a minimum value greater than the first value is determined to obtain the next sampling time of the first transmitting end.

[0078] The next sampling time of the first transmitting end may be a minimum value greater than the first value determined from integer multiples of the sampling period.

[0079] For example, if the first value is 7500 and the sampling period is 2500, then the first minimum value may be 10000.

[0080] For another example, if the first value is 7800 and the sampling period is 2500, then the first minimum value may be 10000.

[0081] In addition, the next sampling time of the second transmitting end can be determined in the following manner:

[0082] First, a minimum value greater than the second value is determined from integer multiple values ​​of the sampling period to obtain the next sampling time of the second transmitting end.

[0083] The next sampling time of the second transmitting end may be a minimum value greater than the second value determined from integer multiples of the sampling period.

[0084] For example, if the second numerical value is 7500 and the sampling period is 2500, then the second minimum numerical value may be 10000.

[0085] For another example, if the second numerical value is 7800 and the sampling period is 2500, then the second minimum numerical value may be 10000.

[0086] As another example, the next sampling time of the first transmitting end and the second transmitting end may also be determined in the following manner:

[0087] First, the moment represented by the first time information, after the duration represented by the sampling period, is used as the first candidate time; the moment represented by the second time information, after the duration represented by the sampling period, is used as the second candidate time.

[0088] The later candidate time between the first candidate time and the second candidate time is determined as the next sampling time of the first transmitting end and the next sampling time of the second transmitting end.

[0089] Step 103: Control the first transmitting end to collect audio data at the next sampling time of the first transmitting end, and control the second transmitting end to collect audio data at the next sampling time of the second transmitting end.

[0090] In this embodiment, after determining the next sampling time of the first transmitting end and the second transmitting end, audio data can be collected at the next sampling time of the first transmitting end, and audio data can be collected at the next sampling time of the second transmitting end.

[0091] In some optional implementations of this embodiment, the following steps may also be performed:

[0092] First, a second data packet sent by the receiving end according to the target sending period is received.

[0093] The second data packet includes the time information of the receiving end, and the time information can be obtained via the RTC (Real_Time Clock) of the receiving end.

[0094] The target sending period may indicate a period during which the receiving end sends the second data packet.

[0095] Afterwards, the second data packet is parsed to obtain time information in the second data packet.

[0096] Then, based on the time information in the second data packet, local time synchronization of the target sending end is performed, wherein the target sending end is the first sending end or the second sending end.

[0097] Here, the local time of the first sending end may be set to the time indicated by the time information in the second data packet.

[0098] It can be understood that in the above optional implementation, the time of the first sending end can be synchronized through the time information in the second data packet sent by the receiving end, thereby further improving the synchronization of audio collection at the sending end.

[0099] In some application scenarios of the above optional implementation manner, the following manner may be adopted to perform local time synchronization of the target sending end based on the time information in the second data packet:

[0100] First, the time difference between the time information in the second data packet and the local time information of the target sending end is determined to obtain a target time difference.

[0101] The target time difference may represent the time difference between the time information in the second data packet and the local time information of the target sending end.

[0102] Afterwards, when the target time difference is greater than or equal to a preset time difference threshold, the sampling frequency of the target sending end is adjusted.

[0103] For example, if the target time difference indicates that the time indicated by the time information in the second data packet is earlier than the time indicated by the local time information of the target sending end, then the sampling frequency of the target sending end may be increased.

[0104] For another example, if the target time difference indicates that the time indicated by the time information in the second data packet is later than the time indicated by the local time information of the target sending end, then the sampling frequency of the target sending end may be reduced.

[0105] Then, audio is collected according to the adjusted sampling frequency to perform local time synchronization of the target sending end.

[0106] It can be understood that in the above application scenario, the sampling frequency of the target sending end can be adjusted only when the time difference between the time information in the second data packet and the local time information of the target sending end is large. In this way, the adjustment frequency of the sampling frequency can be reduced, thereby reducing the resource occupation of the adjustment frequency.

[0107] In some cases of the above application scenarios, the sampling frequency of the target sending end may be adjusted in the following manner:

[0108] First, the sampling times of the target transmitting end within the sampling period are determined to obtain an initial sampling times.

[0109] The initial sampling number may be the sampling number of the target transmitting end, and the initial sampling number may be a preset value.

[0110] Afterwards, the difference between the initial sampling times and the value representing the target time difference is determined to obtain a new sampling times.

[0111] The new sampling number may be the difference between the initial sampling number and the value representing the target time difference.

[0112] Then, within one of the sampling cycles, the target transmitting end is controlled to collect audio at a number of times equal to the new sampling number.

[0113] Then, after one of the sampling cycles, the target transmitting end is controlled to collect audio the number of times equal to the initial sampling number.

[0114] As an example, if the initial sampling number is 1000 times and the target time difference value is 30, then within a sampling cycle, the target transmitter can be controlled to collect audio 970 times, and then the initial sampling number, i.e. 1000, is restored in the next sampling cycle.

[0115] It can be understood that in the above situation, time synchronization between the first transmitting end and the second transmitting end can be achieved within one sampling period.

[0116] In some optional implementations of this embodiment, after controlling the first transmitting end to collect audio data at the next sampling time of the first transmitting end, and controlling the second transmitting end to collect audio data at the next sampling time of the second transmitting end, the collected audio data can also be sent to the receiving end.

[0117] Among them, in the case that there is audio packet loss at the target sending end, the receiving end determines the lost audio data of the target sending end based on the audio data last sent by the target sending end, so as to synchronize the time of the first sending end and the second sending end, and the target sending end is the first sending end or the second sending end.

[0118] The receiving end may adopt a PLC (Packet Loss Concealment) method to determine the audio data lost by the first sending end based on the audio data previously sent by the first sending end.

[0119] It can be understood that in the above optional implementation, when packet loss occurs, the receiving end can ensure the synchronization of timestamps of two or more sending ends by performing packet compensation to avoid jamming.

[0120] The audio data synchronization method provided in the embodiment of the present application can obtain the first time information of the first transmitter and the second time information of the second transmitter, and then, using a preset synchronization algorithm, based on the first time information and the sampling period, determine the next sampling time of the first transmitter, and based on the second time information and the sampling period, determine the next sampling time of the second transmitter, wherein the sampling period represents the sampling period for the first transmitter and the second transmitter to collect audio data, and the preset synchronization algorithm is used to synchronize the next sampling time of the first transmitter and the next sampling time of the second transmitter, and then control the first transmitter to collect audio data at the next sampling time of the first transmitter, and control the second transmitter to collect audio data at the next sampling time of the second transmitter. Thus, the next sampling time of the first transmitter and the second transmitter is synchronized by the preset synchronization algorithm, and then the first transmitter and the second transmitter are controlled to collect audio data at the synchronized sampling time, so that the synchronization of audio collection by the transmitter can be improved.

[0121] Figure 2 A flowchart of another method for synchronizing audio data provided by an embodiment of the present application. Figure 2 As shown, the method specifically includes:

[0122] Step 201, obtaining first time information of a first transmitting end and second time information of a second transmitting end, wherein the first time information includes a first value indicating time, and the second time information includes a second value indicating time.

[0123] In this embodiment, the transmitting end (TX, transport, including the first transmitting end and the second transmitting end) can be an end for sending audio data to the receiving end. For example, each transmitting end (including the first transmitting end and the second transmitting end) can collect audio data (i.e., sampling) according to a preset strategy, and then send the collected audio data to the receiving end (RX, receive). Each transmitting end can be used to transmit one channel of audio data to the above-mentioned receiving end. Each transmitting end can collect audio data according to the same sampling period (i.e., the sampling period represented by the sampling period described later).

[0124] In addition, the number of transmitting ends for transmitting audio data to the receiving end may be at least two. For example, the transmitting end may include a first transmitting end and a second transmitting end. The first transmitting end may be any transmitting end that transmits audio data to the receiving end.

[0125] Each transmitting end may include a timer, and the sampling of the transmitting end is controlled by the timer.

[0126] The first time information of the first transmitting end may be obtained through a timer of the first transmitting end or the receiving end, or may be obtained by adjusting the time provided by a timer included in the first transmitting end.

[0127] The second transmitting end may be one or more transmitting ends different from the first transmitting end among the at least two transmitting ends for transmitting audio data to the receiving end.

[0128] The second time information of the second transmitting end may be obtained by a timer included in the second transmitting end or the receiving end, or may be obtained by adjusting the time provided by the timer included in the second transmitting end.

[0129] Step 202: Determine a minimum value greater than the first value from integer multiple values ​​of the sampling period as the next sampling time of the first transmitting end.

[0130] In this embodiment, the sampling period may include a value, and the sampling period represents a sampling period for the first transmitting end and the second transmitting end to collect audio data.

[0131] For example, if the first value is 7500 and the sampling period is 2500, then the first minimum value may be 10000.

[0132] For another example, if the first value is 7800 and the sampling period is 2500, then the first minimum value may be 10000.

[0133] Step 203 determines the minimum value greater than the second value from the integer multiple values ​​of the sampling period as the next sampling time of the second transmitting end.

[0134] In this embodiment, if the second value is 7500 and the sampling period is 2500, then the second minimum value may be 10000. For another example, if the second value is 7800 and the sampling period is 2500, then the second minimum value may be 10000.

[0135] Step 204: Control the first transmitting end to collect audio data at the next sampling time of the first transmitting end, and control the second transmitting end to collect audio data at the next sampling time of the second transmitting end.

[0136] In this embodiment, in addition to the above contents, the above steps 201-204 can refer to Figure 1 This is implemented in the manner of steps 101-103 in the corresponding embodiment and will not be described in detail here.

[0137] In some optional implementations of this embodiment, before acquiring the first time information of the first transmitting end and the second time information of the second transmitting end, the following steps may also be performed:

[0138] First, a first data packet is received.

[0139] The first data packet is used for the receiving end to establish a connection with the first transmitting end and / or the second transmitting end, and the first data packet includes time information of the receiving end. The time information can be obtained via the RTC (Real Time Clock) of the receiving end. In addition, the first data packet may include a MAC (Media Access Control Address) of a target receiving end. The target transmitting end is the first transmitting end or the second transmitting end.

[0140] Afterwards, the first data packet is parsed to obtain time information in the first data packet.

[0141] Then, the local time of the first sending end and / or the second sending end is adjusted to the time indicated by the time information of the receiving end.

[0142] It can be understood that in the above optional implementation, when the connection between the target transmitting end (such as the first transmitting end or the second transmitting end) and the receiving end is first established, time synchronization between the target transmitting end and the receiving end can be performed, thereby further improving the synchronization of audio collection at the transmitting end.

[0143] In some optional implementations of this embodiment, the sampling period is smaller than a value representing a time synchronization period of the receiving end; the time synchronization period represents a period for the receiving end to send synchronization time to the first sending end or the second sending end.

[0144] It can be understood that in the above optional implementation, time synchronization can be performed within the gap between the time synchronization periods of the data packets sent by the receiving end received by the first transmitting end and the second transmitting end, thereby further improving the synchronization of audio collection at the transmitting end.

[0145] It should be noted that, in addition to the above contents, this embodiment may also include Figure 1 The corresponding technical features described in the corresponding embodiments are realized Figure 1 For details, please refer to the technical effect of the audio data synchronization method shown in Figure 1 For the sake of brevity, the relevant description is not repeated here.

[0146] The audio data synchronization method provided in the embodiment of the present application can, through the first time information and the sampling period, perform time synchronization between the first transmitting end and other transmitting ends without performing data interaction with the receiving end or other transmitting ends except the first transmitting end; and can, through the second time information and the sampling period, perform time synchronization between the second transmitting end and other transmitting ends without performing data interaction with the receiving end or other transmitting ends except the second transmitting end. This reduces the interaction frequency between the ends and ensures the stability of the system.

[0147] The following is an illustrative description of the embodiments of the present application, but it should be noted that the embodiments of the present application may have the features described below, but the following description does not constitute a limitation on the scope of protection of the embodiments of the present application.

[0148] At present, audio synchronization technology is mainly used in two-channel or even multi-channel audio downlink, that is, synchronous playback based on timestamps.

[0149] For wireless mic products, the mic needs synchronization of audio acquisition. Without synchronization, the audio will have a phase difference, affecting the audio experience. For low-latency wireless audio products, due to low-latency synchronization, the audio cache is limited, which makes it more likely to cause problems such as disconnection and freezes due to wireless fluctuations.

[0150] The following uses a dual-channel mic as an example for illustration:

[0151] The implementation of dual-channel low-latency wireless audio synchronization solution mainly consists of the following components:

[0152] 1. The time points of the first audio data sampling of the two TXs (ie, the first transmitting end and the second transmitting end) are aligned.

[0153] 2. On the premise that the time points of the first audio data sampling are aligned, the two TXs ensure that the time points of subsequent periodic sampling are aligned.

[0154] 3. After receiving the audio frames from two TXs, RX performs secondary alignment.

[0155] The specific architecture of the dual-channel low-latency wireless audio synchronization solution is as follows:

[0156] As an example, see Figure 3A , Figure 3A An architectural diagram of an audio data synchronization method provided in an embodiment of the present application.

[0157] like Figure 3A As shown, each device (including the first transmitting end, the second transmitting end, and the receiving end) has two main parts, part 1 and part 2. Part 1 mainly processes audio data sampling, DSP (digital signal processing) noise reduction, and other UI (User Interface) interaction logic, such as turning the mic on and off. Part 2 mainly processes 2.4G communication, RTC synchronization, and other wireless communication related logic. These two parts can be implemented on one SOC (System on Chip), or multiple SOCs can be used.

[0158] 1. There are two TXs (i.e. the first transmitter and the second transmitter) in the entire audio system. Each TX transmits one channel of audio data. After the RX (i.e. the receiving end) receives the two channels of audio, it finally outputs the two channels of audio data through the DAC (Digital to Analog Converter).

[0159] 2. The audio data stream in TX is processed by the following modules in sequence:

[0160] ADC (Analog to Digital Converter) sampling, DSP noise reduction, serial transfer out, serial transfer in, audio sync, encoder, Timer Sync RTC, 2.4G wireless.

[0161] 3. The audio data stream in RX is processed by the following modules in sequence:

[0162] 2.4G wireless, Timer Sync timestamp Sync, serial transfer out, serial transfer in, decoder, audio sync, DAC, AUX out.

[0163] 4. The Timer Sync RTC and Timer Sync timestamp Sync modules in TX and RX are used to achieve RTC synchronization of the three devices. The synchronization and calibration of audio data are all based on the RTC generated by the timer. The RTC synchronization and serial transfer sampling synchronization process of the three devices are as follows:

[0164] (1) Each device starts a timer after it is powered on, and the configuration parameters of the timer must be kept the same. For example, the following configuration parameters of the timer must be kept consistent: frequency, bit width, mode, etc.

[0165] (2) Connection establishment phase:

[0166] Please refer to Figure 3B , Figure 3B A schematic diagram of the interaction between a data sending end and a data receiving end involved in a method for synchronizing audio data provided in an embodiment of the present application.

[0167] 1) The Polling packet (ie, the first data packet) sent by the 2.4G module in the RX during the connection establishment process is accompanied by the RX local RTC (ie, the time information of the receiving end).

[0168] 2) After receiving the Connecting Polling packet, the 2.4G module in TX parses the packet data and sets the RTC to the local time, that is, the local time of the first transmitter and / or the second transmitter is adjusted to the time indicated by the time information of the receiving end. The 2.4G module reports CONNECT_EVT (connection event) to the application layer to indicate that the connection is successful.

[0169] 3) The 2.4G module in RX reports CONNECT_EVT to the application layer to indicate a successful connection.

[0170] (3) After receiving CONNECT_EVT, TX obtains the local RTC, and based on the local RTC, obtains an integer multiple value of P (that is, the sampling period mentioned above) as the next sampling time, that is, from the integer multiple values ​​of the sampling period, determine the minimum value greater than the first value, and obtain the first minimum value; determine the first minimum value as the next sampling time of the first transmitter; from the integer multiple values ​​of the sampling period, determine the minimum value greater than the second value, and obtain the second minimum value; determine the second minimum value as the next sampling time of the second transmitter. And set a corresponding RTC interrupt.

[0171] Among them, P (that is, the sampling period mentioned above) represents the offset between the RTC of the first audio data sampling started by serial transfer in and the RTC of the application layer receiving CONNECT_EVT.

[0172] (4) After the interrupt is generated, the application starts sampling the serial transfer in, and then performs sampling at a period of P.

[0173] like Figure 3B The following is an example:

[0174] If the time point of a serial transfer in sampling after TX1 is connected to RX is when RTC is 7500, and TX2 is connected to RX when RTC is 7800, then the time point of the first serial transfer in sampling of TX2 can be 10000. In this way, when TX2 samples for the first time, TX1 is also sampling, thus ensuring that the first sampling of TX2 is aligned with a certain sampling of TX1, thus ensuring that the sampling of the two TXs is synchronized when they are just connected.

[0175] However, since both TXs are running with their own clocks, deviations will occur over time, and a later resynchronization mechanism is needed to solve this problem.

[0176] (5) Cycle synchronization mechanism:

[0177] The 2.4G module in RX periodically (P1, i.e. the target sending period mentioned above) attaches the local RTC to the Connected Polling packet and sends it to TX. After receiving it, TX parses the RTC and compares it with its own local RTC. If the difference exceeds the defined range T (i.e. the preset time difference threshold mentioned above), it is necessary to fine-tune the sampling frequency of serial transfer in and control the corresponding clock to make the sampling faster or slower to synchronize with the generated deviation. The specific scheme for controlling serial transfer in is as follows.

[0178] 1) RX periodically (corresponding to P1 above) reads the timer count and sends it to TX via polling.

[0179] 2) TX receives the timestamp from RX, compares the local network timer count error, and converts the deviation data into the AUDIO_CLK adjustment value, performs a CLK adjustment, and restores the original CLK. For example, if the serial transfer in frequency is 1MHz, if it is found to be 30us faster by comparing with the RX RTC, then adjust the frequency to 970KHz, and adjust it back to 1MHz after a sampling cycle, so that the slow one in the middle can be pulled back to achieve the purpose of regular synchronization.

[0180] (6)RX audioSync synchronization process:

[0181] When packet loss occurs, RX uses PLC to perform packet compensation to ensure that the timestamps of the two TXs are synchronized to avoid jamming.

[0182] It should be noted that, in addition to the contents recorded above, this embodiment may also include the technical features described in the above embodiments, so as to achieve the technical effects of the audio data synchronization method shown above. Please refer to the above description for details. For the sake of brevity, it will not be repeated here.

[0183] The audio data synchronization method provided in the embodiment of the present application saves bandwidth by synchronizing time through multiple packet periods of RX interval; the TX synchronous audio acquisition mechanism is simple and reliable, with less interaction, ensuring system stability; TX dynamically adjusts the audio sampling speed according to the time difference with RX to ensure synchronization of audio sampling time; when packet loss occurs, RX uses PLC to perform packet compensation to ensure synchronization of timestamps of the two TXs to avoid problems such as stuck sound.

[0184] An embodiment of the present application further provides a data sending end, which includes the first sending end and the second sending end in any of the above-mentioned audio data synchronization methods, and the data sending end is used to execute any of the above-mentioned audio data synchronization methods.

[0185] As an example, see Figure 4 , Figure 4 A structural diagram of a data sending end provided in an embodiment of the present application.

[0186] like Figure 4As shown, the data sending end 10 may include the following modules: ADC sampling, DSP noise reduction, serial transfer out, serial transfer in, audio Sync, encoder, Timer Sync RTC, and 2.4G wireless.

[0187] The audio data stream is processed by the above modules in sequence.

[0188] According to the data transmitting end of the embodiment of the present invention, the data transmitting end determines the next sampling time after synchronization with other transmitting ends through its current sampling time and the sampling period of the transmitting end, so as to collect audio data. In this way, the synchronization of audio collection by the transmitting end can be improved.

[0189] Figure 5 A structural diagram of an audio transceiver system provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, the audio transceiver system 20 includes a data transmitter and a data receiver 23, wherein the data transmitter includes a first transmitter 21 and a second transmitter 22 in any of the above audio data synchronization methods, and the data transmitter is used to execute any of the above audio data synchronization methods. The data receiver 23 is used to receive the audio data collected by the data transmitter.

[0190] According to the audio transceiver system of the embodiment of the present invention, a preset synchronization algorithm is used to synchronize the next sampling time of the first transmitting end and the second transmitting end, and then the first transmitting end and the second transmitting end are controlled to collect audio data at the synchronized sampling time. In this way, the synchronization of audio collection by the transmitting end can be improved.

[0191] The embodiment of the present application also provides a storage medium (computer-readable storage medium). The storage medium here stores one or more programs. The storage medium may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a read-only memory, a flash memory, a hard disk or a solid-state drive; the memory may also include a combination of the above-mentioned types of memory.

[0192] When one or more programs in the storage medium can be executed by one or more processors, the above-mentioned audio data synchronization method executed on the electronic device side can be implemented.

[0193] The processor is used to execute the audio data synchronization program stored in the memory to implement the following steps of the audio data synchronization method performed on the electronic device side:

[0194] Acquire first time information of the first transmitting end and second time information of the second transmitting end;

[0195] Adopting a preset synchronization algorithm, based on the first time information and the sampling period, determining the next sampling time of the first transmitting end, and based on the second time information and the sampling period, determining the next sampling time of the second transmitting end, wherein the sampling period represents the sampling period of the first transmitting end and the second transmitting end for collecting audio data, and the preset synchronization algorithm is used to synchronize the next sampling time of the first transmitting end and the next sampling time of the second transmitting end;

[0196] The first transmitting end is controlled to collect audio data at the next sampling time of the first transmitting end, and the second transmitting end is controlled to collect audio data at the next sampling time of the second transmitting end.

[0197] The professionals should also be further aware that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0198] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented using hardware, a software module executed by a processor, or a combination of the two. The software module may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0199] It should be understood that the terms used herein are only for the purpose of describing specific example embodiments and are not intended to be limiting. Unless the context clearly indicates otherwise, the singular forms "one", "an" and "said" as used herein may also be meant to include plural forms. The terms "include", "comprise", "contain", and "have" are inclusive, and therefore specify the existence of stated features, steps, operations, elements and / or parts, but do not exclude the existence or addition of one or more other features, steps, operations, elements, parts, and / or combinations thereof. The method steps, processes, and operations described herein are not interpreted as necessarily requiring them to be performed in the specific order described or illustrated, unless the execution order is clearly indicated. It should also be understood that additional or alternative steps may be used.

[0200] The foregoing is merely a specific embodiment of the present invention, which enables those skilled in the art to understand or implement the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features claimed herein.

Claims

1. A method for synchronizing audio data, characterized in that: The method comprises: Acquire first time information of the first transmitting end and second time information of the second transmitting end; Adopting a preset synchronization algorithm, based on the first time information and the sampling period, determining the next sampling time of the first transmitting end, and based on the second time information and the sampling period, determining the next sampling time of the second transmitting end, wherein the preset synchronization algorithm is used to synchronize the next sampling time of the first transmitting end and the next sampling time of the second transmitting end; The first transmitting end is controlled to collect audio data at the next sampling time of the first transmitting end, and the second transmitting end is controlled to collect audio data at the next sampling time of the second transmitting end.

2. The method according to claim 1, characterized in that The first time information includes a first value indicating time, and the second time information includes a second value indicating time; and The adopting a preset synchronization algorithm to determine the next sampling time of the first transmitting end based on the first time information and the sampling period, and determining the next sampling time of the second transmitting end based on the second time information and the sampling period, includes: Determine a minimum value greater than the first value from the integer multiple values ​​of the sampling period as the next sampling time of the first transmitting end; A minimum value greater than the second value is determined from integer multiple values ​​of the sampling period as the next sampling time of the second transmitting end.

3. The method according to claim 2, characterized in that Before acquiring the first time information of the first transmitting end and the second time information of the second transmitting end, the method further includes: receiving a first data packet, wherein the first data packet includes time information of the receiving end, wherein the first data packet is used for the receiving end to establish a connection with the first sending end and / or the second sending end; Parsing the first data packet to obtain time information in the first data packet; The local time of the first sending end and / or the second sending end is adjusted to the time indicated by the time information in the first data packet.

4. The method according to claim 2, characterized in that: The sampling period is smaller than a value representing a time synchronization period of the receiving end; the time synchronization period represents a period for the receiving end to send synchronization time to the first sending end or the second sending end.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Receiving a second data packet sent by a receiving end according to a target sending period, wherein the second data packet includes time information of the receiving end; Parsing the second data packet to obtain time information in the second data packet; Based on the time information in the second data packet, local time synchronization of a target sending end is performed, wherein the target sending end is the first sending end or the second sending end.

6. The method according to claim 5, characterized in that The performing local time synchronization of the target sending end based on the time information in the second data packet includes: Determine the time difference between the time information in the second data packet and the local time information of the target sending end to obtain a target time difference; When the target time difference is greater than or equal to a preset time difference threshold, adjusting the sampling frequency of the target sending end to obtain a target sampling frequency; Audio is collected according to the target sampling frequency to perform local time synchronization of the target sending end.

7. The method according to claim 6, characterized in that The adjusting the sampling frequency of the target transmitting end includes: Determine the sampling times of the target transmitting end within the sampling period to obtain an initial sampling times; Determine the difference between the initial sampling number and the value representing the target time difference to obtain a new sampling number; In one of the sampling cycles, controlling the target transmitting end to collect audio at a number of times equal to the new number of sampling times; After one of the sampling cycles, the target transmitting end is controlled to collect audio the number of times equal to the initial sampling number.

8. The method according to any one of claims 1 to 4, characterized in that: After controlling the first transmitting end to collect audio data at the next sampling time of the first transmitting end, and controlling the second transmitting end to collect audio data at the next sampling time of the second transmitting end, the method further includes: The collected audio data is sent to a receiving end, wherein, in the case that audio packet loss occurs at a target sending end, the receiving end determines the audio data lost by the target sending end based on the audio data last sent by the target sending end, so as to synchronize the time of the first sending end and the second sending end, and the target sending end is the first sending end or the second sending end.

9. A data transmitting end, characterized in that: The data sending end includes the first sending end and the second sending end in any one of the methods described in claims 1-8, and the data sending end is used to execute the method described in any one of claims 1-8.

10. An audio transceiver system, characterized in that: The audio transceiver system includes a data transmitting end and a data receiving end, wherein: The data sending end includes the first sending end and the second sending end in any one of the methods described in claims 1 to 8, and the data sending end is used to execute the method described in any one of claims 1 to 8; The data receiving end is used to receive the audio data collected by the first sending end and the second sending end.

Images