Audio data extraction method and apparatus, electronic device, and program product

Abstract

The application provides an audio data extraction method and device, electronic equipment and program product, which are used at a data receiving end. The method comprises the following steps: reading two continuous first audio data in each cycle of data sampling; the first audio data refers to audio data whose data transmission sequence is periodically disturbed; performing audio data extraction based on the two continuous first audio data to obtain second audio data; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission sequence. The method can accurately extract disturbed audio data.

Description

Technical Field

[0001] The present application belongs to the technical field of audio data extraction, and specifically relates to an audio data extraction method, device, electronic device and program product. Background Art

[0002] Currently, when processing audio data, such as spectrum analysis, sampling rate conversion, and echo cancellation, it is necessary to clearly define the audio sampling frequency and the original sampled data. However, during audio data transmission, due to the requirements of the transmission protocol, the original audio data after audio sampling may be split, reorganized, or interpolated, thereby forming audio data with a special structure. Different transmission protocols may have different methods for processing the original data, such as periodically disrupting the audio data, performing invalid interpolation on the original data so that the transmission frequency is inconsistent with the sampling frequency, etc. Therefore, conventional audio processing methods cannot be used to directly process audio data with special structures. In other words, before using conventional data processing, it is necessary to extract valid audio data from the audio data with special structures to restore the original data after audio sampling.

[0003] It should be noted that the above statements are only used to provide background technical information related to this application and do not necessarily constitute prior art. Summary of the Invention

[0004] The present application proposes an audio data extraction method, device, electronic device and program product, which can more accurately extract the original audio data of the disrupted first audio data.

[0005] The first embodiment of the present application provides an audio data extraction method for a data receiving end, the method comprising:

[0006] In each cycle of the data sampling, two consecutive first audio data are read; the first audio data refers to audio data whose data transmission order is periodically disrupted;

[0007] Audio data extraction is performed based on the two consecutive first audio data to obtain second audio data; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission order.

[0008] In some optional embodiments, reading two consecutive first audio data in each data sampling cycle includes:

[0009] In each data sampling cycle, detecting two consecutive byte validity flags of the first audio data;

[0010] In a case where the byte validity identifiers of the two consecutive first audio data indicate that the data are valid, the two consecutive first audio data are read.

[0011] In some optional embodiments, the audio data extraction based on the two continuous first audio data comprises:

[0012] analyzing the validity of each byte in the two continuous first audio data and the channel to which the byte belongs;

[0013] feeding the byte data of the left channel and the byte data of the right channel into the corresponding channel respectively.

[0014] In some optional embodiments, the audio data extraction is performed by using a state machine according to the following steps:

[0015] In the idle state, a bandwidth matching enable signal is detected, and in the case that the data period of the second audio data is determined, the valid flag reading state is entered;

[0016] In the valid flag reading state, the validity flags of two continuous audio bytes are read, and in the case that the validity flags represent that the bytes are valid, the audio data reading state is entered; otherwise, the validity flags of the next two continuous audio bytes are read, and the validity of the next two continuous audio data is determined;

[0017] In the audio data reading state, the first audio data corresponding to the validity flags of the two continuous audio bytes is read, and the data extraction state is entered;

[0018] In the data extraction state, the validity of each byte in the first audio data and the channel to which the byte belongs are analyzed according to the validity flags of the two continuous audio data, and the byte data of the left channel and the byte data of the right channel are fed into the corresponding channel respectively.

[0019] In some optional embodiments, before the two continuous first audio data are read in each period of data sampling, the method further comprises:

[0020] obtaining the current frame sampling time of the first audio data, and the number of valid bytes contained in the first audio data of the current frame;

[0021] determining the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes;

[0022] determining the sampling frequency of the second audio data under the local clock of the data receiving end according to the data period and the local clock frequency of the data receiving end.

[0023] In some optional embodiments, the current frame sampling time of the first audio data is obtained by:

[0024] Detecting that sampling of the first audio data of the previous frame is completed;

[0025] The time from the completion of sampling of the first audio data of the previous frame to the completion of sampling of the first audio data of the current frame is counted and determined as the current frame sampling time.

[0026] In some optional embodiments, the method further includes:

[0027] When a preset number of the first audio data are collected, a flag pulse is generated; the flag pulse is used to indicate that a frame of the first audio data is collected;

[0028] The identification pulse is detected to determine that sampling of the first audio data of the current frame is completed.

[0029] In some optional embodiments, determining the data period of the second audio data based on the current frame sampling time and the number of valid bytes includes:

[0030] When sampling of the current frame audio data is completed within the current frame sampling time, determining a data period of the second audio data of the current frame based on a quotient of the current frame sampling time and the number of valid bytes;

[0031] When the sampling of the current frame audio data is not completed within the current frame sampling time, the preset period default value is determined as the data period of the second audio data of the current frame.

[0032] In some optional embodiments, determining the data period of the second audio data based on a quotient of the current frame sampling time and the number of valid bytes includes:

[0033] When the current frame sampling time is within a preset threshold range, using a divider to calculate a quotient of the current frame sampling time and the number of valid bytes;

[0034] A data period of the second audio data of the current frame is determined based on the quotient and a preset margin.

[0035] In some optional embodiments, before obtaining the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame, the method further includes:

[0036] The received first audio data is written into an audio buffer area of ​​a memory, and the number of the collected first audio data is counted.

[0037] An embodiment of a second aspect of the present application provides an audio data extraction device, the device comprising:

[0038] a data reading module, configured to read two continuous first audio data in each cycle of the data sampling; the first audio data refers to audio data whose data transmission sequence is periodically disturbed;

[0039] an audio extracting module, configured to perform audio data extraction based on the two continuous first audio data to obtain second audio data; the second audio data refers to audio data whose data transmission sequence is normal.

[0040] Embodiments of the third aspect of the present application provide an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method of the first aspect.

[0041] Embodiments of the fourth aspect of the present application provide a computer readable storage medium having a computer program stored thereon, wherein the program is executed by a processor to implement the method of the first aspect.

[0042] Embodiments of the fifth aspect of the present application provide a computer program product comprising a computer program, wherein the computer program is executed by a processor to implement the method of the first aspect.

[0043] The technical solutions provided in the embodiments of the present application have at least the following technical effects or advantages:

[0044] The audio data extraction method provided in the embodiments of the present application reads two continuous first audio data in each cycle of the data sampling; the first audio data refers to audio data whose data transmission sequence is periodically disturbed; audio data extraction is performed based on the two continuous first audio data to obtain second audio data; the second audio data refers to audio data whose data transmission sequence is normal. In this way, since the audio data that is periodically disturbed can usually be in two continuous data, a complete frame of audio data in normal sequence is extracted, so that in one data cycle, the embodiments read two continuous disturbed audio data, which can ensure that the original audio data of the disturbed first audio data can be correctly extracted at the data receiving end, and the actual sampling frequency is matched to send the audio data, so as to ensure that the downstream data receiving end can directly process the audio. BRIEF DESCRIPTION OF DRAWINGS

[0045] Various other advantages and benefits will become apparent to those of ordinary skill in the art upon reading the following detailed description of the preferred embodiments. The accompanying drawings are included to provide a description of the preferred embodiments, and are not meant to limit the scope of the present application. Moreover, the same reference numerals in different figures represent the same or similar components.

[0046] In the drawings:

[0047] Figure 1 A schematic diagram of a flow chart of an audio data extraction method provided in an embodiment of the present application is shown;

[0048] Figure 2 A schematic diagram of the working process of audio data extraction implemented by a state machine according to an embodiment of the present application is shown;

[0049] Figure 3 A schematic diagram of a process for bandwidth matching in an embodiment of the present application is shown;

[0050] Figure 4 A data flow diagram of an audio data extraction method provided by an embodiment of the present application is shown;

[0051] Figure 5 A schematic diagram of a data transmission process in which noise is generated during audio data transmission is shown;

[0052] Figure 6 A schematic diagram of a data transmission process for avoiding noise generation during audio data transmission in one embodiment of the present application is shown;

[0053] Figure 7 A schematic diagram of a data transmission process in which notification of the next frame arrives in advance during audio data transmission is shown;

[0054] Figure 8 A schematic diagram showing a specific flow chart of a bandwidth matching process in an audio data extraction method provided in an embodiment of the present application is shown;

[0055] Figure 9 A schematic diagram showing a specific flow of the audio extraction process in the audio data extraction method provided in one embodiment of the present application is shown;

[0056] Figure 10 A schematic structural diagram of an audio data extraction device in another embodiment of the present application is shown;

[0057] Figure 11 A schematic structural diagram of an audio data extraction device in another embodiment of the present application is shown;

[0058] Figure 12 A schematic diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0059] The following describes exemplary embodiments of the present application in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0060] It should be noted that, unless otherwise specified, the technical or scientific terms used in this application should have the common meanings understood by those skilled in the art to which this application belongs.

[0061] In practical applications, data validity directly affects data usability and audio playback. Bandwidth consistency affects audio quality, and bandwidth mismatches can cause noise. Conventional audio processing methods cannot directly process data based on the transmission frequency, especially for audio structures where dual-channel data is periodically disrupted and the transmission frequency is inconsistent with the actual sampling frequency. Therefore, the audio processing end must not only extract the original audio and match the bandwidth, but also ensure the validity of the extracted data while avoiding noise generated during bandwidth matching (both over- and under-generated data will generate noise). Otherwise, the audio quality output to the audio device will be affected.

[0062] To address the aforementioned issues, this embodiment researched and analyzed related technologies and discovered that in some electronic products requiring audio data transmission, such as but not limited to wired headphones, Bluetooth headphones, and other audio devices, different products utilize different transmission protocols, and therefore the data transmission structure may also differ. For example, in a certain communication protocol, the audio transmission rate is fixed, but the transmitted audio sampling rate is not. When the audio bus transmits 100 data points, it actually only contains 99 scrambled audio data points totaling 594 bytes. Therefore, the data receiving end needs to correctly extract the original audio data and send it out at the audio sampling rate to ensure that the downstream data receiving end can directly process the audio.

[0063] Based on the above findings, the embodiments of the present application provide an audio data extraction method, device, electronic device and program product, which can be used at the data receiving end to read two consecutive first audio data in each data sampling cycle; the first audio data refers to audio data whose data transmission order is periodically disrupted; based on the two consecutive first audio data, audio data extraction is performed to obtain second audio data; the second audio data refers to audio data whose data transmission order is normal. In this way, since the audio data that is periodically disrupted can usually extract a complete frame of audio data in normal order from two consecutive data, this embodiment reads two consecutive disrupted audio data in one data cycle, which can ensure that the data receiving end can correctly extract the original audio data of the disrupted first audio data, and match its actual sampling frequency to send the audio data, so as to ensure that the downstream data receiving end can directly process the audio.

[0064] The audio data extraction device is a processing module on an audio device for executing the audio data extraction method, and may specifically be a processor or a specially configured microprocessor, which is not specifically limited in this embodiment.

[0065] The electronic device may be an audio device to which the above-mentioned audio data extraction method is applied, such as but not limited to headphones, mobile phones, walkmans, etc., or may be an integrated chip on the audio device for extracting audio data.

[0066] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0067] Please refer to Figure 1 , is a flow chart of the audio data extraction method provided in the embodiment of the present application, as shown in the attached Figure 1 As shown, the audio data extraction method includes.

[0068] Step S21 : reading two consecutive first audio data in each data sampling cycle.

[0069] Here, each data sampling cycle can be understood as the sampling cycle of the ADC (analog-to-digital converter) of the data receiving end when receiving audio data to collect audio data, which can also be called a data cycle. The first audio data refers to audio data whose data transmission order is periodically disrupted. It is understandable that even if the order of the audio data is disrupted, when the audio data is transmitted byte by byte after the disruption, in the worst case, the transmission is completed in two audio data packets (12 bytes), that is, two disrupted first audio data packets.

[0070] Two audio data items can be understood as audio data items collected by the data receiving end at two audio sampling points, that is, the audio data items collected by the data receiving end at each audio sampling point can be referred to as one audio data item. Two consecutive first audio data items can be understood as two audio data items collected during two adjacent audio samplings. Furthermore, an audio data packet or a frame of audio data can contain multiple audio data items.

[0071] Step S22: extracting audio data based on two consecutive first audio data to obtain second audio data.

[0072] The second audio data refers to audio data that corresponds to the first audio data and has a normal data transmission order. Here, corresponding to the first audio data means that the valid data bytes contained in the two are the same.

[0073] In this embodiment, the data extraction principle is as follows: first, according to the data scrambling rule, a valid flag of the data byte (left and right channels) is generated and written into a fixed position in the RAM for storage.

[0074] Furthermore, the process of reading two consecutive first audio data items within each data sampling cycle may include the following: within each data sampling cycle, detecting a data byte validity flag for the two consecutive first audio data items; and if the byte validity flag indicates that the two consecutive first audio data items are valid, reading the two consecutive first audio data items. In this way, the validity of the two consecutive first audio data items can be determined at once, effectively shortening audio data processing time and thereby improving audio data processing efficiency.

[0075] During the audio transmission process, the disrupted audio data can be written into the specified area in the RAM for caching. At the same time, in the bandwidth matching stage, the byte validity identifiers of two consecutive audio data are first read in each cycle of the matching data sampling. Based on the byte validity identifier, the two consecutive audio data are judged to be valid and then sent to the shift register for caching. Then, two corresponding audio data are read from the RAM audio buffer area and cached using a 96-bit shift register. Then, the validity of each byte of the audio data is detected by shifting the cached byte validity identifier and the audio data, and the valid bytes are sent to the output cache shift register for output in sequence. After the extraction of one data (48 bits) is completed, the data is sent out at the specified time within the matching data cycle to achieve data extraction and bandwidth matching.

[0076] Specifically, in this embodiment, the two first audio data may have only one byte validity identifier. For example, the order in which the data receiving end receives the data packets is audio data 1, audio data 2, audio data 3, ..., then the first byte validity identifier is generated for audio data 1 and audio data 2, which is used to extract the first audio data; the second byte validity identifier is generated for audio data 2 and audio data 3, which is used to extract the second audio data, and so on. If the first byte validity identifier is 0, indicating that the audio data 1 and audio data 2 corresponding to the identifier are invalid, then determine whether the second byte validity identifier is 0. If it is not 0, start extracting the data in audio data 2 and audio data 3 corresponding to the second byte validity identifier. It should be noted that the audio data here refers to the first audio data transmitted by the protocol.

[0077] It should be noted that the above-described setting of the byte validity flag is only one implementation of this embodiment and is not limited thereto. As long as two consecutive audio data can be determined to be valid based on the validity flag, any validity determination is sufficient. For example, a corresponding byte validity flag may be set for each audio data item, and the two consecutive flags may be determined to determine whether the two consecutive audio data items are valid.

[0078] Specifically, the above-mentioned step of extracting audio data based on two consecutive first audio data to obtain the second audio data may include the following processing: analyzing the validity of each byte in the two consecutive first audio data and the channel to which it belongs; and sending the byte data of the left channel and the byte data of the right channel to their corresponding channels respectively.

[0079] In this embodiment, the cached byte validity flag and audio data can be shifted simultaneously to analyze the validity of each byte of the audio data and the channel to which it belongs, and the byte data of the left and right channels can be sent to their corresponding channels (2'b01: left channel; 2'b10: right channel, otherwise the data is invalid) in sequence to achieve data extraction.

[0080] In other embodiments, a state machine may be used to execute the audio data extraction process according to the following steps: in an idle state, a bandwidth matching enable signal is detected, and when the data period of the second audio data is determined, the valid flag reading state is entered; in the valid flag reading state, two consecutive byte validity identifiers are read, and when it is detected that the byte validity identifier indicates that the byte is valid, the audio data reading state is entered; in the audio data reading state, the first audio data corresponding to the two consecutive byte validity identifiers is read, and the data extraction state is entered; in the data extraction state, the validity of each byte in the first audio data and the channel to which it belongs are analyzed, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channels in sequence.

[0081] In this embodiment, the Figure 2 As shown, the audio extraction process can be implemented using the following four state machines. Specifically, the IDLE state (idle state) is the state in which the state machine remains in the reset state and after audio data recovery is completed. In this state, after the bandwidth match enable signal is detected and the clearing condition of the data synchronization cycle counter is met, the state enters the read audio byte validity flag (DOWN_RD_FLAG) state.

[0082] DOWN_RD_FLAG state (valid flag read state): This state reads the byte validity flag stored in the data RAM and stores it in a 24-bit register for use during the data recovery phase. If the read data in this state is detected as 0, it indicates that the corresponding audio data has been recovered and the data channel flag for the next address is required. In this state, if the state machine detects that the read byte validity flag is valid, it enters the audio data read (DOWN_RD_DATA) state.

[0083] DOWN_RD_DATA state (read audio data state): This state reads the addresses of two audio data stored in RAM and stores the data in a 96-bit register. This state is used in the data recovery state. In this state, the state machine detects that the two data have been read and enters the data extraction (DATA_RECOVER) state.

[0084] DATA_RECOVER state (data extraction state): This state extracts data by simultaneously shifting the cached data byte validity flags and the audio data, analyzing the validity of each audio data byte and the channel to which it belongs. The left and right channel byte data are sequentially sent to their corresponding channels (2'b01: left channel; 2'b10: right channel; otherwise, the byte data is invalid). When data extraction is complete, the state machine re-enters the IDLE state.

[0085] In other embodiments, in each data sampling cycle, before reading two consecutive first audio data, a bandwidth matching process may be performed, such as Figure 3 As shown, the bandwidth matching process includes the following steps: step S11, obtaining the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame; step S12, determining the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes of the current frame; step S13, determining the sampling frequency of the second audio data under the local clock of the data receiving end according to the data period and the local clock frequency of the data receiving end.

[0086] The first audio data of the current frame can be understood as a data packet of the first audio data that has been received and is just received. For example, if a data packet is received, a pulse signal is generated to indicate that the frame audio data has been received. The current frame in this embodiment is the audio frame data corresponding to the latest generated pulse signal. The first audio data can include a plurality of first audio data, for example, 100 audio data. The current frame sampling time can be understood as the sampling time of the first audio data of the current frame. It can be the sampling time of all first audio data included in the current frame, or the sampling time of part of the first audio data included in the current frame.

[0087] Specifically, the number of valid bytes of the first audio data included in the first audio data of the current frame can be preset, or the number of valid bytes can be determined according to the validity flag of the data and the number of valid bytes is counted during the audio data receiving process. This embodiment does not make specific limitation. In addition, in the case of presetting the number of valid bytes, the number of valid bytes of each frame of audio data can be the same. For example, in the case of 100 audio data included in the first audio data of the current frame, the corresponding number of valid bytes can be 99.

[0088] In this embodiment, since the first audio data can include invalid audio data, the actual number of valid bytes in the first audio data of the current frame is less than the total number of bytes in the first audio data of the current frame. The second audio data can be understood as data including only valid audio bytes, and the time of the first audio is the same as the time of the second audio. Therefore, the data period of the second audio data can be directly calculated according to the current frame sampling time and the number of valid bytes, and the data period of each frame of audio data can be obtained, so that the bandwidth matching of the first audio data whose order is disturbed can be performed frame by frame to obtain a more accurate actual sampling frequency of the data receiving end under the local clock.

[0089] It can be understood that each audio data is originally multi-byte, that is, an audio data can include a plurality of valid bytes, so when calculating the data period, the audio sampling depth (or audio data bit width) also needs to be considered.

[0090] Before performing the above step S11, the received first audio data can also be written to the audio cache area of the memory, and the number of collected first audio data is counted.

[0091] Specifically, the embodiment can perform the data flow processing process as shown in Figure 4 The disturbed first audio data received can be directly written to the audio cache area in the memory (RAM), and the number of collected first audio data is counted.

[0092] Furthermore, the first audio data can be written to the RAM cache while being read from the cache. When the first audio data cache has 80 data, bandwidth matching is enabled and bandwidth matching begins. It should be understood that the number 80 is just one example of this embodiment and is not limited thereto. For example, the number 60, 70, 85, 88, and other data may also be used.

[0093] Specifically, it can be set according to the size of the audio buffer in the local memory, because the data size of the audio is uncertain, and leaving a larger space will waste resources. Since the played audio data will no longer require storage space, it can be designed to leave a section of storage space for cyclic writing to realize audio data caching. For example, 100 addresses are left in the memory as audio buffer areas. After the audio is stored in the 100th address, the next cache will be restarted from address 1. Because only one first audio data will be stored in each data cycle, and two audio data need to be read in one data cycle, so in theory, as long as more than 3 and less than 98 first audios are stored, it is possible to avoid reading the played audio data.

[0094] In some embodiments, the above-mentioned process of obtaining the current frame sampling time of the first audio data may include the following processing: detecting that the sampling of the first audio data of the previous frame is completed; counting and statistically analyzing the time from the completion of the sampling of the first audio data of the previous frame to the completion of the sampling of the first audio data of the current frame, and determining it as the current frame sampling time.

[0095] In this embodiment, given the continuous data transmission, the time between the two reception completion times can be used to determine the sampling time of the next frame of data. This eliminates the need for intermediate determination and allows the sampling time of the next frame of data, i.e., the first audio data of the current frame, to be determined by simply recording the count values ​​at the two reception completion times.

[0096] Furthermore, this embodiment can generate an identification pulse when a preset number of first audio data are collected; the identification pulse is used to indicate that a frame of first audio data has been collected and can be used to enable bandwidth matching; and when the identification pulse is detected, it is determined that the sampling of the first audio data of the current frame is completed.

[0097] Specifically, a flag pulse signal may be generated every 100 audio data frames to indicate that a frame of audio data has been received, thereby initiating the bandwidth matching process. It should be understood that the 100 audio data frames are merely one implementation of this embodiment, and this embodiment does not impose a specific limit on this number. This number may be specifically set based on the data structure of the first audio data and may be consistent with the number of bytes contained in a frame of data in the first audio data.

[0098] More specifically, the bandwidth matching counter cnt0 can be used to count the sampling time of a frame of the first audio data frame for bandwidth matching. When the next frame notification flag is detected, it indicates that the previous frame data has ended and the next frame data has begun. At this time, cnt0 has completed the time statistics of the previous frame data.

[0099] In other optional embodiments, the above-mentioned step S12 may specifically include the following processing: when the current frame sampling time includes the sampling time of all audio data of the current frame, determining the data period of the second audio data of the current frame based on the quotient of the current frame sampling time and the number of valid bytes of the current frame; when the current frame sampling time has not been counted, determining the preset period default value as the data period of the second audio data of the current frame.

[0100] In this embodiment, the basic principle of bandwidth matching is that the local clock continuously counts the sampling time of the current frame of audio data, for example, the time it takes to transmit 100 audio data bytes. A divider is then used to calculate the actual bandwidth corresponding to the number of valid bytes (e.g., 99) of audio data, thereby achieving accurate frame-by-frame matching. However, during the transmission of the first frame of data, because the bandwidth matching counter cnt0 has not yet completed counting the data frame, the data cycle counter cnt1 needs to calculate the data cycle based on a default value calculated based on the actual audio sampling frequency combined with the local clock frequency. After the first frame of data transmission is completed, the data cycle counter uses the quotient of the divider to calculate the data cycle.

[0101] In the above-mentioned step S13, in the process of determining the sampling frequency of the second audio data under the local clock of the data receiving end based on the data cycle and the local clock frequency of the data receiving end, the cycle length N of the local clock can be determined based on the local clock frequency f. Then, based on the calculated data cycle multiplied by the cycle length N of the local clock, the cycle length N' of the second audio data under the local clock of the data receiving end can be obtained. The reciprocal of the cycle length N' can be understood as the sampling frequency of the second audio data under the local clock of the data receiving end.

[0102] Furthermore, the data period of the second audio data of the current frame is determined based on the quotient of the current frame sampling time and the number of valid bytes, including: when the current frame sampling time is within a preset threshold range, using a divider to calculate the quotient of the current frame sampling time and the number of valid bytes; and determining the data period of the second audio data of the current frame based on the quotient and a preset margin.

[0103] Specifically, the value of cnt0 can be sent to a divider and divided by the actual number of samples of the audio data, 99, to obtain the number of cycles of the audio data under the local clock.

[0104] Consider not using the remainder of the divider, such as Figure 5In the figure, when the cnt0 statistical value is 10000 during a packet of data, if the divider quotient 98 is used as the matching data cycle, 100 data cycles will be generated, but there are actually only 99 valid data. Therefore, the extra data cycle (such as the shaded part in the figure) will generate a noise point, forming noise.

[0105] To avoid this situation, the divider quotient + 1 is used to calculate the data cycle. This method theoretically ensures that only 99 data will be generated when the next frame notification arrives. Figure 6 As shown in the figure, the number of clock cycles of the last data under the local clock is one clock cycle less than that of the first 98 data. Therefore, it can be seen that the higher the operating frequency of the local clock, the smaller the cycle error of the last data and the higher the bandwidth matching accuracy.

[0106] For audio data of this structure, this method is not applicable in all cases. Let the time for the bus to transmit 100 audio data be y, the quotient of the divider (i.e., the data cycle) be x, and the remainder be n, then:

[0107] y=99*x+n

[0108] =98*(x+1)+(x+n-98)

[0109] Wherein, x and n are both integers greater than 0.

[0110] From the above formula, we can see that 98*(x+1) can be regarded as the total time of the first 98 data, and (x+n-98) is the data period of the last data. Therefore, the value of (x+n-98) must be greater than 0. Without considering the remainder n, the data period x>98.

[0111] Because data extraction requires additional clock cycles, the last data cycle must not only be greater than 0 but also greater than the number of clock cycles required for data processing. Let m be the number of clock cycles required for data processing. This method works only if x - 98 > m, or x > 98 + m. Let k represent the number of valid data points, 99, and x > k + m - 1.

[0112] Considering that the external clock frequency may be offset, the phenomenon is that the number of clock cycles counted by the previous and next frames is different. The situation of generating multiple data cycles is similar to the above and will not be described in detail. For example, when the clock frequency of the data sending end gradually increases, the notification of the next frame will arrive earlier, such as Figure 7As shown, the data time of the previous frame is significantly longer than that of the next frame. In severe cases, this can result in missing data cycles when counting the next frame. Therefore, a margin must be added to the division result to reduce the average data cycle to prevent missing data cycles and data loss. Therefore, the margin value needs to be configurable. When the system detects that the data source clock is off, it increases the margin to reduce the average data cycle, thus preventing data loss. The above diagram is for illustration only. (In practice, due to the clock calibration module, the clock offset is not significant.) The most likely scenario is that the condition x-98>m fails, resulting in the loss of the last data cycle. Therefore, a margin of 1 can resolve this issue.

[0113] After calculating the data period of the second audio data, the sampling frequency of the data receiving end under the local clock can be calculated according to the inverse relationship between the period and the frequency, thereby achieving bandwidth matching of the audio data and accurately extracting the audio data.

[0114] In a specific embodiment, the Figure 8 and Figure 9 The bandwidth matching process and the data extraction process are performed separately, thereby realizing the audio data extraction process provided by this embodiment. Specifically, Figure 8 As shown in the figure, the bandwidth matching process includes the following steps: ① Audio data is written to and read from the RAM buffer simultaneously. When the audio buffer contains 80 data points, the bandwidth matching enable is enabled and bandwidth matching begins. ② Bandwidth matching: The bandwidth matching counter cnt0 counts the data frame time for bandwidth matching. When the next frame notification flag is detected, it indicates that the previous frame of data has ended and the next frame of data has begun. At this point, cnt0 has completed the time counting of the previous frame of data. The cnt0 value is fed into the divider and divided by the actual number of audio data samples, 99, to obtain the number of audio data cycles based on the local clock. ③ During the first frame of data transmission, because the bandwidth matching counter cnt0 has not yet completed the data frame counting, the data cycle counter cnt1 calculates the data cycle based on a default value calculated from the actual audio sampling frequency and the local clock frequency. After the first frame of data transmission is completed, the data cycle counter uses the quotient of the divider to calculate the data cycle.

[0115] like Figure 9As shown, the data extraction process includes the following steps: During audio transmission, the shuffled audio data is written to a designated area in RAM for caching. Meanwhile, during the bandwidth matching phase, within each cycle of the matched data sampling, the byte validity flags of the two audio data are read to determine if the data is valid and then sent to the shift register for caching. Then, two corresponding audio data are read from the RAM audio buffer and cached using a 96-bit shift register. The validity of each byte of the audio data is then checked by shifting the cached byte validity flag and the audio data. If the read data is 0, indicating that the current corresponding audio data has been extracted, the byte validity flag of the next address needs to be obtained. The corresponding first audio data is analyzed and the valid bytes are sequentially sent to the output buffer shift register for output. By simultaneously shifting the cached byte validity flag and audio data, the validity of each byte of the audio data and its corresponding channel are analyzed, and the byte data of the left and right channels are sequentially sent to their corresponding channels (2'b01: left channel; 2'b10: right channel), completing data extraction. After the extraction of a data (48 bits) is completed, the data is sent out at a specified time within the bandwidth matching data cycle to achieve data extraction and bandwidth matching.

[0116] Specifically, it is possible to first determine whether the bandwidth matching process is enabled. If the bandwidth matching process is enabled, it is further determined whether the caching of 80 first audio data has been completed, and whether the data cycle calculation is completed, that is, whether the cnt1 count is cleared. After the data cycle calculation is completed, that is, after the cnt1 count is cleared, the byte validity identifier of the audio data can be read. For example, the byte validity identifiers of two consecutive audio data can be read from RAM address0. If the byte validity identifier indicates that the data is valid, the byte validity identifier of the first audio data is stored in the shift register flg_data[23:0] for caching; if the byte validity identifier indicates that the data is invalid, a data identifier indicating that the data is invalid is generated, such as data_invalid_flg=1, and the byte validity identifiers of the next two consecutive data are read from RAM address0+1 until a byte validity identifier indicating that two consecutive audio data are valid is read. Then, it can be determined whether the audio data at the current address needs to be extracted based on the data identifier. For example, if data_invalid_flg=0, it means that there is data to be extracted from the current cache address address1 and address1+1, then two audio data are read from the corresponding addresses address1 and address1+1 in RAM and stored in the shift register data[95:0] for caching; if data_invalid_flg=1, it means that the data in the addresses address1 and address1+1 have been extracted and are invalid, and the next two consecutive audio data need to be read from RAM address1+1 and address1+2 and stored in the shift register data[95:0] for caching.

[0117] Then, by simultaneously shifting the buffered data's left and right channel (byte validity) flags and the audio data, the validity of each byte of the audio data and the channel to which it belongs are analyzed. The byte data for the left and right channels are then sequentially sent to their corresponding channels (2'b01: left channel; 2'b10: right channel), achieving data extraction. For example, if flg_data[23:22] = 2'b10, it indicates that the current audio data belongs to the right channel and can be sent to the right channel, that is, the audio data is written to the corresponding audio register: audio_data[23:0] = {audio_data[15:0], data[95:88]}. Meanwhile, if flg_data[23:22] = 2'b01, it indicates that the current audio data belongs to the left channel and can be sent to the left channel, that is, the audio data is written to the corresponding audio register: audio_data[47:24] = {audio_data[40:24], data[95:88]}. The circular shifter is then cyclically shifted, i.e., flg_data = {flg_data[21:0], flg_data[23:22]}, data[95:0] = {data[87:0], data[95:88]}, until this occurs 12 times and 6 bytes of data extraction are complete, then the shift ends. It can be further determined whether address1! = the last address. If so, address0 = starting address 0 and address1 = starting address 1. If not, address0 = address0 + 1 and address1 = address1 + 1. It can then be determined again whether the bandwidth matching process has been initiated. If not, the data extraction process ends immediately. If the bandwidth matching process is still initiated, the data period calculation is re-determined, and another round of audio data extraction is performed until the bandwidth matching process ends.

[0118] In summary, the audio data extraction method provided in this embodiment can read two consecutive first audio data in each cycle of the matched data sampling in the bandwidth matching stage, perform audio data extraction on the two consecutive first audio data, obtain audio data with the correct order, and thus accurately extract the audio data. And after completing the extraction of one data (48bit), the data is sent out at a specified time within the bandwidth matching data cycle to achieve data extraction and bandwidth matching. It is also possible to obtain the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame; determine the data cycle of the second audio data based on the current frame sampling time and the number of valid bytes; determine the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end. In this way, the first audio data whose order is disrupted can be accurately bandwidth matched, and the actual sampling frequency of the data receiving end under the local clock is obtained to ensure that the data receiving end can correctly extract the original audio data of the disrupted first audio data, and match its actual sampling frequency to send the audio data to ensure that the downstream data receiving end can directly process the audio.

[0119] Based on the same concept as the above audio data extraction method, this embodiment also provides an audio data extraction method, as shown in the attached Figure 10 As shown, the audio data extraction device includes:

[0120] A data reading module is used to read two consecutive first audio data in each data sampling cycle; the first audio data refers to audio data whose data transmission order is periodically disrupted;

[0121] The data extraction module is used to extract audio data based on two consecutive first audio data to obtain second audio data; the second audio data refers to audio data with normal data transmission sequence.

[0122] The audio data extraction device provided in this embodiment is based on the same concept as the above-mentioned audio data extraction method, so it can at least achieve the above-mentioned beneficial effects, and any of the above-mentioned implementation methods can be applied to the audio data extraction method provided in this embodiment, which will not be repeated here.

[0123] like Figure 11 As shown, this embodiment may further include an audio cache module, which writes the received scrambled audio data directly to the audio cache area in the RAM, counts the number of audio data, and generates a flag pulse signal to notify the frequency determination module every time 100 audio data are counted; the frequency determination module counts the time between two notification flags and uses a divider to calculate the data period of the actual audio, thereby achieving bandwidth matching of the audio data.

[0124] Based on the same concept as the above audio data extraction method, this embodiment further provides a chip on which the above audio data extraction method is integrated.

[0125] The chip provided in this embodiment is based on the same concept as the above-mentioned audio data extraction method, so it can at least achieve the beneficial effects that can be achieved by the above-mentioned audio data extraction method, and any implementation of the above-mentioned audio data extraction method can be applied to the chip provided in this embodiment, which will not be repeated here.

[0126] An embodiment of the present application further provides an electronic device corresponding to the audio data extraction method provided in the aforementioned embodiment, to execute the aforementioned audio data extraction method.

[0127] Please refer to Figure 12 , which shows a schematic diagram of the hardware structure of the electronic device provided by some embodiments of the present application. Figure 12 As shown, the electronic device 6 includes: a processor 600, a memory 601, a bus 602 and a communication interface 603, wherein the processor 600, the communication interface 603 and the memory 601 are connected via the bus 602; the memory 601 stores a computer program that can be run on the processor 600, and when the processor 600 runs the computer program, it executes the audio data extraction method provided in any of the aforementioned embodiments of the present application.

[0128] Memory 601 can be any electronic, magnetic, optical, or other physical storage device and can contain stored information, such as executable instructions, data, and the like. Specifically, memory 601 can be RAM (Random Access Memory), flash memory, a storage drive (such as a hard drive), or similar storage media, or a combination thereof. Communication between the device network element and at least one other network element is achieved via at least one communication interface 603 (which can be wired or wireless), and can utilize the Internet, a wide area network, a local area network, a metropolitan area network, and the like.

[0129] The bus 602 may include an I2C (Inter-integrated Circuit) bus, a MIPI (Mobile Industry Processor Interface) bus, an SPl (Serial Peripheral Interface) bus, an I2S (Integrated Interchip Sound) bus, etc. It may also be divided into an address bus, a data bus, a control bus, etc. according to its function.

[0130] The processor 600 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 600 or by software instructions. The above processor 600 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 601 , and the processor 600 reads the information in the memory 601 and completes the steps of the above method in combination with its hardware.

[0131] The electronic device provided in the embodiment of the present application and the audio data extraction method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented by them.

[0132] An embodiment of the present application also provides a computer-readable storage medium corresponding to the audio data extraction method provided in the aforementioned embodiment, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the audio data extraction method provided in any of the aforementioned embodiments.

[0133] It should be noted that computer-readable storage media may include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0134] An embodiment of the present application further provides a computer program product, including a computer program, which is executed by a processor to implement the audio data extraction method of any of the above embodiments.

[0135] The computer readable storage medium and the computer program product provided by the above embodiments of the present application are based on the same inventive concept as the audio data extraction method provided by the embodiments of the present application, and have the same beneficial effects as the method adopted, run or implemented by the application program stored therein.

[0136] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, but not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement for part or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should be covered in the scope of the claims and the specification of the present application. Especially, as long as there is no structural conflict, each technical feature mentioned in each embodiment can be combined in any way. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for extracting audio data, characterized in that: For a data receiving end, the method includes: Reading two consecutive first audio data in each data sampling cycle; comprising: detecting byte validity flags of the two consecutive first audio data in each data sampling cycle; and reading the two consecutive first audio data when the byte validity flags indicate that the two consecutive first audio data are valid; wherein the first audio data refers to audio data whose data transmission order is periodically disrupted; Audio data extraction is performed based on the two consecutive first audio data to obtain second audio data; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission order.

2. The method according to claim 1, wherein The extracting audio data based on the two consecutive first audio data to obtain second audio data includes: Analyzing the validity of each byte in the two consecutive first audio data and the channel to which the byte belongs; The byte data of the left channel and the byte data of the right channel are sent to their corresponding channels respectively.

3. The method according to claim 1, wherein Using a state machine, the audio data extraction process is performed as follows: In an idle state, a bandwidth matching enable signal is detected, and when a data period of the second audio data is determined, a valid flag reading state is entered; In the valid flag reading state, reading the byte validity flags of two consecutive audio data, and entering the audio data reading state when it is detected that the byte validity flag indicates that the byte is valid; Otherwise, read the byte validity flags of the next two consecutive audio data and determine the validity of the next two consecutive audio data; In the audio data reading state, reading two consecutive first audio data corresponding to the byte validity flag, and entering the data extraction state; In the data extraction state, the validity of each byte in the first audio data and the channel to which it belongs are analyzed according to the byte validity identifiers of the two consecutive audio data, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channels in sequence.

4. The method according to claim 1, wherein The method further includes, before reading two consecutive first audio data in each data sampling cycle: Obtaining a current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame; Determine a data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes of the current frame; A sampling frequency of the second audio data under the local clock of the data receiving end is determined according to the data period and the local clock frequency of the data receiving end.

5. The method according to claim 4, wherein: The obtaining the current frame sampling time of the first audio data includes: Detecting that sampling of the first audio data of the previous frame is completed; The time from the completion of sampling of the first audio data of the previous frame to the completion of sampling of the first audio data of the current frame is counted and determined as the current frame sampling time.

6. The method according to claim 5, wherein The method further comprises: When a preset number of the first audio data are collected, a flag pulse is generated; the flag pulse is used to indicate that a frame of the first audio data is collected; The identification pulse is detected to determine that sampling of the first audio data of the current frame is completed.

7. The method according to claim 4, wherein The determining the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes of the current frame includes: When sampling of the current frame audio data is completed within the current frame sampling time, determining a data period of the second audio data of the current frame based on a quotient of the current frame sampling time and the number of valid bytes of the current frame; When the sampling of the current frame audio data is not completed within the current frame sampling time, the preset period default value is determined as the data period of the second audio data of the current frame.

8. The method according to claim 7, wherein The determining the data period of the second audio data based on the quotient of the current frame sampling time and the number of valid bytes of the current frame includes: When the current frame sampling time is within a preset threshold range, using a divider to calculate a quotient of the current frame sampling time and the number of valid bytes; A data period of the second audio data of the current frame is determined based on the quotient and a preset margin.

9. The method according to claim 4, wherein Before obtaining the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame, the method further includes: The received first audio data is written into an audio buffer area of ​​a memory, and the number of the collected first audio data is counted.

10. An audio data extraction device, characterized in that: Applied to a data receiving end, the device comprises: A data reading module, configured to read two consecutive first audio data within each data sampling cycle; the module comprising: detecting a byte validity flag of the two consecutive first audio data within each data sampling cycle; and reading the two consecutive first audio data if the byte validity flag indicates that the two consecutive first audio data are valid; wherein the first audio data refers to audio data whose data transmission order is periodically disrupted; The audio extraction module is used to extract audio data based on the two consecutive first audio data to obtain second audio data; the second audio data refers to audio data corresponding to the first audio data and with normal data transmission order.

11. An electronic device, characterized in that: The system comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program, characterized in that The computer program is executed by a processor to implement the method according to any one of claims 1 to 9.

Images