Audio data processing method and apparatus, electronic device, and program product

Abstract

The application provides an audio data processing method and device, electronic equipment and program product, which are used at a data receiving end. The method comprises the following steps: obtaining a current frame sampling time of first audio data and a number of valid bytes contained in the first audio data of the current frame; the first audio data refers to audio data whose data transmission sequence is periodically disturbed; determining a data period of second audio data of the current frame based on the current frame sampling time and the number of valid bytes; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission sequence; and determining a sampling frequency of the second audio data under a local clock of the data receiving end according to the data period and the local clock frequency of the data receiving end. The method can quickly and accurately realize bandwidth matching of the audio data, so as to correctly extract the audio data.

Description

Technical Field

[0001] The present application belongs to the field of audio data processing technology, and specifically relates to an audio data processing method, device, electronic device and program product. Background Art

[0002] Currently, processing audio data requires not only the original audio data but also its transmission frequency, which must be consistent with the audio data's sampling frequency. Otherwise, accurate audio processing is impossible. Spectral analysis, sampling rate conversion, and echo cancellation, for example, all rely on the audio data's sampling frequency for accurate analysis and processing. Conventional audio processing methods cannot directly process audio data with unusual structures, such as periodic scrambling or transmission frequency mismatches with the data's actual sampling frequency.

[0003] It should be noted that the above statements are only used to provide background technical information related to this application and do not necessarily constitute prior art. Summary of the Invention

[0004] The present application proposes an audio data processing method, device, electronic device and program product, which can quickly and accurately achieve bandwidth matching of audio data so as to correctly extract audio data.

[0005] The first embodiment of the present application provides an audio data processing method for a data receiving end, the method comprising:

[0006] Obtaining a current frame sampling time of first audio data and the number of valid bytes contained in the first audio data of the current frame; the first audio data refers to audio data whose data transmission order is periodically disrupted;

[0007] determining a data period of second audio data of the current frame based on the current frame sampling time and the number of valid bytes; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission order;

[0008] A sampling frequency of the second audio data under the local clock of the data receiving end is determined according to the data period and the local clock frequency of the data receiving end.

[0009] In some optional embodiments, obtaining the current frame sampling time of the first audio data includes:

[0010] Detecting that sampling of the first audio data of the previous frame is completed;

[0011] The time from the completion of sampling of the first audio data of the previous frame to the completion of sampling of the first audio data of the current frame is counted and determined as the current frame sampling time.

[0012] In some optional embodiments, the method further includes:

[0013] When a preset number of the first audio data are collected, a flag pulse is generated; the flag pulse is used to indicate that a frame of the first audio data is collected;

[0014] The identification pulse is detected to determine that sampling of the first audio data of the current frame is completed.

[0015] In some optional embodiments, determining the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes includes:

[0016] When sampling of the current frame audio data is completed within the current frame sampling time, determining a data period of the second audio data of the current frame based on a quotient of the current frame sampling time and the number of valid bytes;

[0017] When the sampling of the current frame audio data is not completed within the current frame sampling time, the preset period default value is determined as the data period of the second audio data of the current frame.

[0018] In some optional embodiments, determining the data period of the second audio data of the current frame based on the quotient of the current frame sampling time and the number of valid bytes includes:

[0019] When the current frame sampling time is within a preset threshold range, using a divider to calculate a quotient of the current frame sampling time and the number of valid bytes;

[0020] A data period of the second audio data of the current frame is determined based on the quotient and a preset margin.

[0021] In some optional embodiments, before obtaining the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame, the method further includes:

[0022] The received first audio data is written into an audio buffer area of ​​a memory, and the number of the collected first audio data is counted.

[0023] In some optional embodiments, after determining the sampling frequency of the receiving end under the local clock according to the data period, the method further includes:

[0024] In each of the data cycles, reading two consecutive first audio data;

[0025] Audio data extraction is performed based on the two consecutive first audio data to obtain second audio data; the second audio data refers to audio data with normal data transmission order.

[0026] In some optional embodiments, reading two consecutive first audio data in each data period includes:

[0027] In each of the data cycles, detecting two consecutive byte validity flags of the first audio data;

[0028] In a case where the byte validity identifier indicates that the two consecutive first audio data are valid, the two consecutive first audio data are read.

[0029] In some optional embodiments, extracting audio data based on the two consecutive first audio data to obtain second audio data includes:

[0030] Analyzing the validity of each byte in the two consecutive first audio data and the channel to which it belongs;

[0031] The byte data of the left channel and the byte data of the right channel are sent to their corresponding channels respectively.

[0032] In some optional embodiments, a state machine is used to perform the audio data processing process according to the following steps:

[0033] In an idle state, a bandwidth matching enable signal is detected, and when a data period of the second audio data is determined, a valid flag reading state is entered;

[0034] In the valid flag reading state, reading the byte validity flags of two consecutive data, and entering the audio data reading state when it is detected that the byte validity flag indicates that the byte data is valid; otherwise, reading the byte validity flags of the next two consecutive audio data, and determining the validity of the next two consecutive audio data;

[0035] In the audio data reading state, reading two consecutive first audio data corresponding to the byte validity flag, and entering the data extraction state;

[0036] In the data extraction state, the validity of each byte in the first audio data and the channel to which it belongs are analyzed according to the byte validity identifiers of the two consecutive audio data, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channels in sequence.

[0037] An embodiment of a second aspect of the present application provides an audio data processing device, applied to a data receiving end, the device comprising:

[0038] a data acquisition module, configured to acquire a current frame sampling time of first audio data and the number of valid bytes contained in the first audio data of the current frame; the first audio data refers to audio data whose data transmission order is periodically disrupted;

[0039] a data period determination module, configured to determine a data period of second audio data of a current frame based on the current frame sampling time and the number of valid bytes; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission order;

[0040] The frequency determination module is used to determine the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end.

[0041] An embodiment of the third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.

[0042] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium having a computer program stored thereon, which is executed by a processor to implement the method as described in the first aspect.

[0043] An embodiment of the fifth aspect of the present application provides a computer program product, including a computer program, which is executed by a processor to implement the method described in the first aspect.

[0044] The technical solutions provided in the embodiments of this application have at least the following technical effects or advantages:

[0045] The audio data processing method provided in the embodiment of the present application first obtains the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame; determines the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes; and determines the sampling frequency of the data under the local clock of the receiving end based on the data period. In this way, the data period of each frame of audio data can be obtained, so that the bandwidth of the first audio data whose order is disrupted can be matched frame by frame to obtain a more accurate actual sampling frequency of the second audio data under the local clock of the receiving end, and the audio data is sent according to the actual sampling frequency to ensure that the downstream data receiving end can directly process the audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present application. Throughout the accompanying drawings, the same reference numerals are used to denote the same components.

[0047] In the attached figure:

[0048] Figure 1 A flowchart of an audio data processing method according to an embodiment of the present application is shown;

[0049] Figure 2 A data flow diagram of an audio data processing method provided by an embodiment of the present application is shown;

[0050] Figure 3 A schematic diagram of a data transmission process in which noise is generated during audio data transmission is shown;

[0051] Figure 4 A schematic diagram of a data transmission process for avoiding noise generation during audio data transmission in one embodiment of the present application is shown;

[0052] Figure 5 A schematic diagram of a data transmission process in which notification of the next frame arrives in advance during audio data transmission is shown;

[0053] Figure 6 A schematic diagram of the working process of audio data extraction implemented by a state machine according to an embodiment of the present application is shown;

[0054] Figure 7 A schematic diagram showing a specific flow chart of a bandwidth matching process in an audio data processing method provided in an embodiment of the present application is shown;

[0055] Figure 8 A schematic diagram showing a specific flow of an audio extraction process in an audio data processing method provided in an embodiment of the present application is shown;

[0056] Figure 9 A schematic structural diagram of an audio data processing device in another embodiment of the present application is shown;

[0057] Figure 10 A schematic structural diagram of an audio data processing device in another embodiment of the present application is shown;

[0058] Figure 11 A schematic diagram of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0059] The following describes exemplary embodiments of the present application in more detail with reference to the accompanying drawings. Although exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. Instead, these embodiments are provided to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0060] It should be noted that, unless otherwise specified, the technical or scientific terms used in this application should have the common meanings understood by those skilled in the art to which this application belongs.

[0061] In practical applications, data validity directly affects data usability and audio playback. Bandwidth consistency affects audio quality, and bandwidth mismatches can cause noise. Conventional audio processing methods cannot directly process data based on the transmission frequency, especially for audio structures where dual-channel data is periodically disrupted and the transmission frequency is inconsistent with the actual sampling frequency. Therefore, the audio processing end must not only extract the original audio and match the bandwidth, but also ensure the validity of the extracted data while avoiding noise generated during bandwidth matching (both over- and under-generated data will generate noise). Otherwise, the audio quality of the audio output from the audio device will be affected.

[0062] In order to solve the above problems, this embodiment studies and analyzes the data transmitted by the relevant fast charging protocol, and finds that in some electronic products that need to transmit audio data, such as but not limited to wired headphones, Bluetooth headphones, and other audio devices, the transmission protocols of different products are not the same, so the data transmission structure may also be different. For example, in a certain communication protocol, the audio transmission rate is fixed, but the transmitted audio sampling rate is not fixed. When the audio bus transmits 100 data, it actually only contains 99 594 bytes of scrambled audio data. The data receiving end needs to correctly extract the original audio data and send it out at the audio sampling rate to ensure that the downstream data receiving end can directly process the audio.

[0063] Based on the above findings, the embodiments of the present application provide an audio data processing method, device, electronic device, and program product, which is used at the data receiving end to first obtain the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame; determine the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes of the current frame; and determine the sampling frequency of the data under the local clock of the receiving end according to the data period. In this way, the data period of each frame of audio data can be obtained, so that the first audio data whose order is disrupted can be matched with the bandwidth frame by frame to obtain a more accurate actual sampling frequency of the data receiving end under the local clock, and match the actual sampling frequency to send the audio data to ensure that the downstream data receiving end can directly process the audio.

[0064] The audio data processing device is a processing module on an audio device for executing the audio data processing method, and may specifically be a processor or a specially configured microprocessor, which is not specifically limited in this embodiment.

[0065] The electronic device may be an audio device to which the above-mentioned audio data processing method is applied, such as but not limited to headphones, mobile phones, walkmans, etc., or may be an integrated chip on the audio device for performing audio data processing.

[0066] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0067] Please refer to Figure 1 , is a flow chart of the audio data processing method provided in the embodiment of the present application, as shown in the attached Figure 1 As shown, the audio data processing method includes.

[0068] Step S1: Acquire the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame.

[0069] The first audio data refers to audio data whose data transmission order is periodically disrupted. It is understood that even if the order of the audio data is disrupted, when the audio data is transmitted byte by byte after the disruption, in the worst case, the transmission is completed in two audio data packets (12 bytes), that is, two disrupted first audio data packets.

[0070] The first audio data of the current frame can be understood as the data packet of the first audio data that has been received and has just been received. For example, if a pulse signal is generated after a data packet is received, indicating that the reception of the frame of audio data has been completed, then the current frame in this embodiment is the audio frame data corresponding to the most recently generated pulse signal. It can contain multiple first audio data, for example, it can contain 100 audio data. The current frame sampling time can be understood as the sampling time of the first audio data of the current frame, which can be the sampling time of all the first audio data contained in the current frame, or it can be the sampling time of part of the first audio data contained in the current frame.

[0071] Specifically, the number of valid bytes of the first audio data contained in the first audio data of the current frame can be preset, or the byte validity can be determined based on the validity flag of the data during the audio data reception process, and the number of valid bytes can be counted. This embodiment does not specifically limit this. In addition, when the number of valid bytes is preset, the number of valid bytes of each frame of audio data can be the same. For example, when the first audio data of the current frame contains 100 audio data, the corresponding number of valid bytes can be 99.

[0072] Before executing the above step S1, this embodiment may further write the received first audio data into the audio buffer area of ​​the memory, and count the number of the collected first audio data.

[0073] Specifically, this embodiment can be implemented according to Figure 2 In the data stream processing process shown, the received scrambled first audio data may be directly written into the audio buffer area in the memory (RAM), and the number of the collected first audio data may be counted.

[0074] Furthermore, the first audio data can be written to the RAM cache while being read from the cache. When the first audio data cache has 80 data, bandwidth matching is enabled and bandwidth matching begins. It should be understood that 80 is just one implementation of this embodiment and is not limited thereto. A value greater than 50 and less than 100, such as 60, 70, 85, or 88, can also be used.

[0075] Specifically, it can be set according to the size of the audio buffer in the local memory, because the data size of the audio is uncertain, and leaving a larger space will waste resources. Since the played audio data will no longer require storage space, it can be designed to leave a section of storage space for cyclic writing to realize audio data caching. For example, 100 addresses are left in the memory as audio buffer areas. After the audio is stored in the 100th address, the next cache will be restarted from address 1. Because only one first audio data will be stored in each data cycle, and two audio data need to be read in one data cycle, so in theory, as long as more than 3 and less than 98 first audios are stored, it is possible to avoid reading the played audio data.

[0076] In some embodiments, the above-mentioned process of obtaining the current frame sampling time of the first audio data may include the following processing: detecting that the sampling of the first audio data of the previous frame is completed; counting and statistically analyzing the time from the completion of the sampling of the first audio data of the previous frame to the completion of the sampling of the first audio data of the current frame, and determining it as the current frame sampling time.

[0077] In this embodiment, given the continuous data transmission, the time between the two reception completion times can be used to determine the sampling time of the next frame of data. This eliminates the need for intermediate determination and allows the sampling time of the next frame of data, i.e., the first audio data of the current frame, to be determined by simply recording the count values ​​at the two reception completion times.

[0078] Furthermore, this embodiment can generate a flag pulse when a preset number of first audio data are collected; the flag pulse is used to indicate that a frame of first audio data is collected; and when the flag pulse is detected, it is determined that the sampling of the first audio data of the current frame is completed.

[0079] Specifically, a flag pulse signal may be generated every 100 audio data frames to indicate that a frame of audio data has been received, thereby initiating the bandwidth matching process. It should be understood that the 100 audio data frames are merely one implementation of this embodiment, and this embodiment does not impose a specific limit on this number. This number may be specifically set based on the data structure of the first audio data and may be consistent with the number of bytes contained in a frame of data in the first audio data.

[0080] More specifically, the bandwidth matching counter cnt0 can be used to count the sampling time of a frame of the first audio data frame for bandwidth matching. When the next frame notification flag is detected, it indicates that the previous frame data has ended and the next frame data has begun. At this time, cnt0 has completed the time statistics of the previous frame data.

[0081] Step S2: determining a data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes.

[0082] The data cycle can be understood as the sampling period of the ADC (analog-to-digital converter) when the data receiving end receives the audio data. The second audio data refers to audio data that corresponds to the first audio data and has a normal data transmission order. Here, corresponding to the first audio data means that the two audio data contain the same valid data bytes.

[0083] In this embodiment, because the first audio data may contain invalid audio data, the actual number of valid bytes in the first audio data of the current frame is less than the total number of bytes in the first audio data of the current frame. The second audio data can be understood as data containing only valid audio bytes, and the timing of the first audio and the second audio are the same. Therefore, the data period of the second audio data can be directly calculated based on the current frame sampling time and the number of valid bytes.

[0084] It can be understood that each audio data is originally multi-byte, that is, one audio data may include multiple valid bytes, so when calculating the data period, the audio sampling depth (or audio data bit width) also needs to be considered.

[0085] In some embodiments, the above-mentioned step S2 may specifically include the following processing: when the current frame sampling time includes the sampling time of all audio data of the current frame, determining the data period of the second audio data of the current frame based on the quotient of the current frame sampling time and the number of valid bytes; when the current frame sampling time includes the sampling time of part of the audio data of the current frame, determining the preset period default value as the data period of the second audio data of the current frame.

[0086] In this embodiment, the basic principle of bandwidth matching is that the local clock continuously counts the sampling time of the current frame of audio data, such as the time it takes to transmit 100 audio data bytes. A divider is then used to calculate the actual bandwidth corresponding to the number of valid bytes (e.g., 99) of audio data, thereby achieving accurate frame-by-frame matching. However, during the transmission of the first frame of data, since the bandwidth matching counter cnt0 has not yet completed counting the data frame, the data cycle counter cnt1 needs to calculate the data cycle based on a default value calculated based on the actual audio sampling frequency combined with the local clock frequency. After the first frame of data transmission is completed, the data cycle counter uses the quotient of the divider to calculate the data cycle.

[0087] Furthermore, the data period of the second audio data of the current frame is determined based on the quotient of the current frame sampling time and the number of valid bytes of the current frame, including: when the current frame sampling time is within a preset threshold range, using a divider to calculate the quotient of the current frame sampling time and the number of valid bytes; and determining the data period of the second audio data of the current frame based on the quotient and a preset margin.

[0088] Specifically, the value of cnt0 can be sent to a divider and divided by the actual number of samples of the audio data, 99, to obtain the number of cycles of the audio data under the local clock.

[0089] Consider not using the remainder of the divider, such as Figure 3 In the figure, when the cnt0 statistical value is 10000 during a packet of data, if the divider quotient 98 is used as the matching data cycle, 100 data cycles will be generated, but there are actually only 99 valid data. Therefore, the extra data cycle (such as the shaded part in the figure) will generate a noise point, forming noise.

[0090] To avoid this situation, the divider quotient + 1 is used to calculate the data cycle. This method theoretically ensures that only 99 data will be generated when the next frame notification arrives. Figure 4 As shown in the figure, the number of clock cycles of the last data under the local clock is one clock cycle less than that of the first 98 data. Therefore, it can be seen that the higher the operating frequency of the local clock, the smaller the cycle error of the last data and the higher the bandwidth matching accuracy.

[0091] For audio data of this structure, this method is not applicable in all cases. Let the time for the bus to transmit 100 audio data be y, the quotient of the divider (i.e., the data cycle) be x, and the remainder be n, then:

[0092] y=99*x+n

[0093] =98*(x+1)+(x+n-98)

[0094] Wherein, x and n are both integers greater than 0.

[0095] From the above formula, we can see that 98*(x+1) can be regarded as the total time of the first 98 data, and (x+n-98) is the data period of the last data. Therefore, the value of (x+n-98) must be greater than 0. Without considering the remainder n, the data period x>98.

[0096] Because data extraction or processing requires additional clock cycles, the last data cycle must not only be greater than 0 but also greater than the number of clock cycles required for data processing. Assuming the number of clock cycles required for data processing is m, then this method can be used only when x - 98 > m, or x > 98 + m. Let k represent the number of valid data points, 99. This method can be used for bandwidth matching when x > k + m - 1 to avoid noise.

[0097] Considering that the external clock frequency may be offset, the phenomenon is that the number of clock cycles counted by the previous and next frames is different. The situation of generating multiple data cycles is similar to the above and will not be described in detail. For example, when the clock frequency of the data sending end gradually increases, the notification of the next frame will arrive earlier, such as Figure 5 As shown, the data time of the previous frame is significantly longer than that of the next frame. In severe cases, this can result in missing data cycles when counting the next frame. Therefore, a margin must be added to the division result to reduce the average data cycle to prevent missing data cycles and data loss. Therefore, the margin value needs to be configurable. When the system detects that the data source clock is off, it increases the margin to reduce the average data cycle, thus preventing data loss. The above diagram is for illustration only. (In practice, due to the clock calibration module, the clock offset is not significant.) The most likely scenario is that the condition x-98>m fails, resulting in the loss of the last data cycle. Therefore, a margin of 1 can resolve this issue.

[0098] Step S3: Determine the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end.

[0099] After calculating the data period of the second audio data, the sampling frequency at which the data receiving end samples the second audio data under the local clock can be calculated based on the inverse relationship between the period and the frequency, thereby achieving bandwidth matching of the audio data for accurate extraction of the audio data.

[0100] Specifically, in the above step S3, in the process of determining the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end, the cycle length N of the local clock can be determined based on the local clock frequency f. Then, based on the calculated data cycle multiplied by the cycle length N of the local clock, the cycle length N' of the second audio data under the local clock of the data receiving end can be obtained. The reciprocal of the cycle length N' can be understood as the sampling frequency of the second audio data under the local clock of the data receiving end.

[0101] In other embodiments, after determining the sampling frequency under the local clock of the receiving end according to the data cycle, a data extraction step may also be included. The data extraction step may include the following processing: reading two consecutive first audio data in each data cycle; and extracting audio data based on the two consecutive first audio data to obtain second audio data.

[0102] In this embodiment, the data extraction principle is as follows: first, according to the data scrambling rule, a valid flag of the data byte (left and right channels) is generated and written into a fixed position in the RAM for storage.

[0103] Further, the process of reading the two continuous first audio data in each data cycle can include the following processing: detecting the byte validity identification of the two continuous first audio data in each data cycle; and reading the corresponding two continuous first audio data when the byte validity identification of the two continuous first audio data is valid.

[0104] In the audio transmission process, the scrambled audio data is written into the specified area of the RAM for buffering, and in the bandwidth matching stage, the byte validity identification of the two first audio data is read first to determine the validity of the data, and then the data is sent to the shift register for buffering. Then, the two corresponding audio data is read from the RAM audio buffer area and buffered using the 96-bit shift register. The validity of each byte of the audio data is detected by shifting the buffered data byte flag and the audio data, and the valid bytes are sent to the output buffer shift register in sequence. When the extraction of one data (48 bits) is completed, the data is sent out at the specified time in the matching data cycle, realizing data extraction and bandwidth matching.

[0105] Specifically, the step of extracting the audio data based on the two continuous first audio data to obtain the second audio data can include the following processing: analyzing the validity of each byte of the two continuous first audio data and the channel to which the byte belongs; and sending the byte data of the left channel and the byte data of the right channel to the corresponding channel, respectively.

[0106] In this embodiment, the validity of each byte of the audio data and the channel to which the byte belongs can be analyzed by simultaneously shifting the buffered data left and right channel flags and the audio data, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channel (2'b01: left channel; 2'b10: right channel) in sequence, realizing data extraction.

[0107] In other embodiments, a state machine can be used to perform the audio data processing process according to the following steps: in the idle state, detecting the bandwidth matching enable signal, and entering the valid flag reading state when the data cycle of the second audio data is determined; in the valid flag reading state, reading the byte validity identification of the two continuous first audio data, and entering the audio data reading state when the byte validity identification indicates that the byte is valid; in the audio data reading state, reading the first audio data corresponding to the byte validity identification of the two continuous audio data, and entering the data extraction state; in the data extraction state, analyzing the validity of each byte of the first audio data and the channel to which the byte belongs, and sending the byte data of the left channel and the byte data of the right channel to the corresponding channel in sequence, respectively.

[0108] In this embodiment, the validity of each byte of the audio data and the channel to which the byte belongs can be analyzed by simultaneously shifting the buffered data left and right channel flags and the audio data, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channel (2'b01: left channel; 2'b10: right channel) in sequence, realizing data extraction. Figure 6As shown, the audio extraction process is implemented through the following four state machines. Specifically, the IDLE state (idle state) is the reset state and the state machine holds the state after audio data recovery is completed. In this state, after the bandwidth match enable signal is detected and the clearing condition of the data synchronization cycle counter is met, the audio byte validity flag (DOWN_RD_FLAG) state is entered.

[0109] DOWN_RD_FLAG state (valid flag read state): This state reads the left and right channel flags stored in the data RAM and stores them in 24-bit registers for use during the data recovery phase. If the data read in this state is 0, it indicates that the corresponding audio data has been recovered and the data channel flag for the next address is required. In this state, if the state machine detects that the left and right channel flags are valid, it enters the audio data read (DOWN_RD_DATA) state.

[0110] DOWN_RD_DATA state (read audio data state): This state reads the addresses of two audio data stored in RAM and stores the data in a 96-bit register. This state is used in the data recovery state. In this state, the state machine detects that the two data have been read and enters the data recovery (DATA_RECOVER) state.

[0111] DATA_RECOVER state (data extraction state): This state extracts data by simultaneously shifting the left and right channel flags and audio data in the buffered data, analyzing the validity of each audio data byte and the channel to which it belongs. The left and right channel byte data are then sequentially sent to their corresponding channels (2'b01: left channel; 2'b10: right channel). When data extraction is complete, the state machine re-enters the IDLE state.

[0112] In a specific embodiment, the Figure 7 and Figure 8 The bandwidth matching process and the data extraction process are performed separately, thereby realizing the audio data extraction process provided by this embodiment. Specifically, Figure 7As shown in the figure, the bandwidth matching process includes the following steps: ① Audio data is written to and read from the RAM buffer simultaneously. When the audio buffer contains 80 data points, the bandwidth matching enable is enabled and bandwidth matching begins. ② Bandwidth matching: The bandwidth matching counter cnt0 counts the duration of the data frame for bandwidth matching. When the next frame notification flag is detected, it indicates that the previous frame of data has ended and the next frame of data has begun. At this point, cnt0 has completed the time counting of the previous frame of data. The cnt0 value is fed into the divider and divided by the actual number of audio data samples, 99, to obtain the number of audio data cycles based on the local clock. ③ During the first frame of data transmission, since the bandwidth matching counter cnt0 has not yet completed the data frame counting, the data cycle counter cnt1 calculates the data cycle based on a default value calculated from the preset audio sampling frequency and the local clock frequency. After the first frame of data transmission is completed, the data cycle counter uses the quotient of the divider to calculate the data cycle.

[0113] like Figure 8 As shown, the data extraction process includes the following steps: During audio transmission, the scrambled audio data is written to a designated area in RAM for caching. Meanwhile, during the bandwidth matching phase, the byte validity flags of two audio data points are read within each matching data cycle to determine if the data is valid and then sent to the shift register for caching. (If the read data is 0, indicating that the corresponding audio data has been recovered, the byte validity flag of the next address is required.) The two corresponding audio data points are then read from the RAM audio buffer and cached using a 96-bit shift register. The cached data byte validity flags and audio data are shifted and analyzed to determine the validity of each audio data byte and its corresponding channel. The left and right channel byte data are then sequentially sent to their corresponding channels (2'b01: left channel; 2'b10: right channel), completing data extraction. After a data point (48 bits) is extracted, it is sent out at a specified time within the bandwidth matching data cycle, completing data extraction and bandwidth matching.

[0114] In summary, the audio data processing method provided by this embodiment can first obtain the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame; determine the data period of the second audio data based on the current frame sampling time and the number of valid bytes; and determine the sampling frequency of the second audio data under the local clock of the data receiving end according to the data period and the local clock frequency of the data receiving end. In this way, the first audio data whose order is disrupted can be accurately bandwidth matched to obtain the actual sampling frequency of the data under the local clock of the receiving end, and the actual sampling frequency can be matched to send the audio data, that is, the second audio data can be transmitted according to the calculated actual sampling frequency. In the bandwidth matching stage, after the audio data is extracted in each matched data period, the audio data can be sent according to the actual sampling frequency of the audio data to ensure that the downstream data receiving end can directly process the audio.

[0115] Based on the same concept as the above audio data processing method, this embodiment also provides an audio data processing method, as shown in the attached Figure 9 As shown, the audio data processing device includes:

[0116] a data acquisition module, configured to acquire a current frame sampling time of first audio data and the number of valid bytes contained in the first audio data of the current frame; the first audio data refers to audio data whose data transmission order is periodically disrupted;

[0117] A data period determination module, configured to determine a data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes;

[0118] The frequency determination module is used to determine the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end.

[0119] The audio data processing device provided in this embodiment is based on the same concept as the above-mentioned audio data processing method, so it can at least achieve the above-mentioned beneficial effects. Any of the above-mentioned implementation methods can be applied to the audio data processing method provided in this embodiment, and will not be repeated here.

[0120] like Figure 10 As shown, this embodiment may further include an audio cache module, which writes the received scrambled audio data directly to the audio cache area in the RAM, counts the number of audio data, and generates a flag pulse signal to notify the frequency determination module every time 100 audio data are counted; the frequency determination module counts the time between two notification flags and uses a divider to calculate the data period of the actual audio, thereby achieving bandwidth matching.

[0121] Based on the same concept of the audio data processing method, the embodiment further provides a chip integrated with the audio data processing method.

[0122] The chip provided by the embodiment is based on the same concept of the audio data processing method, and thus can at least achieve the beneficial effects of the audio data processing method, and any implementation manner of the audio data processing method can be applied to the chip, which will not be described herein.

[0123] The embodiment further provides an electronic device corresponding to the audio data processing method provided by the foregoing embodiments, to execute the audio data processing method.

[0124] Please refer to Figure 11 , which shows a hardware structure diagram of the electronic device provided by some embodiments of the present application. As Figure 11 shown, the electronic device 6 includes a processor 600, a memory 601, a bus 602 and a communication interface 603, wherein the processor 600, the communication interface 603 and the memory 601 are connected through the bus 602; the memory 601 stores a computer program executable on the processor 600, and the processor 600 executes the computer program to perform the audio data processing method provided by any of the foregoing embodiments of the present application.

[0125] The memory 601 can be any electronic, magnetic, optical or other physical storage device, and can contain stored information such as executable instructions, data, etc. Specifically, the memory 601 can be a RAM (Random Access Memory), a flash memory, a storage drive (such as a hard disk drive), or a similar storage medium, or a combination thereof. The communication between the device network element and at least one other network element is realized through at least one communication interface 603 (which can be wired or wireless), and the Internet, a wide area network, a local network, a metropolitan area network, etc. can be used.

[0126] The bus 602 can include an I2C (Inter-integrated Circuit) bus, an MIPI (Mobile industry Processor Interface) bus, an SPI (Serial Peripheral interface) bus, an I2S (integrated interchip Sound) bus, etc. It can also be divided into address bus, data bus, control bus, etc. according to function.

[0127] The processor 600 may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by hardware integrated logic circuits in the processor 600 or by software instructions. The above processor 600 may be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the various methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor may be a microprocessor or any conventional processor. The steps of the method disclosed in conjunction with the embodiments of this application can be directly implemented and executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. The storage medium is located in the memory 601 , and the processor 600 reads the information in the memory 601 and completes the steps of the above method in combination with its hardware.

[0128] The electronic device provided in the embodiment of the present application and the audio data processing method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, operated or implemented by them.

[0129] An embodiment of the present application also provides a computer-readable storage medium corresponding to the audio data processing method provided in the aforementioned embodiment, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the audio data processing method provided in any of the aforementioned embodiments.

[0130] It should be noted that computer-readable storage media may include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0131] An embodiment of the present application further provides a computer program product, including a computer program, which is executed by a processor to implement the audio data processing method of any of the above embodiments.

[0132] The computer-readable storage medium and computer program product provided in the above-mentioned embodiments of the present application are based on the same inventive concept as the audio data processing method provided in the embodiments of the present application, and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application, and they should all be included in the scope of the claims and specification of the present application. In particular, as long as there is no structural conflict, the various technical features mentioned in the various embodiments can be combined in any way. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions that fall within the scope of the claims.

Claims

1. A method for processing audio data, characterized in that: For a data receiving end, the method includes: Obtaining a current frame sampling time of first audio data and the number of valid bytes contained in the first audio data of the current frame; the first audio data refers to audio data whose data transmission order is periodically disrupted; Determining a data period of second audio data of a current frame based on the current frame sampling time and the number of valid bytes; comprising: if sampling of the current frame audio data is completed within the current frame sampling time, determining the data period of the second audio data of the current frame based on a quotient of the current frame sampling time and the number of valid bytes; if sampling of the current frame audio data is not completed within the current frame sampling time, determining a preset period default value as the data period of the second audio data; wherein the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission order; A sampling frequency of the second audio data under the local clock of the data receiving end is determined according to the data period and the local clock frequency of the data receiving end.

2. The method according to claim 1, wherein The obtaining of the current frame sampling time of the first audio data includes: Detecting that sampling of the first audio data of the previous frame is completed; The time from the completion of sampling of the first audio data of the previous frame to the completion of sampling of the first audio data of the current frame is counted and determined as the current frame sampling time.

3. The method according to claim 2, wherein: The method further comprises: When a preset number of the first audio data are collected, a flag pulse is generated; the flag pulse is used to indicate that a frame of the first audio data is collected; The identification pulse is detected to determine that sampling of the first audio data of the current frame is completed.

4. The method according to claim 1, wherein The determining the data period of the second audio data of the current frame based on the quotient of the current frame sampling time and the number of valid bytes includes: When the current frame sampling time is within a preset threshold range, using a divider to calculate a quotient of the current frame sampling time and the number of valid bytes; A data period of the second audio data of the current frame is determined based on the quotient and a preset margin.

5. The method according to claim 1, wherein Before obtaining the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame, the method further includes: The received first audio data is written into an audio buffer area of ​​a memory, and the number of the collected first audio data is counted.

6. The method according to claim 1, wherein After determining the sampling frequency of the second audio data sampled by the data receiving end under the local clock according to the data cycle, the method further includes: In each of the data cycles, reading two consecutive first audio data; Audio data extraction is performed based on the two consecutive first audio data to obtain second audio data; the second audio data refers to audio data with normal data transmission order.

7. The method according to claim 6, wherein The step of reading two consecutive first audio data in each data cycle includes: In each of the data cycles, detecting two consecutive byte validity flags of the first audio data; In a case where the byte validity identifier indicates that the two consecutive first audio data are valid, the two consecutive first audio data are read.

8. The method according to claim 6, wherein: The extracting audio data based on the two consecutive first audio data to obtain second audio data includes: Analyzing the validity of each byte in the two consecutive first audio data and the channel to which it belongs; The byte data of the left channel and the byte data of the right channel are sent to their corresponding channels respectively.

9. The method according to claim 6, wherein Using a state machine, the audio data processing process is performed according to the following steps: In an idle state, a bandwidth matching enable signal is detected, and when a data period of the second audio data is determined, a valid flag reading state is entered; In the valid flag reading state, reading the byte validity flags of two consecutive first audio data, and entering the audio data reading state when it is detected that the byte validity flag indicates that the byte data is valid; otherwise, reading the byte validity flags of the next two consecutive audio data, and determining the validity of the next two consecutive audio data; In the audio data reading state, reading two consecutive first audio data corresponding to the data byte validity flag, and entering the data extraction state; In the data extraction state, the validity of each byte in the first audio data and the channel to which it belongs are analyzed according to the byte validity identifiers of the two consecutive audio data, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channels in sequence.

10. An audio data processing device, characterized in that: Applied to a data receiving end, the device comprises: a data acquisition module, configured to acquire a current frame sampling time of first audio data and the number of valid bytes contained in the first audio data of the current frame; the first audio data refers to audio data whose data transmission order is periodically disrupted; a data period determination module, configured to determine a data period of second audio data of a current frame based on the current frame sampling time and the number of valid bytes; the module comprising: determining the data period of the second audio data of the current frame based on a quotient of the current frame sampling time and the number of valid bytes when sampling of the current frame audio data is completed within the current frame sampling time; and determining a preset period default value as the data period of the second audio data when sampling of the current frame audio data is not completed within the current frame sampling time; wherein the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission order; The frequency determination module is used to determine the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end.

11. An electronic device, characterized in that: The system comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method according to any one of claims 1 to 9.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 9.

13. A computer program product comprising a computer program, characterized in that The computer program is executed by a processor to implement the method according to any one of claims 1 to 9.

Images