Audio data processing method and device, electronic equipment and program product

Abstract

The invention provides an audio data processing method and device, electronic equipment and a program product, and is used for a data receiving end, and the method comprises the steps: obtaining the sampling time of a current frame of first audio data, and the number of effective bytes contained in the first audio data of the current frame; the first audio data refers to audio data of which the data transmission sequence is periodically disorganized; determining a data period of second audio data of the current frame based on the sampling time of the current frame and the number of effective bytes; the second audio data are audio data corresponding to the first audio data and having a normal data transmission sequence; and determining the sampling frequency of the second audio data under the local clock of the data receiving end according to the data period and the local clock frequency of the data receiving end. According to the method, the bandwidth matching of the audio data can be quickly and accurately realized, so that the audio data can be correctly extracted.

Description

Technical Field

[0001] The present application belongs to the technical field of audio data processing, and specifically relates to an audio data processing method, device, electronic equipment and program product. Background Art

[0002] At present, when processing audio data, not only the original audio data is needed, but also the transmission frequency of the audio data is needed, and the transmission frequency must be consistent with the sampling frequency of the audio data, otherwise the audio cannot be accurately processed. For example, spectrum analysis, sampling rate conversion, echo cancellation, etc. all require the sampling frequency of the audio data to accurately analyze and process the audio data. For audio data with special structures, such as audio data being periodically disrupted, the transmission frequency being inconsistent with the actual sampling frequency of the data, etc., it is impossible to use conventional audio processing methods to directly process the data.

[0003] It should be noted that the above statements are only used to provide background technical information related to the present application and do not necessarily constitute prior art. Summary of the invention

[0004] The present application proposes an audio data processing method, device, electronic device and program product, which can quickly and accurately achieve bandwidth matching of audio data so as to correctly extract audio data.

[0005] The first aspect of the present application provides an audio data processing method for a data receiving end, the method comprising:

[0006] Acquire a current frame sampling time of first audio data and the number of valid bytes contained in the first audio data of the current frame; the first audio data refers to audio data whose data transmission order is periodically disrupted;

[0007] Determine a data period of second audio data of the current frame based on the current frame sampling time and the number of valid bytes; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission sequence;

[0008] According to the data cycle and the local clock frequency of the data receiving end, a sampling frequency of the second audio data under the local clock of the data receiving end is determined.

[0009] In some optional embodiments, obtaining the current frame sampling time of the first audio data includes:

[0010] Detecting that sampling of the first audio data of the previous frame is completed;

[0011] The time from when the first audio data of the previous frame is sampled to when the first audio data of the current frame is sampled is counted and determined as the current frame sampling time.

[0012] In some optional embodiments, the method further includes:

[0013] When a preset number of the first audio data are collected, a marking pulse is generated; the marking pulse is used to indicate that a frame of the first audio data is collected;

[0014] The identification pulse is detected to determine that the sampling of the first audio data of the current frame is completed.

[0015] In some optional embodiments, determining the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes includes:

[0016] When sampling of the audio data of the current frame is completed within the current frame sampling time, determining a data period of the second audio data of the current frame based on a quotient of the current frame sampling time and the number of valid bytes;

[0017] When the current frame audio data is not sampled within the current frame sampling time, the preset period default value is determined as the data period of the second audio data of the current frame.

[0018] In some optional embodiments, determining the data period of the second audio data of the current frame based on the quotient of the current frame sampling time and the number of valid bytes includes:

[0019] When the current frame sampling time is within a preset threshold range, using a divider to calculate a quotient of the current frame sampling time and the number of valid bytes;

[0020] A data period of the second audio data of the current frame is determined based on the quotient value and a preset margin.

[0021] In some optional embodiments, before obtaining the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame, the method further includes:

[0022] The received first audio data is written into an audio buffer area of ​​a memory, and the number of the collected first audio data is counted.

[0023] In some optional embodiments, after determining the sampling frequency of the receiving end under the local clock according to the data period, the method further includes:

[0024] In each of the data cycles, reading two consecutive first audio data;

[0025] Based on the two consecutive first audio data, audio data is extracted to obtain second audio data; the second audio data refers to audio data with normal data transmission sequence.

[0026] In some optional embodiments, reading two consecutive first audio data in each data cycle includes:

[0027] In each of the data cycles, detecting two consecutive byte validity flags of the first audio data;

[0028] In a case where the byte validity identifier indicates that the two consecutive first audio data are valid, the two consecutive first audio data are read.

[0029] In some optional embodiments, the extracting audio data based on the two consecutive first audio data to obtain the second audio data includes:

[0030] Analyzing the validity of each byte in the two consecutive first audio data and the channel to which it belongs;

[0031] The byte data of the left channel and the byte data of the right channel are sent to their corresponding channels respectively.

[0032] In some optional embodiments, a state machine is used to perform the audio data processing process according to the following steps:

[0033] In an idle state, a bandwidth matching enable signal is detected, and when a data period of the second audio data is determined, a valid flag reading state is entered;

[0034] In the valid flag reading state, read the byte validity marks of two consecutive data, and enter the audio data reading state when it is detected that the byte validity mark indicates that the byte data is valid; otherwise, read the byte validity marks of the next two consecutive audio data, and determine the validity of the next two consecutive audio data;

[0035] In the audio data reading state, reading two consecutive first audio data corresponding to the byte validity identifier, and entering the data extraction state;

[0036] In the data extraction state, the validity of each byte in the first audio data and the channel to which it belongs are analyzed according to the byte validity identifiers of the two consecutive audio data, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channels in sequence.

[0037] An embodiment of a second aspect of the present application provides an audio data processing device, applied to a data receiving end, the device comprising:

[0038] A data acquisition module, used to acquire a current frame sampling time of first audio data and the number of valid bytes contained in the first audio data of the current frame; the first audio data refers to audio data whose data transmission order is periodically disrupted;

[0039] a data cycle determination module, configured to determine a data cycle of second audio data of a current frame based on the current frame sampling time and the number of valid bytes; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission sequence;

[0040] The frequency determination module is used to determine the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end.

[0041] An embodiment of the third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.

[0042] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to implement the method described in the first aspect.

[0043] An embodiment of the fifth aspect of the present application provides a computer program product, including a computer program, wherein the computer program is executed by a processor to implement the method described in the first aspect.

[0044] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages:

[0045] The audio data processing method provided in the embodiment of the present application first obtains the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame; determines the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes; and determines the sampling frequency of the data under the local clock of the receiving end according to the data period. In this way, the data period of each frame of audio data can be obtained, so that the bandwidth of the first audio data whose order is disrupted can be matched frame by frame to obtain a more accurate actual sampling frequency of the second audio data under the local clock of the receiving end, and matches the actual sampling frequency to send the audio data to ensure that the downstream data receiving end can directly process the audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.

[0047] In the attached picture:

[0048] Figure 1 A schematic diagram showing a flow chart of an audio data processing method provided by an embodiment of the present application;

[0049] Figure 2 A data flow diagram of an audio data processing method provided by an embodiment of the present application is shown;

[0050] Figure 3 A schematic diagram of a data transmission process in which noise is generated during audio data transmission is shown;

[0051] Figure 4 A schematic diagram of a data transmission process for avoiding noise generation during audio data transmission in an embodiment of the present application is shown;

[0052] Figure 5 A schematic diagram of a data transmission process in which a notification of the next frame will arrive in advance during the audio data transmission process is shown;

[0053] Figure 6 A schematic diagram of a working process of extracting audio data by a state machine provided in an embodiment of the present application is shown;

[0054] Figure 7 A schematic diagram showing a specific flow chart of a bandwidth matching process in an audio data processing method provided in an embodiment of the present application is shown;

[0055] Figure 8 A schematic diagram showing a specific flow of an audio extraction process in an audio data processing method provided in an embodiment of the present application is shown;

[0056] Fig. 9 A schematic diagram showing the structure of an audio data processing device in another embodiment of the present application is shown;

[0057] Fig.10 A schematic diagram showing the structure of an audio data processing device in another embodiment of the present application is shown;

[0058] Fig.11 A schematic diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0059] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0060] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

[0061] In practical applications, data validity directly affects whether the data can be used and whether the audio can be played; whether the bandwidth is consistent will affect the quality of the audio, and bandwidth mismatch will cause audio noise. For special structured audio where dual-channel data is periodically disrupted and the transmission frequency is inconsistent with the actual sampling frequency of the data, conventional audio processing methods cannot directly process data according to the data transmission frequency. Therefore, the audio processing end not only needs to extract the original audio and match the bandwidth, but also needs to ensure the validity of the extracted data while avoiding noise during bandwidth matching (both more and less data will generate noise), otherwise it will affect the sound quality of the audio output by the audio device.

[0062] In order to solve the above problems, this embodiment studies and analyzes the data transmitted by the relevant fast charging protocol, and finds that in some electronic products that need to transmit audio data, such as but not limited to wired headset products, Bluetooth headset products, and other audio devices, the transmission protocols of different products are not the same, so the data transmission structure may also be different. For example, in a certain communication protocol, the audio transmission rate is fixed, but the transmitted audio sampling rate is not fixed. When the audio bus transmits 100 data, it actually only contains 99 594 bytes of shuffled audio data. The data receiving end needs to correctly extract the original audio data and send it out at the audio sampling rate to ensure that the downstream data receiving end can directly process the audio.

[0063] Based on the above findings, the embodiments of the present application provide an audio data processing method, device, electronic device and program product, which is used for the data receiving end, and can first obtain the current frame sampling time of the first audio data, and the number of valid bytes contained in the first audio data of the current frame; determine the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes of the current frame; determine the sampling frequency of the data under the local clock of the receiving end according to the data period. In this way, the data period of each frame of audio data can be obtained, so that the first audio data whose order is disrupted can be matched frame by frame with bandwidth, so as to obtain a more accurate actual sampling frequency of the data receiving end under the local clock, and match the actual sampling frequency to send audio data, so as to ensure that the downstream data receiving end can directly process the audio.

[0064] The audio data processing device is a processing module on an audio device for executing the audio data processing method, and may specifically be a processor or a specially configured microprocessor, which is not specifically limited in this embodiment.

[0065] The electronic device may be an audio device to which the above-mentioned audio data processing method is applied, such as but not limited to headphones, mobile phones, walkmans, etc., or may be an integrated chip on the audio device for performing audio data processing.

[0066] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0067] Please refer to Figure 1 , is a flow chart of the audio data processing method provided in the embodiment of the present application, as shown in the attached Figure 1 As shown, the audio data processing method includes.

[0068] Step S1, obtaining a current frame sampling time of first audio data and the number of valid bytes contained in the first audio data of the current frame.

[0069] The first audio data refers to the audio data whose data transmission order is periodically disrupted. It is understandable that even if the order of the audio data is disrupted, when the audio data is transmitted by bytes after being disrupted, the worst case will be completed in two audio data packets (12 bytes), that is, two disrupted first audios.

[0070] The first audio data of the current frame can be understood as the data packet of the first audio data that has been received and just received. For example, if a pulse signal is generated after a data packet is received, indicating that the frame of audio data has been received, then the current frame in this embodiment is the audio frame data corresponding to the latest generated pulse signal. It can contain multiple first audio data, for example, it can contain 100 audio data. The current frame sampling time can be understood as the sampling time of the first audio data of the current frame, which can be the sampling time of all the first audio data contained in the current frame, or it can be the sampling time of part of the first audio data contained in the current frame.

[0071] Specifically, the number of valid bytes of the first audio data contained in the first audio data of the current frame may be preset, or the byte validity may be judged according to the validity flag of the data during the audio data receiving process, and the number of valid bytes may be counted, which is not specifically limited in this embodiment. In addition, when the number of valid bytes is preset, the number of valid bytes of the audio data of each frame may be the same. For example, when the first audio data of the current frame contains 100 audio data, the corresponding number of valid bytes may be 99.

[0072] In this embodiment, before executing the above step S1, the received first audio data may be written into the audio buffer area of ​​the memory, and the number of the collected first audio data may be counted.

[0073] Specifically, this embodiment can be implemented according to Figure 2 In the data stream processing process shown, the received disrupted first audio data can be directly written into the audio buffer area in the memory (RAM), and the number of the collected first audio data can be counted.

[0074] Furthermore, the first audio data can be written into the RAM cache while being read out from the cache. When the first audio cache has 80 data, the bandwidth matching is enabled and bandwidth matching is started. It can be understood that 80 here is only one implementation of this embodiment, and this embodiment is not limited thereto, and can be a value greater than 50 and less than 100, for example, it can also be 60, 70, 85, 88, etc.

[0075] Specifically, it can be specifically set according to the size of the audio buffer in the local memory, because the data size of the audio is uncertain, and if a larger space is left, resources will be wasted. Since the played audio data will no longer require storage space, it can be designed to reserve a section of storage space for cyclic writing to realize audio data caching. For example, 100 addresses are left in the memory as the audio buffer, and after the audio is stored in the 100th address, the next time it will restart caching from address 1. Because only one first audio data will be stored in each data cycle, and two audio data need to be read in one data cycle, so in theory, as long as more than 3 and less than 98 first audios are stored, it is possible to avoid reading the played audio data.

[0076] In some embodiments, the above process of obtaining the current frame sampling time of the first audio data may include the following processing: detecting that the sampling of the first audio data of the previous frame is completed; counting and statistically analyzing the time from the completion of the sampling of the first audio data of the previous frame to the completion of the sampling of the first audio data of the current frame, and determining it as the current frame sampling time.

[0077] In this embodiment, in view of the continuous data transmission, the time between the two moments when the two data are received can be determined as the sampling time of the next frame of data. In this way, the intermediate judgment process can be omitted, and the sampling time of the next frame of data, that is, the first audio data of the current frame, can be determined by only recording the count values ​​of the two reception completion moments.

[0078] Furthermore, this embodiment can generate a flag pulse when a preset number of first audio data are collected; the flag pulse is used to indicate that a frame of first audio data is collected; and when the flag pulse is detected, it is determined that the sampling of the first audio data of the current frame is completed.

[0079] Specifically, a flag pulse signal may be generated every time 100 audio data are counted, indicating that a frame of audio data has been received, so as to start the bandwidth matching process. It can be understood that the 100 audio data are only one implementation of this embodiment, and this embodiment does not specifically limit the number, which can be specifically set according to the data structure of the first audio data, and is consistent with the number of byte data contained in a frame of data in the first audio data.

[0080] More specifically, the bandwidth matching counter cnt0 can be used to count the sampling time of a frame of the first audio data frame for bandwidth matching. When the next frame notification flag is detected, it indicates that the previous frame data has ended and the next frame data has begun. At this time, cnt0 has completed the time statistics of the previous frame data.

[0081] Step S2, determining the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes.

[0082] The data cycle can be understood as the sampling cycle of the ADC (analog-to-digital converter) of the data receiving end when receiving the audio data. The second audio data refers to the audio data corresponding to the first audio data and having a normal data transmission order, and the corresponding to the first audio data here means that the valid data bytes contained in the two are the same.

[0083] In this embodiment, since the first audio data may contain invalid audio data, the number of actually valid bytes in the first audio data of the current frame is less than the total number of bytes in the first audio data of the current frame. The second audio data can be understood as data containing only valid audio bytes, and the time of the first audio is the same as the time of the second audio, so the data period of the second audio data can be directly calculated according to the current frame sampling time and the number of valid bytes.

[0084] It can be understood that each audio data is originally multi-byte, that is, one audio data may include multiple valid bytes, so when calculating the data period, the audio sampling depth (or audio data bit width) also needs to be considered.

[0085] In some embodiments, the above-mentioned step S2 may specifically include the following processing: when the current frame sampling time includes the sampling time of all audio data of the current frame, the data period of the second audio data of the current frame is determined based on the quotient of the current frame sampling time and the number of valid bytes; when the current frame sampling time includes the sampling time of part of the audio data of the current frame, the preset period default value is determined as the data period of the second audio data of the current frame.

[0086] In this embodiment, the basic principle of bandwidth matching is that the local clock continuously counts the sampling time of the current frame audio data, such as the time to transmit 100 audio data, and then uses a divider to calculate the actual bandwidth of the audio data corresponding to the number of valid bytes (for example, 99), so as to achieve accurate frame-by-frame matching. However, during the transmission of the first frame of data, since the bandwidth matching counter cnt0 has not yet completed the statistics of the data frame, the data cycle counter cnt1 needs to calculate the data cycle based on the default value calculated based on the actual audio sampling frequency combined with the local clock frequency. After the first frame of data transmission is completed, the data cycle counter uses the quotient of the divider to calculate the data cycle.

[0087] Further, the data period of the second audio data of the current frame is determined based on the quotient of the current frame sampling time and the number of valid bytes of the current frame, including: when the current frame sampling time is within a preset threshold range, using a divider to calculate the quotient of the current frame sampling time and the number of valid bytes; and determining the data period of the second audio data of the current frame based on the quotient and a preset margin.

[0088] Specifically, the value of cnt0 can be sent to a divider and divided by the actual sampling number of the audio data, 99, to obtain the number of cycles of the audio data under the local clock.

[0089] Consider not using the remainder of the divider, such as Figure 3 In the figure, when the cnt0 statistical value is 10000 during a packet of data, if the divider quotient 98 is used as the matching data cycle, 100 data cycles will be generated, but only 99 valid data will be generated. Therefore, the extra data cycle (the shaded part in the figure) will generate 1 noise point, forming noise.

[0090] To avoid this situation, the divider quotient + 1 is used to calculate the data cycle. This method theoretically ensures that only 99 data will be generated when the next frame notification arrives. Figure 4 As shown in the figure, the number of clock cycles of the last data under the local clock is 1 clock cycle less than the number of cycles of the first 98 data. Therefore, it can be seen that the higher the operating frequency of the local clock, the smaller the cycle error of the last number, and the higher the accuracy of bandwidth matching.

[0091] For audio data of this structure, this method cannot be used in all cases. Let the time for the bus to transmit 100 audio data be y, the quotient of the divider (i.e., data cycle) be x, and the remainder be n, then:

[0092] y=99*x+n

[0093] =98*(x+1)+(x+n-98)

[0094] Wherein, x and n are both integers greater than 0.

[0095] From the above formula, we can see that 98*(x+1) can be regarded as the total time of the first 98 data, and (x+n-98) is the data period of the last data. Therefore, the value of (x+n-98) must be greater than 0. Without considering the remainder n, the data period x>98.

[0096] Since data extraction or processing requires additional clocks, the last data cycle must not only be greater than 0, but also greater than the number of data clock cycles required for data processing. If the number of clock cycles required for data processing is m, then this method can be used only when x-98>m, that is, x>98+m. If the number of valid data 99 is represented by k, then this method can be used for bandwidth matching when x>k+m-1 to avoid noise.

[0097] Considering that the external clock frequency may be offset, the phenomenon is that the number of clock cycles counted by the previous and next frames is different. The situation of generating multiple data cycles is similar to the above, so I will not repeat it. For example, when the clock frequency of the data sending end gradually increases, the notification of the next frame will arrive in advance, such as Figure 5 As shown in the figure, the statistical time of the previous frame data is larger than that of the next frame. In serious cases, it will lead to less data cycle when the next frame data is counted. Therefore, it is necessary to reduce a certain margin based on the division result to reduce the average cycle of the data to ensure that there is no less data cycle and data loss. Therefore, it is necessary to design the margin value to be configurable. When the system detects that the data source clock is too large, the margin value is increased to reduce the average data cycle, thereby ensuring that data is not lost. The above figure is only for illustration (in fact, because there is a clock calibration module), the clock offset will not be very large. The most likely thing is that the x-98>m condition does not hold, resulting in the loss of the last data. Therefore, this problem can be solved when the margin is 1.

[0098] Step S3, determining the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end.

[0099] After calculating the data period of the second audio data, the sampling frequency at which the data receiving end samples the second audio data under the local clock can be calculated based on the inverse relationship between the period and the frequency, thereby achieving bandwidth matching of the audio data for accurate extraction of the audio data.

[0100] Specifically, in the above step S3, in the process of determining the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end, the cycle length N of the local clock can be determined based on the local clock frequency f, and then based on the calculated data cycle multiplied by the cycle length N of the local clock, the cycle length N' of the second audio data under the local clock of the data receiving end can be obtained, and the inverse of the cycle length N' can be understood as the sampling frequency of the second audio data under the local clock of the data receiving end.

[0101] In other embodiments, after determining the sampling frequency under the local clock of the receiving end according to the data cycle, a data extraction step may also be included, and the data extraction step may include the following processing: reading two consecutive first audio data in each data cycle; extracting audio data based on the two consecutive first audio data to obtain second audio data.

[0102] In this embodiment, the data extraction principle is: first, according to the data scrambling rule, a data byte (left and right channels) valid flag is generated and written to a fixed position in the RAM for storage.

[0103] Furthermore, the above process of reading two consecutive first audio data in each data cycle may include the following processing: in each data cycle, detecting the byte validity identifiers of two consecutive first audio data; when the two consecutive first audio byte validity identifiers are valid, reading the corresponding two consecutive first audio data.

[0104] During the audio transmission process, the disrupted audio data is written to the designated area in the RAM for caching. At the same time, in the bandwidth matching stage, the byte validity flags of the two first audio data are read in each matching data cycle to determine that the data is valid and then sent to the shift register for caching. Then, the two corresponding audio data are read from the RAM audio buffer area and cached using the 96-bit shift register. The validity of each byte of the audio data is detected by shifting the cached data byte flag and audio data, and the valid bytes are sent to the output buffer shift register in sequence. When a data (48 bits) is extracted, the data is sent out at the specified time in the matching data cycle to achieve data extraction and bandwidth matching.

[0105] Specifically, the above-mentioned step of extracting audio data based on two consecutive first audio data to obtain the second audio data may include the following processing: analyzing the validity of each byte in the two consecutive first audio data and the channel to which it belongs; sending the byte data of the left channel and the byte data of the right channel to their corresponding channels respectively.

[0106] In this embodiment, the left and right channel flags of the cached data and the audio data can be shifted simultaneously to analyze the validity of each byte of the audio data and the channel to which it belongs, and the byte data of the left and right channels can be sent to their corresponding channels (2'b01: left channel; 2'b10: right channel) in sequence to achieve data extraction.

[0107] In other embodiments, a state machine may be used to execute the audio data processing process according to the following steps: in an idle state, a bandwidth matching enable signal is detected, and when the data period of the second audio data is determined, the valid flag reading state is entered; in the valid flag reading state, two consecutive byte validity identifiers of the first audio data are read, and when the byte validity identifier is detected to indicate that the byte is valid, the audio data reading state is entered; in the audio data reading state, the first audio data corresponding to the byte validity identifiers of two consecutive audio data are read, and the data extraction state is entered; in the data extraction state, the validity of each byte in the first audio data and the channel to which it belongs are analyzed, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channels in sequence.

[0108] In this embodiment, the Figure 6As shown, the audio extraction process is implemented through the following four state machines. Specifically, IDLE state: the state machine holds the state after the reset state and audio data recovery are completed. In this state, after the bandwidth matching enable signal is detected, the read audio byte validity flag (DOWN_RD_FLAG) state is entered according to the clearing condition of the data synchronization cycle counter when it is met.

[0109] DOWN_RD_FLAG state (valid flag read state): This state reads the left and right channel flags of the data stored in the data ram, and uses a 24-bit register to store and use in the data recovery phase. In this state, if the read data is detected to be 0, it indicates that the current corresponding audio data has been recovered, and the data channel flag of the next address is required. In this state, the state machine enters the audio data reading (DOWN_RD_DATA) state when it detects that the left and right channel flags of the read data are valid.

[0110] DOWN_RD_DATA state (read audio data state): This state reads the address of the two audio data stored in RAM, and uses a 96-bit register to store the data, and is used in the data recovery state. In this state, the state machine detects that the reading of the two data is completed and enters the data recovery (DATA_RECOVER) state.

[0111] DATA_RECOVER state (data extraction state): This state simultaneously shifts the left and right channel flags and audio data of the cached data, analyzes the validity of each byte of the audio data and the channel to which it belongs, and sends the byte data of the left and right channels to their corresponding channels (2'b01: left channel; 2'b10: right channel) in sequence to achieve data extraction. When data extraction is completed, the state machine re-enters the IDLE state.

[0112] In a specific embodiment, the Figure 7 and Figure 8 The bandwidth matching process and the data extraction process are performed respectively, thereby realizing the audio data extraction process provided by this embodiment. Specifically, Figure 7As shown in the figure, the bandwidth matching process includes the following processes: ① The audio data is written into the RAM cache while being read out from the cache. When the audio cache has 80 data, the bandwidth matching enable is turned on and the bandwidth matching begins. ② Bandwidth matching, the bandwidth matching counter cnt0 counts the time of the data frame for bandwidth matching. When the next frame notification flag is detected, it indicates that the previous frame of data has ended and the next frame of data has begun. At this time, cnt0 has completed the time statistics of the previous frame of data. The value of cnt0 is sent to the divider and divided by the actual number of samples of the audio data, 99, to obtain the number of cycles of the audio data under the local clock. ③ During the transmission of the first frame of data, since the bandwidth matching counter cnt0 has not completed the statistics of the data frame, the data cycle counter cnt1 needs to calculate the data cycle based on the default value calculated by the preset sampling frequency of the audio combined with the local clock frequency. After the first frame of data transmission is completed, the data cycle counter uses the quotient of the divider to calculate the data cycle.

[0113] like Figure 8 As shown, the data extraction process includes the following process: during the audio transmission process, the disrupted audio data is written into the designated area in the RAM for caching, and at the bandwidth matching stage, the byte validity identifiers of the two audio data are first read in each matching data cycle to determine whether the data is valid and then sent to the shift register for caching (if the read data is detected to be 0, indicating that the current corresponding audio data has been restored, the byte validity identifier of the next address is required), and then 2 corresponding audio data are read from the RAM audio buffer area and cached using a 96-bit shift register, and the cached data byte validity identifier and audio data are shifted to analyze the validity of each byte of the audio data and the channel to which it belongs, and the byte data of the left and right channels are sequentially sent to their corresponding channels (2'b01: left channel; 2'b10: right channel) to achieve data extraction. When the extraction of a data (48 bits) is completed, the data is sent out at a specified time in the bandwidth matching data cycle to achieve data extraction and bandwidth matching.

[0114] In summary, the audio data processing method provided in this embodiment can first obtain the current frame sampling time of the first audio data, and the number of valid bytes contained in the first audio data of the current frame; determine the data period of the second audio data based on the current frame sampling time and the number of valid bytes; and determine the sampling frequency of the second audio data under the local clock of the data receiving end according to the data period and the local clock frequency of the data receiving end. In this way, the first audio data whose order is disrupted can be accurately bandwidth matched to obtain the actual sampling frequency of the data under the local clock of the receiving end, and the actual sampling frequency can be matched to send the audio data, that is, the second audio data can be transmitted according to the calculated actual sampling frequency. In the bandwidth matching stage, after the audio data is extracted in each matched data period, the audio data can be sent according to the actual sampling frequency of the audio data to ensure that the downstream data receiving end can directly process the audio.

[0115] Based on the same concept as the above audio data processing method, this embodiment also provides an audio data processing method, as shown in the attached Fig. 9 As shown, the audio data processing device comprises:

[0116] A data acquisition module, used to acquire a current frame sampling time of first audio data and the number of valid bytes contained in the first audio data of the current frame; the first audio data refers to audio data whose data transmission order is periodically disrupted;

[0117] A data cycle determination module, used to determine the data cycle of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes;

[0118] The frequency determination module is used to determine the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end.

[0119] The audio data processing device provided in this embodiment is based on the same concept as the above-mentioned audio data processing method, so it can at least achieve the above-mentioned beneficial effects, and any of the above-mentioned implementation modes can be applied to the audio data processing method provided in this embodiment, which will not be repeated here.

[0120] like Fig.10 As shown, this embodiment may also include an audio cache module, which writes the received scrambled audio data directly to the audio cache area in the RAM, and counts the number of audio data, and generates a flag pulse signal to notify the frequency determination module every time 100 audio data are counted; the frequency determination module counts the time between two notification flags and uses a divider to calculate the data period of the actual audio, thereby achieving bandwidth matching.

[0121] Based on the same concept as the above audio data processing method, this embodiment further provides a chip on which the above audio data processing method is integrated.

[0122] The chip provided in this embodiment is based on the same concept as the above-mentioned audio data processing method, so it can at least achieve the beneficial effects that can be achieved by the above-mentioned audio data processing method, and any implementation method of the above-mentioned audio data processing method can be applied to the chip provided in this embodiment, which will not be repeated here.

[0123] The embodiment of the present application also provides an electronic device corresponding to the audio data processing method provided in the above embodiment, so as to execute the above audio data processing method.

[0124] Please refer to Fig.11 , which shows a schematic diagram of the hardware structure of an electronic device provided in some embodiments of the present application. Fig.11 As shown, the electronic device 6 includes: a processor 600, a memory 601, a bus 602 and a communication interface 603, wherein the processor 600, the communication interface 603 and the memory 601 are connected via the bus 602; the memory 601 stores a computer program that can be run on the processor 600, and when the processor 600 runs the computer program, it executes the audio data processing method provided in any of the aforementioned embodiments of the present application.

[0125] The memory 601 may be any electronic, magnetic, optical or other physical storage device, and may contain storage information, such as executable instructions, data, etc. Specifically, the memory 601 may be a RAM (Random Access Memory), a flash memory, a storage drive (such as a hard disk drive), or a similar storage medium, or a combination thereof. The communication connection between the device network element and at least one other network element is realized through at least one communication interface 603 (which may be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. may be used.

[0126] The bus 602 may include an I2C (Inter-integrated Circuit) bus, a MIPI (Mobile industry Processor Interface) bus, an SPl (Serial Peripheral interface) bus, an I2S (Integrated Interchip Sound) bus, etc. It may also be divided into an address bus, a data bus, a control bus, etc. according to its function.

[0127] The processor 600 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 600. The above processor 600 can be a general-purpose processor, including a processor (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to be executed, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 601, and the processor 600 reads the information in the memory 601 and completes the steps of the above method in combination with its hardware.

[0128] The electronic device provided in the embodiment of the present application and the audio data processing method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented therein.

[0129] The embodiments of the present application also provide a computer-readable storage medium corresponding to the audio data processing method provided in the aforementioned embodiments, on which a computer program (i.e., a program product) is stored. When the computer program is run by a processor, it will execute the audio data processing method provided in any of the aforementioned embodiments.

[0130] It should be noted that computer-readable storage media may include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0131] An embodiment of the present application also provides a computer program product, including a computer program, which is executed by a processor to implement the audio data processing method of any of the above embodiments.

[0132] The computer-readable storage medium and computer program product provided in the above-mentioned embodiments of the present application are based on the same inventive concept as the audio data processing method provided in the embodiments of the present application, and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should all be included in the scope of the claims and specification of the present application. In particular, as long as there is no structural conflict, the various technical features mentioned in the various embodiments can be combined in any way. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions that fall within the scope of the claims.

Claims

1. An audio data processing method, characterized in that: For a data receiving end, the method comprises: Acquire a current frame sampling time of first audio data and the number of valid bytes contained in the first audio data of the current frame; the first audio data refers to audio data whose data transmission order is periodically disrupted; Determine a data period of second audio data of the current frame based on the current frame sampling time and the number of valid bytes; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission sequence; According to the data cycle and the local clock frequency of the data receiving end, a sampling frequency of the second audio data under the local clock of the data receiving end is determined.

2. The method according to claim 1, characterized in that The obtaining the current frame sampling time of the first audio data includes: Detecting that sampling of the first audio data of the previous frame is completed; The time from when the first audio data of the previous frame is sampled to when the first audio data of the current frame is sampled is counted and determined as the current frame sampling time.

3. The method according to claim 2, characterized in that The method further comprises: When a preset number of the first audio data are collected, a marking pulse is generated; the marking pulse is used to indicate that a frame of the first audio data is collected; The identification pulse is detected to determine that the sampling of the first audio data of the current frame is completed.

4. The method according to claim 1, characterized in that Determining the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes includes: When sampling of the audio data of the current frame is completed within the current frame sampling time, determining a data period of the second audio data of the current frame based on a quotient of the current frame sampling time and the number of valid bytes; When the current frame audio data is not sampled within the current frame sampling time, the preset period default value is determined as the data period of the second audio data.

5. The method according to claim 4, characterized in that The step of determining the data period of the second audio data of the current frame based on the quotient of the current frame sampling time and the number of valid bytes comprises: When the current frame sampling time is within a preset threshold range, using a divider to calculate a quotient of the current frame sampling time and the number of valid bytes; A data period of the second audio data of the current frame is determined based on the quotient value and a preset margin.

6. The method according to claim 1, characterized in that Before obtaining the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame, the method further includes: The received first audio data is written into an audio buffer area of ​​a memory, and the number of the collected first audio data is counted.

7. The method according to claim 1, characterized in that After determining the sampling frequency at which the data receiving end samples the second audio data under the local clock according to the data cycle, the method further includes: In each of the data cycles, reading two consecutive first audio data; Based on the two consecutive first audio data, audio data is extracted to obtain second audio data; the second audio data refers to audio data with normal data transmission sequence.

8. The method according to claim 7, characterized in that The step of reading two consecutive first audio data in each data cycle includes: In each of the data cycles, detecting two consecutive byte validity flags of the first audio data; When the byte validity identifier indicates that the two consecutive first audio data are valid, the two consecutive first audio data are read.

9. The method according to claim 7, characterized in that The extracting audio data based on the two consecutive first audio data to obtain second audio data includes: Analyzing the validity of each byte in the two consecutive first audio data and the channel to which it belongs; The byte data of the left channel and the byte data of the right channel are sent to their corresponding channels respectively.

10. The method according to claim 7, characterized in that Using a state machine, the audio data processing process is performed according to the following steps: In an idle state, a bandwidth matching enable signal is detected, and when a data period of the second audio data is determined, a valid flag reading state is entered; In the valid flag reading state, read the byte validity flags of two consecutive first audio data, and enter the audio data reading state when it is detected that the byte validity flag indicates that the byte data is valid; otherwise, read the byte validity flags of the next two consecutive audio data, and determine the validity of the next two consecutive audio data; In the audio data reading state, read two consecutive first audio data corresponding to the data byte validity identifier, and enter the data extraction state; In the data extraction state, the validity of each byte in the first audio data and the channel to which it belongs are analyzed according to the byte validity identifiers of the two consecutive audio data, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channels in sequence.

11. An audio data processing device, characterized in that: Applied to a data receiving end, the device comprises: A data acquisition module, used to acquire a current frame sampling time of first audio data and the number of valid bytes contained in the first audio data of the current frame; the first audio data refers to audio data whose data transmission order is periodically disrupted; a data cycle determination module, configured to determine a data cycle of second audio data of a current frame based on the current frame sampling time and the number of valid bytes; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission sequence; The frequency determination module is used to determine the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end.

12. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 10.

14. A computer program product comprising a computer program, characterized in that The computer program is executed by a processor to implement the method according to any one of claims 1 to 10.

Images