Audio data extraction method and device, electronic equipment and program product

Abstract

The invention provides an audio data extraction method and device, electronic equipment and a program product, and is used for a data receiving end, and the method comprises the steps: reading two consecutive first audio data in each period of data sampling; the first audio data refers to audio data of which the data transmission sequence is periodically disorganized; performing audio data extraction based on the two continuous first audio data to obtain second audio data; the second audio data are audio data which correspond to the first audio data and are normal in data transmission sequence. According to the method, the disordered audio data can be accurately extracted.

Description

Technical Field

[0001] The present application belongs to the technical field of audio data extraction, and specifically relates to an audio data extraction method, device, electronic equipment and program product. Background Art

[0002] At present, when processing audio data, such as spectrum analysis, sampling rate conversion, echo cancellation, etc., it is necessary to clarify the frequency of audio sampling and the original data of the sampling. However, when transmitting audio data, due to the requirements of the transmission protocol, the original data after audio sampling may be split, reorganized or interpolated to form audio data with a special structure. Different transmission protocols may have different processing methods for the original data, such as periodic disruption of audio data, invalid interpolation of original data to make the transmission frequency inconsistent with the sampling frequency, etc. Therefore, conventional audio processing methods cannot be used to directly process audio data with special structures, or before using conventional data processing, effective audio data needs to be extracted from the audio data with special structures to restore the original data after audio sampling.

[0003] It should be noted that the above statements are only used to provide background technical information related to the present application and do not necessarily constitute prior art. Summary of the invention

[0004] The present application proposes an audio data extraction method, device, electronic device and program product, which can more accurately extract the original audio data of the disrupted first audio data.

[0005] The first aspect of the present application provides an audio data extraction method for a data receiving end, the method comprising:

[0006] In each cycle of the data sampling, two consecutive first audio data are read; the first audio data refers to audio data whose data transmission order is periodically disrupted;

[0007] Based on the two consecutive first audio data, audio data is extracted to obtain second audio data; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission order.

[0008] In some optional embodiments, reading two consecutive first audio data in each cycle of data sampling includes:

[0009] In each cycle of data sampling, detecting two consecutive byte validity flags of the first audio data;

[0010] In a case where the byte validity identifier of the two consecutive first audio data indicates that the data is valid, the two consecutive first audio data are read.

[0011] In some optional embodiments, the extracting audio data based on the two consecutive first audio data to obtain the second audio data includes:

[0012] Analyzing the validity of each byte in the two consecutive first audio data and the channel to which it belongs;

[0013] The byte data of the left channel and the byte data of the right channel are sent to their corresponding channels respectively.

[0014] In some optional embodiments, a state machine is used to perform the audio data extraction process according to the following steps:

[0015] In an idle state, a bandwidth matching enable signal is detected, and when a data period of the second audio data is determined, a valid flag reading state is entered;

[0016] In the valid flag reading state, read two consecutive audio byte validity flags, and enter the audio data reading state when it is detected that the byte validity flag indicates that the byte is valid; otherwise, read the byte validity flags of the next two consecutive audio data, and determine the validity of the next two consecutive audio data;

[0017] In the audio data reading state, reading the first audio data corresponding to the two consecutive audio byte validity identifiers, and entering the data extraction state;

[0018] In the data extraction state, the validity of each byte in the first audio data and the channel to which it belongs are analyzed according to the byte validity identifiers of the two consecutive audio data, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channels in sequence.

[0019] In some optional embodiments, before reading two consecutive first audio data in each cycle of data sampling, the method further includes:

[0020] Acquire a current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame;

[0021] Determine a data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes;

[0022] According to the data cycle and the local clock frequency of the data receiving end, a sampling frequency of the second audio data under the local clock of the data receiving end is determined.

[0023] In some optional embodiments, obtaining the current frame sampling time of the first audio data includes:

[0024] Detecting that sampling of the first audio data of the previous frame is completed;

[0025] The time from when the first audio data of the previous frame is sampled to when the first audio data of the current frame is sampled is counted and determined as the current frame sampling time.

[0026] In some optional embodiments, the method further includes:

[0027] When a preset number of the first audio data are collected, a marking pulse is generated; the marking pulse is used to indicate that a frame of the first audio data is collected;

[0028] The identification pulse is detected to determine that the sampling of the first audio data of the current frame is completed.

[0029] In some optional embodiments, determining the data period of the second audio data based on the current frame sampling time and the number of valid bytes includes:

[0030] When sampling of the audio data of the current frame is completed within the current frame sampling time, determining a data period of the second audio data of the current frame based on a quotient of the current frame sampling time and the number of valid bytes;

[0031] When the current frame audio data is not sampled within the current frame sampling time, the preset period default value is determined as the data period of the second audio data of the current frame.

[0032] In some optional embodiments, determining the data period of the second audio data based on the quotient of the current frame sampling time and the number of valid bytes includes:

[0033] When the current frame sampling time is within a preset threshold range, using a divider to calculate a quotient of the current frame sampling time and the number of valid bytes;

[0034] A data period of the second audio data of the current frame is determined based on the quotient value and a preset margin.

[0035] In some optional embodiments, before obtaining the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame, the method further includes:

[0036] The received first audio data is written into an audio buffer area of ​​a memory, and the number of the collected first audio data is counted.

[0037] The second aspect of the present application provides an audio data extraction device, the device comprising:

[0038] A data reading module, used for reading two consecutive first audio data in each cycle of the data sampling; the first audio data refers to audio data whose data transmission order is periodically disrupted;

[0039] The audio extraction module is used to extract audio data based on the two consecutive first audio data to obtain second audio data; the second audio data refers to audio data with normal data transmission sequence.

[0040] An embodiment of the third aspect of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in the first aspect.

[0041] An embodiment of the fourth aspect of the present application provides a computer-readable storage medium on which a computer program is stored. The program is executed by a processor to implement the method described in the first aspect.

[0042] An embodiment of the fifth aspect of the present application provides a computer program product, including a computer program, wherein the computer program is executed by a processor to implement the method described in the first aspect.

[0043] The technical solution provided in the embodiments of the present application has at least the following technical effects or advantages:

[0044] The audio data extraction method provided in the embodiment of the present application reads two consecutive first audio data in each cycle of data sampling; the first audio data refers to the audio data whose data transmission order is periodically disrupted; based on the two consecutive first audio data, audio data extraction is performed to obtain the second audio data; the second audio data refers to the audio data whose data transmission order is normal. In this way, since the audio data that is periodically disrupted can usually extract a complete frame of audio data in normal order from two consecutive data, this embodiment reads two consecutive disrupted audio data in one data cycle, which can ensure that the data receiving end can correctly extract the original audio data of the disrupted first audio data, and match its actual sampling frequency to send the audio data, so as to ensure that the downstream data receiving end can directly process the audio. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] By reading the detailed description of the preferred embodiment below, various other advantages and benefits will become clear to those of ordinary skill in the art. The accompanying drawings are only used for the purpose of illustrating the preferred embodiment and are not considered to be limitations of the present application. In addition, the same reference symbols are used to represent the same components throughout the accompanying drawings.

[0046] In the attached picture:

[0047] Figure 1 A schematic diagram of a flow chart of an audio data extraction method provided by an embodiment of the present application is shown;

[0048] Figure 2 A schematic diagram of a working process of extracting audio data by a state machine provided in an embodiment of the present application is shown;

[0049] Figure 3 A schematic diagram of a process for bandwidth matching in an embodiment of the present application is shown;

[0050] Figure 4 A data flow diagram of an audio data extraction method provided by an embodiment of the present application is shown;

[0051] Figure 5 A schematic diagram of a data transmission process in which noise is generated during audio data transmission is shown;

[0052] Figure 6 A schematic diagram of a data transmission process for avoiding noise generation during audio data transmission in an embodiment of the present application is shown;

[0053] Figure 7 A schematic diagram of a data transmission process in which a notification of the next frame will arrive in advance during the audio data transmission process is shown;

[0054] Figure 8 A schematic diagram showing a specific flow chart of a bandwidth matching process in an audio data extraction method provided in an embodiment of the present application is shown;

[0055] Fig. 9 A schematic diagram showing a specific flow of an audio extraction process in an audio data extraction method provided in an embodiment of the present application is shown;

[0056] Fig.10 A schematic diagram of the structure of an audio data extraction device in another embodiment of the present application is shown;

[0057] Fig.11 A schematic diagram showing the structure of an audio data extraction device in another embodiment of the present application is shown;

[0058] Fig.12 A schematic diagram of an electronic device provided by an embodiment of the present application is shown. DETAILED DESCRIPTION

[0059] The exemplary embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although the exemplary embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be limited by the embodiments described herein. On the contrary, these embodiments are provided in order to enable a more thorough understanding of the present application and to fully convey the scope of the present application to those skilled in the art.

[0060] It should be noted that, unless otherwise specified, the technical terms or scientific terms used in this application should have the common meanings understood by technicians in the field to which this application belongs.

[0061] In practical applications, data validity directly affects whether the data can be used and whether the audio can be played; whether the bandwidth is consistent will affect the quality of the audio, and bandwidth mismatch will cause audio noise. For special structured audio where dual-channel data is periodically disrupted and the transmission frequency is inconsistent with the actual sampling frequency of the data, conventional audio processing methods cannot directly process data according to the data transmission frequency. Therefore, the audio processing end not only needs to extract the original audio and match the bandwidth, but also needs to ensure the validity of the extracted data while avoiding noise during bandwidth matching (both more and less data will generate noise), otherwise it will affect the audio quality output to the audio device.

[0062] In order to solve the above problems, this embodiment studies and analyzes the relevant technologies and finds that in some electronic products that need to transmit audio data, such as but not limited to wired headset products, Bluetooth headset products, and other audio devices, the transmission protocols of different products are not the same, so the data transmission structure may also be different. For example, in a certain communication protocol, the transmission rate of audio is fixed, but the audio sampling rate of transmission is not fixed. When the audio bus transmits 100 data, it actually only contains 99 594 bytes of shuffled audio data. The data receiving end needs to correctly extract the original audio data and match the audio sampling rate to send it out to ensure that the downstream data receiving end can directly process the audio.

[0063] Based on the above findings, the embodiments of the present application provide an audio data extraction method, device, electronic device and program product, which can be used for a data receiving end to read two consecutive first audio data in each cycle of data sampling; the first audio data refers to audio data whose data transmission order is periodically disrupted; based on the two consecutive first audio data, audio data extraction is performed to obtain second audio data; the second audio data refers to audio data with normal data transmission order. In this way, considering that the audio data that is periodically disrupted can usually extract a complete frame of audio data in normal order from two consecutive data, this embodiment reads two consecutive disrupted audio data in one data cycle, which can ensure that the data receiving end can correctly extract the original audio data of the disrupted first audio data, and match its actual sampling frequency to send the audio data, so as to ensure that the downstream data receiving end can directly process the audio.

[0064] The audio data extraction device is a processing module on an audio device for executing the audio data extraction method, and may specifically be a processor or a specially configured microprocessor, which is not specifically limited in this embodiment.

[0065] The electronic device may be an audio device to which the above-mentioned audio data extraction method is applied, such as but not limited to headphones, mobile phones, walkmans, etc., or may be an integrated chip on the audio device for extracting audio data.

[0066] The embodiments of the present application are described in detail below with reference to the accompanying drawings.

[0067] Please refer to Figure 1 , is a flow chart of the audio data extraction method provided in the embodiment of the present application, as shown in the attached Figure 1 As shown, the audio data extraction method includes.

[0068] Step S21, reading two consecutive first audio data in each data sampling cycle.

[0069] Among them, each cycle of data sampling can be understood as the sampling cycle of the ADC (analog-to-digital converter) of the data receiving end for audio data acquisition when receiving audio data, which can also be called a data cycle. The first audio data refers to the audio data whose data transmission order is periodically disrupted. It can be understood that even if the order of the audio data is disrupted, when the audio data is transmitted by bytes after being disrupted, the worst case will be completed in two audio data packets (12 bytes), that is, two disrupted first audios.

[0070] The two audio data may be understood as the audio data collected by the data receiving end at two audio sampling points, that is, the audio data collected by the data receiving end at each audio sampling point may be referred to as one audio data. The two consecutive first audio data may be understood as two audio data collected in two adjacent audio samplings. Furthermore, one audio data packet or one frame of audio data may contain multiple audio data.

[0071] Step S22: extracting audio data based on two consecutive first audio data to obtain second audio data.

[0072] The second audio data refers to audio data corresponding to the first audio data and having a normal data transmission sequence. Here, corresponding to the first audio data means that the valid data bytes contained in the two are the same.

[0073] In this embodiment, the principle of data extraction is: first, according to the data scrambling rule, a data byte (left and right channels) valid flag is generated and written into a fixed position in the RAM for storage.

[0074] Furthermore, the process of reading two consecutive first audio data in each data sampling cycle may include the following processing: in each data sampling cycle, detecting the data byte validity mark of two consecutive first audio data; when the byte validity mark indicates that the two consecutive first audio data are valid, reading the two consecutive first audio data. In this way, the validity of two consecutive first audio data can be determined at one time, which can effectively shorten the audio data processing time and thus improve the audio data processing efficiency.

[0075] During the audio transmission process, the disrupted audio data can be written into the specified area in the RAM for caching. At the same time, in the bandwidth matching stage, the byte validity identifiers of two consecutive audio data are first read in each cycle of the matching data sampling, and the two consecutive audio data are judged to be valid based on the byte validity identifier and then sent to the shift register for caching. Then, 2 corresponding audio data are read from the RAM audio buffer area and cached using a 96-bit shift register. Then, the validity of each byte of the audio data is detected by shifting the cached byte validity identifier and the audio data, and the valid bytes are sent to the output cache shift register for output in sequence. After the extraction of one data (48 bits) is completed, the data is sent out at a specified time in the matching data cycle to achieve data extraction and bandwidth matching.

[0076] Specifically, in this embodiment, the two first audio data may have only one byte validity identifier. For example, the order in which the data receiving end receives the data packets is audio data 1, audio data 2, audio data 3, ..., then the first byte validity identifier is generated for audio data 1 and audio data 2, which is used to extract the first audio data; the second byte validity identifier is generated for audio data 2 and audio data 3, which is used to extract the second audio data, and so on. If the first byte validity identifier is 0, indicating that the audio data 1 and audio data 2 corresponding to the identifier are invalid, then determine whether the second byte validity identifier is 0. If it is not 0, start extracting the data in audio data 2 and audio data 3 corresponding to the second byte validity identifier. It should be noted that the audio data here refers to the first audio data transmitted by the protocol.

[0077] It should be noted that the setting of the above-mentioned byte validity flag is only one implementation of this embodiment, and this embodiment is not limited thereto, as long as it can be determined that two consecutive audio data are valid based on the validity flag. For example, corresponding byte validity flags can be set for each audio data, and two consecutive flags are judged each time to determine whether two consecutive audio data are valid.

[0078] Specifically, the above-mentioned step of extracting audio data based on two consecutive first audio data to obtain the second audio data may include the following processing: analyzing the validity of each byte in the two consecutive first audio data and the channel to which it belongs; sending the byte data of the left channel and the byte data of the right channel to their corresponding channels respectively.

[0079] In this embodiment, the cached byte validity flag and audio data can be shifted simultaneously to analyze the validity of each byte of the audio data and the channel to which it belongs, and the byte data of the left and right channels can be sent to their corresponding channels (2'b01: left channel; 2'b10: right channel, otherwise the data is invalid) in sequence to achieve data extraction.

[0080] In other embodiments, a state machine may be used to execute the audio data extraction process according to the following steps: in an idle state, a bandwidth matching enable signal is detected, and when the data period of the second audio data is determined, the valid flag reading state is entered; in the valid flag reading state, two consecutive byte validity identifiers are read, and when the byte validity identifier is detected to indicate that the byte is valid, the audio data reading state is entered; in the audio data reading state, the first audio data corresponding to the two consecutive byte validity identifiers are read, and the data extraction state is entered; in the data extraction state, the validity of each byte in the first audio data and the channel to which it belongs are analyzed, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channels in sequence.

[0081] In this embodiment, the Figure 2 As shown, the audio extraction process can be implemented by the following four state machines. Specifically, IDLE state: the state machine holds the state after the reset state and audio data recovery are completed. In this state, after the bandwidth matching enable signal is detected, the state of reading the audio byte validity flag (DOWN_RD_FLAG) is entered when the clearing condition of the data synchronization cycle counter is met.

[0082] DOWN_RD_FLAG state (valid flag read state): This state reads the byte validity flag stored in the data ram, and uses a 24-bit register to store it and use it in the data recovery phase. In this state, if the read data is detected to be 0, it indicates that the current corresponding audio data has been recovered, and the data channel flag of the next address is required. In this state, the state machine enters the audio data read (DOWN_RD_DATA) state when it detects that the read byte validity flag is valid.

[0083] DOWN_RD_DATA state (read audio data state): This state reads the address of the two audio data stored in RAM, and uses a 96-bit register to store the data, and is used in the data recovery state. In this state, the state machine detects that the reading of the two data is completed and enters the data extraction (DATA_RECOVER) state.

[0084] DATA_RECOVER state (data extraction state): This state simultaneously shifts the cached data byte validity flag and the audio data, analyzes the validity of each byte of the audio data and the channel to which it belongs, and sends the byte data of the left and right channels to their corresponding channels in sequence (2'b01: left channel; 2'b10: right channel; otherwise the byte data is invalid) to achieve data extraction. When data extraction is completed, the state machine re-enters the IDLE state.

[0085] In some other embodiments, in each data sampling cycle, before reading two consecutive first audio data, a bandwidth matching process may be performed, such as Figure 3 As shown, the bandwidth matching process includes the following steps: step S11, obtaining the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame; step S12, determining the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes of the current frame; step S13, determining the sampling frequency of the second audio data under the local clock of the data receiving end according to the data period and the local clock frequency of the data receiving end.

[0086] Among them, the first audio data of the current frame can be understood as the data packet of the first audio data that has been received and just received. For example, if a pulse signal is generated after a data packet is received, indicating that the frame of audio data has been received, then the current frame in this embodiment is the audio frame data corresponding to the latest generated pulse signal. It can contain multiple first audio data, for example, it can contain 100 audio data. The current frame sampling time can be understood as the sampling time of the first audio data of the current frame, which can be the sampling time of all the first audio data contained in the current frame, or it can be the sampling time of part of the first audio data contained in the current frame.

[0087] Specifically, the number of valid bytes of the first audio data contained in the first audio data of the current frame may be preset, or the byte validity may be judged according to the validity flag of the data during the audio data receiving process, and the number of valid bytes may be counted, which is not specifically limited in this embodiment. In addition, when the number of valid bytes is preset, the number of valid bytes of the audio data of each frame may be the same. For example, when the first audio data of the current frame contains 100 audio data, the corresponding number of valid bytes may be 99.

[0088] In this embodiment, since the first audio data may contain invalid audio data, the number of actually valid bytes in the first audio data of the current frame is less than the total number of bytes in the first audio data of the current frame. The second audio data can be understood as data containing only valid audio bytes, and the time of the first audio is the same as the time of the second audio, so the data period of the second audio data can be directly calculated according to the current frame sampling time and the number of valid bytes, and the data period of each frame of audio data can be obtained, so that the first audio data whose order is disrupted can be matched frame by frame with respect to bandwidth, so as to obtain a more accurate actual sampling frequency of the data receiving end under the local clock.

[0089] It can be understood that each audio data is originally multi-byte, that is, one audio data may include multiple valid bytes, so when calculating the data period, the audio sampling depth (or audio data bit width) also needs to be considered.

[0090] In this embodiment, before executing the above step S11, the received first audio data may be written into the audio buffer area of ​​the memory, and the number of the collected first audio data may be counted.

[0091] Specifically, this embodiment can be implemented according to Figure 4 In the data stream processing process shown, the received disrupted first audio data can be directly written into the audio buffer area in the memory (RAM), and the number of the collected first audio data can be counted.

[0092] Furthermore, the first audio data can be written into the RAM cache while being read out from the cache. When the first audio cache has 80 data, the bandwidth matching is enabled and bandwidth matching is started. It can be understood that 80 here is only one implementation of this embodiment, and this embodiment is not limited thereto, for example, it can also be 60, 70, 85, 88, etc. data.

[0093] Specifically, it can be specifically set according to the size of the audio buffer in the local memory, because the data size of the audio is uncertain, and if a larger space is left, resources will be wasted. Since the played audio data will no longer require storage space, it can be designed to reserve a section of storage space for cyclic writing to realize audio data caching. For example, 100 addresses are left in the memory as the audio buffer, and after the audio is stored in the 100th address, the next time it will restart caching from address 1. Because only one first audio data will be stored in each data cycle, and two audio data need to be read in one data cycle, so in theory, as long as more than 3 and less than 98 first audios are stored, it is possible to avoid reading the played audio data.

[0094] In some embodiments, the above process of obtaining the current frame sampling time of the first audio data may include the following processing: detecting that the sampling of the first audio data of the previous frame is completed; counting and statistically analyzing the time from the completion of the sampling of the first audio data of the previous frame to the completion of the sampling of the first audio data of the current frame, and determining it as the current frame sampling time.

[0095] In this embodiment, in view of the continuous data transmission, the time between the two moments when the two data are received can be determined as the sampling time of the next frame of data. In this way, the intermediate judgment process can be omitted, and the sampling time of the next frame of data, that is, the first audio data of the current frame, can be determined by only recording the count values ​​of the two reception completion moments.

[0096] Furthermore, this embodiment can generate an identification pulse when a preset number of first audio data are collected; the identification pulse is used to indicate that a frame of first audio data is collected, which can be used as a bandwidth matching enable; and when the identification pulse is detected, it is determined that the sampling of the first audio data of the current frame is completed.

[0097] Specifically, a flag pulse signal may be generated every time 100 audio data are counted, indicating that a frame of audio data has been received, so as to start the bandwidth matching process. It can be understood that the 100 audio data are only one implementation of this embodiment, and this embodiment does not specifically limit the number, which can be specifically set according to the data structure of the first audio data, and is consistent with the number of byte data contained in a frame of data in the first audio data.

[0098] More specifically, the bandwidth matching counter cnt0 can be used to count the sampling time of a frame of the first audio data frame for bandwidth matching. When the next frame notification flag is detected, it indicates that the previous frame data has ended and the next frame data has begun. At this time, cnt0 has completed the time statistics of the previous frame data.

[0099] In some other optional embodiments, the above-mentioned step S12 may specifically include the following processing: when the current frame sampling time includes the sampling time of all audio data of the current frame, determining the data period of the second audio data of the current frame based on the quotient of the current frame sampling time and the number of valid bytes of the current frame; when the current frame sampling time has not been counted, determining the preset period default value as the data period of the second audio data of the current frame.

[0100] In this embodiment, the basic principle of bandwidth matching is that the local clock continuously counts the sampling time of the current frame audio data, such as the time to transmit 100 audio data, and then uses a divider to calculate the actual bandwidth of the audio data corresponding to the number of valid bytes (for example, 99), so as to achieve accurate matching frame by frame. However, during the transmission of the first frame of data, since the bandwidth matching counter cnt0 has not yet completed the statistics of the data frame, the data cycle counter cnt1 needs to calculate the data cycle based on the default value calculated based on the actual audio sampling frequency combined with the local clock frequency. After the first frame of data transmission is completed, the data cycle counter uses the quotient of the divider to calculate the data cycle.

[0101] In the above step S13, in the process of determining the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end, the cycle length N of the local clock can be determined based on the local clock frequency f, and then the cycle length N' of the second audio data under the local clock of the data receiving end can be obtained based on the calculated data cycle multiplied by the cycle length N of the local clock. The reciprocal of the cycle length N' can be understood as the sampling frequency of the second audio data under the local clock of the data receiving end.

[0102] Further, the data period of the second audio data of the current frame is determined based on the quotient of the current frame sampling time and the number of valid bytes, including: when the current frame sampling time is within a preset threshold range, using a divider to calculate the quotient of the current frame sampling time and the number of valid bytes; and determining the data period of the second audio data of the current frame based on the quotient and a preset margin.

[0103] Specifically, the value of cnt0 can be sent to a divider and divided by the actual sampling number of the audio data, 99, to obtain the number of cycles of the audio data under the local clock.

[0104] Consider not using the remainder of the divider, such as Figure 5In the figure, when the cnt0 statistical value is 10000 during a packet of data, if the divider quotient 98 is used as the matching data cycle, 100 data cycles will be generated, but only 99 valid data will be generated. Therefore, the extra data cycle (the shaded part in the figure) will generate 1 noise point, forming noise.

[0105] To avoid this situation, the divider quotient + 1 is used to calculate the data cycle. This method theoretically ensures that only 99 data will be generated when the next frame notification arrives. Figure 6 As shown in the figure, the number of clock cycles of the last data under the local clock is 1 clock cycle less than the number of cycles of the first 98 data. Therefore, it can be seen that the higher the operating frequency of the local clock, the smaller the cycle error of the last number, and the higher the accuracy of bandwidth matching.

[0106] For audio data of this structure, this method cannot be used in all cases. Let the time for the bus to transmit 100 audio data be y, the quotient of the divider (i.e., data cycle) be x, and the remainder be n, then:

[0107] y=99*x+n

[0108] =98*(x+1)+(x+n-98)

[0109] Wherein, x and n are both integers greater than 0.

[0110] From the above formula, we can see that 98*(x+1) can be regarded as the total time of the first 98 data, and (x+n-98) is the data period of the last data. Therefore, the value of (x+n-98) must be greater than 0. Without considering the remainder n, the data period x>98.

[0111] Since data extraction requires additional clocks, the last data cycle must not only be greater than 0, but also greater than the number of data clock cycles required for data processing. Suppose the number of clock cycles required for data processing is m, then this method can be used when x-98>m, that is, x>98+m. The number of valid data 99 is represented by k, and this method can be used when x>k+m-1.

[0112] Considering that the external clock frequency may be offset, the phenomenon is that the number of clock cycles counted by the previous and next frames is different. The situation of generating multiple data cycles is similar to the above, so I will not repeat it. For example, when the clock frequency of the data sending end gradually increases, the notification of the next frame will arrive in advance, such as Figure 7As shown in the figure, the statistical time of the previous frame data is larger than that of the next frame. In serious cases, it will lead to less data cycle when the next frame data is counted. Therefore, it is necessary to reduce a certain margin based on the division result to reduce the average cycle of the data to ensure that there is no less data cycle and data loss. Therefore, it is necessary to design the margin value to be configurable. When the system detects that the data source clock is too large, the margin value is increased to reduce the average data cycle, thereby ensuring that data is not lost. The above figure is only for illustration (in fact, because there is a clock calibration module), the clock offset will not be very large. The most likely thing is that the x-98>m condition does not hold, resulting in the loss of the last data. Therefore, this problem can be solved when the margin is 1.

[0113] After calculating the data period of the second audio data, the sampling frequency of the data receiving end under the local clock can be calculated according to the inverse relationship between the period and the frequency, thereby achieving bandwidth matching of the audio data and accurately extracting the audio data.

[0114] In a specific embodiment, the Figure 8 and Fig. 9 The bandwidth matching process and the data extraction process are performed respectively, thereby realizing the audio data extraction process provided by this embodiment. Specifically, Figure 8 As shown in the figure, the bandwidth matching process includes the following processes: ① The audio data is written into the RAM cache while being read out from the cache. When the audio cache has 80 data, the bandwidth matching enable is turned on and the bandwidth matching begins. ② Bandwidth matching, the bandwidth matching counter cnt0 counts the time of the data frame for bandwidth matching. When the next frame notification flag is detected, it indicates that the previous frame of data has ended and the next frame of data has begun. At this time, cnt0 has completed the time statistics of the previous frame of data. The value of cnt0 is sent to the divider and divided by the actual number of samples of the audio data, 99, to obtain the number of cycles of the audio data under the local clock. ③ During the transmission of the first frame of data, since the bandwidth matching counter cnt0 has not completed the statistics of the data frame, the data cycle counter cnt1 needs to calculate the data cycle based on the default value calculated by the actual audio sampling frequency combined with the local clock frequency. After the first frame of data transmission is completed, the data cycle counter uses the quotient of the divider to calculate the data cycle.

[0115] like Fig. 9As shown, the data extraction process includes the following process: during the audio transmission process, the disrupted audio data is written into the designated area in the RAM for caching, and during the bandwidth matching stage, in each cycle of the matched data sampling, the byte validity identifiers of the two audio data are first read to determine the data validity and then sent to the shift register for caching, and then the two corresponding audio data are read from the RAM audio buffer area and cached using the 96-bit shift register, and then the validity of each byte of the audio data is detected by shifting the cached byte validity identifier and the audio data; if the read data is detected to be 0, it indicates that the current corresponding audio data has been extracted, and it is necessary to obtain the byte validity identifier of the next address. Analyze the corresponding first audio data and send the valid bytes to the output buffer shift register in sequence for output. By simultaneously shifting the cached byte validity identifier and audio data, analyze the validity of each byte of the audio data and the channel to which it belongs, and send the byte data of the left and right channels to their corresponding channels (2'b01: left channel; 2'b10: right channel) in sequence to achieve data extraction. After the extraction of a data (48 bits) is completed, the data is sent out at a specified time within the bandwidth matching data cycle to achieve data extraction and bandwidth matching.

[0116] Specifically, it can be determined whether the bandwidth matching process is turned on. If the bandwidth matching process is turned on, it is further determined whether the cache of 80 first audio data has been completed, and whether the data cycle calculation is completed, that is, whether the count of cnt1 is cleared. After the data cycle calculation is completed, that is, the count of cnt1 is cleared, the byte validity mark of the audio data can be read. For example, the byte validity mark of two consecutive audio data can be read from RAM address0. If the byte validity mark indicates that the data is valid, the byte validity mark of the first audio data is stored in the shift register flg_data[23:0] for caching; if the byte validity mark indicates that the data is invalid, a data mark indicating that the data is invalid is generated, such as data_invalid_flg=1, and the byte validity mark of the next two consecutive data is read from RAM address0+1 until the byte validity mark indicating that the two consecutive audio data are valid is read. Then, it can be determined whether the audio data at the current address needs to be extracted based on the data mark. For example, if data_invalid_flg=0, it means that there is data to be extracted from the current cache address address1 and address1+1, then two audio data are read from the corresponding addresses address1 and address1+1 in the RAM and stored in the shift register data[95:0] for caching; if data_invalid_flg=1, it means that the data in the addresses address1 and address1+1 have been extracted and are invalid, and the next two consecutive audio data need to be read from the RAM address1+1 and address1+2 and stored in the shift register data[95:0] for caching.

[0117] Then, by shifting the cached data left and right channel (byte validity) flags and audio data simultaneously, analyzing the validity of each byte of the audio data and the channel to which it belongs, the byte data of the left and right channels are sequentially sent to their corresponding channels (2'b01: left channel; 2'b10: right channel) to achieve data extraction. For example, flg_data[23:22]=2'b10, indicating that the current audio data belongs to the right channel, it can be sent to the right channel, that is, the audio data is written to the corresponding audio register, audio data audio_data[23:0]={audio_data[15:0],data[95:88]}. And flg_data[23:22]=2'b01 indicates that the current audio data belongs to the left channel, it can be sent to the left channel, that is, the audio data is written to the corresponding audio register, audio data audio_data[47:24]={audio_data[40:24],data[95:88]}. Then the circular shifter is circularly shifted, that is, flg_data = {flg_data[21:0], flg_data[23:22]}, data[95:0] = {data[87:0], data[95:88]}, until it occurs 12 times, the 6-byte data extraction is completed, and the shift is completed. It can be further determined that address1! = the last address. If so, address0 = starting address 0, and address1 = starting address 1. If not, address0 = address0+1, and address1 = address1+1 can be set. After that, it can be determined again whether the bandwidth matching process is started. If not, the data extraction process is terminated directly. If the bandwidth matching process is still started, it is re-determined whether the data cycle calculation is completed, and then another round of audio data extraction is performed until the bandwidth matching process ends.

[0118] In summary, the audio data extraction method provided in this embodiment can read two consecutive first audio data in each cycle of the matched data sampling in the bandwidth matching stage, extract the audio data of the two consecutive first audio data, and obtain the audio data with the correct order, so that the audio data can be accurately extracted. And when the extraction of one data (48bit) is completed, the data is sent out at a specified time in the data cycle of the bandwidth matching to achieve data extraction and bandwidth matching. It is also possible to obtain the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame; determine the data cycle of the second audio data based on the current frame sampling time and the number of valid bytes; determine the sampling frequency of the second audio data under the local clock of the data receiving end according to the data cycle and the local clock frequency of the data receiving end. In this way, the first audio data whose order is disrupted can be accurately matched with bandwidth, and the actual sampling frequency of the data receiving end under the local clock is obtained to ensure that the data receiving end can correctly extract the original audio data of the disrupted first audio data, and match its actual sampling frequency to send the audio data, so as to ensure that the downstream data receiving end can directly process the audio.

[0119] Based on the same concept as the above audio data extraction method, this embodiment also provides an audio data extraction method, as shown in the attached Fig.10 As shown, the audio data extraction device comprises:

[0120] A data reading module, used for reading two consecutive first audio data in each cycle of data sampling; the first audio data refers to audio data whose data transmission sequence is periodically disrupted;

[0121] The data extraction module is used to extract audio data based on two consecutive first audio data to obtain second audio data; the second audio data refers to audio data with normal data transmission sequence.

[0122] The audio data extraction device provided in this embodiment is based on the same concept as the above-mentioned audio data extraction method, so it can at least achieve the above-mentioned beneficial effects, and any of the above-mentioned implementation modes can be applied to the audio data extraction method provided in this embodiment, which will not be repeated here.

[0123] like Fig.11 As shown, this embodiment may also include an audio cache module, which writes the received scrambled audio data directly to the audio cache area in the RAM, and counts the number of audio data, and generates a flag pulse signal to notify the frequency determination module every time 100 audio data are counted; the frequency determination module counts the time between two notification flags and uses a divider to calculate the data period of the actual audio, thereby achieving bandwidth matching of the audio data.

[0124] Based on the same concept as the above audio data extraction method, this embodiment also provides a chip on which the above audio data extraction method is integrated.

[0125] The chip provided in this embodiment is based on the same concept as the above-mentioned audio data extraction method, so it can at least achieve the beneficial effects that the above-mentioned audio data extraction method can achieve, and any implementation method of the above-mentioned audio data extraction method can be applied to the chip provided in this embodiment, which will not be repeated here.

[0126] The embodiment of the present application also provides an electronic device corresponding to the audio data extraction method provided in the above embodiment to execute the above audio data extraction method.

[0127] Please refer to Fig.12 , which shows a schematic diagram of the hardware structure of an electronic device provided in some embodiments of the present application. Fig.12 As shown, the electronic device 6 includes: a processor 600, a memory 601, a bus 602 and a communication interface 603, wherein the processor 600, the communication interface 603 and the memory 601 are connected via the bus 602; the memory 601 stores a computer program that can be run on the processor 600, and when the processor 600 runs the computer program, it executes the audio data extraction method provided in any of the aforementioned embodiments of the present application.

[0128] The memory 601 may be any electronic, magnetic, optical or other physical storage device, and may contain storage information, such as executable instructions, data, etc. Specifically, the memory 601 may be a RAM (Random Access Memory), a flash memory, a storage drive (such as a hard disk drive), or a similar storage medium, or a combination thereof. The communication connection between the device network element and at least one other network element is realized through at least one communication interface 603 (which may be wired or wireless), and the Internet, a wide area network, a local area network, a metropolitan area network, etc. may be used.

[0129] The bus 602 may include an I2C (Inter-integrated Circuit) bus, a MIPI (Mobile industry Processor Interface) bus, an SPl (Serial Peripheral interface) bus, an I2S (Integrated Interchip Sound) bus, etc. It may also be divided into an address bus, a data bus, a control bus, etc. according to its function.

[0130] The processor 600 may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by the hardware integrated logic circuit or software instructions in the processor 600. The above processor 600 may be a general-purpose processor, including a processor (Central Processing Unit, referred to as CPU), a network processor (Network Processor, referred to as NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a readily available programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc. The steps of the method disclosed in the embodiments of the present application can be directly embodied as a hardware decoding processor to execute, or the hardware and software modules in the decoding processor can be executed. The software module can be located in a mature storage medium in the field such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, etc. The storage medium is located in the memory 601, and the processor 600 reads the information in the memory 601 and completes the steps of the above method in combination with its hardware.

[0131] The electronic device provided in the embodiment of the present application and the audio data extraction method provided in the embodiment of the present application are based on the same inventive concept and have the same beneficial effects as the methods adopted, run or implemented therein.

[0132] The embodiments of the present application also provide a computer-readable storage medium corresponding to the audio data extraction method provided in the aforementioned embodiments, on which a computer program (i.e., a program product) is stored. When the computer program is executed by a processor, it will execute the audio data extraction method provided in any of the aforementioned embodiments.

[0133] It should be noted that computer-readable storage media may include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other optical or magnetic storage media, which are not listed here one by one.

[0134] An embodiment of the present application also provides a computer program product, including a computer program, which is executed by a processor to implement the audio data extraction method of any of the above embodiments.

[0135] The computer-readable storage medium and computer program product provided in the above-mentioned embodiments of the present application are based on the same inventive concept as the audio data extraction method provided in the embodiments of the present application, and have the same beneficial effects as the methods adopted, run or implemented by the application programs stored therein.

[0136] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present application, and they should all be included in the scope of the claims and specification of the present application. In particular, as long as there is no structural conflict, the various technical features mentioned in the various embodiments can be combined in any way. The present application is not limited to the specific embodiments disclosed herein, but includes all technical solutions that fall within the scope of the claims.

Claims

1. A method for extracting audio data, characterized in that: For a data receiving end, the method comprises: In each cycle of data sampling, two consecutive first audio data are read; the first audio data refers to audio data whose data transmission order is periodically disrupted; Based on the two consecutive first audio data, audio data is extracted to obtain second audio data; the second audio data refers to audio data corresponding to the first audio data and having a normal data transmission order.

2. The method according to claim 1, characterized in that The step of reading two consecutive first audio data in each data sampling cycle includes: In each cycle of data sampling, detecting byte validity flags of the two consecutive first audio data; In a case where the byte validity identifier indicates that the two consecutive first audio data are valid, the two consecutive first audio data are read.

3. The method according to claim 1, characterized in that The extracting audio data based on the two consecutive first audio data to obtain second audio data includes: Analyzing the validity of each byte in the two consecutive first audio data and the channel to which it belongs; The byte data of the left channel and the byte data of the right channel are sent to their corresponding channels respectively.

4. The method according to claim 1, characterized in that Using a state machine, the audio data extraction process is performed as follows: In an idle state, a bandwidth matching enable signal is detected, and when a data period of the second audio data is determined, a valid flag reading state is entered; In the valid flag reading state, read the byte validity marks of two consecutive audio data, and enter the audio data reading state when it is detected that the byte validity mark indicates that the byte is valid; Otherwise, read the byte validity flags of the next two consecutive audio data, and determine the validity of the next two consecutive audio data; In the audio data reading state, reading two consecutive first audio data corresponding to the byte validity identifier, and entering the data extraction state; In the data extraction state, the validity of each byte in the first audio data and the channel to which it belongs are analyzed according to the byte validity identifiers of the two consecutive audio data, and the byte data of the left channel and the byte data of the right channel are sent to the corresponding channels in sequence.

5. The method according to claim 1, characterized in that The method further includes, before reading two consecutive first audio data in each cycle of data sampling: Acquire a current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame; Determine a data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes of the current frame; According to the data cycle and the local clock frequency of the data receiving end, a sampling frequency of the second audio data under the local clock of the data receiving end is determined.

6. The method according to claim 5, characterized in that The obtaining the current frame sampling time of the first audio data includes: Detecting that sampling of the first audio data of the previous frame is completed; The time from when the first audio data of the previous frame is sampled to when the first audio data of the current frame is sampled is counted and determined as the current frame sampling time.

7. The method according to claim 6, characterized in that The method further comprises: When a preset number of the first audio data are collected, a marking pulse is generated; the marking pulse is used to indicate that a frame of the first audio data is collected; The identification pulse is detected to determine that the sampling of the first audio data of the current frame is completed.

8. The method according to claim 5, characterized in that The step of determining the data period of the second audio data of the current frame based on the current frame sampling time and the number of valid bytes of the current frame comprises: When sampling of the audio data of the current frame is completed within the current frame sampling time, determining a data period of the second audio data of the current frame based on a quotient of the current frame sampling time and the number of valid bytes of the current frame; When the current frame audio data is not sampled completely within the current frame sampling time, the preset period default value is determined as the data period of the second audio data of the current frame.

9. The method according to claim 8, characterized in that The determining the data period of the second audio data based on the quotient of the current frame sampling time and the number of valid bytes of the current frame comprises: When the current frame sampling time is within a preset threshold range, a divider is used to calculate a quotient of the current frame sampling time and the number of valid bytes; A data period of the second audio data of the current frame is determined based on the quotient value and a preset margin.

10. The method according to claim 5, characterized in that Before obtaining the current frame sampling time of the first audio data and the number of valid bytes contained in the first audio data of the current frame, the method further includes: The received first audio data is written into an audio buffer area of ​​a memory, and the number of the collected first audio data is counted.

11. An audio data extraction device, characterized in that: Applied to a data receiving end, the device comprises: A data reading module, used for reading two consecutive first audio data in each cycle of data sampling; the first audio data refers to audio data whose data transmission order is periodically disrupted; The audio extraction module is used to extract audio data based on the two consecutive first audio data to obtain second audio data; the second audio data refers to audio data corresponding to the first audio data and with normal data transmission order.

12. An electronic device, characterized in that: The invention comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method according to any one of claims 1 to 10.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: The program is executed by a processor to implement the method according to any one of claims 1 to 10.

14. A computer program product comprising a computer program, characterized in that The computer program is executed by a processor to implement the method according to any one of claims 1 to 10.

Images