Audio caching method, device, microphone device, electronic device and storage medium

Abstract

The present disclosure relates to an audio caching method, apparatus, microphone device, electronic device and storage medium. The audio caching method includes: in response to determining that a cache area meets a set condition, determining a continuous first audio frame and a second audio frame from the collected audio frames; fusing the first audio frame and the second audio frame to obtain a continuously changing fused audio frame, wherein the beginning of the fused audio frame is the same as the beginning of the first audio frame, and the end of the fused audio frame is the same as the end of the second audio frame; and storing the fused audio frame in the cache area.

Description

Technical Field

[0001] The present disclosure relates to the field of short video technology, and in particular to an audio caching method, apparatus, microphone device, electronic device and storage medium. Background Art

[0002] In office environments, online meetings are increasingly used. Professional conference microphone equipment is favored by users because of its wide sound pickup range and good playback sound quality. Conference microphone equipment often uses a USB interface to connect to a computer or tablet. This method is simple to connect and the method of use is very intuitive for users.

[0003] However, as a slave device of a computer or tablet, the conference microphone has an independent clock signal from the master device. The slight difference between them, after a long period of accumulation, will cause the buffer area of ​​the conference microphone to be filled up or insufficient data. Generally speaking, when the buffer area is full, the latest frame of the signal received will often be discarded. After the earliest frame of the signal is taken away by the master device, the buffer area will make room and continue to receive signals normally until the buffer area is filled again. If the buffer area does not have enough data, some all-zero data will have to be sent to complete it. Both methods will cause discontinuity in speech, and the human ear is very sensitive to discontinuity in speech, so it is easy to perceive obvious noise, affecting the quality of communication. Summary of the invention

[0004] The present disclosure provides an audio caching method, apparatus, microphone device, electronic device and storage medium to at least solve the problem of low communication quality in the related art, and may not solve any of the above problems.

[0005] According to a first aspect of the present disclosure, an audio caching method is provided, the audio caching method comprising: in response to determining that a cache area meets a set condition, determining a continuous first audio frame and a second audio frame from the collected audio frames; fusing the first audio frame and the second audio frame to obtain a continuously changing fused audio frame, the beginning of the fused audio frame is the same as the beginning of the first audio frame, and the end of the fused audio frame is the same as the end of the second audio frame; and storing the fused audio frame in the cache area.

[0006] Optionally, in response to determining that the cache area meets the set conditions, determining the continuous first audio frame and the second audio frame from the captured audio frames includes: in response to capturing a new audio frame and determining that the available capacity in the cache area is less than the set capacity, recording the latest audio frame in the cache area as the first audio frame, and recording the captured new audio frame as the second audio frame; storing the fused audio frame in the cache area includes: overwriting the first audio frame with the fused audio frame.

[0007] Optionally, the set capacity is equal to the size of an audio frame.

[0008] Optionally, in response to determining that the cache area satisfies a set condition, determining a continuous first audio frame and a second audio frame from the collected audio frames includes: in response to determining that there are at least two audio frames in the cache area and the available capacity in the cache area is greater than the size of at least one audio frame, recording the latest audio frame of the at least two audio frames as the first audio frame, and recording the previous audio frame of the first audio frame as the second audio frame; storing the fused audio frame in the cache area includes: storing the fused audio frame between the second audio frame and the first audio frame.

[0009] Optionally, determining that there are at least two audio frames in the buffer area and that the available space in the buffer area is larger than the size of at least one audio frame includes: determining that there are only two audio frames in the buffer area.

[0010] Optionally, the fusing processing of the first audio frame and the second audio frame to obtain a continuously changing fused audio frame includes: aligning the start time and the end time of the first audio frame and the second audio frame respectively; determining the product of the first audio frame and the first coefficient to obtain first audio data, and determining the product of the second audio frame and the second coefficient to obtain second audio data; wherein the first coefficient gradually changes from 1 to 0 from the start time to the end time, and the second coefficient gradually changes from 0 to 1 from the start time to the end time, and at the same time, the sum of the first coefficient and the second coefficient is 1; superimposing the first audio data and the second audio data to obtain the fused audio frame.

[0011] According to a second aspect of the present disclosure, an audio caching device is provided, comprising: a determination unit, configured to: in response to determining that a cache area satisfies a set condition, determine a continuous first audio frame and a second audio frame from the collected audio frames; a fusion unit, configured to: perform fusion processing on the first audio frame and the second audio frame to obtain a continuously changing fused audio frame, the beginning of the fused audio frame is the same as the beginning of the first audio frame, and the end of the fused audio frame is the same as the end of the second audio frame; a storage unit, configured to: store the fused audio frame in the cache area.

[0012] Optionally, the determination unit is further configured to: in response to collecting a new audio frame and determining that the available capacity in the cache area is less than a set capacity, record the latest audio frame in the cache area as the first audio frame, and record the collected new audio frame as the second audio frame; the storage unit is further configured to: use the fused audio frame to overwrite the first audio frame.

[0013] Optionally, the set capacity is equal to the size of an audio frame.

[0014] Optionally, the determination unit is further configured to: in response to determining that there are at least two audio frames in the buffer area and the available capacity in the buffer area is greater than the size of at least one audio frame, record the latest audio frame of the at least two audio frames as the first audio frame, and record the previous audio frame of the first audio frame as the second audio frame; the storage unit is further configured to: store the fused audio frame between the second audio frame and the first audio frame.

[0015] Optionally, the determination unit is further configured to: in response to determining that there are only two audio frames in the buffer area, record the latest audio frame of the two audio frames as the first audio frame, and record the previous audio frame of the first audio frame as the second audio frame.

[0016] Optionally, the fusion unit is also configured to: align the starting time and the ending time of the first audio frame and the second audio frame respectively; determine the product of the first audio frame and the first coefficient to obtain the first audio data, determine the product of the second audio frame and the second coefficient to obtain the second audio data; wherein, the first coefficient gradually changes from 1 to 0 from the starting time to the ending time, and the second coefficient gradually changes from 0 to 1 from the starting time to the ending time, and at the same time, the sum of the first coefficient and the second coefficient is 1; superimpose the first audio data and the second audio data to obtain the fused audio frame.

[0017] According to a third aspect of the present disclosure, a microphone device is provided, comprising the audio buffering apparatus according to the present disclosure.

[0018] According to a fourth aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and at least one memory storing computer executable instructions, wherein the computer executable instructions, when executed by the at least one processor, prompt the at least one processor to execute the audio caching method according to the present disclosure.

[0019] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to execute the audio caching method according to the present disclosure.

[0020] According to a sixth aspect of the present disclosure, a computer program product is provided, comprising computer instructions, which implement the audio caching method according to the present disclosure when executed by at least one processor.

[0021] The technical solution provided by the embodiments of the present disclosure brings at least the following beneficial effects:

[0022] According to the audio caching method and audio caching device of the embodiments of the present disclosure, by determining two consecutive audio frames from the currently collected audio frames, and merging the two into a new, continuously changing fused audio frame, and ensuring that the beginning of the fused audio frame is the same as the beginning of the first audio frame of the two, and the end of the fused audio frame is the same as the end of the second audio frame of the two, the beginning of the fused audio frame can be kept continuous with the previous audio frame of the first audio frame, and the end of the fused audio frame can be kept continuous with the next audio frame of the second audio frame, thereby achieving a continuous transition between the newly generated fused audio frame and the currently collected audio frame, reducing audio noise, and helping to ensure communication quality.

[0023] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute improper limitations on the present disclosure.

[0025] Figure 1 is a flowchart illustrating an audio caching method according to an exemplary embodiment of the present disclosure.

[0026] Figure 2 is a schematic diagram showing downsampling according to an exemplary embodiment of the present disclosure.

[0027] Figure 3 is a schematic diagram showing upsampling according to an exemplary embodiment of the present disclosure.

[0028] Figure 4 is a block diagram illustrating an audio caching apparatus according to an exemplary embodiment of the present disclosure.

[0029] Figure 5 is a block diagram illustrating a microphone apparatus according to an exemplary embodiment of the present disclosure.

[0030] Figure 6 is a block diagram illustrating an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION

[0031] In order to enable ordinary persons in the art to better understand the technical solutions of the present disclosure, the technical solutions in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the accompanying drawings.

[0032] It should be noted that the terms "first", "second", etc. in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The implementation methods described in the following examples do not represent all implementation methods consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the attached claims.

[0033] It should be noted that the phrase "at least one of the items" in the present disclosure includes three types of parallel situations: "any one of the items", "a combination of any number of the items", and "all of the items". For example, "including at least one of A and B" includes the following three parallel situations: (1) including A; (2) including B; (3) including A and B. Another example is "executing at least one of step 1 and step 2" which means the following three parallel situations: (1) executing step 1; (2) executing step 2; (3) executing step 1 and step 2.

[0034] In office environments, online meetings are increasingly used. If multiple people hold online meetings with others in a conference room, professional conference microphones are often needed to obtain a better sound pickup experience. Compared with ordinary microphones, conference microphones have a wider sound pickup range and better sound quality. Conference microphones are often connected to computers or tablets using USB interfaces. This method is simple to connect and the method of use is very intuitive for users.

[0035] As a slave device of a computer or tablet, the conference microphone will collect audio signals according to its own clock signal, store them in the buffer, and then the master device will take the audio signals in the buffer according to its own clock signal. However, the master and slave devices have independent clock signals, and the slight difference between them will cause the buffer of the conference microphone to be filled up or insufficient data after a long period of accumulation.

[0036] Specifically, if the clock of the slave device is faster than that of the master device, the rate at which the buffer inputs audio signals (i.e., the rate at which the slave device collects audio signals) will be greater than the rate at which it outputs audio signals (i.e., the rate at which the master device takes away audio signals), and the master device fails to take away all signals in time, so the audio signals accumulate in the buffer. When the buffer is full, the latest received frame of signal is often discarded. After the earliest frame of signal is taken away by the master device, the buffer will free up space and can continue to receive signals normally until the buffer is filled again.

[0037] If the clock of the slave device is slower than that of the master device, the rate of the input audio signal of the buffer will be lower than the rate of its output audio signal, and the collected signal is insufficient, resulting in the buffer being empty. In this regard, the relevant technology often sends some all-zero data to fill it.

[0038] The existing processing methods for the above two situations will cause speech discontinuity, and the human ear is very sensitive to speech discontinuity, so it is easy to perceive obvious noise, affecting the communication quality.

[0039] According to the audio caching method and audio caching device of the exemplary embodiments of the present disclosure, by determining two consecutive audio frames from the currently collected audio frames, and merging the two into a new, continuously changing fused audio frame, and ensuring that the beginning of the fused audio frame is the same as the beginning of the first audio frame of the two, and the end of the fused audio frame is the same as the end of the second audio frame of the two, the beginning of the fused audio frame can be kept continuous with the previous audio frame of the first audio frame, and the end of the fused audio frame can be kept continuous with the next audio frame of the second audio frame, thereby achieving a continuous transition between the newly generated fused audio frame and the currently collected audio frame, reducing audio noise, and helping to ensure communication quality. Specifically, in terms of timing, when the first audio frame is in front and the second audio frame is in the back, the two audio frames can be directly compressed into a fused audio frame to achieve downsampling, which can meet the situation where the clock of the slave device is faster than the clock of the master device; when the second audio frame is in front and the first audio frame is in the back, the beginning of the fused audio frame is continuous with the second audio frame, and the end is continuous with the first audio frame, and it can be smoothly inserted between the second audio frame and the first audio frame to achieve upsampling, which can meet the situation where the clock of the slave device is slower than the clock of the master device. Therefore, according to the audio caching method and audio caching device of the exemplary embodiments of the present disclosure, it is possible to achieve downsampling or upsampling without affecting the audio quality to reduce or complete the sampling point data of the audio.

[0040] Next, we will refer to Figures 1 to 6 An audio caching method and an audio caching device according to exemplary embodiments of the present disclosure are described in detail.

[0041] Figure 1 is a flow chart showing an audio caching method according to an exemplary embodiment of the present disclosure. It should be understood that the audio caching method according to an exemplary embodiment of the present disclosure can be implemented in a microphone device.

[0042] Reference Figure 1In step 101, in response to determining that the buffer area meets the set condition, a continuous first audio frame and a second audio frame are determined from the collected audio frames. For situations where the relationship between the clock speed is different, corresponding set conditions are configured respectively, and the first audio frame and the second audio frame can be reasonably determined accordingly to meet the needs of upsampling or downsampling. It should be understood that the first audio frame can be before the second audio frame or after the second audio frame to achieve different sampling, as long as the two are guaranteed to be continuous.

[0043] In step 102, the first audio frame and the second audio frame are fused to obtain a continuously changing fused audio frame, the beginning of the fused audio frame is the same as the beginning of the first audio frame, and the end of the fused audio frame is the same as the end of the second audio frame. In other words, the fused audio frame obtained by the fusion process has two characteristics. First, its beginning is the same as the beginning of the first audio frame, and its end is the same as the end of the second audio frame, which can ensure that the beginning of the fused audio frame is continuous with the previous audio frame of the first audio frame, and the end is continuous with the next audio frame of the second audio frame, thereby achieving a continuous transition between the newly generated fused audio frame and the currently collected audio frame; second, the obtained fused audio frame itself is continuously changing, thereby ensuring that after adding it to the collected audio frame, the overall audio is continuous, which can reduce audio noise and ensure communication quality.

[0044] Optionally, step 102 includes: aligning the start time and the end time of the first audio frame and the second audio frame respectively, for example, returning the start time of both to zero, so that the time of the two audio frames that were originally in sequence is synchronized; determining the product of the first audio frame and the first coefficient to obtain the first audio data, determining the product of the second audio frame and the second coefficient to obtain the second audio data; wherein the first coefficient gradually changes from 1 to 0 from the start time to the end time, and the second coefficient gradually changes from 0 to 1 from the start time to the end time, and at the same time, the sum of the first coefficient and the second coefficient is 1; superimposing the first audio data and the second audio data to obtain a fused audio frame. By performing weighted summation on the data of the time-synchronized first audio frame and the second audio frame at each moment, and ensuring that the first coefficient (i.e., the weight of the first audio frame) and the second coefficient (i.e., the weight of the second audio frame) gradually change between 0 and 1 in opposite trends, and that the sum of the two at the same moment is 1, balanced fusion of the first audio frame and the second audio frame can be achieved, continuous change, and ensuring that the beginning of the fused audio frame is the same as the beginning of the first audio frame, and the end is the same as the end of the second audio frame, thereby ensuring that the fusion effect meets the requirements.

[0045] It should be noted that as long as it can be ensured that the first coefficient and the second coefficient keep changing gradually from the beginning to the end, that is, the change is continuous, the specific change characteristics are not limited, for example, it can be a linear change or a nonlinear change.

[0046] In step 103, the fused audio frame is stored in the buffer area. Finally, the generated fused audio frame is stored in the buffer area, and the current audio buffering can be realized. It should be understood that for downsampling and upsampling, the storage method is different, which will be described below in conjunction with the specific situation.

[0047] Figure 2 is a schematic diagram showing downsampling according to an exemplary embodiment of the present disclosure.

[0048] Figure 3 is a schematic diagram showing upsampling according to an exemplary embodiment of the present disclosure.

[0049] Next, combine Figure 2 and Figure 3 , step 101 and step 103 are specifically introduced for the cases where the clock of the slave device is faster and slower than the clock of the master device, respectively.

[0050] In the case where the clock of the slave device is faster than the clock of the master device, as mentioned above, the audio signal will accumulate more and more in the buffer area. In this regard, step 101 specifically includes: in response to the acquisition of a new audio frame (corresponding to Figure 2 n) and determine that the available capacity in the buffer is less than the set capacity, the latest audio frame in the buffer (corresponding to Figure 2 The audio frame n-1 in the buffer is recorded as the first audio frame, and the newly collected audio frame is recorded as the second audio frame. That is to say, if the available capacity in the buffer is limited and less than the set capacity when the audio frame n is collected, then in step 102, the previous audio frame n-1 and the newly collected audio frame n that has not yet been stored in the buffer are merged to obtain a fused audio frame. Correspondingly, step 103 is specifically performed as follows: overwriting the first audio frame with the fused audio frame. That is to say, unlike the direct discarding of audio frame n in the related art, according to an exemplary embodiment of the present disclosure, audio frame n and audio frame n-1 can be merged, and the merged fused audio frame can be used to overwrite the existing audio frame n-1. There is no need to store the separate audio frame n in the buffer, and downsampling can be achieved. After the fused audio frame is stored in the buffer, the earliest audio frame in the buffer is taken away by the main device, so that the buffer has space again, and the signal can continue to be received normally. Refer to Figure 2 , the audio frame n+1 may be stored after the audio frame is fused. It should be understood that storing the audio frame n+1 after the audio frame is fused means that corresponding timing identifiers are configured for the two when storing.

[0051] It should be noted that the set capacity reflects a standard of a smaller available capacity of the buffer area, which can be set as needed and is not limited by the present disclosure. As an example, the set capacity is equal to the size of an audio frame, that is, when the buffer area is not large enough to store a complete audio frame, the newly collected audio frame may be discarded. Fusion processing can be performed to timely implement downsampling, reduce unnecessary operations, and improve operating efficiency.

[0052] In the case where the clock of the slave device is slower than the clock of the master device, as described above, the collected signal will be insufficient, resulting in the buffer becoming empty. In this regard, step 101 specifically includes: in response to determining that there are at least two audio frames in the buffer and the available capacity in the buffer is greater than the size of at least one audio frame, the latest audio frame (corresponding to Figure 3 The audio frame n in the first audio frame is recorded as the first audio frame, and the previous audio frame of the first audio frame (corresponding to Figure 3 That is, if the current buffer still contains at least audio frame n-1 and audio frame n, and the remaining available capacity of the buffer is sufficient to store a new audio frame, then in step 102, the subsequent audio frame n and the previous audio frame n-1 are merged to obtain a merged audio frame that can be smoothly inserted between the two. Figure 3 , step 103 is specifically performed as follows: storing the fused audio frame between the second audio frame and the first audio frame. It should be understood that storing the fused audio frame between the second audio frame and the first audio frame means configuring the timing mark of the fused audio frame to be between the timing mark of the second audio frame and the timing mark of the first audio frame when saving.

[0053] It should be noted that the setting condition in step 101 at this time is that there are at least two audio frames in the buffer area and the available space in the buffer area is greater than the size of at least one audio frame, which can ensure that there are enough audio frames for generating fused audio frames, and ensure that there is enough space in the buffer area for storing newly generated fused audio frames. As long as this condition is met, the upsampling operation can be performed. As an example, the setting condition may be that there are only two audio frames in the buffer area. At this time, the audio frames in the buffer area are just enough to generate a fused audio frame, which can effectively avoid sending an all-zero signal to the main device, and can perform upsampling operations only when necessary, reducing invalid operations and improving operating efficiency.

[0054] In summary, according to the audio caching method of the exemplary embodiment of the present disclosure, it is possible to directly select downsampling or upsampling on the slave device according to the state of the buffer area, merge or generate a frame of data, so that the data can meet the master device's demand for the slave device's audio data. In addition, this method can keep the data continuous, and the human ear cannot hear its changes, and does not affect the sound quality.

[0055] Figure 4is a block diagram showing an audio buffering device according to an exemplary embodiment of the present disclosure. It should be understood that the audio buffering device according to the exemplary embodiment of the present disclosure can be implemented in a microphone device in a software, hardware, or a combination of software and hardware.

[0056] Reference Figure 4 The audio caching device 400 includes a determining unit 401, a fusion unit 402, and a storage unit 403.

[0057] The determination unit 401 may determine the continuous first audio frame and the second audio frame from the collected audio frames in response to determining that the buffer area meets the set conditions. For situations where the clock speed relationship is different, the corresponding set conditions are configured respectively, and the first audio frame and the second audio frame can be reasonably determined accordingly to meet the needs of upsampling or downsampling. It should be understood that the first audio frame can be before the second audio frame or after the second audio frame to achieve different sampling, as long as the two are guaranteed to be continuous.

[0058] The fusion unit 402 can perform fusion processing on the first audio frame and the second audio frame to obtain a continuously changing fused audio frame, the beginning of the fused audio frame is the same as the beginning of the first audio frame, and the end of the fused audio frame is the same as the end of the second audio frame. In other words, the fused audio frame obtained by the fusion processing has two characteristics. First, its beginning is the same as the beginning of the first audio frame, and its end is the same as the end of the second audio frame, which can ensure that the beginning of the fused audio frame is continuous with the previous audio frame of the first audio frame, and the end is continuous with the next audio frame of the second audio frame, thereby achieving a continuous transition between the newly generated fused audio frame and the currently collected audio frame; second, the obtained fused audio frame itself is continuously changing, thereby ensuring that after adding it to the collected audio frame, the overall audio is continuous, which can reduce audio noise and ensure communication quality.

[0059] Optionally, the fusion processing performed by the fusion unit 402 specifically includes: aligning the start time and the end time of the first audio frame and the second audio frame, for example, returning the start time of both to zero, so that the time of the two audio frames that were originally in sequence is synchronized; determining the product of the first audio frame and the first coefficient to obtain the first audio data, determining the product of the second audio frame and the second coefficient to obtain the second audio data; wherein the first coefficient gradually changes from 1 to 0 from the start time to the end time, and the second coefficient gradually changes from 0 to 1 from the start time to the end time, and at the same time, the sum of the first coefficient and the second coefficient is 1; superimposing the first audio data and the second audio data to obtain a fused audio frame. By performing weighted summation on the data of the time-synchronized first audio frame and the second audio frame at each moment, and ensuring that the first coefficient (i.e., the weight of the first audio frame) and the second coefficient (i.e., the weight of the second audio frame) gradually change between 0 and 1 in opposite trends, and that the sum of the two at the same moment is 1, balanced fusion of the first audio frame and the second audio frame can be achieved, continuous change, and ensuring that the beginning of the fused audio frame is the same as the beginning of the first audio frame, and the end is the same as the end of the second audio frame, thereby ensuring that the fusion effect meets the requirements.

[0060] It should be noted that as long as it can be ensured that the first coefficient and the second coefficient keep changing gradually from the beginning to the end, that is, the change is continuous, the specific change characteristics are not limited, for example, it can be a linear change or a nonlinear change.

[0061] The storage unit 403 can store the fused audio frame in the buffer area. Finally, the generated fused audio frame is stored in the buffer area, and the current audio buffer can be realized. It should be understood that for downsampling and upsampling, the storage method is different, which will be described below in conjunction with specific circumstances.

[0062] In the case where the clock of the slave device is faster than the clock of the master device, as described above, the audio signal will accumulate more and more in the buffer area. In this regard, the determination unit 401 may respond to the acquisition of a new audio frame (corresponding to Figure 2 n) and determine that the available capacity in the buffer is less than the set capacity, the latest audio frame in the buffer (corresponding to Figure 2The audio frame n-1 in the buffer is recorded as the first audio frame, and the newly collected audio frame is recorded as the second audio frame. That is to say, if when the audio frame n is collected, the available capacity in the buffer is limited and less than the set capacity, the fusion unit 402 may fuse the previous audio frame n-1 and the newly collected audio frame n that has not yet been stored in the buffer to obtain a fused audio frame. Accordingly, the storage unit 403 may overwrite the first audio frame with the fused audio frame. That is to say, unlike the direct discarding of audio frame n in the related art, according to an exemplary embodiment of the present disclosure, audio frame n and audio frame n-1 may be merged, and the merged fused audio frame may be used to overwrite the existing audio frame n-1. There is no need to store the separate audio frame n in the buffer, and downsampling may be achieved. After the fused audio frame is stored in the buffer, the earliest audio frame in the buffer is taken away by the master device, so that space is freed up in the buffer and the signal can continue to be received normally. Refer to Figure 2 , the storage unit 403 may continue to store the audio frame n+1 after the fused audio frame. It should be understood that storing the audio frame n+1 after the fused audio frame means configuring corresponding timing identifiers for the two when storing.

[0063] It should be noted that the set capacity reflects a standard of a smaller available capacity of the buffer area, which can be set as needed and is not limited by the present disclosure. As an example, the set capacity is equal to the size of an audio frame, that is, when the buffer area is not large enough to store a complete audio frame, the newly collected audio frame may be discarded. Fusion processing can be performed to timely implement downsampling, reduce unnecessary operations, and improve operating efficiency.

[0064] In the case where the clock of the slave device is slower than the clock of the master device, as described above, the collected signal will be insufficient, resulting in the buffer becoming empty. In response to determining that there are at least two audio frames in the buffer and the available capacity in the buffer is greater than the size of at least one audio frame, the determining unit 401 may store the latest audio frame (corresponding to Figure 3 The audio frame n in the first audio frame is recorded as the first audio frame, and the previous audio frame of the first audio frame (corresponding to Figure 3 That is, if the current buffer still contains at least audio frame n-1 and audio frame n, and the remaining available capacity of the buffer is sufficient to store a new audio frame, the fusion unit 402 may fuse the subsequent audio frame n with the previous audio frame n-1 to obtain a fused audio frame that can be smoothly inserted between the two. Figure 3 The storage unit 403 may store the fused audio frame between the second audio frame and the first audio frame. It should be understood that storing the fused audio frame between the second audio frame and the first audio frame means configuring the timing mark of the fused audio frame to be between the timing mark of the second audio frame and the timing mark of the first audio frame when storing.

[0065] It should be noted that the setting condition used by the determination unit 401 at this time is that there are at least two audio frames in the buffer area and the available space in the buffer area is greater than the size of at least one audio frame, which can ensure that there are enough audio frames for generating fused audio frames, and ensure that there is enough space in the buffer area for storing the newly generated fused audio frames. As long as this condition is met, the upsampling operation can be performed. As an example, the setting condition may be that there are only two audio frames in the buffer area. At this time, the audio frames in the buffer area are just enough to generate a fused audio frame, which can effectively avoid sending an all-zero signal to the main device, and can perform upsampling operations only when necessary, reducing invalid operations and improving operating efficiency.

[0066] Figure 5 is a block diagram illustrating a microphone apparatus according to an exemplary embodiment of the present disclosure.

[0067] Reference Figure 5 , the microphone device 500 includes the audio buffer device 400 according to an exemplary embodiment of the present disclosure.

[0068] As an example, the microphone device 500 can be applied to video conferencing, including a signal acquisition module, a calculation module, and a cache module. The signal acquisition module is used to collect audio signals and obtain multiple continuous audio frames. The cache module is used to cache these audio frames for use by the main device connected to the microphone device 500. The calculation module corresponds to the audio cache device 400 according to the exemplary embodiment of the present disclosure, and can perform the aforementioned downsampling or upsampling operation in combination with the state of the cache module, and store the obtained fused audio frame into the cache module. The three cooperate with each other to keep the audio data continuous during the video conference, and the human ear cannot hear its changes, which does not affect the communication sound quality.

[0069] Figure 6 is a block diagram of an electronic device according to an exemplary embodiment of the present disclosure.

[0070] Reference Figure 6 The electronic device 600 includes at least one memory 601 and at least one processor 602, wherein the at least one memory 601 stores a set of computer executable instructions. When the set of computer executable instructions is executed by the at least one processor 602, an audio caching method according to an exemplary embodiment of the present disclosure is executed.

[0071] As an example, the electronic device 600 may be a PC, a tablet device, a personal digital assistant, a smart phone, or other device capable of executing the above instruction set. Here, the electronic device 600 is not necessarily a single electronic device, but may also be any device or circuit capable of executing the above instruction (or instruction set) individually or in combination. The electronic device 600 may also be part of an integrated control system or system manager, or may be configured as a portable electronic device interconnected with a local or remote (e.g., via wireless transmission) interface.

[0072] In the electronic device 600, the processor 602 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller or a microprocessor. As an example and not limitation, the processor may also include an analog processor, a digital processor, a microprocessor, a multi-core processor, a processor array, a network processor, etc.

[0073] The processor 602 may execute instructions or codes stored in the memory 601, which may also store data. Instructions and data may also be sent and received over a network via a network interface device, which may employ any known transmission protocol.

[0074] The memory 601 may be integrated with the processor 602, for example, by placing RAM or flash memory within an integrated circuit microprocessor or the like. In addition, the memory 601 may include a separate device, such as an external disk drive, a storage array, or any other storage device that can be used by a database system. The memory 601 and the processor 602 may be operatively coupled, or may communicate with each other, such as through an I / O port, a network connection, etc., so that the processor 602 can read files stored in the memory.

[0075] In addition, the electronic device 600 may further include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, a mouse, a touch input device, etc.) All components of the electronic device 600 may be connected to each other via a bus and / or a network.

[0076] According to an exemplary embodiment of the present disclosure, a computer-readable storage medium may also be provided, and when the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the audio caching method according to the exemplary embodiment of the present disclosure. Examples of computer-readable storage media here include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disk storage, hard disk drive (HDD), solid state drive (SSD), card storage (such as, multimedia card, secure digital (SD) card or extreme digital (XD) card), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid state disk and any other device, any other device is configured to store computer programs and any associated data, data files and data structures in a non-transitory manner and provide the computer programs and any associated data, data files and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the above-mentioned computer-readable storage medium can be run in an environment deployed in a computer device such as a client, a host, an agent device, a server, etc. In addition, in one example, the computer program and any associated data, data files and data structures are distributed on a networked computer system, so that the computer program and any associated data, data files and data structures are stored, accessed and executed in a distributed manner by one or more processors or computers.

[0077] According to an exemplary embodiment of the present disclosure, a computer program product may also be provided. The computer program product includes computer instructions. When the computer instructions are executed by at least one processor, the at least one processor is prompted to perform the audio caching method according to the exemplary embodiment of the present disclosure.

[0078] According to the audio caching method, apparatus, microphone device, electronic device and storage medium of the exemplary embodiments of the present disclosure, two consecutive audio frames are determined from the currently collected audio frames, and the two are merged into a new, continuously changing fused audio frame, and the beginning of the fused audio frame is ensured to be the same as the beginning of the first audio frame of the two, and the end of the fused audio frame is the same as the end of the second audio frame of the two. The beginning of the fused audio frame can be kept continuous with the previous audio frame of the first audio frame, and the end of the fused audio frame can be kept continuous with the next audio frame of the second audio frame, thereby achieving a continuous transition between the newly generated fused audio frame and the currently collected audio frame, reducing audio noise, and being able to reduce or complete the audio sampling point data by downsampling or upsampling without affecting the audio quality.

[0079] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the following claims.

[0080] It should be understood that the present disclosure is not limited to the exact structures that have been described above and shown in the drawings, and that various modifications and changes may be made without departing from the scope thereof. The scope of the present disclosure is limited only by the appended claims.

Claims

1. An audio caching method, characterized in that: The audio caching method comprises: In response to determining that the buffer area satisfies a set condition, determining a first audio frame and a second audio frame that are consecutive from the collected audio frames; Performing a fusion process on the first audio frame and the second audio frame to obtain a continuously changing fused audio frame, wherein a beginning of the fused audio frame is the same as a beginning of the first audio frame, and an end of the fused audio frame is the same as an end of the second audio frame; Storing the fused audio frame in the buffer; The fusing the first audio frame and the second audio frame to obtain a continuously changing fused audio frame includes: Aligning the start time and the end time of the first audio frame and the second audio frame respectively; Determine the product of the first audio frame and the first coefficient to obtain first audio data, and determine the product of the second audio frame and the second coefficient to obtain second audio data; wherein the first coefficient gradually changes from 1 to 0 from the start time to the end time, and the second coefficient gradually changes from 0 to 1 from the start time to the end time, and at the same time, the sum of the first coefficient and the second coefficient is 1; The first audio data and the second audio data are superimposed to obtain the fused audio frame.

2. The audio caching method according to claim 1, characterized in that: In response to determining that the buffer area satisfies a set condition, determining a continuous first audio frame and a second audio frame from the collected audio frames includes: In response to collecting a new audio frame and determining that the available capacity in the buffer area is less than a set capacity, recording the latest audio frame in the buffer area as the first audio frame, and recording the collected new audio frame as the second audio frame; The step of storing the fused audio frame into the buffer area comprises: The first audio frame is overwritten with the fused audio frame.

3. The audio caching method according to claim 2, characterized in that: The set capacity is equal to the size of one audio frame.

4. The audio caching method according to claim 1, wherein: In response to determining that the buffer area satisfies a set condition, determining a continuous first audio frame and a second audio frame from the collected audio frames includes: In response to determining that at least two audio frames are stored in the buffer and the available capacity in the buffer is greater than the size of at least one audio frame, recording a latest audio frame of the at least two audio frames as the first audio frame, and recording an audio frame preceding the first audio frame as the second audio frame; The step of storing the fused audio frame into the buffer area comprises: The fused audio frame is stored between the second audio frame and the first audio frame.

5. The audio caching method according to claim 4, characterized in that: The determining that there are at least two audio frames in the buffer area and that the available space in the buffer area is larger than the size of at least one audio frame includes: It is determined that there are only two audio frames in the buffer area.

6. An audio caching device, characterized in that: The audio caching device comprises: A determining unit is configured to: in response to determining that the buffer area satisfies a set condition, determine a continuous first audio frame and a second audio frame from the collected audio frames; a fusion unit configured to: perform fusion processing on the first audio frame and the second audio frame to obtain a continuously changing fused audio frame, wherein a beginning of the fused audio frame is the same as a beginning of the first audio frame, and an end of the fused audio frame is the same as an end of the second audio frame; A storage unit is configured to: store the fused audio frame into the buffer area; The fusion unit is further configured to: Aligning the start time and the end time of the first audio frame and the second audio frame respectively; Determine the product of the first audio frame and the first coefficient to obtain first audio data, and determine the product of the second audio frame and the second coefficient to obtain second audio data; wherein the first coefficient gradually changes from 1 to 0 from the start time to the end time, and the second coefficient gradually changes from 0 to 1 from the start time to the end time, and at the same time, the sum of the first coefficient and the second coefficient is 1; The first audio data and the second audio data are superimposed to obtain the fused audio frame.

7. The audio buffering device according to claim 6, characterized in that: The determining unit is further configured to: in response to collecting a new audio frame and determining that the available capacity in the buffer area is less than a set capacity, record the latest audio frame in the buffer area as the first audio frame, and record the collected new audio frame as the second audio frame; The storage unit is further configured to: overwrite the first audio frame with the fused audio frame.

8. The audio buffering device according to claim 7, characterized in that: The set capacity is equal to the size of one audio frame.

9. The audio buffering device according to claim 6, wherein: The determining unit is further configured to: in response to determining that there are at least two audio frames in the buffer area and the available capacity in the buffer area is greater than the size of at least one audio frame, record the latest audio frame of the at least two audio frames as the first audio frame, and record the previous audio frame of the first audio frame as the second audio frame; The storage unit is further configured to store the fused audio frame between the second audio frame and the first audio frame.

10. The audio buffering device according to claim 9, characterized in that: The determining unit is further configured to: In response to determining that there are only two audio frames in the buffer area, the latest audio frame of the two audio frames is recorded as the first audio frame, and the previous audio frame of the first audio frame is recorded as the second audio frame.

11. A microphone device, characterized in that: include: An audio buffering device as claimed in any one of claims 6 to 10.

12. An electronic device, characterized in that: include: at least one processor; at least one memory storing computer executable instructions, When the computer executable instructions are executed by the at least one processor, the at least one processor is prompted to perform the audio caching method according to any one of claims 1 to 5.

13. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by at least one processor, the at least one processor is prompted to perform the audio caching method according to any one of claims 1 to 5.

14. A computer program product comprising computer instructions, characterized in that When the computer instructions are executed by at least one processor, the audio buffering method according to any one of claims 1 to 5 is implemented.

Images