Audio signal processing method and apparatus, communication device, communication system, and storage medium

Abstract

The present disclosure provides an audio signal processing method and apparatus, a communication device, a communication system, and a storage medium. The method comprises: determining a window function length value required for processing a first audio signal, and determining a first window function, wherein the first window function comprises a reference variable value related to the window function length value, the reference variable value is less than the window function length value, and the reference variable value is used for determining a delay required for restoring the first audio signal; and processing the first audio signal on the basis of the first window function to obtain a spectral coefficient related to the first audio signal. The method of the present disclosure can effectively reduce the delay for audio signal processing.

Description

Audio signal processing method and device, communication equipment, communication system, and storage medium Technical Field

[0001] The present disclosure relates to the field of communication technology, and in particular to an audio signal processing method and apparatus, communication equipment, a communication system, and a storage medium. Background Art

[0002] Digitized audio signals typically have large data volumes, making them unsuitable for storage and transmission. Therefore, digital audio compression coding techniques are often used to compress and encode audio signals. This technology has become a crucial audio signal processing technique.

[0003] Summary of the Invention

[0004] The embodiments of the present disclosure provide an audio signal processing method, apparatus, device, chip system, storage medium, computer program, and computer program product, which can be applied in the field of communication technology. When processing an audio signal, the audio signal is first divided into audio signals of predetermined time periods, so as to solve the technical problem that "the compression and encoding process of the audio signal will introduce a delay, thereby affecting the timeliness of audio signal recovery."

[0005] The present disclosure provides an audio signal processing method and apparatus, a communication device, a communication system, and a storage medium.

[0006] According to a first aspect of an embodiment of the present disclosure, a method for processing an audio signal is proposed, including: determining a window function length value required to process a first audio signal; determining a first window function, wherein the first window function includes: a reference variable value related to the window function length value, the reference variable value being less than the window function length value, and the reference variable value being used to determine a time delay required to restore the first audio signal; and processing the first audio signal according to the first window function to obtain spectral coefficients related to the first audio signal.

[0007] According to a second aspect of an embodiment of the present disclosure, a method for processing an audio signal is proposed, including: obtaining spectral coefficients related to a first audio signal; determining a first window function, wherein the first window function includes: a reference variable value related to a window function length value used to process the first audio signal, the reference variable value being less than the window function length value, and the reference variable value being used to determine a time delay required to restore the first audio signal; and processing the spectral coefficients according to the first window function to obtain the first audio signal.

[0008] According to a third aspect of an embodiment of the present disclosure, a method for processing an audio signal is proposed, including: a first communication device, used to determine a window function length value required to process a first audio signal, and determine a first window function, wherein the first window function includes: a reference variable value related to the window function length value, the reference variable value is less than the window function length value, the reference variable value is used to determine the time delay required to restore the first audio signal, and process the first audio signal according to the first window function to obtain spectral coefficients related to the first audio signal; a second communication device, used to obtain spectral coefficients related to the first audio signal, and process the spectral coefficients according to the first window function to obtain the first audio signal.

[0009] According to a fourth aspect of an embodiment of the present disclosure, an audio signal processing device is proposed, including: a processing module, used to determine a window function length value required to process a first audio signal, and determine a first window function, wherein the first window function includes: a reference variable value related to the window function length value, the reference variable value is less than the window function length value, the reference variable value is used to determine the time delay required to restore the first audio signal, and process the first audio signal according to the first window function to obtain spectral coefficients related to the first audio signal.

[0010] According to a fifth aspect of an embodiment of the present disclosure, an audio signal processing device is proposed, including: a processing module, used to obtain spectral coefficients related to a first audio signal and determine a first window function, wherein the first window function includes: a reference variable value related to a window function length value used to process the first audio signal, the reference variable value is less than the window function length value, the reference variable value is used to determine a time delay required to restore the first audio signal, and processing the spectral coefficients according to the first window function to obtain the first audio signal.

[0011] According to the sixth aspect of an embodiment of the present disclosure, a communication device is proposed, comprising: one or more processors; wherein the processor is used to call instructions to enable the communication device to execute the audio signal processing method of any one of the first aspect, the second aspect, and the third aspect.

[0012] According to the seventh aspect of an embodiment of the present disclosure, a communication system is proposed, characterized in that it includes a first communication device and a second communication device, wherein the first communication device is configured to implement the audio signal processing method of the first aspect, and the second communication device is configured to implement the audio signal processing method of the second aspect.

[0013] According to an eighth aspect of an embodiment of the present disclosure, a storage medium is proposed, which stores instructions, and is characterized in that when the instructions are executed on a communication device, the communication device executes an audio signal processing method as described in any one of the first, second, and third aspects. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the background technology, the drawings required for use in the embodiments of the present disclosure or the background technology will be described below.

[0015] FIG1 is a schematic diagram of the architecture of a communication system according to an embodiment of the present disclosure;

[0016] FIG2 is a schematic diagram of windowing a signal in the MDCT module in AVS3;

[0017] FIG3A is an interactive schematic diagram illustrating an audio signal processing method according to an embodiment of the present disclosure;

[0018] FIG3B is an interactive schematic diagram illustrating an audio signal processing method according to another embodiment of the present disclosure;

[0019] FIG4A is an interactive schematic diagram illustrating an audio signal processing method according to another embodiment of the present disclosure;

[0020] FIG4B is an interactive schematic diagram illustrating an audio signal processing method according to another embodiment of the present disclosure;

[0021] FIG5A is an interactive schematic diagram illustrating an audio signal processing method according to another embodiment of the present disclosure;

[0022] FIG5B is an interactive schematic diagram illustrating an audio signal processing method according to another embodiment of the present disclosure;

[0023] FIG6A is a schematic diagram of a flow chart of a time domain signal encoding process according to an embodiment of the present disclosure;

[0024] FIG6B is a schematic diagram of a function of a low latency window in an embodiment of the present disclosure;

[0025] FIG6C is a schematic diagram of a decoding process flow diagram according to an embodiment of the present disclosure;

[0026] FIG6D is a schematic diagram of signal overlap-addition in an embodiment of the present disclosure;

[0027] FIG7A is a waveform diagram of a Wav test file 1 according to an embodiment of the present disclosure;

[0028] FIG7B is a schematic diagram of an output signal 1 corresponding to the waveform of Wav test file 1 in an embodiment of the present disclosure;

[0029] FIG7C is a schematic diagram of an output signal 2 corresponding to the waveform of Wav test file 1 according to an embodiment of the present disclosure;

[0030] FIG7D is a waveform diagram of a second Wav test file according to an embodiment of the present disclosure;

[0031] FIG7E is a schematic diagram of an output signal 1 corresponding to the waveform of Wav test file 2 in an embodiment of the present disclosure;

[0032] FIG7F is a schematic diagram of an output signal 2 corresponding to the waveform of the Wav test file 2 in an embodiment of the present disclosure;

[0033] FIG7G is a waveform diagram of Wav test file 3 according to an embodiment of the present disclosure;

[0034] FIG7H is a schematic diagram of an output signal 1 corresponding to the waveform of Wav test file 3 in an embodiment of the present disclosure;

[0035] FIG7I is a schematic diagram of an output signal 2 corresponding to the waveform of Wav test file 3 in an embodiment of the present disclosure;

[0036] FIG8A is a schematic structural diagram of an audio signal processing device proposed in an embodiment of the present disclosure;

[0037] FIG8B is a schematic structural diagram of an audio signal processing device proposed in an embodiment of the present disclosure;

[0038] FIG9A is a schematic structural diagram of a communication device proposed in an embodiment of the present disclosure;

[0039] FIG9B is a schematic diagram of the structure of the chip proposed in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0040] The present disclosure provides an audio signal processing method and apparatus, a communication device, a communication system, and a storage medium. In some embodiments, the terms "audio signal processing method" and "information processing method" and "communication method" are interchangeable; the terms "audio signal processing apparatus" and "information processing apparatus" and "communication apparatus" are interchangeable; and the terms "information processing system" and "communication system" are interchangeable.

[0041] The embodiments of the present disclosure are not exhaustive and are merely illustrative of some embodiments, and are not intended to be a specific limitation on the scope of protection of the present disclosure. In the absence of contradiction, each step in a certain embodiment can be implemented as an independent embodiment, and the steps can be arbitrarily combined. For example, a solution after removing some steps in a certain embodiment can also be implemented as an independent embodiment, and the order of the steps in a certain embodiment can be arbitrarily exchanged. In addition, the optional implementation methods in a certain embodiment can be arbitrarily combined; in addition, the embodiments can be arbitrarily combined. For example, some or all steps of different embodiments can be arbitrarily combined, and a certain embodiment can be arbitrarily combined with the optional implementation methods of other embodiments.

[0042] In each embodiment of the present disclosure, unless otherwise specified or provided for by logic, the terms and / or descriptions between the embodiments are consistent and can be referenced by each other. The technical features in different embodiments can be combined to form a new embodiment based on their inherent logical relationships.

[0043] The terms used in the embodiments of the present disclosure are only for the purpose of describing specific embodiments and are not intended to limit the present disclosure.

[0044] In the embodiments of the present disclosure, unless otherwise specified, elements expressed in the singular, such as "a", "an", "the", "above", "said", "the", "the", etc., may mean "one and only one", or "one or more", "at least one", etc. For example, when using articles such as "a", "an", "the" in English in translation, the noun following the article may be understood as a singular expression or a plural expression.

[0045] In the embodiments of the present disclosure, “plurality” refers to two or more.

[0046] In some embodiments, the terms "at least one of", "at least one of", "at least one of", "one or more", "a plurality of", "multiple", etc. can be used interchangeably.

[0047] In the embodiments of the present disclosure, descriptions such as “at least one of A, B, C…”, “A and / or B and / or C…”, etc. include the situation where any one of A, B, C… exists alone, and also include any combination of any multiple of A, B, C…, and each situation can exist alone; for example, “at least one of A, B, C” includes the situation where A exists alone, B exists alone, C exists alone, the combination of A and B, the combination of A and C, the combination of B and C, and the combination of A, B, and C; for example, A and / or B includes the situation where A exists alone, B exists alone, and the combination of A and B.

[0048] In some embodiments, descriptions such as "in one case A, in another case B," or "in response to one case A, in response to another case B," may include the following technical solutions depending on the situation: executing A independently of B (in some embodiments, A); executing B independently of A (in some embodiments, B); selectively executing A and B (in some embodiments, selecting between A and B); and executing both A and B (in some embodiments, A and B). The same applies when there are more branches, such as A, B, and C.

[0049] The prefixes such as "first" and "second" in the embodiments of the present disclosure are only used to distinguish different description objects and do not constitute any restriction on the position, order, priority, quantity or content of the description objects. For the statement of the description object, please refer to the description in the context of the claims or embodiments, and no unnecessary restriction should be constituted due to the use of prefixes. For example, if the description object is a "field", the ordinal number before the "field" in the "first field" and the "second field" does not limit the position or order between the "fields". "First" and "second" do not limit whether the "fields" they modify are in the same message, nor do they limit the order of the "first field" and the "second field". For another example, if the description object is a "level", the ordinal number before the "level" in the "first level" and the "second level" does not limit the priority between the "levels". For another example, the number of description objects is not limited by the ordinal number and can be one or more. Taking "first device" as an example, the number of "devices" can be one or more. In addition, the objects modified by different prefixes can be the same or different. For example, if the description object is "device", then the "first device" and the "second device" can be the same device or different devices, and their types can be the same or different; for another example, if the description object is "information", then the "first information" and the "second information" can be the same information or different information, and their contents can be the same or different.

[0050] In some embodiments, “including A,” “comprising A,” “used to indicate A,” and “carrying A” can be interpreted as directly carrying A or indirectly indicating A.

[0051] In some embodiments, terms such as "in response to...", "in response to determining...", "in the case of...", "at the time of...", "when...", "if...", "if...", etc. can be used interchangeably.

[0052] In some embodiments, terms such as "greater than", "greater than or equal to", "not less than", "more than", "more than or equal to", "not less than", "higher than", "higher than or equal to", "not less than", and "above" can be replaced with each other, and terms such as "less than", "less than or equal to", "not greater than", "less than", "less than or equal to", "not more than", "lower than", "lower than or equal to", "not higher than", and "below" can be replaced with each other.

[0053] In some embodiments, devices, etc. can be interpreted as physical or virtual, and their names are not limited to the names recorded in the embodiments. Terms such as "device", "equipment", "device", "circuit", "network element", "node", "function", "unit", "section", "system", "network", "chip", "chip system", "entity", and "subject" can be used interchangeably.

[0054] In some embodiments, "network" can be interpreted as devices included in the network (eg, access network equipment, core network equipment, etc.).

[0055] In some embodiments, the terms "access network device (AN device)", "radio access network device (RAN device)", "base station (BS)", "radio base station" "fixed station", "node", "access point", "transmission point (TP)", "reception point (RP)", "transmission / reception point (TRP)", "panel", "antenna panel", "antenna array", "cell", "macro cell", "small cell", "femto cell", "pico cell", "sector", "cell group", "carrier", "component carrier", "bandwidth part (BWP)" and the like may be used interchangeably.

[0056] In some embodiments, the terms "terminal", "terminal device", "user equipment (UE)", "user terminal", "mobile station (MS)", "mobile terminal (MT)", subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, etc. can be used interchangeably.

[0057] In some embodiments, obtaining data, information, etc. may comply with the laws and regulations of the country where the data is obtained.

[0058] In some embodiments, data, information, etc. may be obtained with the user's consent.

[0059] In addition, each element, each row, or each column in the table of the embodiment of the present disclosure can be implemented as an independent embodiment, and the combination of any elements, any rows, and any columns can also be implemented as an independent embodiment.

[0060] The correspondences shown in the tables of the present disclosure can be configured or predefined. The values ​​of the information in each table are merely examples and can be configured to other values, which are not limited by the present disclosure. When configuring the correspondences between information and parameters, it is not necessarily required to configure all the correspondences shown in each table. For example, in the tables of the present disclosure, the correspondences shown in certain rows may not be configured. For another example, appropriate deformation adjustments can be made based on the above tables, such as splitting, merging, etc. The names of the parameters shown in the titles of the above tables may also adopt other names that can be understood by the communication device, and the values ​​or representations of the parameters may also adopt other values ​​or representations that can be understood by the communication device. When implementing the above tables, other data structures may also be used, such as arrays, queues, containers, stacks, linear lists, pointers, linked lists, trees, graphs, structures, classes, heaps, hash tables or hash tables, etc.

[0061] The predefined in the present disclosure may be understood as defined, predefined, stored, pre-stored, pre-negotiated, pre-configured, solidified, or pre-burned.

[0062] Figure 1 is a schematic diagram of the architecture of a communication system according to an embodiment of the present disclosure. As shown in Figure 1, the communication system 100 may include a terminal 101 and a network device 102. The network device 102 may include at least one of an access network device and a core network device.

[0063] In some embodiments, the terminal 101 includes, for example, a mobile phone, a wearable device, an Internet of Things device, a car with communication function, a smart car, a tablet computer, a computer with wireless transceiver function, a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal device in industrial control, a wireless terminal device in self-driving, a wireless terminal device in remote medical surgery, a wireless terminal device in a smart grid, a wireless terminal device in transportation safety, a wireless terminal device in a smart city, and at least one of a wireless terminal device in a smart home, but is not limited thereto.

[0064] In some embodiments, the access network device is, for example, a node or device that accesses a terminal to a wireless network. The access network device may include an evolved NodeB (eNB) in a 5G communication system, a next generation evolved NodeB (ng-eNB), a next generation NodeB (gNB), a node B (NB), a home node B (HNB), a home evolved nodeB (HeNB), a wireless backhaul device, a radio network controller (RNC), a base station controller (BSC), a base transceiver station (BTS), a base band unit (BBU), a mobile switching center, a base station in a 6G communication system, an open base station (Open RAN), a cloud base station (Cloud RAN), a base station in other communication systems, and at least one of an access node in a WiFi system, but is not limited thereto.

[0065] In some embodiments, the technical solution of the present disclosure can be applied to the Open RAN architecture. In this case, the interfaces between or within the access network devices involved in the embodiments of the present disclosure can be transformed into internal interfaces of the Open RAN, and the processes and information interactions between these internal interfaces can be implemented through software or programs.

[0066] In some embodiments, the access network device can be composed of a centralized unit (CU) and a distributed unit (DU), where the CU can also be called a control unit. The CU-DU structure can be used to split the protocol layer of the access network device, with the functions of some protocol layers centrally controlled by the CU, and the functions of the remaining part or all of the protocol layers distributed in the DU, which is centrally controlled by the CU, but is not limited to this.

[0067] In some embodiments, a core network device may be a single device comprising one or more network elements, or may be a plurality of devices or device groups, each comprising all or part of one or more network elements. A network element may be virtual or physical. The core network may, for example, comprise at least one of an Evolved Packet Core (EPC), a 5G Core Network (5GCN), or a Next Generation Core (NGC).

[0068] It can be understood that the communication system described in the embodiment of the present disclosure is for the purpose of more clearly illustrating the technical solution of the embodiment of the present disclosure, and does not constitute a limitation on the technical solution proposed in the embodiment of the present disclosure. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solution proposed in the embodiment of the present disclosure is also applicable to similar technical problems.

[0069] The following embodiments of the present disclosure may be applied to the communication system 100 shown in Figure 1, or a portion thereof, but are not limited thereto. The entities shown in Figure 1 are illustrative only. The communication system may include all or part of the entities shown in Figure 1, or may include other entities outside of Figure 1. The number and form of the entities may be arbitrary. The connection relationship between the entities is illustrative only. The entities may be connected or disconnected, and the connection may be in any manner, including direct or indirect, wired or wireless.

[0070] The embodiments of the present disclosure can be applied to Long Term Evolution (LTE), LTE-Advanced (LTE-A), LTE-Beyond (LTE-B), SUPER 3G, IMT-Advanced, 4th generation mobile communication system (4G), 5th generation mobile communication system (5G), 5G new radio (NR), future radio access (FRA), new radio access technology (RAT), new radio (NR), new radio access (NX), future generation radio access (FX), Global System for Mobile communications (GSM (registered trademark)), CDMA2000, Ultra Mobile Broadband (UMB), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, Ultra-WideBand (UWB), Bluetooth (registered trademark), Public Land Mobile Network (PLMN) networks, Device-to-Device (D2D) systems, Machine-to-Machine (M2M) systems, Internet of Things (IoT) systems, Vehicle-to-Everything (V2X), systems utilizing other communication methods, and next-generation systems based on and extending these methods. Furthermore, multiple systems may be combined (for example, a combination of LTE or LTE-A with 5G).

[0071] Digitized audio signals typically have large data volumes, making them unsuitable for storage and transmission. Therefore, digital audio compression coding techniques are often used to compress and encode audio signals. This technology has become a crucial audio signal processing technique.

[0072] Optionally, the compression coding technology for digital audio can be, for example, a modified discrete cosine transform (MDCT), which is an algorithm used for multimedia signal compression. MDCT is based on the theory of time-frequency decomposition and is used to divide frequency domain signals into multiple sub-bands for analysis and processing. A window function can be used to divide the time domain signal into a series of time periods set by the signal with a predetermined time period length, and a discrete cosine transform (DCT) is performed on the signal in each time period, and then the spectrum is formed. Due to the use of the window function, audio artifacts caused by DCT mutations can be reduced, thereby improving the quality of the audio.

[0073] Alternatively, Advanced Audio Coding (AAC), Moving Picture Experts Group Audio Layer III (MP3), and AC-3 (AC-3 is a digital multimedia technology) all use the MDCT algorithm for audio signal compression. MDCT technology is also widely used in video coding standards such as H.264 (H.264 is a highly compressed digital video codec standard) and High Efficiency Video Coding (HEVC) for video signal compression. MDCT has also been widely used in spectral analysis, artificial speech synthesis, and imaging.

[0074] As shown in FIG2 , FIG2 is a schematic diagram of windowing a signal in the MDCT module in AVS3, wherein AVS3 is an audio and video source coding standard. For an N-point sequence X(n), the mathematical expression of its MDCT is:

[0075] in, x(n) represents the input signal.

[0076] The mathematical expression of IMDCT is:

[0077] Where n=0, 1, ..., N-1.

[0078] The window functions commonly used in audio encoding and decoding are:

[0079] Where w(n) represents the window function.

[0080] Optionally, most audio encoders window the signal before performing MDCT processing. Taking AVS3 as an example, there are four main types of windows, of which long-short windows and short-long windows are switching windows, also known as transition windows. When information changes suddenly, a short window can be used for MDCT processing. At this time, a transition window can be used to transition from a long window to a short window. There are also two types of windows, long windows and short windows, which are the most commonly used windows in audio coding. The long window is generally 2048 points, and the short window is 256 points. Taking the long window as an example, the signal is windowed and MDCT processed at the encoding end, and IMDCT and windowed at the decoding end. Time Domain Aliasing Cancellation (TDAC) is used to perform a 50% overlap-addition on the signal, allowing the signal to be perfectly reconstructed.

[0081] Optionally, to perfectly reconstruct the signal, overlap-addition can be performed on the signal. For example, 2N points can be used for MDCT processing to obtain N spectral coefficients. The 2N points consist of N points from the previous frame and N points from the current frame, as shown in Figure 2. When the decoder recovers the signal, N points are obtained by overlapping and adding the N points from the previous frame and the N points from the current frame. The window length is 2N, which means there is a 50% overlap-addition. However, when processing the 2N points from the current frame, only the N points from the current frame can be recovered, and the remaining N points are recovered in the next frame. In this case, the algorithm delay generated is N. Therefore, in AVS3, windowing will generate an inherent algorithm delay of N. In other words, the compression and encoding process of the audio signal will introduce a delay, which will affect the timeliness of audio signal recovery.

[0082] The method shown in the embodiment of the present disclosure reduces the delay introduced by the compression encoding process of the audio signal by changing the shape and parameters of the window, thereby effectively improving the timeliness of audio signal recovery.

[0083] The method shown in the embodiment of the present disclosure can be applied to various fields such as audio coding, video coding, audio synthesis, etc. In addition, it can also be used in application scenarios such as signal and image analysis and processing, communication modulation and demodulation, and mathematical analysis.

[0084] Figure 3A is an interactive schematic diagram of an audio signal processing method according to an embodiment of the present disclosure. As shown in Figure 3A, the embodiment of the present disclosure relates to an audio signal processing method, which can be used in a communication system 100. This embodiment can be applied to the process of encoding and processing a first audio signal. The terminal 101 and / or network device 102 in the above-mentioned communication system 100 can encode and process the first audio signal. Of course, any other possible equipment, devices, components, modules, etc. with encoding and processing functions can also use the audio signal processing method provided in the embodiment of the present disclosure to encode and process the first audio signal. The above-mentioned method includes:

[0085] Step S3101, determine the energy threshold.

[0086] In some embodiments, the energy threshold refers to a threshold value for determining the energy level of each sub-signal block in a signal. For example, if the energy of a sub-signal block is greater than or equal to the energy threshold, it indicates that the energy of the sub-signal block is relatively large; if the energy of the sub-signal block is less than the energy threshold, it indicates that the energy of the sub-signal block is relatively small.

[0087] In some embodiments, the energy threshold may be adaptively adjusted according to audio signal processing requirements, or may be based on protocol agreement.

[0088] Step S3102: Determine an energy value corresponding to each sub-signal block in the first audio signal.

[0089] In some embodiments, the first audio signal refers to a signal to be processed. The first audio signal can be divided into multiple sub-signal blocks, and the energy of each sub-signal block can be analyzed separately. The energy of each sub-signal block can be expressed using a quantized energy value. The energy value of each sub-signal block can be used to determine the window function length required to process the first audio signal. The window function length determines whether to use a long window or a short window to process the first audio signal.

[0090] In some embodiments, the energy value of each sub-signal block may be analyzed based on any possible energy analysis method.

[0091] In some embodiments, a sub-signal block may also be referred to as a sub-block.

[0092] Step S3103: Determine the window function length value according to the energy threshold and multiple energy values.

[0093] In some embodiments, the window function length value may be represented by N, and the window function length value may be used to represent the size of the window.

[0094] In some embodiments, after determining the energy threshold and the energy value of each sub-signal block, the window function length value may be determined with reference to the energy threshold and the multiple energy values.

[0095] In some embodiments, a comparison between each energy value and the energy threshold may be performed, and a suitable window function length value may be selected according to the comparison.

[0096] In some embodiments, the size of the window can be determined. For example, the window size can be obtained by comparing the energy threshold and the energy of each sub-block (an optional example of the energy value of the sub-signal block). For example, when the energy of one sub-block is greater than the energy threshold, it is determined to be a short window, and when the energies of all sub-blocks are less than or equal to the energy threshold, it is determined to be a long window, wherein the short window and the long window can be distinguished by different window function length values. For example, the window function length value of the long window can be a first window function length value, and the window function length value of the short window can be a second window function length value, and the second window function length value can be much smaller than the first window function length value.

[0097] In some embodiments, if the first audio signal is divided into multiple sub-signal blocks, and multiple energy values ​​corresponding to the multiple sub-signal blocks are all less than or equal to an energy threshold, then the window function length value is determined to be a first window function length value, indicating that the first audio signal can be processed based on a long window. In some embodiments, if the first audio signal is divided into multiple sub-signal blocks, and at least one energy value among the multiple energy values ​​corresponding to the multiple sub-signal blocks is greater than the energy threshold, then the window function length value is determined to be a second window function length value, wherein the second window function length value is less than the first window function length value, indicating that the first audio signal can be processed based on a short window.

[0098] Therefore, by performing energy analysis on the first audio signal and determining a suitable window function length value for processing the first audio signal, a suitable window is selected to perform truncation processing on the first audio signal, thereby improving the accuracy and encoding effect of the first audio signal encoding processing.

[0099] Step S3104 : determining a first window function according to the window function length value, a reference variable value related to the window function length value, the first variable, and the second window function.

[0100] In some embodiments, the first window function is a window function used in the embodiments of the present disclosure to perform truncation processing on the first audio signal.

[0101] In some embodiments, the first window function includes: a reference variable value related to the window function length value, the reference variable value is smaller than the window function length value, and the reference variable value is used to determine the delay required to restore the first audio signal.

[0102] In some embodiments, the second window function may be, for example, an existing window function. The second window function may be, for example, a window function commonly used in audio coding and decoding:

[0103] Wherein, w(n) represents the second window function.

[0104] It is understandable that after the first audio signal is truncated and frequency-domain transformed using a window function commonly used in audio coding and decoding, if the first audio signal needs to be restored, there will usually be a certain time delay, which is usually related to N. The larger N is, the longer the time delay is, and the smaller N is, the smaller the time delay is. In order to effectively reduce the time delay required to restore the first audio signal, the above-mentioned window function can be improved in certain embodiments of the present disclosure. For example, the second window function can be improved with reference to the window function length value, the reference variable value related to the window function length value, and the first variable, and the window function obtained by the improved processing is used as the first window function.

[0105] In some embodiments, the window function length value can be represented by N, the reference variable value related to the window function length value can be represented by X, the first variable can be represented by n, the first variable can also be called the serial number of the discrete time axis, the reference variable value is less than the window function length value (that is, X is less than N), and the reference variable value is used to determine the time delay required to restore the first audio signal. In other words, in the improved window function in the embodiment of the present disclosure, the reference variable value X is related to the time delay required to restore the first audio signal (that is, the time delay required to restore the first audio signal is determined by the reference variable value X rather than the window function length value N), and because the reference variable value X is less than the window function length value N. Since the time delay required to restore the first audio signal is reduced. Therefore. By changing the shape and parameters of the window, the delay introduced by the compression encoding process of the audio signal is reduced, thereby effectively improving the timeliness of audio signal recovery.

[0106] In the embodiment of the present disclosure, the second window function is improved by using a reference window function length value, a reference variable value related to the window function length value, and a first variable, and the improved window function is used as the first window function. This achieves the goal of minimizing the delay required to restore the first audio signal while effectively avoiding the introduction of excessive computational overhead.

[0107] In some embodiments, the first window function satisfies a first condition, which includes: the sum of the first value and the second value is 1; wherein the first value is the square of the value obtained by processing the first variable based on the first window function, the second value is the square of the value obtained by processing the second variable based on the first window function, and the second variable is the sum of the first variable and the window function length value.

[0108] In some embodiments, the first condition is a condition that the first window function must meet to support reducing the delay required to recover the first audio signal and effectively recover the first audio signal. The first variable is n, the second variable is n+N, the first value is the square of a value obtained by processing the first variable with the first window function, the second value is the square of a value obtained by processing the second variable with the first window function, and the sum of the first value and the second value must be 1.

[0109] In some embodiments, by configuring the first condition, the first window function can be flexibly configured and processed based on the first condition, effectively applicable to personalized audio signal processing scenarios, and improving signal coding processing flexibility and coding processing effect.

[0110] In some embodiments, the first condition is the following formula: W(n) 2 +W(n+N) 2 =1; wherein n represents the first variable, N represents the window function length value, and W() represents the first window function, thereby improving the configuration convenience of the first condition.

[0111] In some embodiments, the first window function includes at least one of the following: a first function portion corresponding to a first variable, wherein the first variable belongs to a first value range; a second function portion corresponding to the first variable, wherein the first variable belongs to a second value range; a third function portion corresponding to the first variable, wherein the first variable belongs to a third value range; and a fourth function portion corresponding to the first variable, wherein the first variable belongs to a fourth value range; wherein there is no intersection between different value ranges, and the value range is obtained by dividing the reference value range according to the window function length value and the reference variable value, the minimum value of the reference value range is zero, and the maximum value is twice the window function length, the different function portions are different from each other, at least one function portion is related to the second window function, and at least one function portion is a constant value. After the first audio signal is truncated based on the improved first window function, the delay introduced by the compression encoding process of the audio signal is reduced, thereby effectively improving the timeliness of audio signal recovery.

[0112] In some embodiments, the first window function may be a piecewise function, including different function parts. When the first variable n is within different value ranges, different function parts may be correspondingly adopted. For example, the first value range is 0≤n<N-X, that is, the first variable n is greater than or equal to 0 and less than the difference between the window function length value and the reference variable value; the second value range is N-X≤n<N, that is, the first variable n is greater than or equal to the difference between the window function length value and the reference variable value and less than the window function length value; the third value range is N≤n<2N-X, that is, the first variable n is greater than or equal to the window function length value and less than the difference between twice the window function length value and the reference variable value; the fourth value range is 2N-X≤n<2N, that is, the first variable n is greater than or equal to the difference between twice the window function length value and the reference variable value and less than twice the window function length value; correspondingly, corresponding to the foregoing different value ranges, the first window function may specifically correspond to different function parts. By way of example, corresponding to the first value range, the first function part may be a constant value 0; corresponding to the second value range, the second function part may be w(n-N+X); corresponding to the third value range, the third function part may be a constant value 1; corresponding to the fourth value range, the fourth function part may be w(n-2N+2X).

[0113] In some embodiments, the first window function may specifically be, for example, the following formula:

[0114] where n represents the first variable, N represents the window function length value, X represents the reference variable value, and W D () represents the first window function, and w() represents the second window function, so as to accurately and clearly express the first window function, such that the first window function can be accurately used for truncating the first audio signal.

[0115] In some embodiments, the reference variable value is related to the window function length value. That is to say, the reference variable value may be determined by referring to the window function length value required for processing the first audio signal. The relationship between the reference variable value and the window function length value may be expressed by the following formula:

[0116] where N represents the window function length value, X represents the reference variable value, and m is a positive integer.

[0117] It can be seen from the above formula that the reference variable value is less than the window function length value, and the existing second window function is improved to obtain the first window function, such that the first window function includes the reference variable value, and the delay required for restoring the first audio signal is determined by the reference variable value. Therefore, the delay introduced in the process of compression encoding the audio signal is reduced, thereby effectively improving the timeliness of audio signal restoration.

[0118] Step S3105: Process the first audio signal according to the first window function to obtain an intermediate audio signal.

[0119] In some embodiments, after determining the first window function, the first audio signal may be truncated using the first window function, and the resulting signal may be referred to as an intermediate audio signal. For example, the first window function may be multiplied by the first audio signal to obtain the intermediate audio signal.

[0120] In some embodiments, processing the first audio signal according to the first window function may also be referred to as windowing. For example, the input signal (an optional example of the first audio signal) is multiplied by the window function (an optional example of the first window function), i.e., X(n) = W(n) * input(n), where W(n) is the first window function, input(n) is the input signal, and X(n) is the windowed signal (an optional example of the intermediate audio signal).

[0121] Step S3106: Process the intermediate audio signal according to the first processing information to obtain spectral coefficients related to the first audio signal, wherein the first processing information includes: a reference variable value.

[0122] In some embodiments, the intermediate audio signal may be processed using the first processing information to obtain spectral coefficients related to the first audio signal, and the first processing information is used to perform modified discrete cosine transform processing on the intermediate audio signal.

[0123] In some embodiments, the first processing information refers to information required for performing a modified discrete cosine transform process.

[0124] Since the first window function is an improved window function, the first window function includes a reference variable value. In order to effectively determine the spectral coefficient related to the first audio signal based on the intermediate audio signal, the reference variable value can also be included in the first processing information, so that the first processing information can be effectively applied to the transformation processing of the intermediate audio signal.

[0125] In some embodiments, the first processing information is the following formula:

[0126] Where k = 0, 1, ..., N-1, n represents the first variable, N represents the window function length, X represents the reference variable value, x(n) represents the intermediate audio signal, and X(k) represents the spectral coefficient. Thus, the reference variable value is included in the improved discrete cosine transform formula, making the improved discrete cosine transform formula effectively applicable to performing improved discrete cosine transform processing on the intermediate audio signal.

[0127] In some embodiments, after the encoder processes the intermediate audio signal according to the first processing information to obtain spectral coefficients related to the first audio signal, the encoder can transmit the spectral coefficients related to the first audio signal to the decoder, and the decoder can restore the first audio signal based on the spectral coefficients related to the first audio signal.

[0128] The audio signal processing method involved in the embodiments of the present disclosure may include at least one of steps S3101 to S3106. For example, step S3101 can be implemented as an independent embodiment, step S3102 can be implemented as an independent embodiment, and so on, but the present invention is not limited thereto. Steps S3101+S3102 can be implemented as independent embodiments, and steps S3101+S3102+S3103 can be implemented as independent embodiments, but the present invention is not limited thereto.

[0129] In this embodiment or example, unless there is any contradiction, each step can be independent, arbitrarily combined or exchanged in order, the optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other embodiments or other examples.

[0130] In this embodiment, a window function length value required for processing a first audio signal is determined, and a first window function is determined, wherein the first window function includes a reference variable value associated with the window function length value, the reference variable value being less than the window function length value, and the reference variable value is used to determine the delay required to recover the first audio signal; and the first audio signal is processed according to the first window function to obtain spectral coefficients associated with the first audio signal. By changing the shape and parameters of the window, the delay introduced by the compression encoding process of the audio signal is reduced, thereby effectively improving the timeliness of audio signal recovery.

[0131] Figure 3B is an interactive schematic diagram of an audio signal processing method according to another embodiment of the present disclosure. As shown in Figure 3B, the embodiment of the present disclosure relates to an audio signal processing method, which can be used in a communication system 100. This embodiment can be applied to the process of decoding a first audio signal. The terminal 101 and / or the network device 102 in the above-mentioned communication system 100 can decode the first audio signal. Of course, any other possible equipment, devices, components, modules, etc. with decoding processing functions can also use the audio signal processing method provided in the embodiment of the present disclosure to decode the first audio signal. In addition, the encoding process and the decoding process can be implemented by the same device, or can also be implemented by different devices, without limitation. The above method includes:

[0132] Step S3201: Acquire spectral coefficients related to the first audio signal.

[0133] In some embodiments, spectral coefficients obtained by encoding the first audio signal at the encoder may be obtained. For example, after the encoder processes the intermediate audio signal according to the first processing information to obtain spectral coefficients associated with the first audio signal, the encoder may transmit the spectral coefficients associated with the first audio signal to the decoder. The decoder can then obtain the spectral coefficients associated with the first audio signal from the encoder and restore the first audio signal based on the spectral coefficients associated with the first audio signal.

[0134] Step S3202: Determine a first window function according to the window function length value used for processing the first audio signal, a reference variable value related to the window function length value, the first variable, and the second window function.

[0135] In some embodiments, the reference variable value is less than the window function length value, and the reference variable value is used to determine the time delay required to recover the first audio signal. In this embodiment, the description of the window function length value used to process the first audio signal, the reference variable value related to the window function length value, the first variable, and the second window function, as well as the description of determining the first window function, can be found in the above embodiment and are not repeated here.

[0136] Step S3203: Process the spectral coefficients according to the second processing information to obtain an intermediate audio signal, wherein the second processing information includes: a reference variable value.

[0137] In some embodiments, the second processing information refers to information required for performing an inverse modified discrete cosine transform process.

[0138] In some embodiments, the second processing information may be used to process the spectral coefficients to obtain an intermediate audio signal, and the second processing information is used to perform inverse modified discrete cosine transform processing on the spectral coefficients.

[0139] Since the first window function is an improved window function, the first window function includes a reference variable value. In order to effectively restore the intermediate audio signal based on the spectral coefficients related to the first audio signal, the reference variable value can also be included in the second processing information, so that the second processing information can be effectively applied to the transformation processing of the spectral coefficients related to the first audio signal.

[0140] In some embodiments, the second processing information is the following formula:

[0141] Where n = 0, 1, ..., 2N-1, n represents the first variable, N represents the window function length, X represents the reference variable value, x(n) represents the intermediate audio signal, and X(k) represents the spectral coefficient. Thus, the reference variable value is included in the inverse modified discrete cosine transform formula, making the inverse modified discrete cosine transform formula effectively applicable to performing inverse modified discrete cosine transform processing on the spectral coefficients associated with the first audio signal.

[0142] Step S3204: Process the intermediate audio signal according to the first window function to obtain a first audio signal.

[0143] In some embodiments, after processing spectral coefficients associated with the first audio signal using the second processing information to obtain an intermediate audio signal, the intermediate audio signal may be windowed to recover the first audio signal. For example, windowing the intermediate audio signal involves multiplying the input signal (e.g., the intermediate audio signal) by a window function (an optional example of the first window function) using the following formula: X(n) = W(n) * input(n), where W(n) is the first window function, input(n) is the input signal, and X(n) is the windowed signal.

[0144] In some embodiments, the windowed signal can be overlap-added to recover the first audio signal. For example, X points from the previous frame can be overlap-added with Nx to N points from the current frame to recover X points. Adding these points to the Nx points from the current frame (the Nx points do not need to be overlap-added), a total of N points are recovered. However, the X points from the 2N-X to 2N portion of the current frame cannot be recovered until the next frame, resulting in a delay of X.

[0145] The audio signal processing method involved in the embodiments of the present disclosure may include at least one of steps S3201 to S3204. For example, step S3201 can be implemented as an independent embodiment, step S3202 can be implemented as an independent embodiment, and so on, but the present invention is not limited thereto. Steps S3201+S3202 can be implemented as independent embodiments, and steps S3201+S3202+S3203 can be implemented as independent embodiments, but the present invention is not limited thereto.

[0146] In this embodiment or example, unless there is any contradiction, each step can be independent, arbitrarily combined or exchanged in order, the optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other embodiments or other examples.

[0147] In this embodiment, spectral coefficients associated with a first audio signal are obtained and a first window function is determined, wherein the first window function includes a reference variable value associated with a window function length value used to process the first audio signal, the reference variable value being less than the window function length value, the reference variable value being used to determine the delay required to recover the first audio signal, and the spectral coefficients are processed according to the first window function to obtain the first audio signal. By changing the shape and parameters of the window, the delay introduced by the compression encoding process of the audio signal is reduced, thereby effectively improving the timeliness of audio signal recovery.

[0148] FIG4A is an interactive diagram of an audio signal processing method according to another embodiment of the present disclosure. As shown in FIG4A , the present disclosure embodiment relates to an audio signal processing method that can be used in an encoding process. The method includes:

[0149] Step S4101: Determine a window function length value required for processing a first audio signal.

[0150] Step S4102: Determine a first window function, wherein the first window function includes: a reference variable value related to the window function length value, the reference variable value is smaller than the window function length value, and the reference variable value is used to determine the delay required to restore the first audio signal.

[0151] Step S4103: Process the first audio signal according to the first window function to obtain spectral coefficients related to the first audio signal.

[0152] The audio signal processing method involved in the embodiments of the present disclosure may include at least one of steps S4101 to S4103. For example, step S4101 can be implemented as an independent embodiment, step S4102 can be implemented as an independent embodiment, and so on, but the present invention is not limited thereto. Steps S4101+S4102 can be implemented as independent embodiments, and steps S4101+S4102+S4103 can be implemented as independent embodiments, but the present invention is not limited thereto.

[0153] In this embodiment or example, unless there is any contradiction, each step can be independent, arbitrarily combined or exchanged in order, the optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other embodiments or other examples.

[0154] FIG4B is an interactive diagram of an audio signal processing method according to another embodiment of the present disclosure. As shown in FIG4B , the present disclosure embodiment relates to an audio signal processing method that can be used in an encoding process. The method includes:

[0155] Step S4201, determine the energy threshold.

[0156] Step S4202: Determine an energy value corresponding to each sub-signal block in the first audio signal.

[0157] Step S4203: Determine the window function length value according to the energy threshold and multiple energy values.

[0158] Step S4204 : determining a first window function according to the window function length value, a reference variable value related to the window function length value, the first variable, and the second window function.

[0159] Step S4205: Process the first audio signal according to the first window function to obtain an intermediate audio signal.

[0160] Step S4206: Process the intermediate audio signal according to the first processing information to obtain spectral coefficients related to the first audio signal, wherein the first processing information includes: a reference variable value.

[0161] The audio signal processing method involved in the embodiments of the present disclosure may include at least one of steps S4201 to S4206. For example, step S4201 can be implemented as an independent embodiment, step S4202 can be implemented as an independent embodiment, and so on, but the present invention is not limited thereto. Steps S4201+S4202 can be implemented as independent embodiments, and steps S4201+S4202+S4203 can be implemented as independent embodiments, but the present invention is not limited thereto.

[0162] In this embodiment or example, unless there is any contradiction, each step can be independent, arbitrarily combined or exchanged in order, the optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other embodiments or other examples.

[0163] FIG5A is an interactive diagram of an audio signal processing method according to another embodiment of the present disclosure. As shown in FIG5A , the present disclosure embodiment relates to an audio signal processing method that can be used in a decoding process. The method includes:

[0164] Step S5101: Acquire spectral coefficients related to the first audio signal.

[0165] Step S5102: Determine a first window function, where the first window function includes: a reference variable value related to a window function length value used to process the first audio signal, the reference variable value is less than the window function length value, and the reference variable value is used to determine a delay required to restore the first audio signal.

[0166] Step S5103: Process the spectral coefficients according to the first window function to obtain a first audio signal.

[0167] The audio signal processing method involved in the embodiments of the present disclosure may include at least one of steps S5101 to S5103. For example, step S5101 can be implemented as an independent embodiment, step S5102 can be implemented as an independent embodiment, and so on, but the present invention is not limited thereto. Steps S5101 + S5102 can be implemented as independent embodiments, and steps S5101 + S5102 + S5103 can be implemented as independent embodiments, but the present invention is not limited thereto.

[0168] In this embodiment or example, unless there is any contradiction, each step can be independent, arbitrarily combined or exchanged in order, the optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other embodiments or other examples.

[0169] FIG5B is an interactive diagram of an audio signal processing method according to another embodiment of the present disclosure. As shown in FIG5B , the present disclosure embodiment relates to an audio signal processing method that can be used in a decoding process. The method includes:

[0170] Step S5201: Acquire spectral coefficients related to the first audio signal.

[0171] Step S5202: Determine the first window function based on the window function length value used to process the first audio signal, the reference variable value related to the window function length value, the first variable, and the second window function, wherein the reference variable value is less than the window function length value, and the reference variable value is used to determine the delay required to restore the first audio signal.

[0172] Step S5203: Process the spectral coefficients according to the second processing information to obtain an intermediate audio signal, wherein the second processing information includes: a reference variable value.

[0173] Step S5204: Process the intermediate audio signal according to the first window function to obtain a first audio signal.

[0174] The audio signal processing method involved in the embodiments of the present disclosure may include at least one of steps S5201 to S5204. For example, step S5201 can be implemented as an independent embodiment, step S5202 can be implemented as an independent embodiment, and so on, but the present invention is not limited thereto. Steps S5201+S5202 can be implemented as independent embodiments, and steps S5201+S5202+S5203 can be implemented as independent embodiments, but the present invention is not limited thereto.

[0175] In this embodiment or example, unless there is any contradiction, each step can be independent, arbitrarily combined or exchanged in order, the optional methods or optional examples can be arbitrarily combined, and can be arbitrarily combined with any steps of other embodiments or other examples.

[0176] The following is an exemplary introduction to the above method.

[0177] Optional embodiment:

[0178] As shown in Figure 6A, Figure 6A is a schematic diagram of a time domain signal encoding process according to an embodiment of the present disclosure. As shown in Figure 6A, in an embodiment of the present disclosure, an encoding end process is shown. The encoding end is an encoder for example.

[0179] In some embodiments, the encoder input terminal reads N points each time, taking N as 1024 points as an example, and converts the input data from integer data to floating-point data.

[0180] In some embodiments, the encoder processes the first audio signal in units of frames and calculates the energy of a frame of the first audio signal read. The calculation formula is as follows: blockEnergy[i]+=input[i*nblockSize+j]*input[i*nblockSize+j];

[0181] Where i = 0, 1, ..., 7 indicates that a frame of the first audio signal is divided into 8 sub-blocks (an optional example of a sub-signal block), blockEnergy[i] indicates that the energy of each sub-block is calculated, j = 0, 1, ..., 127 indicates that the size of each sub-block is 128 points, nblockSize indicates the length of the sub-block, and input indicates the input signal. The above formula divides 1024 points into 8 sub-blocks and calculates the energy of each sub-block.

[0182] In some embodiments, the size of the window can be determined by comparing the energy of the threshold value (an optional example of the energy threshold) and the energy size of each sub-block (an optional example of the energy value), and the size of the window (an optional example of the window function length value) can be obtained. When the energy of a sub-block is greater than the energy of the threshold value, it is determined to be a short window, otherwise it is a long window.

[0183] In some embodiments, a long window is used as an example. In the disclosed embodiments, the delay can be reduced by changing the length of the overlap-add portion of the window function. The improved window function needs to meet the requirements of perfect signal reconstruction. The formula for the long window is as follows:

[0184] Where X is a variable parameter (an optional example of the above reference variable value), and its value is Where m is a positive integer. The window function in the embodiment of the present disclosure is divided into four parts. The first part is zeroed, with n ranging from 0 to NX. The second part is w(), with n ranging from NX to N. The third part is a constant gain of 1, with n ranging from N to 2N-X. The fourth part has n ranging from 2N-X to 2N. The shape of the window is shown in Figure 6B below, which is a functional diagram of the low-latency window in the embodiment of the present disclosure.

[0185] In some embodiments, the first audio signal may be windowed. For example, the input signal (an optional example of the first audio signal) is multiplied by a window function (an optional example of the first window function), i.e., X(n) = W(n) * input(n), where W(n) is the first window function, input(n) is the input signal, and X(n) is the windowed signal (an optional example of the intermediate audio signal).

[0186] In some embodiments, the windowed signal may be subjected to an MDCT transform. The MDCT formula is as follows:

[0187] Here, k = 0, 1, ..., N-1, n represents the first variable, N represents the window function length, X represents the reference variable, x(n) represents the intermediate audio signal, and X(k) represents the spectral coefficients. In other words, performing the MDCT transform on 2N points yields N MDCT coefficients.

[0188] As shown in Figure 6C, Figure 6C is a schematic diagram of a decoding process according to an embodiment of the present disclosure. As shown in Figure 6C, in an embodiment of the present disclosure, a decoding process is shown.

[0189] In some embodiments, the MDCT coefficients (an optional example of the spectral coefficients of the first audio signal) obtained at the encoding end may be subjected to an inverse modified discrete cosine transform (IMDCT) to obtain a time domain signal. The formula of IMDCT is as follows:

[0190] Where n = 0, 1, ..., 2N-1, n represents the first variable, N represents the window function length, X represents the reference variable, x(n) represents the intermediate audio signal, and X(k) represents the spectral coefficient. That is, the N-point frequency domain coefficients are converted into a 2N-point time domain signal.

[0191] In some embodiments, the window type can be determined by reading the information transmitted from the encoding end, and then the signal is windowed, that is, the input signal is multiplied by the window function. For example, the intermediate audio signal is windowed, that is, the input signal (for example, the intermediate audio signal) is multiplied by the window function (an optional example of the first window function). The formula is as follows: X(n) = W(n)*input(n), where W(n) is the first window function, input(n) is the input signal, and X(n) is the windowed signal.

[0192] In some embodiments, after windowing, overlap-add is performed, overlapping and adding the X points of the previous frame with the Nx to N points of the current frame to recover X points. Adding the Nx points of the current frame (without overlap-add), a total of N points are recovered. However, the X points in the 2n-x to 2n portion of the current frame cannot be recovered until the next frame, resulting in a delay of X. An illustration of overlap-add is shown in FIG6D , which is a schematic diagram of signal overlap-add in an embodiment of the present disclosure.

[0193] In the disclosed embodiment, the original long window can be replaced with the low-latency window described above (an optional example of the first window function). First, the use of the low-latency window is verified to determine whether perfect reconstruction is possible. X = N / 4 is selected for verification. Experimental verification shows that the method used in the disclosed embodiment can fully recover the original signal, achieving perfect reconstruction, but the delay is increased from the original N to X(N / 4).

[0194] Experimental plan: Encode and decode the original audio signal through the existing long window to obtain output signal 1. Then encode and decode the original audio signal through the improved low-latency window to obtain output signal 2. Compare the waveforms of the two. Take X = N / 32.

[0195] As shown in Figures 7A-7C, Figure 7A is a schematic diagram of the waveform of Wav test file 1 in an embodiment of the present disclosure. Figure 7B is a schematic diagram of output signal 1 corresponding to the waveform of Wav test file 1 in an embodiment of the present disclosure. Figure 7C is a schematic diagram of output signal 2 corresponding to the waveform of Wav test file 1 in an embodiment of the present disclosure.

[0196] As shown in Figures 7D-7F, Figure 7D is a schematic diagram of the waveform of Wav test file 2 according to an embodiment of the present disclosure. Figure 7E is a schematic diagram of output signal 1 corresponding to the waveform of Wav test file 2 according to an embodiment of the present disclosure. Figure 7F is a schematic diagram of output signal 2 corresponding to the waveform of Wav test file 2 according to an embodiment of the present disclosure.

[0197] As shown in Figures 7G-7I, Figure 7G is a schematic diagram of the waveform of Wav test file 3 according to an embodiment of the present disclosure. Figure 7H is a schematic diagram of output signal 1 corresponding to the waveform of Wav test file 3 according to an embodiment of the present disclosure. Figure 7I is a schematic diagram of output signal 2 corresponding to the waveform of Wav test file 3 according to an embodiment of the present disclosure.

[0198] It can be seen from the above Figures 7A to 7I that the output signal 1 and the output signal 2 are basically consistent with the original signal. The method provided in the embodiment of the present disclosure can effectively ensure the quality of audio processing while reducing the algorithm delay.

[0199] An embodiment of the present disclosure also provides an audio signal processing method, wherein a first communication device determines a window function length value required to process a first audio signal and determines a first window function, wherein the first window function includes: a reference variable value related to the window function length value, the reference variable value is less than the window function length value, the reference variable value is used to determine the time delay required to restore the first audio signal, and processes the first audio signal according to the first window function to obtain spectral coefficients related to the first audio signal; a second communication device obtains the spectral coefficients related to the first audio signal, and processes the spectral coefficients according to the first window function to obtain the first audio signal.

[0200] Among them, the first communication device is, for example, a communication device that performs an encoding process, and the second communication device is, for example, a communication device that performs a decoding process. The first communication device can be a terminal or a network device, and the second communication device can also be a terminal or a network device. The first communication device and the second communication device can be the same or different communication devices.

[0201] FIG8A is a schematic diagram of the structure of an audio signal processing device according to an embodiment of the present disclosure. As shown in FIG8A , the audio signal processing device includes: a processing module for determining a window function length value required to process a first audio signal, and determining a first window function, wherein the first window function includes: a reference variable value associated with the window function length value, the reference variable value being less than the window function length value, the reference variable value being used to determine the time delay required to restore the first audio signal; and processing the first audio signal according to the first window function to obtain spectral coefficients associated with the first audio signal.

[0202] FIG8B is a schematic diagram of the structure of an audio signal processing device proposed in an embodiment of the present disclosure. As shown in FIG8B , the audio signal processing device includes: a processing module for obtaining spectral coefficients associated with a first audio signal and determining a first window function, wherein the first window function includes: a reference variable value associated with a window function length value used to process the first audio signal, the reference variable value being less than the window function length value, the reference variable value being used to determine the time delay required to restore the first audio signal; and processing the spectral coefficients according to the first window function to obtain the first audio signal.

[0203] It should be understood that the division of the various units or modules in the above device is merely a division of logical functions. In actual implementation, they may be fully or partially integrated into a physical entity, or they may be physically separated. In addition, the units or modules in the device may be implemented in the form of a processor calling software: for example, the device includes a processor, the processor is connected to a memory, and the memory stores instructions. The processor calls the instructions stored in the memory to implement any of the above methods or implement the functions of the various units or modules of the above device, wherein the processor is, for example, a general-purpose processor, such as a central processing unit (CPU) or a microprocessor, and the memory is a memory within the device or a memory outside the device. Alternatively, the units or modules in the device can be implemented in the form of hardware circuits, and the functions of some or all of the units or modules can be realized by designing the hardware circuits. The above-mentioned hardware circuits can be understood as one or more processors; for example, in one implementation, the above-mentioned hardware circuit is an application-specific integrated circuit (ASIC), which realizes the functions of some or all of the above units or modules by designing the logical relationship of the components in the circuit; for example, in another implementation, the above-mentioned hardware circuit can be realized by a programmable logic device (PLD). Taking a field programmable gate array (FPGA) as an example, it can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by configuring the configuration file, thereby realizing the functions of some or all of the above units or modules. All units or modules of the above devices can be realized in the form of software called by the processor, or in the form of hardware circuits, or in part by the form of software called by the processor, and the rest by hardware circuits.

[0204] As shown in Figure 9A, Figure 9A is a schematic diagram of the structure of the communication device proposed in an embodiment of the present disclosure. The communication device 9100 includes one or more processors 9101. The processor 9101 can be a general-purpose processor or a dedicated processor, for example, a baseband processor or a central processing unit. The baseband processor can be used to process the communication protocol and communication data, and the central processing unit can be used to control the communication device (such as a base station, a baseband chip, a terminal device, a terminal device chip, a DU or a CU, etc.), execute programs, and process program data. The processor 9101 is used to call instructions to enable the communication device 9100 to execute any of the above methods.

[0205] In some embodiments, the communication device 9100 further includes one or more memories 9102 for storing instructions. Optionally, all or part of the memories 9102 may be located outside the communication device 9100.

[0206] In some embodiments, the communication device 9100 further includes one or more transceivers 9103. When the communication device 9100 includes one or more transceivers 9103, the communication steps such as sending and receiving in the above method are performed by the transceiver 9103, and the other steps are performed by the processor 9101.

[0207] In some embodiments, a transceiver may include a receiver and a transmitter, which may be separate or integrated. Optionally, the terms transceiver, transceiver unit, transceiver, and transceiver circuit may be used interchangeably; the terms transmitter, transmitting unit, transmitter, and transmitting circuit may be used interchangeably; and the terms receiver, receiving unit, receiver, and receiving circuit may be used interchangeably.

[0208] Optionally, the communication device 9100 further includes one or more interface circuits 9104, which are connected to the memory 9102. The interface circuits 9104 can be used to receive signals from the memory 9102 or other devices, and can be used to send signals to the memory 9102 or other devices. For example, the interface circuits 9104 can read instructions stored in the memory 9102 and send the instructions to the processor 9101.

[0209] The communication device 9100 described in the above embodiments may be a network device or a terminal, but the scope of the communication device 9100 described in the present disclosure is not limited thereto, and the structure of the communication device 9100 may not be limited by FIG. 9A. The communication device may be an independent device or may be part of a larger device. For example, the communication device may be: 1) an independent integrated circuit IC, or a chip, or a chip system or subsystem; (2) a collection of one or more ICs, optionally, the above IC collection may also include a storage component for storing data or programs; (3) an ASIC, such as a modem; (4) a module that can be embedded in other devices; (5) a receiver, a terminal device, an intelligent terminal device, a cellular phone, a wireless device, a handheld device, a mobile unit, an in-vehicle device, a network device, a cloud device, an artificial intelligence device, etc.; (6) others, etc.

[0210] Figure 9B is a schematic diagram of the structure of a chip proposed in an embodiment of the present disclosure. If the communication device 9100 can be a chip or a chip system, please refer to the schematic diagram of the structure of the chip 9200 shown in Figure 9B, but the present invention is not limited thereto. The chip 9200 includes one or more processors 9201, which are used to call instructions to cause the chip 9200 to execute any of the above methods.

[0211] In some embodiments, chip 9200 further includes one or more interface circuits 9202, which are connected to memory 9203. Interface circuits 9202 can be used to receive signals from memory 9203 or other devices, and can be used to send signals to memory 9203 or other devices. For example, interface circuit 9202 can read instructions stored in memory 9203 and send the instructions to processor 9201. Optionally, the terms interface circuit, interface, transceiver pin, and transceiver are interchangeable.

[0212] In some embodiments, the chip 9200 further includes one or more memories 9203 for storing instructions. Alternatively, all or part of the memories 9203 may be located outside the chip 9200.

[0213] The present disclosure also proposes a storage medium having instructions stored thereon, which, when executed on the communication device 9100, causes the communication device 9100 to execute any of the above methods. Optionally, the storage medium is an electronic storage medium. Optionally, the storage medium is a computer-readable storage medium, but is not limited thereto and may also be a storage medium readable by other devices. Optionally, the storage medium may be a non-transitory storage medium, but is not limited thereto and may also be a temporary storage medium.

[0214] In an embodiment of the present disclosure, the processor is a circuit capable of processing audio signals. In one implementation, the processor may be a circuit capable of reading and executing instructions, such as a central processing unit (CPU), a microprocessor, a graphics processing unit (GPU) (which may be understood as a microprocessor), or a digital signal processor (DSP); in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the above-mentioned hardware circuit may be fixed or reconfigurable, such as a hardware circuit implemented by an application-specific integrated circuit (ASIC) or a programmable logic device (PLD), such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the configuration of the hardware circuit may be understood as the process of the processor loading instructions to implement the functions of some or all of the above units or modules. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as ASIC, such as a neural network processing unit (NPU), a tensor processing unit (TPU), a deep learning processing unit (DPU), etc.

[0215] The present disclosure also provides a program product, which, when executed by the communication device 9100, enables the communication device 9100 to perform any of the above methods. Optionally, the program product is a computer program product.

[0216] The present disclosure also proposes a computer program, which, when executed on a computer, causes the computer to perform any one of the above methods.

[0217] In the above embodiments, all or part of the embodiments can be implemented by software, hardware, firmware or any combination thereof. When implemented using software, all or part of the embodiments can be implemented in the form of a computer program product. The computer program product includes one or more computer programs. When the computer program is loaded and executed on a computer, all or part of the processes or functions described in the embodiments of the present disclosure are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer program can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer program can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated therein. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a high-density digital video disc (DVD)), or a semiconductor medium (eg, a solid state disk (SSD)).

[0218] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this disclosure.

[0219] Those skilled in the art will clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0220] The above description is merely a specific embodiment of the present disclosure, but the scope of protection of the present disclosure is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in this disclosure should be included in the scope of protection of the present disclosure. Therefore, the scope of protection of the present disclosure should be based on the scope of protection of the claims.

Claims

1. An audio signal processing method, characterized in that: The method comprises: Determining a window function length value required for processing the first audio signal; Determine a first window function, wherein the first window function includes: a reference variable value related to the window function length value, the reference variable value is less than the window function length value, and the reference variable value is used to determine a time delay required to restore the first audio signal; and The first audio signal is processed according to the first window function to obtain spectral coefficients related to the first audio signal.

2. The method according to claim 1, characterized in that The determining of the window function length value required for processing the first audio signal comprises: Determine energy threshold; Determining an energy value corresponding to each sub-signal block in the first audio signal; The window function length value is determined according to the energy threshold and a plurality of the energy values.

3. The method according to claim 2, characterized in that The step of determining the window function length value according to the energy threshold and the plurality of energy values ​​comprises: If the plurality of energy values ​​are all less than or equal to the energy threshold, determining the window function length value to be a first window function length value; If at least one of the energy values ​​is greater than the energy threshold, the window function length value is determined to be a second window function length value, wherein the second window function length value is smaller than the first window function length value.

4. The method according to any one of claims 1 to 3, characterized in that: The first window function satisfies a first condition, which includes: the sum of the first value and the second value is 1; wherein the first value is the square of a value obtained by processing a first variable with the first window function, the second value is the square of a value obtained by processing a second variable with the first window function, and the second variable is the sum of the first variable and the length value of the window function.

5. The method according to claim 4, characterized in that The first condition is the following formula: W(n) 2 +W(n+N) 2 =1; Wherein, n represents the first variable, N represents the length value of the window function, and W() represents the first window function.

6. The method according to any one of claims 1 to 4, characterized in that: The determining of the first window function comprises: The first window function is determined according to the window function length value, the reference variable value, the first variable and the second window function.

7. The method according to claim 6, characterized in that The first window function includes at least one of the following: a first function portion corresponding to the first variable, wherein the first variable belongs to a first value range; a second function portion corresponding to the first variable, wherein the first variable belongs to a second value range; a third function portion corresponding to the first variable, wherein the first variable belongs to a third value range; a fourth function portion corresponding to the first variable, wherein the first variable belongs to a fourth value range; There is no intersection between the different value ranges, and the value range is obtained by dividing the reference value range according to the window function length value and the reference variable value, the minimum value of the reference value range is zero, and the maximum value is twice the window function length, the different function parts are different from each other, at least one function part is related to the second window function, and at least one function part is a constant value.

8. The method according to claim 6 or 7, characterized in that The first window function is the following formula: Wherein, n represents the first variable, N represents the length value of the window function, X represents the reference variable value, and W D () represents the first window function, and w() represents the second window function.

9. The method according to any one of claims 1 to 8, characterized in that The step of processing the first audio signal according to the first window function to obtain a spectral coefficient related to the first audio signal includes: Processing the first audio signal according to the first window function to obtain an intermediate audio signal; The intermediate audio signal is processed according to first processing information to obtain the spectral coefficient related to the first audio signal, wherein the first processing information includes: the reference variable value.

10. The method according to claim 9, characterized in that The first processing information is used to perform a modified discrete cosine transform process on the intermediate audio signal.

11. The method according to claim 9 or 10, characterized in that The first processing information is the following formula: Wherein, k=0, 1, ..., N-1, n represents the first variable, N represents the window function length value, X represents the reference variable value, x(n) represents the intermediate audio signal, and X(k) represents the spectral coefficient.

12. The method according to any one of claims 1 to 11, characterized in that: The reference variable value is determined by the following formula: Wherein, N represents the length value of the window function, X represents the reference variable value, and m is a positive integer.

13. An audio signal processing method, characterized in that: The method comprises: Obtaining spectral coefficients associated with the first audio signal; Determine a first window function, wherein the first window function includes: a reference variable value related to a window function length value used to process the first audio signal, the reference variable value is less than the window function length value, and the reference variable value is used to determine a time delay required to restore the first audio signal; The spectral coefficients are processed according to the first window function to obtain the first audio signal.

14. The method according to claim 13, characterized in that The first window function satisfies a first condition, which includes: the sum of the first value and the second value is 1; wherein the first value is the square of a value obtained by processing a first variable with the first window function, the second value is the square of a value obtained by processing a second variable with the first window function, and the second variable is the sum of the first variable and the length value of the window function.

15. The method according to claim 14, characterized in that The first condition is the following formula: W(n) 2 +W(n+N) 2 =1; Wherein, n represents the first variable, N represents the length value of the window function, and W() represents the first window function.

16. The method according to any one of claims 13 to 15, characterized in that: The determining of the first window function comprises: The first window function is determined according to the window function length value, the reference variable value, the first variable and the second window function.

17. The method according to claim 16, characterized in that The first window function includes at least one of the following: a first function portion corresponding to the first variable, wherein the first variable belongs to a first value range; a second function portion corresponding to the first variable, wherein the first variable belongs to a second value range; a third function portion corresponding to the first variable, wherein the first variable belongs to a third value range; a fourth function portion corresponding to the first variable, wherein the first variable belongs to a fourth value range; There is no intersection between the different value ranges, and the value range is obtained by dividing the reference value range according to the window function length value and the reference variable value, the minimum value of the reference value range is zero, and the maximum value is twice the window function length, the different function parts are different from each other, at least one function part is related to the second window function, and at least one function part is a constant value.

18. The method according to claim 16 or 17, characterized in that The first window function is the following formula: Wherein, n represents the first variable, N represents the length value of the window function, X represents the reference variable value, and W D () represents the first window function, and w() represents the second window function.

19. The method according to any one of claims 13 to 18, characterized in that: The step of processing the spectral coefficients according to the first window function to obtain the first audio signal includes: Process the spectral coefficients according to second processing information to obtain an intermediate audio signal, wherein the second processing information includes: the reference variable value; The intermediate audio signal is processed according to the first window function to obtain the first audio signal.

20. The method of claim 19, wherein: The second processing information is used to perform inverse modified discrete cosine transform processing on the spectral coefficients.

21. The method according to claim 19 or 20, characterized in that The second processing information is the following formula: Among them, n=0, 1, ..., 2N-1, n represents the first variable, N represents the window function length value, X represents the reference variable value, x(n) represents the intermediate audio signal, and X(k) represents the spectral coefficient.

22. The method according to any one of claims 1 to 21, characterized in that The reference variable value is determined by the following formula: Wherein, N represents the length value of the window function, X represents the reference variable value, and m is a positive integer.

23. An audio signal processing method, characterized in that: The method comprises: A first communication device, configured to determine a window function length value required for processing a first audio signal, and determine a first window function, wherein the first window function includes: a reference variable value related to the window function length value, the reference variable value is less than the window function length value, the reference variable value is used to determine a time delay required to restore the first audio signal, and process the first audio signal according to the first window function to obtain a spectral coefficient related to the first audio signal; The second communication device is configured to obtain spectral coefficients related to the first audio signal, and process the spectral coefficients according to the first window function to obtain the first audio signal.

24. An audio signal processing device, characterized in that: The device comprises: A processing module, used to determine a window function length value required for processing a first audio signal, and determine a first window function, wherein the first window function includes: a reference variable value related to the window function length value, the reference variable value is less than the window function length value, the reference variable value is used to determine a time delay required to restore the first audio signal, and process the first audio signal according to the first window function to obtain a spectral coefficient related to the first audio signal.

25. An audio signal processing device, characterized in that: The device comprises: A processing module, used to obtain spectral coefficients related to a first audio signal and determine a first window function, wherein the first window function includes: a reference variable value related to a window function length value used to process the first audio signal, the reference variable value is less than the window function length value, the reference variable value is used to determine a time delay required to restore the first audio signal, and process the spectral coefficients according to the first window function to obtain the first audio signal.

26. A communication device, characterized in that: include: One or more processors; The processor is used to call instructions so that the communication device executes the method according to any one of claims 1 to 23.

27. A communication system, characterized in that: The invention comprises a first communication device and a second communication device, wherein the first communication device is configured to implement the method according to any one of claims 1 to 12, and the second communication device is configured to implement the method according to any one of claims 13 to 22.

28. A storage medium storing instructions, characterized in that: When the instruction is executed on a communication device, the communication device is caused to execute the method according to any one of claims 1 to 23.

Images