Efficient time delay synthesis

Abstract

A method for adjusting timing of output audio signals to achieve a desired inter-channel time difference (ITD) between the output audio signals is provided. A method includes receiving a current ITD value and an audio frame, and determining a transition time t1, t2 to perform a time shift applied to at least one of a first output signal and a second output signal based on an ITD of the current frame and an ITD of a previous frame. When generating the first output signal and the second output signal, a time shift is applied within the determined transition times t1, t2.

Description

Technical Field

[0001] This disclosure relates generally to communications, and more particularly to communication methods that support audio encoding and decoding, as well as related devices and nodes. Background Technology

[0002] Spatial audio is a description of the sound field for an immersive listener. There are several formats for spatial audio. The most common format is stereo, where the sound field is rendered by two speakers or a set of headphones. In scenarios played back on larger sets of speakers such as 5.1, 7.1+4, or 22.2, spatial audio is often referred to as multichannel audio. There are also spatial audio formats that describe the sound field itself, independent of the speaker system layout. This includes wave field synthesis (WFS), where the sound field is captured by a microphone array and symmetrically reproduced by a speaker array. Another popular format is stereo reverb, which relies on spherical harmonics captured using a compact microphone array. Currently, stereo reverb has recently become increasingly popular because it is well-suited for listener-centric rendering, such as virtual reality (VR) and augmented reality (AR) audio rendering, and it is inherently suitable for rotation. They can also be coupled with 360-degree video capture for reconstructing experiential scenes.

[0003] Multichannel audio formats can be played back directly on the speaker setups they are designed for. However, if no speaker configuration exists, audio cannot be played back without a suitable audio fit. This fit is often referred to as rendering the spatial audio of the playback system. If a 22.2 multichannel signal or a stereo reverberation signal is available, it can be rendered, for example, for playback on a 5.1 system or a set of headphones. When rendering for headphones, the audio reaching the ears is typically modeled using a head-related filter (HRF) or head-related transfer function (HRTF). The filter models the direction of arrival (DoA) of the sound source so that the listener perceives the sound coming from that direction. This is achieved through spectral coloring, horizontal differences between ears, and time differences caused by the difference in path length to the left and right ears. This time difference is often referred to as inter-ear time difference or inter-channel time difference (ITD). Inter-channel time differences can be created by filtering one or both channels using Dirac pulses.

[0004] However, it is necessary to handle transitions between different time shifts, such as those for mobile sources. Summary of the Invention

[0005] When modeling HRF, filters can be used for spectral coloring, and time differences can be generated through time shifting. This disclosure applies time shifting in an efficient manner when crossing zero boundaries used for shifting.

[0006] When the sign of the time delay parameter is changed, a shift operation needs to be performed on both output channels. To limit the complexity of the shift operation, the total conversion length is shared between the channels. This sharing is done proportionally to the size of the shift on each side of the zero point.

[0007] According to a first aspect, a method for adjusting the timing of an output audio signal to achieve a desired inter-channel time difference (ITD) between the output audio signals is proposed. The method includes: receiving a current ITD value and an audio frame, and determining a transition time for performing a time shift on at least one of a first output signal and a second output signal based on the ITD of the current frame and the ITD of a previous frame. t 1. t 2. During the generation of the first and second output signals, at the determined conversion time... t 1. t 2. Application time shift.

[0008] According to a second aspect, an apparatus is proposed for adjusting the timing of an output audio signal to achieve a desired interchannel time difference (ITD) between the output audio signals. The apparatus is adapted to: receive a current ITD value and an audio frame, and, based on the ITD of the current frame and the ITD of the previous frame, determine a transition time for performing a time shift on at least one of a first output signal and a second output signal. t 1. t 2. The device is suitable for: generating the first output signal and the second output signal at a determined switching time. t 1. t 2. Application time shift.

[0009] According to a third aspect, an apparatus is provided, comprising: processing circuitry and a memory coupled to the processing circuitry, wherein the memory includes instructions that, when executed by the processing circuitry, cause the apparatus to perform an operation comprising: receiving a current inter-channel time difference (ITD) value and an audio frame; and determining a conversion time for performing a time shift to execute an application on at least one of a first output signal and a second output signal, based on the ITD of the current frame and the ITD of a previous frame. t 1. t 2; and during the generation of the first output signal and the second output signal, at the determined conversion time t 1. t 2. Application time shift.

[0010] According to the fourth aspect, a computer program is proposed, which includes program code to be executed by the processing circuitry of the device, whereby the execution of the program code causes the device to perform the operation of the first aspect.

[0011] According to the fifth aspect, a computer program product is proposed, comprising a non-transitory storage medium including program code to be executed by the processing circuitry of a device, wherein execution of the program code causes the device to perform the operation of the first aspect. Certain embodiments may provide one or more of the following technical advantages. For switching across zero boundaries, the adjustment speed remains consistent, and computational complexity remains low. This method aims to utilize time delays to generate two channels, where the time delay can be updated per frame. Updates to the time delay can be accomplished with minimal transformation artifacts. Attached Figure Description

[0012] The accompanying drawings, included to provide a further understanding of this disclosure and incorporated herein by reference and forming part of this application, illustrate certain non-limiting embodiments of the inventive concept. In the drawings: Figure 1 This is a block diagram illustrating an environment in which various embodiments of the present disclosure may be implemented; Figure 2 This is a block diagram of an audio object renderer according to some embodiments of the present disclosure; Figure 3 This is a block diagram of a parametric stereo decoder according to some embodiments of the present disclosure; Figure 4 This is a flowchart illustrating the operation of an ITD synthesizer according to some embodiments of the present disclosure; Figure 5 This is a block diagram of an ITD synthesizer according to some embodiments of the present disclosure; Figure 6 This is a flowchart illustrating the operation of an ITD synthesizer according to some embodiments of the present disclosure; Figure 7 This is an illustration of buffer operations performed by an ITD synthesizer according to some embodiments of the present disclosure; Figure 8 This is a flowchart illustrating the operation of an ITD synthesizer according to some embodiments of the present disclosure; Figures 9 to 11 This is an illustration of buffer operations performed by an ITD synthesizer according to some embodiments of the present disclosure; Figure 12 This is a diagram of the sinc resampling function for processing the compressed and extended segments of a signal; Figure 13 This is a block diagram of an audio object renderer according to some embodiments; Figure 14 This is a block diagram of a host computer communicating with an encoder and / or decoder according to some embodiments; and Figure 15 This is a block diagram of a virtualized environment according to some embodiments. Detailed Implementation

[0013] Some embodiments contemplated herein will now be described more fully with reference to the accompanying drawings. Embodiments are provided by way of example to convey the scope of the subject matter to those skilled in the art, wherein examples of embodiments of the inventive concept are shown. However, the inventive concept can be embodied in many different forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the inventive concept to those skilled in the art. It should also be noted that these embodiments are not mutually exclusive. Components from one embodiment may be suitably assumed to be present / used in another embodiment.

[0014] Figure 1 Examples of operating environments in which various embodiments of this disclosure can be implemented are shown. Go to Figure 1 In the example operating environment 100, encoder 102 receives data (such as audio files) to be encoded from an entity and / or from storage device 108 via network 104 (such as host 106). In some embodiments, host 106 may communicate directly with encoder 102. Encoder 102 encodes the audio file as described herein and stores the encoded audio file in storage device 108, or transmits the encoded audio file via network 110 to decoder 112 having audio object renderer 114. Audio object renderer 114 within decoder 112 renders the decoded audio file and transmits the rendered decoded audio file to audio player 116 for playback. For example, audio player 116 may play the rendered decoded audio file as a spatial audio representation of a virtual reality conference or computer game. Audio player 116 may be a user device, terminal, mobile phone, etc., or be included in a user device, terminal, mobile phone, etc. In other embodiments, host 106 may transmit the encoded audio file to audio object renderer 114 via network 110. In some embodiments, where the decoded audio is not directly “fit” for the audio player 116, the audio object renderer 114 may be a separate device between the decoder 112 and the audio player 116, for example, to render a decoded audio file for headphones.

[0015] As mentioned earlier, changing the sign of the time delay parameter requires a shift operation on both output channels. To limit the complexity of the shift operation, the total conversion length is shared between the channels. This sharing is performed proportionally to the magnitude of the shift on each side of the zero point, according to the following formula:

[0016] in and It is the time difference between the audio channels. It is a subframe index. It is the total conversion length, and and This is the conversion length during which a time stretching or compression operation is performed. Time stretching or compression operations can also be called time shifting or resampling operations. The conversion length can be expressed in seconds or the number of samples in the discrete sampled audio signal. The conversion length can also be referred to as the conversion time.

[0017] The above formula can be simplified (rounding down to the nearest integer) as follows:

[0018] In some embodiments of this disclosure, such as Figure 2 As shown, the method operates within an ITD synthesizer implemented within an audio object renderer 114. In other embodiments, the ITD synthesizer may be implemented within a parametric stereo decoder. The method operates on audio segments called frames, where each frame or subframe... Depend on It consists of 10 samples.

[0019]

[0020] Frames can, for example, constitute audio segments from a decoded audio object, a mono downmix channel in a parametric stereo decoder, or an input channel to an audio object renderer. Here, the audio object renderer 114 receives an audio object that includes an audio signal and location metadata describing the location of the audio object. This location can be absolute or relative to the listener's location. The location metadata is input to an HR filter module 210, which provides ITD values ​​and a set of HR filters for the left and right channels. For frames... The time delay parameter is a range Integers within.

[0021] When the frame comes from a parametric stereo decoder This can be found by analyzing the input channels of the stereo encoder. Preferably, this is done by compensating before generating the lower mixed channel. Align the input channel. The lower mix channel will be encoded along with the stereo parameters, including decoding and reconstruction in the parametric stereo decoder. The parametric stereo decoder will reconstruct the downmixed signal; stereo parameters include at least... Reconstruction and corresponding Combine two output channels.

[0022] Audio and location metadata can come from, for example, an audio object decoder or be generated by a 3D audio engine for spatial audio representation, such as for virtual reality conferences or computer games. The HR filter module 210 can be, for example, a database storing filter and ITD values, or it can be a model-based database that generates filter and ITD values ​​for a given location. ITD values ​​are input to an ITD synthesizer 220, which generates two output signals based on the input audio frame, where the output signals have the desired ITD. The two channels are filtered by a left filter 230 and a right filter 240 to produce synthesized left and right channels. Audio objects can be added along with one or more additional objects. The output left and right channels can be forwarded to an audio device for playback.

[0023] This is in the operation of method 400. Figure 4 As shown in the flowchart, the ITD synthesizer 220 performs in some embodiments. In block 401, the ITD synthesizer 220 receives the current ITD and audio frames, where each frame include Each sample. Upon receiving an audio frame or at any time during frame processing, the ITD synthesizer 220 may store at least a portion of the current input audio frame to be used for ITD processing in a subsequent audio frame in box 403.

[0024] In block 405, ITD synthesizer 220 determines the transition time for performing the time shift applied to at least one of output signal 0 and output signal 1 based on the sign of the interchannel time difference ITD of the current audio frame and the ITD of the previous audio frame. t1 , t2 ITD comes from Figure 2 The HRF filter module 210 is located within the context of the ITD synthesizer. In other embodiments where the ITD synthesizer is implemented as part of a parametric stereo decoder, the ITD is reconstructed from the bitstream. Here, time shift represents the operation of smoothly transitioning to the target ITD.

[0025] In block 407, when generating output signal 0 and output signal 1, ITD synthesizer 220 applies time shift within the determined conversion times t1 and t2.

[0026] Before describing further details of the ITD synthesizer 220, Figure 3An embodiment in which an ITD synthesizer can be implemented within a parametric stereo decoder 300 is shown. In the parametric stereo decoder 300, stereo parameters, including ITD parameters, are decoded by a parametric decoder 310, and optionally, the reconstructed residual signal is generated by a residual decoder 320. A downmixer decoder 330 is configured to decode and reconstruct the encoded downmixer signal to output a reconstructed downmixer signal, wherein time shifting may be applied. The reconstructed downmixer, the reconstructed stereo parameters, and optionally the reconstructed residual signal are fed to a stereo upmixer 340 to produce a reconstructed stereo signal. An ITD synthesizer 220 is part of the stereo upmixer 340.

[0027] ITD Synthesizer 220 in Figure 5 The details are described further in the text, and it is also executed. Figure 6 The operation shown. Go to Figure 6 In step 601, the current input audio frame is used. The signal memory 520 is used to fill the processing buffer 510. (Memory) The length should be at least the maximum time shift and the look-back / look-ahead memory required for the resampling function The sum of .

[0028]

[0029] Figure 7 The image shows the processing buffer 510, with the intermediate curve illustrating the processing buffer. ,in The first value corresponds to the current frame. The input frame is also fed to the signal memory 520 for use in the next frame. In step 603, the current frame... The ITD value along with the values ​​from the ITD memory 540 The data was input into the conversion length calculator 530. Based on... and Calculate the conversion length. First, based on the frame length required by resamplers 570 and 580. and ahead of time To calculate the total conversion length. If Therefore, a small portion of the processing buffer must be reserved to accommodate lead time without introducing processing latency due to resampling. The transformation can be divided into three parts. , and Conversion length This can be considered as the buffer length to avoid reading memory during resampling operations. The resampler must leave at least [a certain length] at the end of the buffer. The samples are then subjected to resampling or filtering operations. The transformation length is calculated using the following formula:

[0030] Then the total conversion time for

[0031] in This is the maximum allowed conversion length. It can be set to... This means that the entire frame time is allowed to be used to perform the conversion. However, if If it is large, then it may be necessary to use Limit the maximum allowed conversion length to achieve faster conversion and potentially lower complexity.

[0032] Figure 8 The operation performed by ITD synthesizer 220 in determining the total conversion length is illustrated. In block 801, ITD synthesizer 220 is based on frames... Frame length and the advance memory required for resampling To calculate the total transformation length, the total transformation is divided into sections including... t 1 and t 2 has two parts.

[0033] In block 803, ITD synthesizer 220 is based on advance memory. Determine conversion time t 3. In box 805, the ITD synthesizer 220 is based on the maximum permissible conversion length. and conversion time t 3. Determine the total conversion length.

[0034] The following time-shift operations can be divided into two groups: 1. and The signs are the same or one of them is zero.

[0035] 2. The sign of ITD is non-zero and changes, that is... .

[0036] Case 1 - ITDs have the same sign or one of them is zero if and If the signs are the same or one of them is zero, the shift can be handled by processing only one channel, which means processing step 605, where the resampler 570 adjusts the processing buffer 510 to fill the output buffer A 550. This can be achieved by allocating the entire conversion length to... And To achieve this, set it to zero, that is,

[0037] in This indicates that the current input subframe is assumed to be in The starting index of each time shift segment, starting from [location]. It is a resampling segment and The length of the buffer. In this case, the input signal portion of the processing buffer is simply copied to the output buffer B 560.

[0038] When the time delay of the current frame is the same as that of the previous frame, that is Output time delay synthesis is generated by pointing to the corresponding starting point in the processing buffer. The sign determines which of the two channels the delay is applied in. For example, a positive value... It can be indicated that the left channel of a stereo pair is in front of the right channel. In this case, the right channel should be delayed, and the left channel should be output without delay. Figure 7 This situation is illustrated in the diagram. In this case, the resampling operation on the output buffer A 550 has the same input and output lengths and is equivalent to a copy operation.

[0039] When the absolute value of the time delay of the current frame is greater than the absolute value of the time delay of the previous frame, This generates a transformation to allow for smooth conversion between delayed values. Figure 9 This situation is illustrated in [the image]. By changing the frame length from [the previous text]... of Extended to length The conversion is completed by outputting frames. If the conversion time... Greater than zero (i.e.) Then it simply copies the last output channel from the processing buffer. The sample reached The delay Here, it can also be noted that if If the input length is the same as the output length, then resampling will be equivalent to a copy operation.

[0040] When the absolute value of the time delay decreases, that is The expression for the input frame length remains the same. However, the length... Now it will be greater than the obtained length. And resampling corresponds to shortening the frame length. This is in Figure 10 As shown in the example. This means that the last output buffer A is copied from the processing buffer 510. sample.

[0041] Case 2 - ITD has a different sign and is non-zero If the sign of ITD changes, that is, if In this case, a shift operation must be performed on both channels. The total conversion length in this scenario... It is divided into two parts according to the following formula: in This indicates rounding to the nearest integer. Next, as... Figure 11 As shown, using conversion time and The signal is time-shifted to assemble output buffer A and output buffer B. First, from the length... of Start to the first output buffer A The sample is resampled from the buffer. The remaining portion of the processing buffer is then copied to the output buffer A to fill the final buffer. Samples. Then, the first sample from the processing buffer, starting at index 0, is copied to the first sample from the output buffer B. Sample. By analyzing the length In the processing buffer Begin resampling the samples to create the next output buffer B. Sample. Finally, the last part of the buffer will be processed. The sample is copied to output buffer B. The last input frame. The samples are stored in memory for processing the next subframe. Output buffer A and output buffer B are allocated to output channel 0 and output channel 1 for left HRIR filtering and right HRIR filtering, respectively. The allocation of output channels is based on the following... and Completed using symbols:

[0042] in This represents the logical AND. This indicates the presence of OR. The resampling operation is implemented using a polyphase filter with a sinc function derived from a lookup table. The benefit of dividing the conversion length between the two channels is that the computational complexity of the resampler is proportional to the conversion length, and by constraining the total conversion length... The overall complexity is kept below a certain limit. Furthermore, the conversion speeds on the left and right channels remain approximately the same (approximately due to integer rounding). Since conversion artifacts are lower at lower conversion speeds, this keeps conversion artifacts to a minimum.

[0043] Resampling and copying operations can also be described with reference to the buffer index as follows. When the ITD symbols are different, the first conversion is from... arrive Turn on output buffer A 550 and move it from arrive Turn on the output buffer B 560 shift. An example of this process is in Figure 11 As shown in the diagram. In step 605, the resampler 570 fills the first output buffer A550. Sample. This is achieved by resampling the length. samples Samples suitable for output buffer A 550 To complete this. In the same step, the sample... Copy to the corresponding index in output buffer B 560 In step 607, the resampler 580 is adapted. Sample length To fit the length in the output buffer B560 samples .exist In the case of step 609, the last sample Copy from processing buffer 510 to output buffer B 560. Note that output buffers 550 and 560 may have different alignments, causing the index to shift according to the processing buffer. In this case, the aforementioned indexes associated with output buffers 550 and 560 will be shifted by that amount, but the segment will still be appended in the same manner as described herein. For example, Figure 7 and Figure 9 The output buffer A550 in the middle can be offset Alignment with the processing buffer index. Furthermore, resamplers 570 and 580 can be implemented using the same resampling function for different input operations.

[0044] In step 611, similar to cases 1 and 2 above, output buffer A 550 and output buffer B 560 are assigned to output 0 and output 1, respectively. In intermediate buffers A and B, buffer A always corresponds to the channel currently having a non-zero ITD and being delayed, while buffer B corresponds to the channel having a zero ITD. These assumptions simplify the processing of the intermediate buffers, and output assignment is a simple step that can be completed last to assign the processed buffers to the correct output channels. The assignment of the output buffers depends on the following pseudocode. and Symbols: if

[0045] if

[0046] Output buffer A (550) outputs 1, output buffer B (560) outputs 0. otherwise Output buffer A (550) outputs 0, output buffer B (560) outputs 1. otherwise if

[0047] Output buffer A (550) outputs 1, output buffer B (560) outputs 0. otherwise Output buffer 550 outputs 0, output buffer 560 outputs 1. It can also be simplified to:

[0048] in This indicates a logical AND and also indicates that OR is included. This means assigning output buffer A 550 to output 1 and output buffer B 560 to output 0, and This means that output buffer A 550 is assigned to output 0 and output buffer B 560 is assigned to output 1. Output buffer 0 and output buffer 1 can correspond to the left and right binaural channels respectively. They can also be numbered differently, for example, output buffer 1 and output buffer 2.

[0049] In other words, if If it is zero, then the current ITD Used to determine which buffer is delayed. If If positive, then output 0 precedes output 1, and output 1 should be delayed. If If the value is negative, then the output 1 should precede the output 0, and the output 0 should be delayed. If If it is not zero, then the previous ITD Determine which buffer to shift first. If If the result is positive, first shift the output to 1 in buffer A 550, then shift the output to 0 in buffer B 560. If... If the value is negative, the output is first shifted to 0 in buffer A550, and then shifted to 1 in buffer B560. It should be noted that... The definition of the symbol can be reversed, in which case the output 0 and output 1 will switch their positions.

[0050] It should be noted that step 611 can occur before step 605 by having output buffer A 550 and output buffer B 560 already assigned to designated output 1 and output 2, such that the outputs are filled during steps 605 to 611. In an embodiment, output 0 and output 1 may correspond to the left channel and right channel, respectively.

[0051] Resampling with sinc function The described method depends on the resampling function used to handle the compressed and expanded segments of the signal. This can be achieved using the sinc resampling function, such as... Figure 12 As shown. Given length and output length input signal These steps can be indexed by fractions. The input signal is resampled. For each ,calculate

[0052] in, This indicates the floor function.

[0053] Because the sinc function is computationally complex, it might be desirable to store it in a table with a predefined resolution. For example, The resolution implies that 64 samples between the zero crossovers of the sinc function might be appropriate (see [reference]). Figure 12 Then you can find the corresponding index in the sinc table at the following location.

[0054]

[0055] Output value It can be found by the following summation

[0056] in

[0057]

[0058]

[0059] Note that although the above embodiments are described using an audio object renderer (e.g., a decoder), the various embodiments described above can also be done at the encoder, where the shifted outputs (i.e., outputs 0 and 1) are shifted at the encoder rather than at the audio object renderer.

[0060] Figure 13An audio object renderer 114 (e.g., a decoder) according to some embodiments is illustrated, wherein the audio object renderer 114 is implemented as a stand-alone device. As used herein, an audio object renderer refers to a device capable of, configured, arranged, and / or operable to decode encoded objects and communicate with network nodes, encoders, and / or decoders. Examples of audio object renderers include, but are not limited to, smartphones, mobile phones, cellular phones, Voice over IP (VoIP) phones, wireless local loop phones, desktop computers, personal digital assistants (PDAs), wireless cameras, game consoles or devices, storage devices, playback devices, wearable terminal devices, wireless endpoints, mobile stations, tablet computers, laptops, laptop embedded devices (LEEs), laptop devices (LMEs), smart devices, wireless customer premises equipment (CPEs), in-vehicle or vehicle embedded / integrated wireless devices, etc.

[0061] Audio object renderers can support device-to-device (D2D) communication, for example, by implementing 3GPP standards for sidelink communication, dedicated short-range communication (DSRC), vehicle-to-vehicle (V2V), vehicle-to-infrastructure (V2I), or vehicle-to-everything (V2X). In other examples, the decoder may not necessarily have a user in the sense of a human user who owns and / or operates the relevant device.

[0062] The audio object renderer 114 includes processing circuitry 1302, which is operatively coupled via bus 1304 to an input / output interface 1306, a power supply 1308, a memory 1310, a communication interface 1312, and / or any other component or any combination thereof. Certain decoders may utilize... Figure 13 The components shown may be all or a subset. The level of integration between components can vary from one decoder to another. Furthermore, some decoders may contain multiple instances of components, such as multiple processors, memories, transceivers, transmitters, receivers, etc.

[0063] Processing circuitry 1302 is configured to process instructions and data and can be configured to implement any sequential state machine operable to execute instructions stored in memory 1310 as a machine-readable computer program. Processing circuitry 1302 can be implemented as one or more hardware-implemented state machines (e.g., in discrete logic, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), etc.); programmable logic along with appropriate firmware; one or more stored computer programs, a general-purpose processor (such as a microprocessor or digital signal processor (DSP)) along with appropriate software; or any combination of the foregoing. For example, processing circuitry 1302 may include multiple central processing units (CPUs).

[0064] In this example, the input / output interface 1306 can be configured to provide one or more interfaces to input devices, output devices, or one or more input and / or output devices. Examples of output devices include speakers, sound cards, video cards, displays, monitors, actuators, transmitters, smart cards, other output devices, or any combination thereof. Input devices can allow a user to capture information into the audio object renderer 114. Examples of input devices include touch-sensitive or presence-sensitive displays, cameras (e.g., digital cameras, digital camcorders, webcams, etc.), microphones, sensors, steering wheels, touchpads, scroll wheels, smart cards, etc. Presence-sensitive displays may include capacitive or resistive touch sensors to sense input from the user. Sensors may be, for example, accelerometers, gyroscopes, tilt sensors, force sensors, magnetometers, optical sensors, proximity sensors, biometric sensors, etc., or any combination thereof. Output devices can use the same type of interface port as input devices. For example, a Universal Serial Bus (USB) port can be used to provide both input and output devices.

[0065] In some embodiments, power supply 1308 is configured as a battery or battery pack. Other types of power sources may be used, such as external power sources (e.g., power outlets), photovoltaic devices, or batteries. Power supply 1308 may also include power circuitry for delivering power from power supply 1308 itself and / or external power sources to various parts of audio object renderer 114 via input circuitry or an interface such as a power cable. Power delivery may be, for example, charging of power supply 1308. The power circuitry may perform any formatting, transformation, or other modifications on the power from power supply 1308 to adapt the power to the corresponding components of audio object renderer 114 to which it is supplied power.

[0066] Memory 1310 may be or be configured to include memory such as random access memory (RAM), read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), disk, optical disk, hard disk, removable cartridge, flash drive, etc. In one example, memory 1310 includes one or more applications 1314 (such as an operating system, web browser application, gadget, gadget engine, or other application) and corresponding data 1316. Memory 1310 may store any of various operating systems or combinations of operating systems for use by audio object renderer 114.

[0067] Memory 1310 can be configured to include multiple physical drive units, such as a Redundant Array of Independent Disks (RAID), flash memory, a USB flash drive, an external hard drive, a thumb drive, a pen drive, a key drive, a high-density digital multifunction disc (HD-DVD) optical disc drive, an internal hard drive, a Blu-ray disc drive, a holographic digital data storage (HDDS) optical disc drive, an external micro dual in-line memory module (DIMM), synchronous dynamic random access memory (SDRAM), external micro DIMM SDRAM, smart card memory (such as a tamper-proof module in the form of a universal integrated circuit card (UICC), including one or more subscriber identity modules (SIMs), such as USIM and / or ISIM), other memory, or any combination thereof. The UICC may be, for example, an embedded UICC (eUICC), an integrated UICC (iUICC), or a removable UICC commonly referred to as a "SIM card." Memory 1310 can allow the audio object renderer 114 to access instructions, applications, etc., stored on transient or non-transient memory media to unload or upload data. Articles of art utilizing communication systems may be tangibly embodied in or in memory 1310, which may be or include a device-readable storage medium.

[0068] Processing circuitry 1302 can be configured to communicate with an access network or other network using communication interface 1312. Communication interface 1312 may include one or more communication subsystems and may include or be communicatively coupled to antenna 1322. Communication interface 1312 may include one or more transmitters for communication, such as through one or more remote transmitters communicating with another device capable of wireless communication (e.g., another UE or a network node in the access network). Each transceiver may include transmitter 1318 and / or receiver 1320 adapted to provide network communication (e.g., optical, electrical, frequency allocation, etc.). Furthermore, transmitter 1318 and receiver 1320 may be coupled to one or more antennas (e.g., antenna 1322) and may share circuitry, software, or firmware, or alternatively, be implemented separately.

[0069] In the illustrated embodiments, the communication functions of the communication interface 1312 may include cellular communication, Wi-Fi communication, LPWAN communication, data communication, voice communication, multimedia communication, short-range communication such as Bluetooth, near-field communication, location-based communication such as using a Global Positioning System (GPS) to determine location, another similar communication function, or any combination thereof. Communication may be implemented according to one or more communication protocols and / or standards, such as IEEE 802.11, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), GSM, LTE, New Radio (NR), UMTS, WiMax, Ethernet, Transmission Control Protocol / Internet Protocol (TCP / IP), Synchronous Optical Networking (SONET), Asynchronous Transfer Mode (ATM), QUIC, Hypertext Transfer Protocol (HTTP), etc.

[0070] Regardless of the type of sensor, the audio object renderer can provide the output of decoded data via its communication interface 1312 through a wireless connection to the network node.

[0071] When an audio object renderer is in the form of an Internet of Things (IoT) device, it can be a device for use in one or more application domains, including but not limited to urban wearable technology, extended industrial applications, and healthcare. Non-limiting examples of such IoT devices include devices embedded in: connected refrigerators or freezers, TVs, connected lighting fixtures, electricity meters, robotic vacuum cleaners, voice-controlled smart speakers, home security cameras, thermostats, electric door locks, connected doorbells, autonomous vehicles, surveillance systems, weather monitoring devices, vehicle parking monitoring devices, electric vehicle charging stations, smartwatches, fitness trackers, head-mounted displays for augmented reality (AR) or virtual reality (VR), and wearable devices for haptic or sensory enhancement. The decoder in the form of an IoT device, in addition to [the above], [is also relevant to the following]. Figure 13 In addition to the other components described in the audio object renderer 114 shown, it also includes circuitry and / or software depending on the intended application of the IoT device.

[0072] Figure 14 This is a block diagram of host 1400 according to the various aspects described herein. As used herein, host 1400 can be or include various combinations of hardware and / or software, including processing resources in a standalone server, blade server, cloud-implemented server, distributed server, virtual machine, container, or server cluster. Host 1400 can provide one or more services to one or more UEs.

[0073] Host 1400 includes processing circuitry 1402 operably coupled via bus 1404 to input / output interface 1406; network interface 1408; power supply 1410; and memory 1412. Other components may be included in other embodiments. The features of these components may be substantially similar to those described in the preceding figures (such as...). Figure 13 The characteristics of the device description make it generally applicable to the corresponding components of the host 1400.

[0074] Memory 1412 may include one or more computer programs, including one or more host applications 1414 and data 1416, which may include user data, such as data generated by the UE for the host 1400 or data generated by the host 1400 for the UE. Embodiments of the host 1400 may utilize only a subset or all of the illustrated components. The host application 1414 may be implemented in a container-based architecture and may provide support for video codecs (e.g., Multi-Function Video Codec (VVC), High Efficiency Video Codec (HEVC), Advanced Video Codec (AVC), MPEG, VP9) and audio codecs (e.g., EVS, IVAS, FLAC, Advanced Audio Codec (AAC), MPEG, G.711), including transcoding for multiple different categories, types, or implementations of the UE (e.g., mobile phones, desktop computers, wearable display systems, head-up display systems). The host application 1414 may also provide user authentication and authorization checks and may periodically report health, routing, and content availability to a central node (such as a device at the edge of the core network or on the edge). Therefore, host 1400 can select and / or indicate different hosts for the over-the-top service of the UE. Main application 1414 can support various protocols, such as HTTP Real-Time Streaming (HLS), Real-Time Messaging Protocol (RTMP), Real-Time Streaming Protocol (RTSP), HTTP Dynamic Adaptive Streaming (MPEG-DASH), etc.

[0075] Figure 15This is a block diagram illustrating a virtualization environment 1500, in which functionality implemented by some embodiments of or components of the audio object renderer 114 can be virtualized. In this context, virtualization means creating a virtual version of a device or apparatus, which may include virtualized hardware platforms, storage devices, and network resources. As used herein, virtualization can be applied to any device or component thereof described herein, and involves at least a portion of its functionality being implemented as an implementation of one or more virtual components. Some or all of the functionality described herein can be implemented as virtual components executed by one or more virtual machines (VMs) in one or more virtual environments 1500 hosted by one or more hardware nodes, such as hardware computing devices operating as decoders, encoders, network nodes, UEs, core network nodes, or hosts. Furthermore, in embodiments where virtual nodes do not require radio connectivity (e.g., core network nodes or hosts), the nodes can be fully virtualized.

[0076] Application 1502 (which may alternatively be referred to as a software instance, virtual device, network function, virtual node, virtual network function, etc.) runs in virtualization environment 1500 to implement some of the features, functions and / or benefits of some embodiments disclosed herein.

[0077] Hardware 1504 includes processing circuitry, memory storing software and / or instructions executable by the hardware processing circuitry, and / or other hardware devices as described herein, such as network interfaces, input / output interfaces, etc. Software may be executed by the processing circuitry to instantiate one or more virtualization layers 1506 (also referred to as hypervisors or virtual machine monitors (VMMs)), providing VMs 1508A and 1508B (one or more of which may generally be referred to as VM 1508), and / or performing any functionality, features, and / or benefits described with respect to some embodiments described herein. Virtualization layer 1506 may present a virtual operating platform that resembles networked hardware as VM 1508.

[0078] VM 1508 includes virtual processing, virtual memory, virtual networking or interfaces, and virtual storage, and can be run by a corresponding virtualization layer 1506. Different embodiments of instances of virtual device 1502 can be implemented on one or more VMs 1508, and can be implemented in different ways. Hardware virtualization is referred to as Network Functions Virtualization (NFV) in some contexts. NFV can be used to consolidate many network device types onto industry-standard high-capacity server hardware, physical switches, and physical storage that can reside in data centers and customer premises.

[0079] In the context of NFV, VM 1508 can be a software implementation of the physical machine running the program as if it were running on a physical, non-virtualized machine. Each of the VM 1508, along with the portion of the hardware 1504 that executes the VM (which is dedicated to that VM and / or shared by that VM and other VMs within it), forms a separate virtual network element. Still within the context of NFV, the virtual network function is responsible for handling the specific network function running on one or more VMs 1508 above the hardware 1504 and corresponding to the application 1502.

[0080] Hardware 1504 can be implemented in a standalone network node with general or specific components. Hardware 1504 can implement some functions via virtualization. Alternatively, hardware 1504 can be part of a larger hardware cluster (e.g., in a data center or CPE), where many hardware nodes work together and are managed via management and orchestration 1510, which in particular oversees the lifecycle management of application 1502. In some embodiments, hardware 1504 is coupled to one or more radio units, each radio unit including one or more transmitters and one or more receivers that can be coupled to one or more antennas. The radio units can communicate directly with other hardware nodes via one or more suitable network interfaces and can be used in combination with virtual components to provide a radio-capable virtual node, such as a radio access node or base station. In some embodiments, a control system 1512 can be used to provide some signaling, which can alternatively be used for communication between the hardware nodes and the radio units.

[0081] While the computing devices described herein (e.g., decoders, audio object renderers, encoders, hosts) may include the combinations of hardware components shown, other embodiments may include computing devices with different combinations of components. It should be understood that these computing devices may include any suitable combination of hardware and / or software required to perform the tasks, features, functions, and methods disclosed herein. The determination, computation, acquisition, or similar operations described herein may be performed by processing circuitry that processes information by, for example, converting acquired information into other information, comparing acquired or converted information with information stored in network nodes, and / or performing one or more operations based on the acquired or converted information, and making a determination as a result of said processing. Furthermore, although components are depicted as single boxes located within larger boxes or nested within multiple boxes, in practice, computing devices may include multiple different physical components that make up a single illustrated component, and functionality may be partitioned between individual components. For example, a communication interface may be configured to include any of the components described herein, and / or the functionality of a component may be partitioned between processing circuitry and the communication interface. In another example, non-computationally intensive functions of any such component may be implemented in software or firmware, and computationally intensive functions may be implemented in hardware.

[0082] In some embodiments, some or all of the functions described herein may be provided by processing circuitry that executes instructions stored in memory; in some embodiments, this processing circuitry may be a computer program product in the form of a non-transitory computer-readable storage medium. In alternative embodiments, some or all of the functions may be provided by processing circuitry without executing instructions stored on a separate or discrete device-readable storage medium, such as in a hard-wired manner. In any of those particular embodiments, the processing circuitry may be configured to perform the described functions regardless of whether instructions stored on a non-transitory computer-readable storage medium are executed. The benefits provided by such functions are not limited to separate processing circuitry or other components of the computing device, but are generally enjoyed by the computing device and / or typically by end users and wireless networks.

[0083] Example Implementation 1. A method in an inter-channel time difference (ITD) synthesizer (220, 340, 1502), the method comprising: Receive (401) the current ITD and audio frames, where each frame include One sample; Store (403) at least a portion of the current input audio frame in the signal memory; Based on the interchannel time difference (ITD) of the current input audio frame and the ITD of the previous input audio frame, determine (405) the transition time for performing the time shift applied to at least one of output signal 0 and output signal 1. t 1. t 2; and When generating output signal 0 and output signal 1, at the determined transition time t 1. t 2. Application (407) time shift.

[0084] 2. The method according to embodiment 1, wherein the audio frame is a portion of an object audio signal having location metadata describing the location relative to a listener, the method further comprising obtaining a time shift from the location metadata.

[0085] 3. The method according to any one of Embodiments 1 to 2 further includes: Based on frames Frame length and advance memory required for resampling To calculate the total conversion length (801), the total conversion length is divided into including t 1 and t 2 has two parts; Based on advanced memory Determine the length of the (803) buffer. t 3; and Based on the maximum allowed conversion length and buffer length t 3. Determine the total conversion length (805).

[0086] 4. The method according to Example 3, wherein the buffer length is determined. t 3. The buffer length is determined according to the following formula. t 3: , The total conversion length is determined according to the following formula:

[0087] in .

[0088] 5. The method according to any one of Examples 1 to 4, wherein the conversion time is determined. t 1. t 2 includes: In response to the fact that the symbol of the current ITD is the same as that of the previous ITD, the total transformation length is allocated to the transformation time. t 1. t Set one of the conversion times in step 2 and set the other conversion time to zero.

[0089] 6. The method according to any one of Examples 1 to 4, wherein the conversion time is determined. t 1. t2 includes: In response to the fact that the sign of the current ITD is different from that of the previous ITD, the transition time is determined by splitting the total transition length into two parts. t 1. t 2. Apply a shift operation to both output signal 0 and output signal 1.

[0090] 7. The method according to Example 6, wherein the total conversion length is split into two parts to determine the conversion time. t 1. t 2. The total conversion length is divided according to the following formula:

[0091] in This represents the rounding operation to the nearest integer.

[0092] 8. The method according to any one of embodiments 3 to 7 further includes: Use the current input audio frame and signal memory of the object audio to fill the processing buffer (510). The determined switching time is applied when generating output signal 0 and output signal 1. t 1. t 2 includes: The response is that the sign of the current ITD is the same as the sign of the previous ITD, or the response is that one of the current ITD and the previous ITD is zero: By allocating the total conversion length to t 1 and will t 2 is set to zero to adjust the processing buffer (510) to fill the first output buffer (550); and The input signal portion of the processing buffer (510) is copied to the second output buffer (560).

[0093] 9. The method according to Example 8, wherein the determined switching time is applied when generating output signal 0 and output signal 1. t 1. t 2 also includes: In response to Furthermore, the sign of one of the current ITD and the previous ITD is negative, thereby indicating that the total conversion length of the output buffer delay will include either the first output buffer (550) or the second output buffer (560) associated with the other of the output signals 0 and 1, which precedes the other of the output signals 0 and 1.

[0094] 10. The method according to any one of embodiments 8 to 9, wherein the determined conversion time is applied when generating output signal 0 and output signal 1. t 1. t 2 also includes: In response to The conversion is generated in the following way: The length of the frame in the processing buffer (510) is changed from the length of Extended to length The output frame; and Response to buffer length Greater than zero, the last output channel is added by copying from the processing buffer (510). sample, .

[0095] 11. The method according to any one of Examples 8 to 10, wherein the determined conversion time is applied when generating output signal 0 and output signal 1. t 1. t 2 also includes: In response to The last part of the first output buffer (550) is added by copying from the processing buffer (510). t 3 samples.

[0096] 12. The method according to any one of Examples 8 to 11, wherein the determined switching time is applied when generating output signal 0 and output signal 1. t 1. t 2 also includes: In response to The total conversion length is divided according to the following formula:

[0097] in This represents the rounding operation to the nearest integer, where the total conversion length includes: For length samples Resample to fit the samples of the first output buffer (550). ; Sample Copy to the corresponding index in the second output buffer (560); For length samples Resample to fit the length in the second output buffer (560) samples .

[0098] 13. The method according to any one of Examples 8 to 12, wherein the determined switching time is applied when generating output signal 0 and output signal 1. t 1. t 2 also includes: In response to and The first output buffer (550) is assigned to output signal 1 and the second output buffer (560) is assigned to output signal 2. In response to and The first output buffer (550) is assigned to output signal 0 and the second output buffer (560) is assigned to output signal 1. In response to The first output buffer (550) is assigned to output signal 1 and the second output buffer (560) is assigned to output signal 0; and In response to The first output buffer (550) is assigned to output signal 0 and the second output buffer (560) is assigned to output signal 1.

[0099] 14. An apparatus (114, 1502) having an ITD synthesizer adapted to: Receive (401) the current ITD and audio frames, where each frame include One sample; Store (403) at least a portion of the current input audio frame in the signal memory; Based on the interchannel time difference (ITD) of the current input audio frame and the ITD of the previous input audio frame, determine (405) the transition time for performing the time shift applied to at least one of output signal 0 and output signal 1. t 1. t 2; and When generating output signal 0 and output signal 1, at the determined transition time t 1. t 2. Application (407) time shift.

[0100] 15. The apparatus (114, 300, 1502) according to Example 14, wherein the ITD synthesizer (220, 340, 1502) is also adapted to perform according to any one of Examples 2 to 12.

[0101] 16. An apparatus (114, 300, 1502) having an inter-channel time difference (ITD) synthesizer (220, 340, 1502), comprising: Processing circuit (1202); and A memory (1210), coupled to processing circuitry, includes instructions that, when executed by the processing circuitry, cause the ITD synthesizer (220, 340, 1502) to perform operations including: Receive (401) the current ITD and audio frames, where each frame include One sample; Store (403) at least a portion of the current input audio frame in the signal memory; Based on the interchannel time difference (ITD) of the current input audio frame and the ITD of the previous input audio frame, determine (405) the transition time for performing the time shift applied to at least one of output signal 0 and output signal 1. t 1. t 2; and When generating output signal 0 and output signal 1, at the determined transition time t 1. t 2. Application (407) time shift.

[0102] 17. The apparatus (114, 300, 1502) according to embodiment 16, wherein the memory includes additional instructions that, when executed by the processing circuitry, cause the ITD synthesizer (220, 340, 1502) to perform the operation according to any one of embodiments 2 to 13.

[0103] 18. A computer program comprising program code to be executed by a processing circuitry (1202) of a device (112, 300, 1502) having an inter-channel time difference (ITD) synthesizer (220, 340, 1502), wherein execution of the program code causes the ITD synthesizer (220, 340, 1502) to perform operations including: Receive (401) the current ITD and audio frames, where each frame include One sample; Store (403) at least a portion of the current input audio frame in the signal memory; Based on the interchannel time difference (ITD) of the current input audio frame and the ITD of the previous input audio frame, determine (405) the transition time for performing the time shift applied to at least one of output signal 0 and output signal 1. t 1. t 2; and When generating output signal 0 and output signal 1, at the determined transition time t 1. t 2. Application (407) time shift.

[0104] 19. The computer program of claim 18, comprising additional program code, wherein execution of the program code causes the ITD synthesizer (220, 340, 1502) to perform according to any one of embodiments 2 to 13.

[0105] 20. A computer program product comprising a non-transitory storage medium including program code to be executed by processing circuitry (1202) of a device (112, 300, 1502) having an inter-channel time difference (ITD) synthesizer (220, 340, 1502), wherein execution of the program code causes the ITD synthesizer (220, 340, 1502) to perform operations including: Receive (401) the current ITD and audio frames, where each frame include One sample; Store (403) at least a portion of the current input audio frame in the signal memory; Based on the interchannel time difference (ITD) of the current input audio frame and the ITD of the previous input audio frame, determine (405) the transition time for performing the time shift applied to at least one of output signal 0 and output signal 1. t 1. t 2; and When generating output signal 0 and output signal 1, at the determined transition time t 1. t 2. Application (407) time shift.

[0106] 21. The computer program product of claim 19, wherein the non-transitory storage medium includes additional program code, the execution of which causes the ITD synthesizer (220, 340, 1502) to perform according to any one of embodiments 2 to 13.

Claims

1. A method for adjusting the timing of an output audio signal to achieve a desired interchannel time difference (ITD) between the output audio signals, the method comprising: Receive (401) the current ITD value and the audio frame; Based on the ITD of the current frame and the ITD of the previous frame, determine (405) the conversion time for performing the time shift applied to at least one of the first output signal and the second output signal. t 1. t 2, wherein the conversion time is determined. t 1. t 2 includes: In response to the fact that the sign of the current ITD is different from that of the previous ITD, a shift operation is applied to both the first output signal and the second output signal by splitting the total conversion length into two parts according to the following formula: in This is the current ITD. It is the aforementioned previous ITD and This represents the rounding operation to the nearest integer. as well as When generating the first output signal and the second output signal, at the determined conversion time t 1. t The time shift described in 2 (407) is applied.

2. The method of claim 1, wherein at least a portion of the audio frame is stored (403) in a memory for use in synthesizing the ITD in subsequent frames.

3. The method according to any one of claims 1 to 2, wherein the audio frame is a portion of an audio object, the audio object comprising an audio signal and location metadata describing the location of the object, the method further comprising obtaining the time shift from the location metadata.

4. The method according to any one of claims 1 to 3, further comprising: Based on the current frame Frame length and advance memory required for resampling To calculate the total conversion length (801), the total conversion length is divided into including t 1 and t 2 has two parts; Based on the advanced memory Determine the (803) conversion length t 3; and Based on the maximum allowed conversion length and the conversion length t 3. Determine the total conversion length as described in (805).

5. The method of claim 4, wherein the conversion length is determined. t 3. The conversion length is determined according to the following formula. t 3: ,in It is the ITD of the current audio frame, which includes N samples, and the total conversion length is determined according to the following formula: ,in , in It is the maximum allowed conversion length.

6. The method according to any one of claims 1 to 5, wherein the conversion time is determined. t 1. t 2 includes: In response to the fact that the symbol of the current ITD is the same as that of the previous ITD, the total transformation length is allocated to the transformation time. t 1. t Set one of the conversion times in step 2 and set the other conversion time to zero.

7. The method according to any one of claims 4 to 6, further comprising: The audio frames are used to fill the processing buffer (510). The determined switching time is applied when generating the first output signal and the second output signal. t 1. t 2 includes: In response to the current ITD having the same sign as the previous ITD, or in response to either the current ITD or the previous ITD being zero: By allocating the total conversion length to t 1 and will t 2 is set to zero to adjust the processing buffer (510) to fill the first output buffer (550); and The input signal portion of the processing buffer (510) is copied to the second output buffer (560).

8. The method of claim 7, wherein the determined conversion time is applied when generating the first output signal and the second output signal. t 1. t 2 also includes: In response to Furthermore, the sign of one of the current ITD and the previous ITD is negative, thereby indicating that one of the first output signal and the second output signal precedes the other of the first output signal and the second output signal, and the output buffer of either the first output buffer (550) or the second output buffer (560) associated with the other of the first output signal and the second output signal will be delayed by the total conversion length.

9. The method according to any one of claims 7 to 8, wherein the determined conversion time is applied when generating the first output signal and the second output signal. t 1. t 2 also includes: In response to The conversion is generated in the following way: The length of the frame in the processing buffer (510) is changed from the length of Extended to length The output frame; as well as In response to the conversion length Greater than zero, the last output channel is added by copying from the processing buffer (510). sample, .

10. The method according to any one of claims 7 to 9, wherein the determined conversion time is applied when generating the first output signal and the second output signal. t 1. t 2 also includes: In response to The last part of the first output buffer (550) is added by copying from the processing buffer (510). t 3 samples.

11. The method according to any one of claims 7 to 10, wherein the determined conversion time is applied when generating the first output signal and the second output signal. t 1. t 2 also includes: In response to The total conversion length is divided according to the following formula: in This represents the rounding operation to the nearest integer, where the total conversion length is broken down into: For length samples Resample to fit the samples of the first output buffer (550). ; Sample Copy to the corresponding index in the second output buffer (560); For length samples Resample to fit the length in the second output buffer (560) samples .

12. The method according to any one of claims 7 to 11, wherein generating the second output signal further comprises: In response to the conversion length Greater than zero, the last output channel is added by copying from the processing buffer (510). sample, .

13. The method according to any one of claims 7 to 12, wherein the determined conversion time is applied when generating the first output signal and the second output signal. t 1. t 2 also includes: In response to and The first output buffer (550) is assigned to the second output signal and the second output buffer (560) is assigned to the first output signal; In response to and The first output buffer (550) is assigned to the first output signal and the second output buffer (560) is assigned to the second output signal; In response to The first output buffer (550) is assigned to the second output signal and the second output buffer (560) is assigned to the first output signal; as well as In response to The first output buffer (550) is assigned to the first output signal and the second output buffer (560) is assigned to the second output signal.

14. An apparatus (112, 300, 1502) for adjusting the timing of an output audio signal to achieve a desired interchannel time difference (ITD) between the output audio signals, said apparatus being adapted to: Receive the current ITD value and audio frame; Based on the ITD of the current frame and the ITD of the previous frame, determine the transition time for performing the time shift applied to at least one of the first and second output signals. t 1. t 2, wherein the conversion time is determined. t 1. t 2 includes: In response to the fact that the sign of the current ITD is different from that of the previous ITD, a shift operation is applied to both the first output signal and the second output signal by splitting the total conversion length into two parts according to the following formula: in This is the current ITD. It is the aforementioned previous ITD and This represents the rounding operation to the nearest integer. as well as When generating the first output signal and the second output signal, at the determined conversion time t 1. t The time shift described in section 2 is applied.

15. The apparatus (112, 300, 1502) of claim 14, wherein at least a portion of the audio frame is stored (403) in a memory for use in synthesizing the ITD in subsequent frames.

16. The apparatus (112, 300, 1502) according to any one of claims 14 to 15, wherein the audio frame is a portion of an audio object, the audio object including an audio signal and location metadata describing the location of the object, and the method further comprising obtaining the time shift from the location metadata.

17. The apparatus (112, 300, 1502) according to any one of claims 14 to 16, wherein the apparatus (112, 300, 1502) is further adapted to: Based on the current frame Frame length and advance memory required for resampling To calculate the total conversion length (801), the total conversion length is divided into including t 1 and t 2 has two parts; Based on the advanced memory Determine the (803) conversion length t 3; and Based on the maximum allowed conversion length and the conversion length t 3. Determine the total conversion length as described in (805).

18. The apparatus (112, 300, 1502) according to claim 17, wherein the conversion length is determined. t 3. The conversion length is determined according to the following formula. t 3: ,in It is the ITD of the current audio frame, which includes N samples, and the total conversion length is determined according to the following formula: ,in , in It is the maximum allowed conversion length.

19. The apparatus (112, 300, 1502) according to any one of claims 14 to 18, wherein the conversion time is determined. t 1. t 2 includes: In response to the fact that the symbol of the current ITD is the same as that of the previous ITD, the total transformation length is allocated to the transformation time. t 1. t Set one of the conversion times in step 2 and set the other conversion time to zero.

20. The apparatus (112, 300, 1502) according to any one of claims 17 to 19, wherein the apparatus (112, 300, 1502) is further adapted to: The audio frames are used to fill the processing buffer (510). The determined switching time is applied when generating the first output signal and the second output signal. t 1. t 2 includes: In response to the current ITD having the same sign as the previous ITD, or in response to either the current ITD or the previous ITD being zero: By allocating the total conversion length to t 1 and will t 2 is set to zero to adjust the processing buffer (510) to fill the first output buffer (550); and The input signal portion of the processing buffer (510) is copied to the second output buffer (560).

21. The apparatus (112, 300, 1502) according to claim 20, wherein the determined switching time is applied when generating the first output signal and the second output signal. t 1. t 2 also includes: In response to Furthermore, the sign of one of the current ITD and the previous ITD is negative, thereby indicating that one of the first output signal and the second output signal precedes the other of the first output signal and the second output signal, and the output buffer of either the first output buffer (550) or the second output buffer (560) associated with the other of the first output signal and the second output signal will be delayed by the total conversion length.

22. The apparatus (112, 300, 1502) according to any one of claims 20 to 21, wherein the determined switching time is applied when generating the first output signal and the second output signal. t 1. t 2 also includes: In response to The conversion is generated in the following way: The length of the frame in the processing buffer (510) is changed from the length of Extended to length The output frame; as well as In response to the conversion length Greater than zero, the last output channel is added by copying from the processing buffer (510). sample, .

23. The apparatus (112, 300, 1502) according to any one of claims 20 to 22, wherein the determined switching time is applied when generating the first output signal and the second output signal. t 1. t 2 also includes: In response to The last part of the first output buffer (550) is added by copying from the processing buffer (510). t 3 samples.

24. The apparatus (112, 300, 1502) according to any one of claims 20 to 23, wherein the determined switching time is applied when generating the first output signal and the second output signal. t 1. t 2 also includes: In response to The total conversion length is divided according to the following formula: in This represents the rounding operation to the nearest integer, where the total conversion length is broken down into: For length samples Resample to fit the samples of the first output buffer (550). ; Sample Copy to the corresponding index in the second output buffer (560); For length samples Resample to fit the length in the second output buffer (560) samples .

25. The apparatus (112, 300, 1502) according to any one of claims 20 to 24, wherein generating the second output signal further comprises: In response to the conversion length Greater than zero, the last output channel is added by copying from the processing buffer (510). sample, .

26. The apparatus (112, 300, 1502) according to any one of claims 20 to 25, wherein the determined switching time is applied when generating the first output signal and the second output signal. t 1. t 2 also includes: In response to and The first output buffer (550) is assigned to the second output signal and the second output buffer (560) is assigned to the first output signal; In response to and The first output buffer (550) is assigned to the first output signal and the second output buffer (560) is assigned to the second output signal; In response to The first output buffer (550) is assigned to the second output signal and the second output buffer (560) is assigned to the first output signal; as well as In response to The first output buffer (550) is assigned to the first output signal and the second output buffer (560) is assigned to the second output signal.

Images