Device and method for adjusting number of enabled vector quantization stages according to quality of communication

Abstract

This electronic device (101) comprises: a communication circuit for establishing communication with an external device (104); at least one processor (120) comprising processing circuitry; and a memory (130) including one or more storage media for storing instructions, wherein, when executed by the at least one processor (120), the instructions can instruct the electronic device (101) to: determine, on the basis of the quality of communication, whether to enable a plurality of vector quantization stages; use an enabled vector quantization stage from among the plurality of vector quantization stages so as to quantize a latent vector (215) extracted from an audio frame (211); and transmit, to the external device (104), a quantization result (219) and information related to the vector quantization stage used for the quantization.

Description

Device and method for adjusting the number of enabled vector quantization steps according to the quality of communication

[0001] An electronic device is disclosed below for adjusting the number of enabled vector quantization steps according to the quality of communication.

[0002] With the advancement of wireless communication technology, various types of wearable devices have emerged and are widely used alongside smartphones. For example, regarding headsets used for listening to music or making calls, products that connect wirelessly to smartphones using wireless communication technologies such as Bluetooth are more commonly used than products that connect via wires.

[0003] The information described above may be provided as related art for the purpose of aiding understanding of this document. None of the foregoing is to be claimed as prior art related to this document, nor is it to be used to determine prior art.

[0004] An electronic device (101) includes a communication circuit for establishing communication with an external device (104); at least one processor (120) including a processing circuit; and a memory (130) including one or more storage media for storing instructions. When the instructions are executed by the at least one processor (120), the electronic device (101) may determine whether to enable a plurality of vector quantization stages based on the quality of the communication, quantize a latent vector (215) extracted from an audio frame (211) using the enabled vector quantization stage among the plurality of vector quantization stages, and transmit information regarding the quantization result (219) and the vector quantization stage used for quantization to the external device (104).

[0005] A method performed by an electronic device (101) may include: an operation (410) of determining whether to enable a plurality of vector quantization stages based on the quality of communication between the electronic device (101) and an external device (104); an operation (420) of quantizing a latent vector (215) extracted from an audio frame (211) using an enabled vector quantization stage among the plurality of vector quantization stages; and an operation (430) of transmitting information regarding the quantization result (219) and the vector quantization stage used for quantization to the external device (104).

[0006] FIG. 1 is a block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0007] FIG. 2 is a diagram illustrating examples of the operation of compressing an audio signal and the operation of restoring the quantization result, performed by a first electronic device or a second electronic device according to various embodiments.

[0008] FIG. 3 is a diagram illustrating an example of the operation of a quantization module that quantizes a vector according to various embodiments.

[0009] FIG. 4 is a flowchart illustrating an example of a method in which an electronic device according to various embodiments adjusts the compression operation of an audio frame according to the quality of communication.

[0010] FIG. 5 is a flowchart illustrating an example of an operation in which an electronic device according to various embodiments determines whether to enable a plurality of vector quantization steps.

[0011] FIG. 6 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments enables or disables at least one quantization step.

[0012] Hereinafter, embodiments will be described in detail with reference to the attached drawings. In the description with reference to the attached drawings, identical components are given the same reference numeral regardless of the drawing number, and redundant descriptions thereof will be omitted.

[0013] FIG. 1 is a block diagram illustrating an exemplary configuration of an electronic device according to various embodiments.

[0014] FIG. 1 is a block diagram of an electronic device (101) in a network environment (100) according to various embodiments. Referring to FIG. 1, in the network environment (100), the electronic device (101) may communicate with an electronic device (102) through a first network (198) (e.g., a short-range wireless communication network) or may communicate with at least one of an electronic device (104) or a server (108) through a second network (199) (e.g., a long-range wireless communication network). According to one embodiment, the electronic device (101) may communicate with the electronic device (104) through a server (108).

[0015] According to one embodiment, the electronic device (101) may include a processor (120), memory (130), input module (150), sound output module (155), display (160), audio module (170), sensor (176), interface (177), connection terminal (178), haptic module (179), camera (180), power management module (188), battery (189), communication module (190), subscriber identification module (196), or antenna module (197). In some embodiments, at least one of these components (e.g., connection terminal (178)) may be omitted from the electronic device (101), or one or more other components may be added. In some embodiments, some of these components (e.g., sensor (176), camera (180), or antenna module (197)) may be integrated into a single component (e.g., display (160)).

[0016] The processor (120) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing operations. The processor (120) may include at least one electrical circuit (e.g., a processing circuit) and may perform distributed processing (e.g., execution) of instructions (or programs (140), data, etc.) stored in memory (130) individually or collectively. The processor (120) may include a processor assembly comprising one or more processing circuits. The processor (120) may include any processing circuit that is operative to control the performance and operation of one or more components of the electronic device (101) (e.g., memory (130), display (160), camera (180), communication circuit, and / or sensor (176)).

[0017] The processor (120) can control at least one other component (e.g., a hardware or software component) of the electronic device (101) connected to the processor (120) by executing software (e.g., a program (140)), for example, and can perform various data processing or operations. According to one embodiment, as at least part of the data processing or operations, the processor (120) can store commands or data received from other components (e.g., a sensor (176) or a communication module (190)) in volatile memory (132), process the commands or data stored in volatile memory (132), and store the resulting data in non-volatile memory (134). According to one embodiment, the processor (120) may include a main processor (121) (e.g., a central processing unit or an application processor) or an auxiliary processor (123) that can operate independently or together with it (e.g., a graphics processing unit, a neural processing unit (NPU), an image signal processor, a sensor hub processor, or a communication processor). For example, if the electronic device (101) includes a main processor (121) and an auxiliary processor (123), the auxiliary processor (123) may be configured to use lower power than the main processor (121) or to be specialized for a designated function. The auxiliary processor (123) may be implemented separately from the main processor (121) or as part thereof.

[0018] The auxiliary processor (123) may control at least some of the functions or states associated with at least one component of the electronic device (101) (e.g., display (160), sensor (176), or communication module (190)) on behalf of the main processor (121) while the main processor (121) is in an inactive (e.g., sleep) state, or together with the main processor (121) while the main processor (121) is in an active (e.g., application execution) state. According to one embodiment, the auxiliary processor (123) (e.g., image signal processor or communication processor) may be implemented as part of another functionally related component (e.g., camera (180) or communication module (190)). According to one embodiment, the auxiliary processor (123) (e.g., neural network processing unit) may include a hardware structure specialized for processing an artificial intelligence model. The artificial intelligence model may be generated through machine learning. Such learning may be performed, for example, on the electronic device (101) itself where the artificial intelligence model is executed, or through a separate server (e.g., server (108)). The learning algorithm may include, for example, supervised learning, unsupervised learning, semi-supervised learning, or reinforcement learning, but is not limited to the examples described above. The artificial intelligence model may include a plurality of artificial neural network layers.An artificial neural network may be a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), a restricted Boltzmann machine (RBM), a deep belief network (DBN), a bidirectional recurrent deep neural network (BRDNN), a deep Q-network, or a combination of two or more of the above, but is not limited to the examples described above. In addition to the hardware structure, the artificial intelligence model may include a software structure, either additionally or substantially.

[0019] The memory (130) can store various data used by at least one component of the electronic device (101) (e.g., processor (120) or sensor (176)). The data may include, for example, input data or output data for software (e.g., program (140)) and related instructions. The memory (130) may include volatile memory (132) or non-volatile memory (134).

[0020] The program (140) may be stored as software in memory (130) and may include, for example, an operating system (142), middleware (144), or an application (146).

[0021] The input module (150) can receive commands or data to be used for a component of the electronic device (101) (e.g., processor (120)) from outside the electronic device (101) (e.g., user). The input module (150) may include, for example, a microphone, a mouse, a keyboard, a key (e.g., a button), or a digital pen (e.g., a stylus pen).

[0022] The sound output module (155) can output a sound signal to the outside of the electronic device (101). The sound output module (155) may include, for example, a speaker or a receiver. The speaker may be used for general purposes, such as multimedia playback or recording playback. The receiver may be used to receive incoming calls. According to one embodiment, the receiver may be implemented separately from the speaker or as part thereof.

[0023] A display (160) (e.g., a display) can visually provide information to an external (e.g., a user) outside of the electronic device (101). The display (160) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling said device. According to one embodiment, the display (160) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of the force generated by said touch.

[0024] The audio module (170) can convert sound into an electrical signal or, conversely, convert an electrical signal into sound. According to one embodiment, the audio module (170) can acquire sound through the input module (150) or output sound through the sound output module (155) or an external electronic device (e.g., electronic device (102)) (e.g., speaker or headphones) connected directly or wirelessly to the electronic device (101).

[0025] The sensor (176) can detect the operating state of the electronic device (101) (e.g., power or temperature) or the external environmental state (e.g., user state) and generate an electrical signal or data value corresponding to the detected state. According to one embodiment, the sensor (176) may include, for example, a gesture sensor, a gyroscope sensor, a barometric pressure sensor, a magnetic sensor, an accelerometer sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biosensor, a temperature sensor, a humidity sensor, or an illuminance sensor. For example, the sensor (176) may include an inertial measurement unit (IMU).

[0026] The interface (177) may support one or more specified protocols that can be used for the electronic device (101) to be connected directly or wirelessly to an external electronic device (e.g., electronic device (102)). According to one embodiment, the interface (177) may include, for example, a high definition multimedia interface (HDMI), a universal serial bus (USB) interface, an SD card interface, or an audio interface.

[0027] The connection terminal (178) may include a connector through which the electronic device (101) can be physically connected to an external electronic device (e.g., electronic device (102)). According to one embodiment, the connection terminal (178) may include, for example, an HDMI connector, a USB connector, an SD card connector, or an audio connector (e.g., a headphone connector).

[0028] The haptic module (179) can convert an electrical signal into a mechanical stimulus (e.g., vibration or movement) or an electrical stimulus that can be perceived by the user through tactile or kinesthetic senses. According to one embodiment, the haptic module (179) may include, for example, a motor, a piezoelectric element, or an electric stimulation device.

[0029] The camera (180) can capture still images and video. According to one embodiment, the camera (180) may include one or more lenses, one or more image sensors, one or more image signal processors, or one or more flashes.

[0030] The power management module (188) can manage power supplied to the electronic device (101). According to one embodiment, the power management module (188) can be implemented, for example, as at least part of a power management integrated circuit (PMIC).

[0031] The battery (189) can supply power to at least one component of the electronic device (101). According to one embodiment, the battery (189) may include, for example, a non-rechargeable primary battery, a rechargeable secondary battery, or a fuel cell.

[0032] A communication module (190) can support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between an electronic device (101) and an external electronic device (e.g., electronic device (102), electronic device (104), or server (108)), and the performance of communication through the established communication channel. The communication module (190) may include one or more communication circuits. The communication module (190) may include one or more communication processors (CP) that operate independently of a processor (120) (e.g., application processor) and support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication module (190) may include a wireless communication module (192) (e.g., cellular communication module, short-range wireless communication module, or GNSS (global navigation satellite system) communication module) or a wired communication module (194) (e.g., LAN (local region network) communication module, or power line communication module). The corresponding communication module among these communication modules can communicate with an external electronic device (104) through a first network (198) (e.g., a short-range communication network such as Bluetooth, WiFi (wireless fidelity) direct, or IrDA (infrared data relation)) or a second network (199) (e.g., a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips). The wireless communication module (192) can identify or authenticate the electronic device (101) within a communication network such as the first network (198) or the second network (199) using subscriber information (e.g., International Mobile Subscriber Identifier (IMSI)) stored in the subscriber identification module (196).

[0033] The wireless communication module (192) can support 5G networks and next-generation communication technologies following 4G networks, for example, new radio access technology. NR access technology can support high-speed transmission of high-capacity data (enhanced mobile broadband (eMBB)), minimization of terminal power and connection of multiple terminals (massive machine type communications (mMTC)), or high reliability and low latency (ultra-reliable and low-latency communications (URLLC)). The wireless communication module (192) can support a high-frequency band (e.g., mmWave band) to achieve a high data transmission rate, for example. The wireless communication module (192) can support various technologies for securing performance in the high-frequency band, such as beamforming, massive MIMO (multiple-input and multiple-output), full-dimensional MIMO (FD-MIMO), array antenna, analog beam-forming, or large-scale antenna. The wireless communication module (192) can support various requirements specified in the electronic device (101), external electronic device (e.g., electronic device (104)), or network system (e.g., second network (199)). According to one embodiment, the wireless communication module (192) may support a Peak data rate (e.g., 20 Gbps or more) for eMBB realization, loss coverage (e.g., 164 dB or less) for mMTC realization, or U-plane latency (e.g., downlink (DL) and uplink (UL) each 0.5 ms or less, or round trip 1 ms or less) for URLLC realization.

[0034] An antenna module (197) can transmit a signal or power to or from an external source (e.g., an external electronic device). According to one embodiment, the antenna module (197) may include an antenna comprising a radiator made of a conductor or a conductive pattern formed on a substrate (e.g., a PCB). According to one embodiment, the antenna module (197) may include a plurality of antennas (e.g., an array antenna). In this case, at least one antenna suitable for a communication method used in a communication network, such as a first network (198) or a second network (199), may be selected from the plurality of antennas, for example, by a communication module (190). A signal or power may be transmitted or received between the communication module (190) and an external electronic device through the selected at least one antenna. According to some embodiments, in addition to the radiator, other components (e.g., a radio frequency integrated circuit (RFIC)) may be additionally formed as part of the antenna module (197).

[0035] According to various embodiments, the antenna module (197) may form a mmWave antenna module. According to one embodiment, the mmWave antenna module may include a printed circuit board, an RFIC disposed on or adjacent to a first surface (e.g., bottom surface) of the printed circuit board and capable of supporting a specified high frequency band (e.g., mmWave band), and a plurality of antennas (e.g., array antennas) disposed on or adjacent to a second surface (e.g., top surface or side surface) of the printed circuit board and capable of transmitting or receiving a signal of the specified high frequency band.

[0036] At least some of the above components can be connected to each other via a communication method between peripheral devices (e.g., bus, GPIO (general purpose input and output), SPI (serial peripheral interface), or MIPI (mobile industry processor interface)) and exchange signals (e.g., commands or data) with each other.

[0037] According to one embodiment, commands or data may be transmitted or received between an electronic device (101) and an external electronic device (104) through a server (108) connected to a second network (199).

[0038] Each of the external electronic devices (102, 104) and the server (108) may be the same or different type of device as the electronic device (101). According to one embodiment, all or part of the operations performed on the electronic device (101) may be performed on one or more of the external electronic devices (102, 104) or the server (108). For example, if the electronic device (101) needs to perform a function or service automatically or in response to a request from a user or another device, the electronic device (101) may request one or more external electronic devices to perform at least part of the function or service instead of performing the function or service itself or additionally. One or more external electronic devices that receive the request may perform at least part of the requested function or service, or additional functions or services related to the request, and transmit the result of the execution to the electronic device (101). The electronic device (101) may provide the result as is or additionally processed as at least part of the response to the request.

[0039]

[0040] FIG. 2 is a diagram illustrating examples of the operation of compressing an audio signal and the operation of restoring the quantization result, performed by a first electronic device or a second electronic device according to various embodiments.

[0041] A first electronic device according to one embodiment (e.g., electronic device (101) of FIG. 1) transmits an audio signal (sound signal) (e.g., audio frame (211)) to a second electronic device (e.g., electronic device (104) of FIG. 1), and the second electronic device can play a sound corresponding to the received audio signal. The first electronic device may be a device connected to the second electronic device by wireless communication. In various embodiments of the present disclosure, the second electronic device may be represented as an external device with respect to the first electronic device, and likewise, the first electronic device may be represented as an external device with respect to the second electronic device.

[0042] A first electronic device according to one embodiment may transmit the compression result of an audio frame (211) to an external device instead of transmitting the audio frame (211) as is to a second electronic device. The second electronic device may receive the compression result and play sound based on the restored audio frame (239) obtained by restoring the received compression result. In FIG. 2, blocks illustrating the operation (210) of compressing the audio frame (211) by the first electronic device and the operation (230) of restoring the compression result by the second electronic device are shown. The operation (210) of compressing the audio frame (211) and the operation (230) of restoring the compression result according to one embodiment may be performed based on a neural network-based audio codec.

[0043] In operation (210), the first electronic device may apply an encoder (213) to an audio frame (211). The encoder (213) may represent a model created and / or trained to output a latent vector (215) by being applied to the audio frame (211). The first electronic device may obtain a quantization result (219, 231) based on the result of applying a quantization module (217) to the latent vector (215).

[0044] A quantization module (217) according to one embodiment may mean a module that performs quantization of a vector (e.g., a latent vector (215)). For example, the quantization module (217) may quantize the latent vector (215) based on RVQ (residual vector quantization), GVQ (group vector quantization), and / or GRVQ (group residual vector quantization), which quantizes the vector using at least one of a plurality of vector quantization steps.

[0045] The quantization module (217) may be a hardware module or a software module. If the quantization module (217) is implemented as a software module, it may include at least one instruction that is stored in memory and executed by a processor. The operation of the software module may be understood as the operation of the processor. The vector quantization operation performed based on the quantization module (217) is described in more detail later in FIG. 3. The first electronic device may obtain a quantization result (219, 231) of quantizing a potential vector (215) from the quantization module (217).

[0046] The first electronic device can transmit the quantization result (219, 231) obtained after performing the operation (210) to the second electronic device.

[0047] In operation (230), the second electronic device may apply a dequantization module (233) to the quantization result (219, 231). According to one embodiment, the dequantization module (233) may mean a module that restores the quantization result (219, 231). The dequantization module (233) may be a hardware module or a software module. If the dequantization module (233) is implemented as a software module, it may include at least one instruction that is stored in memory and executed by a processor. The operation of the software module may be understood as the operation of the processor. The second electronic device may obtain a restored vector (235) by applying the dequantization module (233) to the quantization result (219, 231). The vector restoration operation performed based on the dequantization module (233) is described in more detail later in FIG. 3.

[0048] The second electronic device may apply a decoder (237) to the restored vector (235). The decoder (237) may represent a model created and / or trained to output a restored audio frame (239) by applying it to the vector (e.g., the restored vector (235)). The second electronic device may obtain the restored audio frame (239) based on the result of applying an inverse quantization module (233) to the restored vector (235). The second electronic device may play sound based on the restored audio frame (239).

[0049]

[0050] FIG. 3 is a diagram illustrating an example of the operation of a quantization module that quantizes a vector according to various embodiments.

[0051] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) may include a quantization module (317) (e.g., the quantization module (217) of FIG. 2). The quantization module (317) may include a plurality of vector quantization steps. In FIG. 3, exemplarily, the quantization module (317) may include N vector quantization steps (e.g., a first vector quantization step (310-1), a second vector quantization step (310-2), a third vector quantization step (310-3), ..., or an Nth vector quantization step (310-N)). Here, N may be an integer greater than or equal to 2.

[0052] Each of the plurality of vector quantization steps may have a level indicating the order in which the vectors are applied to the quantization module (317). For example, the first vector quantization step (310-1) may have a first level. The second vector quantization step (310-2) may have a second level. The third vector quantization step (310-3) may have a third level. The earlier a vector quantization step is applied, the lower the level of the vector quantization step may be. For example, the first level may be lower than the second level, and the second level may be lower than the third level.

[0053] As will be described later, a vector quantization step of a specific level may be used to quantize the error of the quantization result of a vector quantization step of a level lower than the specific level. In various embodiments of the present disclosure, a vector quantization step of a lower level (e.g., a level below a threshold level) (e.g., a first vector quantization step (310-1)) may be used for coarse-quantization, and a vector quantization step of a higher level (e.g., a second vector quantization step (310-2), a third vector quantization step (310-3), ..., a Nth vector quantization step (310-N)) may be used for fine-quantization.

[0054] According to one embodiment, the electronic device can quantize a vector based on RVQ.

[0055] For example, for each vector quantization step, the electronic device may select as the target vector the vector most similar to the input vector input to the corresponding vector quantization step among the candidate vectors included in the codebook of the corresponding vector quantization step. The input vector of the lowest level vector quantization step among the plurality of vector quantization steps (e.g., the first vector quantization step (310-1)) may be a latent vector extracted from an audio frame (e.g., the latent vector (215) of FIG. 2). As will be described later, the input vector of each of the remaining vector quantization steps among the plurality of vector quantization steps (e.g., the second vector quantization step (310-2), the third vector quantization step (310-3), ..., or the Nth vector quantization step (310-N)) may be a residual vector between the input vector and the target vector of the vector quantization step preceding the corresponding vector quantization step.

[0056] A codebook may refer to a set of candidate vectors containing multiple candidate vectors for each vector quantization step. The codebooks of the vector quantization steps may differ from one another. For example, the size of the codebook for the first vector quantization step (310-1) may differ from the size of the codebook for the second vector quantization step (310-2). The size of the codebook may refer to the number of candidate vectors included in the codebook. For example, the multiple candidate vectors included in the codebook of the first vector quantization step (310-1) may differ at least partially from the multiple candidate vectors included in the codebook of the second vector quantization step (310-2).

[0057] According to one embodiment, the lower the level of the vector quantization step, the larger the size of the codebook of the vector quantization step may be. For example, among a plurality of vector quantization steps, the number of candidate vectors included in the codebook of the first quantization step of the first level may be greater than the number of candidate vectors included in the codebook of the second quantization step of the second level, which is higher than the first level.

[0058] The target vector may refer to the vector most similar to the input vector among a plurality of candidate vectors included in the codebook. The similarity between the candidate vector and the input vector may be determined based on the distance between the vectors (e.g., L1 norm, L2 norm) and / or the cosine similarity level between the vectors. For each vector quantization step, the electronic device may obtain information regarding the target vector selected in that vector quantization step as at least part of the quantization result. The information regarding the target vector may include a step identifier for identifying the vector quantization step (e.g., a level identifier of the vector quantization step), and information indicating the target vector among a plurality of candidate vectors in the codebook of the vector quantization step (e.g., a vector identifier, an index of the target vector).

[0059] For each vector quantization step, the electronic device may use the residual vector between the input vector of the corresponding vector quantization step and the target vector of the corresponding vector quantization step as the input vector of the subsequent vector quantization step of the corresponding vector quantization step.

[0060] Referring to FIG. 3, an electronic device according to one embodiment may apply a first vector quantization step (310-1) to a first input vector (x1), which is a potential vector. The electronic device may select a first target vector (y1) from the codebook of the first vector quantization step (310-1). The electronic device may use the difference between the first input vector (x1) and the first target vector (y1) (e.g., x1-y1) (e.g., residual vector) as the second input vector (x2) of the second vector quantization step (310-2). Similar to or identical to the first vector quantization step (310-1), the electronic device may apply a second vector quantization step (310-2) to the second input vector (x2). The electronic device may select a second target vector (y2) from the codebook of the second vector quantization step (310-2). The electronic device may use the difference between the second input vector (x2) and the second target vector (y2) (e.g., x2-y2) (e.g., residual vector) as the third input vector (x3) of the third vector quantization step (310-3). The electronic device may apply the third vector quantization step (310-3) to the third input vector (x3). The electronic device may obtain the Nth input vector based on the first vector quantization step (310-1) through the N-1st vector quantization step (not shown). As a result of applying the Nth vector quantization step (310-N) to the Nth input vector, the electronic device obtains the Nth target vector (y) from the codebook of the Nth vector quantization step (310-N). N ) can be selected. The electronic device comprises a plurality of target vectors selected in a plurality of vector quantization steps (e.g., a first target vector (y1), a second target vector (y2), a third target vector (y3), ..., and an Nth target vector (y N Information regarding )) can be obtained as a quantization result. Information regarding each target vector may include the level of the vector quantization step in which the target vector is selected and the vector identifier of the target vector (e.g., the index of the target vector in the codebook).

[0061] Although not explicitly shown in FIG. 3, when an electronic device obtains a quantization result from an external device, it can obtain a vector restored from the quantization result using codebooks of multiple vector quantization steps.

[0062] For example, when an electronic device obtains a quantization result, it may determine a target vector for each vector quantization step using information regarding a target vector included in the quantization result. The electronic device may store or access codebooks for multiple vector quantization steps. The electronic device may determine target vectors using information regarding each target vector included in the quantization result and the codebooks. The electronic device may obtain the sum of the target vectors as a restored vector.

[0063]

[0064] FIG. 4 is a flowchart illustrating an example of a method in which an electronic device according to various embodiments adjusts the compression operation of an audio frame according to the quality of communication.

[0065] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) may include a communication circuit that establishes communication (e.g., wireless communication) with an external device (e.g., the electronic device (104) of FIG. 1). The electronic device may adjust a compression operation regarding an audio frame to be transmitted to the external device based on the quality of communication with the external device.

[0066] In operation (410), the electronic device can determine whether to enable a plurality of vector quantization steps based on the quality of communication.

[0067] According to one embodiment, an electronic device may be able to transmit data at a higher bit rate as the quality level of communication increases. Accordingly, as the quality level of communication increases, the electronic device may utilize a greater number of vector quantization steps for quantization. As described above in FIG. 3, since the quantization result includes the target vector of each vector quantization step, the amount of data included in the quantization result may be greater as the number of vector quantization steps increases. Therefore, the electronic device may enable (e.g., activate) a greater number of vector quantization steps as the quality of communication improves. Similarly, the electronic device may enable a smaller number of vector quantization steps as the quality of communication worsens.

[0068] In one embodiment, the electronic device may determine the quality of communication with an external device based on at least one of the state of a transmission buffer corresponding to the external device or the received signal strength indication (RSSI) of a signal received from the external device.

[0069] According to one embodiment, an electronic device can determine the quality of communication with an external device based on the state of a transmission buffer corresponding to the external device. A transmission buffer may refer to a temporary storage space that stores data packets divided into packet units (e.g., packetization of data) from data in order to transmit data from the electronic device to the external device. Data packets may be deleted from the transmission buffer after being transmitted to the external device. A transmission buffer corresponding to the external device may refer to a storage space where data packets to be transmitted from the electronic device to the external device are waiting.

[0070] According to one embodiment, when an electronic device communicates with a plurality of external devices, it may use a transmission buffer corresponding to each external device (e.g., one-to-one correspondence). For example, when the electronic device communicates with a first external device and a second external device, a first transmission buffer for the first external device may be configured (e.g., defined) independently of a second transmission buffer for the second external device. As the quality of communication deteriorates, it may take more time for data packets to be transmitted from the electronic device to the external device, and more data packets may be stored (e.g., accumulated) in the transmission buffer. The state of the transmission buffer may include the number of data packets stored in the transmission buffer and / or the memory size of the data packets (e.g., memory size in bytes (B) and / or kilobytes (KB)). The electronic device may determine a higher quality of communication as the number of data packet(s) stored in the transmission buffer decreases and / or the memory size of the data packet(s) stored in the transmission buffer decreases.

[0071] For example, an electronic device can determine the quality of communication based on the RSSI of a signal received from an external device. The electronic device can determine the quality of communication with an external device based on the result of measuring the RSSI of a signal received from an external device.

[0072] In various embodiments of the present disclosure, the quality of communication is primarily described as being determined based on the state of the transmission buffer and / or RSSI, but is not limited thereto. For example, an electronic device may determine the quality of communication based on the number of external devices connected to the electronic device via wireless communication and / or the distance between the external devices and the electronic device.

[0073] According to one embodiment, the electronic device can determine whether to enable vector quantization steps based on the result of comparing a target quality level corresponding to the quality of communication with a comparison quality level range based on whether the vector quantization steps of the quantization module are enabled.

[0074] The target quality level may refer to the level of communication quality that is achievable when transmitting data packets from an electronic device to an external device, considering the quality of communication between the electronic device and the external device. For example, if the quality of communication is high—that is, if the communication between the electronic device and the external device is smooth—the target quality level may be determined to be high. The communication quality level may be expressed by at least one of the bitrate (e.g., target bitrate), the state of the transmission buffer, or the RSSI (received signal strength indication). As the communication quality level increases, the bitrate may be expressed as a larger value, the state of the transmission buffer (e.g., the number of data packets stored in the buffer) as a lower value, or the RSSI as a larger value.

[0075] The comparative quality level range may refer to the level of communication quality required for an electronic device to transmit the quantization result of a latent vector using a quantization module to an external device, depending on the enablement or disablement of each of the multiple vector quantization steps included in the quantization module.

[0076] The operation of determining whether to enable the vector quantization step based on a comparison of the target quality level and the comparison quality level range is described in more detail later in FIGS. 5 and 6.

[0077] According to one embodiment, the electronic device can determine whether to enable a plurality of vector quantization steps based on the sampling rate of an audio frame.

[0078] For example, an electronic device may determine a limit on the number of enableable vector quantization steps among a plurality of vector quantization steps based on the sampling rate of an audio frame. The electronic device may determine to enable vector quantization steps among the plurality of vector quantization steps that are less than or equal to the limit. For example, the sampling rate of the audio frame may be one of 8 kHz, 22.05 kHz, 44.1 kHz, 48 kHz, or 96 kHz. In one example, for each sampling rate of the audio frame, a limit on the number of vector quantization steps among the plurality of vector quantization steps may be determined. The larger the value of the sampling rate of the audio frame, the greater the limit on the number may be set to a value equal to or greater than the limit.

[0079] For example, for a sampling rate of 8 kHz, the number limit among the multiple vector quantization steps may be set to 1, for a sampling rate of 22.05 kHz, the number limit may be set to 2, for a sampling rate of 44.1 kHz, the number limit may be set to 3, for a sampling rate of 48 kHz, the number limit may be set to 4, or for a sampling rate of 96 kHz, the number limit may be set to 8.

[0080] An electronic device according to one embodiment can efficiently manage the amount of computation of an electronic device or an external device by enabling a specific number of quantization steps only for audio frames with a sampling rate exceeding a threshold, and enabling fewer than a specific number of quantization steps for audio frames with a sampling rate exceeding a threshold, instead of unnecessarily enabling a large number of quantization steps for audio frames with a sampling rate below a threshold even when wireless communication is smooth.

[0081] According to one embodiment, even when the electronic device satisfies the condition of enabling at least one of the disabled vector quantization steps (e.g., when the target quality level in FIG. 5 is higher than the comparison quality level range), if the number of vector quantization steps already enabled in the quantization module is equal to the number limit, the enabling of the vector quantization steps can be omitted and the decision on whether to enable can be completed.

[0082] In operation (420), the electronic device can quantize a potential vector extracted from an audio frame using an enabled vector quantization step among a plurality of vector quantization steps.

[0083] In one embodiment, among a plurality of vector quantization steps included in a quantization module, the electronic device may apply a vector quantization step enabled to a potential vector and omit the application of a disabled vector quantization step.

[0084] For example, the quantization module may include a first vector quantization step (e.g., the first vector quantization step (310-1) of FIG. 3), a second vector quantization step (e.g., the second vector quantization step (310-2) of FIG. 3), a third vector quantization step (e.g., the third vector quantization step (310-3) of FIG. 3), and a fourth vector quantization step (e.g., the fourth vector quantization step (310-4) of FIG. 3). The electronic device may decide to enable the first vector quantization step and the second vector quantization step, and disable the third vector quantization step and the fourth vector quantization step.

[0085] The electronic device may apply a first vector quantization step and a second vector quantization step to a potential vector, and may omit (e.g., limit) applying a third vector quantization step and a fourth vector quantization step. The electronic device may obtain information regarding a first target vector (e.g., the first target vector (y1) in FIG. 3) and a second target vector (e.g., the second target vector (y2) in FIG. 3) selected in each of the enabled first vector quantization step and the second vector quantization step as a quantization result.

[0086] As described above in FIG. 3, the electronic device can quantize a potential vector using enabled vector quantization steps based on RVQ, GVQ, and / or GRVQ. For example, the electronic device can quantize the input vector of the first vector quantization step into a target vector similar to the input vector among candidate vectors included in the codebook of the first vector quantization step in a first vector quantization step of a first level among at least one enabled vector quantization step. The electronic device can obtain the residual vector between the target vector and the input vector of the first vector quantization step as the input vector of a second vector quantization step of a second level higher than the first level.

[0087] According to one embodiment, in each vector quantization step excluding the lowest level vector quantization step, an operation to quantize the residual vector of the vector quantization step preceding the corresponding vector quantization step may be performed. Accordingly, the enabling of the vector quantization steps may be performed sequentially starting from the lower level among the disabled vector quantization steps. The disabling of the vector quantization steps may be performed sequentially starting from the higher level among the enabled vector quantization steps. The enabling and / or disabling of the vector quantization steps performed sequentially is described in more detail later in FIGS. 5 and 6.

[0088] In operation (430), the electronic device can transmit information regarding the quantization result and the vector quantization step used for quantization to an external device.

[0089] As described above in FIG. 3, the quantization result may include information indicating the target vector of each enabled vector quantization step (e.g., an index corresponding to the target vector in the codebook of the corresponding enabled vector quantization step).

[0090] Information regarding the vector quantization step used for quantization may include information (e.g., level) indicating the vector quantization step actually used for quantization (e.g., enabled vector quantization step) among a plurality of vector quantization steps included in the quantization module.

[0091] In various embodiments of the present disclosure, an electronic device (e.g., the electronic device (101) of FIG. 1) is primarily described as applying all enabled vector quantization steps to a potential vector and not applying disabled vector quantization steps, but is not limited thereto. According to one embodiment, the electronic device may omit the application of at least one enabled vector quantization step following a specific vector quantization step based on a residual vector between the input vector and the target vector of a specific vector quantization step.

[0092] An electronic device can obtain a residual vector between an input vector and a target vector of a vector quantization step from the result of applying at least one of the enabled vector quantization steps to a potential vector. Based on the fact that the magnitude of the residual vector is less than or equal to a threshold size, the electronic device may skip the application of a vector quantization step of a higher level than the at least one vector quantization step applied to the potential vector.

[0093] For example, the quantization module may include an enabled first vector quantization step, an enabled second vector quantization step, a disabled third vector quantization step, and a disabled fourth vector quantization step. The electronic device may obtain a first target vector by applying a first input vector, which is a potential vector, to the first vector quantization step.

[0094] If the magnitude of the residual vector (e.g., difference (x1-y1) in FIG. 3) between the first input vector (e.g., the first input vector (x1) in FIG. 3) and the first target vector (e.g., the first target vector (y1) in FIG. 3) is less than or equal to a threshold size, the electronic device may omit the application of the second vector quantization step and obtain information regarding the first target vector as a quantization result. The electronic device may transmit information regarding the first vector quantization step used for quantization to an external device, and omit (e.g., not transmit) information regarding the second vector quantization step not used for quantization even if enabled to an external device.

[0095] If the magnitude of the residual vector exceeds a threshold size, the electronic device may apply a second vector quantization step to the second input vector (e.g., the second input vector (x2) in FIG. 3), which is the residual vector. The electronic device may obtain information regarding the first target vector obtained from the first vector quantization step and the second target vector obtained from the second vector quantization step as a quantization result. Along with the quantization result, the electronic device may transmit information regarding the first vector quantization step and the second vector quantization step to an external device as information regarding the vector quantization step used for quantization.

[0096] According to one embodiment, when the power of the electronic device or the external device is below a threshold power, the electronic device may compare the magnitude of the residual vector with the threshold power and omit the application of a subsequent vector quantization step based on the comparison result. For example, when the power of the electronic device or the external device exceeds the threshold power (e.g., while the electronic device is in a first mode), the electronic device may perform quantization of the latent vector using all of the enabled vector quantization steps independently of the magnitude of the residual vector. When the power of the electronic device or the external device is below a threshold power (e.g., while the electronic device is in a second mode), the electronic device may omit the application of at least some of the enabled vector quantization steps based on the magnitude of the residual vector. When the application of the vector quantization step is omitted based on the magnitude of the residual vector, the amount of computation and / or power of the electronic device required for compressing the latent vector and the amount of computation and / or power of the external device required for restoring the quantization result can be saved with minimal loss.

[0097] According to one embodiment, the electronic device may transmit information regarding the quantization result and the vector quantization step used for quantization to an external device at a quality level equal to or lower than the target quality level. For example, the target quality level may be determined by a target bit rate based on the bit rate. The electronic device may transmit information regarding the quantization result and the vector quantization step used for quantization to an external device at a bit rate lower than or equal to the target bit rate.

[0098]

[0099] FIG. 5 is a flowchart illustrating an example of an operation in which an electronic device according to various embodiments determines whether to enable a plurality of vector quantization steps.

[0100] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) can compare a target quality level representing the quality of communication with an external device (e.g., the electronic device (104) of FIG. 1) with a comparison quality level range corresponding to an already enabled vector quantization step. Based on the result of the comparison, the electronic device can disable the highest level of the vector quantization step among the already enabled vector quantization steps, or enable the lowest level of the quantization step among the disabled vector quantization steps among the plurality of vector quantization steps.

[0101] In operation (510), the electronic device can compare the target quality level with the comparison quality level range.

[0102] As described above in FIG. 4, the target quality level may refer to a quality level of communication that is supported when transmitting data packets from an electronic device to an external device, when considering the quality of communication between an electronic device and an external device.

[0103] The comparative quality level range may refer to a level of communication quality required to transmit information regarding the quantization result and the vector quantization step used when quantizing a potential vector using an already enabled vector quantization step among a plurality of vector quantization steps of an electronic device (e.g., quantization module (217) of FIG. 2, quantization module (317) of FIG. 3).

[0104] For example, the range of comparative quality levels may be greater than or equal to a first comparative quality level and less than a second comparative quality level. The first comparative quality level may refer to a level of communication quality required to transmit the quantization result of a latent vector using an already enabled vector quantization step. The second comparative quality level may refer to a level of communication quality required to transmit the quantization result of a latent vector using the enabled vector quantization steps, by enabling one additional vector quantization step (e.g., the lowest level vector quantization step among the disabled vector quantization steps) in addition to the already enabled vector quantization steps.

[0105] For example, two vector quantization steps (e.g., a first vector quantization step and a second vector quantization step) may be enabled in the quantization module. The first comparative quality level of the comparative quality level range may be a quality level required to transmit the quantization result using the two vector quantization steps. The second comparative quality level of the comparative quality level range may be a quality level required to transmit the quantization result using three vector quantization steps (e.g., a first vector quantization step, a second vector quantization step, and a third vector quantization step).

[0106] An electronic device may maintain the enable or disable of multiple vector quantization steps when the target quality level is included in the range of comparative quality levels. The target quality level being included in the range of comparative quality levels may include the target quality level being equal to or higher than a first comparative quality level and the target quality level being lower than a second comparative quality level. Being equal to or higher than the first comparative quality level may mean that the quantization result using the current quantization module (e.g., the first comparative quality level is required) can be transmitted to the communication state between the electronic device and the external device (e.g., the target quality level). Being lower than the second comparative quality level may mean that if the vector quantization step additionally disabled in the current quantization module is enabled, it is difficult to transmit the quantization result using the quantization module (e.g., the second comparative quality level is required) to the communication state between the electronic device and the external device (e.g., the target quality level). Accordingly, the electronic device may decide to maintain the enable or disable of each vector quantization step without changing it.

[0107] In operation (520), if the target quality level is higher than the comparison quality level range, the electronic device can enable the lowest level of vector quantization step among the disabled vector quantization steps among the plurality of vector quantization steps.

[0108] A target quality level being higher than the range of comparative quality levels may mean that the target quality level is equal to or higher than the second comparative quality level. A target quality level being equal to or higher than the second comparative quality level may mean that even if additional disabled vector quantization steps in the current quantization module are enabled, the quantization result using the quantization module (e.g., the second comparative quality level is required) can be transmitted to a communication state between the electronic device and an external device (e.g., the target quality level). Therefore, the electronic device can enable the lowest level of vector quantization steps among the disabled vector quantization steps.

[0109] In operation (530), the electronic device may disable the highest level of vector quantization steps among the already enabled vector quantization steps if the target quality level is lower than the comparison quality level range. A target quality level lower than the comparison quality level range may mean that the target quality level is lower than the first comparison quality level. A target quality level lower than the first comparison quality level may mean that it is difficult to transmit the quantization result using the current quantization module (e.g., the first comparison quality level is required) due to the communication state between the electronic device and the external device (e.g., the target quality level). Therefore, the electronic device may disable the highest level of vector quantization steps among the enabled vector quantization steps.

[0110] In operation (540), the electronic device can update the comparison quality level range.

[0111] For example, if the electronic device enables the disabled vector quantization steps through operation (520), the quantization module includes more enabled vector quantization steps than before, so the range of comparative quality levels can be increased.

[0112] For example, if the electronic device disables the enabled vector quantization step through operation (530), the quantization module contains fewer enabled vector quantization steps than before, so the range of the comparison quality level may be lowered.

[0113] An electronic device according to one embodiment may perform operation (510) again based on updating the comparison quality level range. The electronic device may repeat at least one of operation (510), operation (520), operation (530), or operation (540) until the target range is included in the comparison quality level range.

[0114] Although not explicitly illustrated in FIG. 5, the electronic device may maintain the enablement of the lowest level of vector quantization steps among the plurality of vector quantization steps (e.g., the first vector quantization step (310-1) of FIG. 3) independently of the quality of communication. Disabling the lowest level of vector quantization steps is substantially equivalent to disabling all vector quantization steps included in the quantization module and not performing quantization of the latent vector, so transmitting the latent vector from the electronic device to an external device instead of the quantization result may require a higher level of communication quality. Therefore, the electronic device may maintain the enablement of one vector quantization step even when the quality of communication is very low (e.g., lower than the threshold quality). For example, the electronic device may maintain the enablement of the first vector quantization step even if the target comparative quality level is lower than the range of comparative quality levels of the quantization module including the enabled first vector quantization step and the disabled remaining vector quantization steps.

[0115]

[0116] FIG. 6 is a diagram illustrating an example of an operation in which an electronic device according to various embodiments enables or disables at least one quantization step.

[0117] An electronic device according to one embodiment (e.g., the electronic device (101) of FIG. 1) may include a quantization module (e.g., the quantization module (217) of FIG. 2, the quantization module (317) of FIG. 3) that includes a plurality of vector quantization steps. In FIG. 6, exemplarily, the quantization module may include four vector quantization steps (e.g., a first vector quantization step (611), a second vector quantization step (612), a third vector quantization step (613), and a fourth vector quantization step (614)).

[0118] In state (610), the electronic device may include a quantization module in which three vector quantization steps (e.g., a first vector quantization step (611), a second vector quantization step (612), and a third vector quantization step (613)) are enabled. When the electronic device quantizes an input vector (x1) (e.g., a potential vector), it may obtain information regarding a first target vector (y1) based on the first vector quantization step (611), a second target vector (y2) based on the second vector quantization step (612), and a third target vector (y3) based on the third vector quantization step (613) as a quantization result. The electronic device may omit the quantization through the fourth vector quantization step (614) for the residual vector between the input and output of the third vector quantization step (613) (e.g., the third target vector (y3)) based on the fourth vector quantization step (614) being disabled.

[0119] In state (620), the electronic device may enable the fourth vector quantization step (614) that was disabled based on the fact that the quality of communication with an external device is improved (e.g., improved) compared to state (610). The electronic device may obtain the fourth target vector (y4) by further applying the fourth vector quantization step (614) to the residual vector between the input and output of the third vector quantization step (613), unlike in state (610). Consequently, the electronic device may obtain information regarding the first target vector (y1), the second target vector (y2), the third target vector (y3), and the fourth target vector (y4) as the quantization result of the input vector (x1).

[0120] In state (630), the electronic device may disable the third vector quantization step (613) that was enabled based on the fact that the quality of communication with the external device is degraded (e.g., worsened) compared to state (610). The electronic device may omit applying the third vector quantization step (613) to the residual vector between the input and output of the second vector quantization step (612), unlike in state (610). As a result, the electronic device may obtain information regarding the first target vector (y1) and the second target vector (y2) as the quantization result of the input vector (x1).

[0121]

[0122] An electronic device (101) includes a communication circuit for establishing communication with an external device (104); at least one processor (120) including a processing circuit; and a memory (130) including one or more storage media for storing instructions. When the instructions are executed by the at least one processor (120), the electronic device (101) may determine whether to enable a plurality of vector quantization stages based on the quality of the communication, quantize a latent vector (215) extracted from an audio frame (211) using the enabled vector quantization stage among the plurality of vector quantization stages, and transmit information regarding the quantization result (219) and the vector quantization stage used for quantization to the external device (104).

[0123] When the above instructions are executed by the at least one processor (120), the electronic device (101) may determine a target bitrate to transmit a signal to the external device (104) based on the quality of the communication, determine whether to enable the plurality of vector quantization steps based on the determined target bitrate, and transmit the quantization result (219) and information regarding the vector quantization steps used for quantization to the external device (104) at a bitrate lower than or equal to the target bitrate.

[0124] When the above instructions are executed by the at least one processor (120), the electronic device (101) may determine a target quality level of the communication, compare a range of comparison quality levels corresponding to a vector quantization step already enabled among the plurality of vector quantization steps with the determined target quality level, and based on the result of the comparison, disable the highest level of vector quantization step among the already enabled vector quantization steps, or enable the lowest level of quantization step among the disabled vector quantization steps among the plurality of vector quantization steps.

[0125] When the above instructions are executed by the at least one processor (120), the electronic device (101) may be configured to maintain the enable or disable of the plurality of vector quantization steps when the target quality level is included in the comparison quality level range, enable the lowest level vector quantization step among the disabled vector quantization steps among the plurality of vector quantization steps when the target quality level is higher than the comparison quality level range, and disable the highest level vector quantization step among the already enabled vector quantization steps when the target quality level is lower than the comparison quality level range.

[0126] When the above instructions are executed by the at least one processor (120), the electronic device (101) may be configured to quantize the input vector of the first vector quantization step (310-1; 611) of the first level of the enabled vector quantization steps into the target vector most similar to the input vector among the candidate vectors included in the codebook of the first vector quantization step (310-1; 611), obtain the residual vector between the target vector and the input vector as the input vector of the second vector quantization step (310-2; 612) of the second level higher than the first level, and transmit to the external device (104) information indicating the enabled vector quantization steps among the plurality of vector quantization steps, and information indicating the target vector of each enabled vector quantization step.

[0127] When the above instructions are executed by the at least one processor (120), the electronic device (101) may maintain the enable of the lowest level of the vector quantization step among the plurality of vector quantization steps, independently of the quality of the communication.

[0128] When the above instructions are executed by the at least one processor (120), the electronic device (101) may apply at least one of the vector quantization steps enabled to the potential vector (215), obtain a residual vector between the input vector and the target vector of the at least one applied vector quantization step, and skip the application of a vector quantization step of a higher level than the at least one applied vector quantization step based on the fact that the magnitude of the residual vector is less than or equal to a threshold size.

[0129] Among the plurality of vector quantization steps above, the number of candidate vectors included in the codebook of the first quantization step of the first level may be greater than the number of candidate vectors included in the codebook of the second quantization step of the second level, which is higher than the first level.

[0130] When the above instructions are executed by the at least one processor (120), the electronic device (101) may determine a limit on the number of vector quantization steps that can be enabled among the plurality of vector quantization steps based on the sampling rate of the audio frame (211), and determine to enable vector quantization steps among the plurality of vector quantization steps that are less than or equal to the limit.

[0131] When the above instructions are executed by the at least one processor (120), the electronic device (101) may determine the quality of the communication between the electronic device (101) and the external device (104) based on at least one of the state of the transmission buffer corresponding to the external device (104) or the RSSI (received signal strength indication) of the signal received from the external device (104).

[0132] A method performed by an electronic device (101) may include: an operation (410) of determining whether to enable a plurality of vector quantization stages based on the quality of communication between the electronic device (101) and an external device (104); an operation (420) of quantizing a latent vector (215) extracted from an audio frame (211) using an enabled vector quantization stage among the plurality of vector quantization stages; and an operation (430) of transmitting information regarding the quantization result (219) and the vector quantization stage used for quantization to the external device (104).

[0133] The operation (410) for determining whether to enable the above includes: determining a target bitrate to transmit a signal to the external device (104) based on the quality of the communication; and determining whether to enable the plurality of vector quantization steps based on the determined target bitrate. The operation (430) for transmitting the quantization result (219) and the information may include transmitting the quantization result (219) and information regarding the vector quantization steps used for quantization to the external device (104) at a bitrate lower than or equal to the target bitrate.

[0134] The operation (410) for determining whether to enable the above may include: an operation of determining a target quality level of the communication; an operation of comparing a comparison quality level range corresponding to a vector quantization step that is already enabled among the plurality of vector quantization steps with the determined target quality level; and an operation of disabling the highest level of vector quantization step among the already enabled vector quantization steps or enabling the lowest level of vector quantization step among the disabled vector quantization steps among the plurality of vector quantization steps based on the result of the comparison.

[0135] The operation (410) for determining whether to enable the above may include: maintaining the enable or disable of the plurality of vector quantization steps when the target quality level is included in the comparison quality level range; enabling the lowest level vector quantization step among the disabled vector quantization steps among the plurality of vector quantization steps when the target quality level is higher than the comparison quality level range; and disabling the highest level vector quantization step among the already enabled vector quantization steps when the target quality level is lower than the comparison quality level range.

[0136] The operation (420) of quantizing the potential vector (215) comprises, in the first vector quantization step (310-1; 611) of the first level among the enabled vector quantization steps, quantizing the input vector of the first vector quantization step (310-1; 611) into the target vector most similar to the input vector among the candidate vectors included in the codebook of the first vector quantization step (310-1; 611); The operation of obtaining a residual vector between the target vector and the input vector as an input vector of a second vector quantization step (310-2; 612) of a second level higher than the first level, and the operation of transmitting the quantization result (219) and the information (430) may include transmitting to the external device (104) information indicating the enabled vector quantization steps among the plurality of vector quantization steps, and information indicating the target vector of each enabled vector quantization step.

[0137] The operation (410) for determining whether to enable the above may include an operation of maintaining the enable of the lowest level of the vector quantization step among the plurality of vector quantization steps, independently of the quality of the communication.

[0138] The operation (420) of quantizing the potential vector (215) may include: applying at least one of the vector quantization steps enabled to the potential vector (215), and then obtaining a residual vector between the input vector and the target vector of the at least one applied vector quantization step; and skipping the application of a vector quantization step of a higher level than the at least one applied vector quantization step based on the fact that the magnitude of the residual vector is less than or equal to a threshold size.

[0139] Among the plurality of vector quantization steps above, the number of candidate vectors included in the codebook of the first quantization step of the first level may be greater than the number of candidate vectors included in the codebook of the second quantization step of the second level, which is higher than the first level.

[0140] The operation (410) for determining whether to enable the above may include: an operation for determining a limit on the number of vector quantization steps that can be enabled among the plurality of vector quantization steps based on the sampling rate of the audio frame (211); and an operation for determining to enable vector quantization steps among the plurality of vector quantization steps that are less than or equal to the number limit.

[0141]

[0142] The electronic device according to the various embodiments disclosed in this document may be of various forms. The electronic device may include, for example, a portable communication device (e.g., a smartphone), a computer device, a portable multimedia device, a portable medical device, a camera, a wearable device, or a consumer electronics device. The electronic device according to the embodiments of this document is not limited to the devices described above.

[0143] The various embodiments of this document and the terms used therein are not intended to limit the technical features described in this document to specific embodiments, and should be understood to include various modifications, equivalents, or substitutions of said embodiments. In connection with the description of the drawings, similar reference numerals may be used for similar or related components. The singular form of a noun corresponding to an item may include one or more of said items unless the relevant context clearly indicates otherwise. In this document, phrases such as "A or B," "at least one of A and B," "at least one of A or B," "A, B or C," "at least one of A, B and C," and "at least one of A, B, or C" may each include any one of the items listed together in the corresponding phrase, or all possible combinations thereof. Terms such as "first," "second," or "first" or "second" may be used simply to distinguish said components from other said components and do not limit said components in any other aspect (e.g., importance or order). Where any (e.g., 1st) component is referred to as "coupled" or "connected" to another (e.g., 2nd) component, with or without the terms "functionally" or "communicationly," it means that said any component may be connected to said other component directly (e.g., via a wire), wirelessly, or through a third component.

[0144] The term “module” as used in the various embodiments of this document may include a unit implemented in hardware, software, or firmware, and may be used interchangeably with terms such as logic, logic block, component, or circuit, for example. A module may be a component formed integrally, or a minimum unit of said component or a part thereof that performs one or more functions. For example, according to one embodiment, a module may be implemented in the form of an application-specific integrated circuit (ASIC).

[0145] Various embodiments of the present document may be implemented as software (e.g., program (140)) comprising one or more instructions stored in a storage medium (e.g., internal memory (136) or external memory (138)) readable by a machine (e.g., electronic device (101)). For example, a processor (e.g., processor (120)) of the machine (e.g., electronic device (101)) may call at least one of the one or more instructions stored in the storage medium and execute it. This enables the machine to be operated to perform at least one function according to the at least one called instruction. The one or more instructions may include code generated by a compiler or code that can be executed by an interpreter. The storage medium readable by the machine may be provided in the form of a non-transitory storage medium. Here, 'non-temporary' simply means that the storage medium is a tangible device and does not contain a signal (e.g., electromagnetic waves), and the term does not distinguish between cases where data is stored semi-permanently and cases where it is stored temporarily.

[0146] According to one embodiment, the method according to the various embodiments disclosed herein may be provided as included in a computer program product. The computer program product may be traded between a seller and a buyer as a product. The computer program product may be distributed in the form of a device-readable storage medium (e.g., compact disc read-only memory (CD-ROM)), or distributed online (e.g., download or upload) through an application store (e.g., Play Store™) or directly between two user devices (e.g., smartphones). In the case of online distribution, at least a portion of the computer program product may be temporarily stored or temporarily created on a device-readable storage medium, such as the memory of a manufacturer's server, an application store's server, or a relay server.

[0147] According to various embodiments, each component (e.g., module or program) of the components described above may include a singular or multiple entities, and some of the multiple entities may be separated and placed in other components. According to various embodiments, one or more of the components or operations of the aforementioned components may be omitted, or one or more other components or operations may be added. Generally or additionally, multiple components (e.g., module or program) may be integrated into a single component. In this case, the integrated component may perform one or more functions of each of the multiple components in the same or similar manner as those performed by the corresponding component among the multiple components prior to integration. According to various embodiments, operations performed by the module, program, or other components may be executed sequentially, in parallel, iteratively, or heuristically, or one or more of the operations may be executed in a different order, omitted, or one or more other operations may be added.

[0148] The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware and software components. For example, the devices, methods, and components described in the embodiments may be implemented using a general-purpose computer or a special-purpose computer, such as, for example, a processor, a controller, an arithmetic logic unit (ALU), a digital signal processor, a microcomputer, a field programmable gate array (FPGA), a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing unit may execute an operating system (OS) and software applications executed on said operating system. Additionally, the processing unit may access, store, manipulate, process, and generate data in response to the execution of the software. For ease of understanding, the processing unit may be described as being used as a single unit, but those skilled in the art will understand that the processing unit may include multiple processing elements and / or multiple types of processing elements. For example, the processing unit may include multiple processors or one processor and one controller. In addition, other processing configurations, such as parallel processors, are also possible.

[0149] Software may include computer programs, code, instructions, or a combination of one or more of these, and may configure a processing unit to operate as desired or command the processing unit independently or collectively. Software and / or data may be permanently or temporarily embodied in any type of machine, component, physical device, virtual equipment, or computer storage medium or device so as to be interpreted by the processing unit or to provide instructions or data to the processing unit. Software may be distributed over networked computer systems and may be stored or executed in a distributed manner. Software and data may be stored on computer-readable recording media.

[0150] The method according to the embodiment may be implemented in the form of program instructions that can be executed through various computer means and recorded on a computer-readable medium. The computer-readable medium may include program instructions, data files, data structures, etc., either alone or in combination, and the program instructions recorded on the medium may be those specifically designed and configured for the embodiment or those known and available to those skilled in the art of computer software. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tapes; optical recording media such as CD-ROMs and DVDs; magneto-optical media such as floptical disks; and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, and flash memory. Examples of program instructions include machine code, such as that generated by a compiler, as well as high-level language code that can be executed by a computer using an interpreter, etc.

[0151] The hardware device described above may be configured to operate as one or more software modules to perform the operation of the embodiment, and vice versa.

Claims

1. In an electronic device (101), A communication circuit that establishes communication with an external device (104); At least one processor (120) including a processing circuit; and It includes a memory (130) comprising one or more storage media for storing instructions, and When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Based on the quality of the above communication, determine whether to enable a plurality of vector quantization stages, and A latent vector (215) extracted from an audio frame (211) is quantized using an enabled vector quantization step among the plurality of vector quantization steps, and Information regarding the quantization result (219) and the vector quantization step used for the quantization is transmitted to the external device (104). making, Electronic device (101).

2. In Paragraph 1, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Based on the quality of the above communication, a target bitrate to transmit a signal to the external device (104) is determined, and Based on the above-determined target bitrate, the enable status of the plurality of vector quantization steps is determined, and Information regarding the quantization result (219) and the vector quantization step used for the quantization is transmitted to the external device (104) at a bit rate lower than or equal to the target bit rate. making, Electronic device (101).

3. In any one of paragraphs 1 to 2, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Determine the target quality level of the above communication, and A comparison quality level range corresponding to an already enabled vector quantization step among the plurality of vector quantization steps is compared with the determined target quality level, and Based on the result of the above comparison, disable the highest level vector quantization step among the already enabled vector quantization steps, or enable the lowest level quantization step among the disabled vector quantization steps among the plurality of vector quantization steps. making, Electronic device (101).

4. In any one of paragraphs 1 through 3, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Based on the fact that the target quality level is included in the comparison quality level range, the enable or disable of the plurality of vector quantization steps is maintained, Based on the fact that the above target quality level is higher than the above comparison quality level range, the lowest level vector quantization step among the disabled vector quantization steps among the plurality of vector quantization steps is enabled, and Based on the fact that the above target quality level is lower than the above comparison quality level range, disable the highest level vector quantization step among the already enabled vector quantization steps. making, Electronic device (101).

5. In any one of paragraphs 1 through 4, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, In the first vector quantization step (310-1; 611) of the first level among the enabled vector quantization steps, the input vector of the first vector quantization step (310-1; 611) is quantized into the target vector most similar to the input vector among the candidate vectors included in the codebook of the first vector quantization step (310-1; 611), and The residual vector between the target vector and the input vector of the first vector quantization step (310-1; 611) is obtained as the input vector of a second vector quantization step (310-2; 612) of a second level higher than the first level, and Information indicating the enabled vector quantization steps among the plurality of vector quantization steps, and information indicating the target vector of each enabled vector quantization step, are transmitted to the external device (104). making, Electronic device (101).

6. In any one of paragraphs 1 through 5, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Independently of the quality of the above communication, maintain the enable of the lowest level vector quantization step among the plurality of vector quantization steps. making, Electronic device (101).

7. In any one of paragraphs 1 through 6, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, After applying at least one of the enabled vector quantization steps to the above potential vector (215), a residual vector between the input vector and the target vector of the at least one applied vector quantization step is obtained, and Based on the fact that the magnitude of the above residual vector is less than or equal to a threshold size, the application of a vector quantization step at a level higher than the level of the at least one applied vector quantization step is skipped. making, Electronic device (101).

8. In any one of paragraphs 1 through 7, Among the plurality of vector quantization steps above, the number of candidate vectors included in the codebook of the first quantization step of the first level is, Larger than the number of candidate vectors included in the codebook of the second quantization step of the second level, which is higher than the first level mentioned above. Electronic device (101).

9. In any one of paragraphs 1 through 8, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, Based on the sampling rate of the above audio frame (211), a limit on the number of enableable vector quantization steps among the plurality of vector quantization steps is determined, and Determining to enable vector quantization steps, among the plurality of vector quantization steps, that are less than or equal to the number limit. making, Electronic device (101).

10. In any one of paragraphs 1 through 9, When the above instructions are executed by the at least one processor (120), the electronic device (101) is made to, The quality of the communication between the electronic device (101) and the external device (104) is determined based on at least one of the state of the transmission buffer corresponding to the external device (104) or the RSSI (received signal strength indication) of the signal received from the external device (104). making, Electronic device (101).

11. In a method performed by an electronic device (101), An operation (410) for determining whether to enable a plurality of vector quantization stages based on the quality of communication between the electronic device (101) and the external device (104); An operation (420) of quantizing a latent vector (215) extracted from an audio frame (211) using an enabled vector quantization step among the plurality of vector quantization steps; and The operation (430) of transmitting information regarding the quantization result (219) and the vector quantization step used for the quantization to the external device (104) including, method.

12. In Paragraph 11, The operation (410) for determining whether to enable the above is, An operation to determine a target bitrate to transmit a signal to the external device (104) based on the quality of the above communication; Based on the above-determined target bit rate, the method includes an operation to determine whether to enable the plurality of vector quantization steps, and The operation (430) of transmitting the above quantization result (219) and the above information is, The operation of transmitting information regarding the quantization result (219) and the vector quantization step used for the quantization to the external device (104) at a bit rate lower than or equal to the target bit rate, method.

13. In any one of paragraphs 11 to 12, The operation (410) for determining whether to enable the above is, An operation to determine the target quality level of the above communication; The operation of comparing a comparison quality level range corresponding to a vector quantization step already enabled among the plurality of vector quantization steps with the determined target quality level; and Based on the result of the above comparison, the operation of disabling the highest level vector quantization step among the already enabled vector quantization steps, or enabling the lowest level quantization step among the disabled vector quantization steps among the plurality of vector quantization steps. method.

14. In any one of paragraphs 11 through 13, The operation (410) for determining whether to enable the above is, If the target quality level is included in the above comparison quality level range, the operation of maintaining the enable or disable of the plurality of vector quantization steps; If the above target quality level is higher than the above comparison quality level range, the operation of enabling the lowest level vector quantization step among the disabled vector quantization steps among the plurality of vector quantization steps; and If the above target quality level is lower than the above comparison quality level range, the operation of disabling the highest level vector quantization step among the already enabled vector quantization steps is included. method.

15. A computer-readable recording medium storing one or more computer programs comprising instructions for performing the method of any one of paragraphs 11 through 14.

Images