Method for providing call quality information and electronic device therefor

Abstract

Disclosed is an electronic device comprising a display, a speaker, a communication circuit, a processor, and a memory electrically connected to the processor. The electronic device may: establish a connection for a call with an external electronic device by using the communication circuit; receive call data from the external electronic device through the connection; decode the received call data or perform error recovery processing on the received call data to generate decoded call data from the received call data; calculate an error rate of the decoded call data on the basis of an amount of data subjected to error recovery among the decoded call data; and provide output quality information based on the error rate to the external electronic device through the connection so that the external electronic device is able to recognize output quality information of the decoded call data that is output through the electronic device.

Description

Method for providing call quality information and electronic device for the same

[0001] The embodiments disclosed in this document relate to a method for providing real-time call quality information and an electronic device for the same.

[0002] With the recent development of digital technology, electronic devices that support calls, such as mobile communication terminals, smartphones, or tablet PCs (personal computers), are becoming widely available.

[0003] Most electronic devices supporting calls display signal strength based on the electric field strength between the electronic device and a nearby base station through signal strength notifications shown as multiple bars. If the condition of the network (or communication network) connected to the electronic device in a call is poor, the electronic device can identify that the transmission and reception status of call data may deteriorate by utilizing status information related to the network's electric field conditions and the electronic device's signal strength (e.g., RSRP (reference signals received power) and / or RSSI (received signal strength indicator)). While the electronic device can check its own transmission and reception status using the information related to the network electric field conditions and the electronic device's signal strength, it cannot know the call status of the counterparty, i.e., the receiving electronic device. For example, if there is a problem with the transmission and reception status of call data due to degradation in the receiving network, the transmitting electronic device cannot detect this.

[0004] Each electronic device participating in a call may transmit information related to the transmission and reception status (e.g., packet loss rate, latency, and / or jitter) to the other electronic device using the RTCP RR (real-time transport control protocol receiver report) and / or RTCP SR (real-time transport control protocol sender report) protocols. Each electronic device may use the RTCP RR and / or RTCP SR information to predict the field conditions of the network connected to each electronic device during the call, but it cannot know whether errors have occurred in the call data finally output by the receiving electronic device. The RTCP protocol information may include information about the state of the network, but may not include information regarding errors that may occur after the call data is received from the network (e.g., degradation occurring during the output device and the processing of the received call data).

[0005] The information described above may be provided as related art for the purpose of aiding understanding of the present disclosure. No claim or determination is made as to whether any of the foregoing may be applied as prior art in relation to the present disclosure.

[0006] An electronic device according to one embodiment disclosed herein may include a display, at least one speaker, at least one communication circuit, at least one processor, and a memory electrically connected to at least one processor. The memory may store instructions that, when executed individually or collectively by the at least one processor, cause the electronic device to create a connection for a call with an external electronic device using the at least one communication circuit. The memory may store instructions that, when executed individually or collectively by the at least one processor, cause the electronic device to receive call data from the external electronic device through the connection. The memory may store instructions that, when executed individually or collectively by the at least one processor, cause the electronic device to decode the received call data or perform error recovery processing to generate decoded call data from the received call data. The memory may store instructions that, when executed individually or in combination by the at least one processor, cause the electronic device to calculate the error rate of the decoded call data based on the amount of error-recovered data among the decoded call data. The memory may store instructions that, when executed individually or in combination by the at least one processor, cause the electronic device to provide output quality information based on the error rate to the external electronic device through the connection, so that the electronic device can know the output quality information of the decoded call data output from the external electronic device through the electronic device.

[0007] A method for generating output quality information for an electronic device to transmit to an external electronic device according to an embodiment disclosed in this document may include: generating a connection for a call with the external electronic device using at least one communication circuit of the electronic device; receiving call data from the external electronic device through the connection; generating decoded call data from the received call data by decoding or error recovery processing the received call data; calculating an error rate of the decoded call data based on the amount of error-recovered data among the decoded call data; and providing output quality information based on the error rate to the external electronic device through the connection so that the external electronic device can know the quality information of the decoded call data output through the electronic device.

[0008] A computer-readable storage medium according to one embodiment disclosed in this document may store instructions that cause an electronic device to perform a method of generating output quality information for transmitting to an external electronic device.

[0009] An electronic device according to one embodiment disclosed in this document may include at least one camera, at least one microphone, at least one communication circuit, at least one processor electrically connected to the at least one camera, the at least one microphone, and the at least one communication circuit and including a processing circuit, and a memory electrically connected to the at least one processor. The memory may store instructions that, when executed individually or collectively by the at least one processor, cause the electronic device to create a connection for a call with an external electronic device using the at least one communication circuit. The memory may store instructions that, when executed individually or collectively by the at least one processor, cause the electronic device to acquire call data from at least one of the at least one camera, the at least one microphone, or a wireless electronic device capable of communicating through the at least one communication circuit. The memory may store instructions that, when executed individually or collectively by the at least one processor, cause the electronic device to transmit the acquired call data to the external electronic device through the connection. The memory may store instructions that, when executed individually or in combination by the at least one processor, allow the electronic device to receive output quality information associated with the quality when the call data is output through the external electronic device via the connection.The memory may store instructions such that, when executed individually or in combination by the at least one processor, the electronic device provides information about the quality of the call data when it is output through the external electronic device based on the output quality information.

[0010] A method for an electronic device to receive and provide output quality information from an external electronic device according to an embodiment disclosed in this document may include: an operation of creating a connection for a call with the external electronic device using at least one communication circuit; an operation of acquiring call data from at least one camera of the electronic device, at least one microphone of the electronic device, or at least one wireless electronic device capable of communicating through the at least one communication circuit; an operation of transmitting the acquired call data to the external electronic device through the connection; an operation of receiving output quality information associated with the quality when the call data is output through the external electronic device through the connection; and an operation of providing information about the quality when the call data is output through the external electronic device based on the output quality information.

[0011] A computer-readable storage medium according to one embodiment disclosed in this document may store instructions that cause an electronic device to perform a method of receiving and providing output quality information from an external electronic device.

[0012] FIG. 1 illustrates a communication scenario between electronic devices according to one embodiment.

[0013] FIG. 2 illustrates a block diagram of an electronic device according to one embodiment.

[0014] FIG. 3 illustrates a block diagram showing a data transfer path and an electronic device according to one embodiment.

[0015] FIG. 4 illustrates a block diagram showing a data transfer path of a receiving electronic device and a receiving electronic device according to one embodiment.

[0016] FIG. 5 illustrates received data and output data according to one embodiment.

[0017] FIG. 6 illustrates an RTCP APP packet message format according to one embodiment.

[0018] FIG. 7 is a flowchart of the operation of a receiving electronic device for providing call quality information according to one embodiment.

[0019] FIG. 8 is a flowchart of the operation of a transmitting electronic device for providing call quality information according to one embodiment.

[0020] FIG. 9 is a flowchart of a method for providing call quality information according to one embodiment.

[0021] FIG. 10 illustrates a user interface (UI) providing call quality information according to one embodiment.

[0022] FIG. 11 is a block diagram of an exemplary electronic device capable of performing the operations described in this document.

[0023] Hereinafter, various embodiments of the present invention are described with reference to the accompanying drawings. However, this is not intended to limit the present invention to specific embodiments and should be understood to include various modifications, equivalents, and / or alternatives of the embodiments of the present invention.

[0024] FIG. 1 illustrates a communication scenario between electronic devices according to one embodiment.

[0025] Referring to FIG. 1, according to one embodiment, the electronic device (10, 20) may include any portable electronic device. The electronic device (10, 20) may include, for example, at least one of a mobile phone, a smart watch, smart glasses, a smart ring, and / or an IoT (Internet of Things) device.

[0026] In one example, an electronic device (10, 20) may be connected to a communication network to perform a call. The communication network may include a mobile communication network and / or a packet-switched network. For example, a packet-switched network may include a voice over internet protocol network and / or a video over internet protocol network. The communication network may mean a network that supports a call through a mobile communication network or a call through a packet-switched network.

[0027] In one example, the electronic device (10, 20) may be a device configured to perform a call. For example, the electronic device (10, 20) may be configured to perform a call based on packet data. For example, the electronic device (10, 20) may be configured to perform a video call and / or voice call based on packet data. In one example, the electronic device (10, 20) may be configured to perform a packet-based call over a network. For example, the electronic device (10, 20) may be configured to perform a voice call based on VoLTE (voice over long-term evolution). For example, the electronic device (10, 20) may be configured to perform a video call based on ViLTE (video over long-term evolution). For example, the electronic device (10, 20) may be configured to perform voice calls based on VoWiFi (voice over wireless fidelity). For example, the electronic device (10, 20) may be configured to perform video calls based on ViWiFi (video over wireless fidelity).

[0028] In one example, the transmitting electronic device (10) can acquire call inputs (101) (e.g., audio input and video input). For example, the transmitting electronic device (10) can acquire audio input after establishing a connection with the receiving electronic device (20) to perform a voice call. The audio input may include an analog audio signal input via the microphone of the transmitting electronic device (10). For example, the transmitting electronic device (10) can acquire audio input and video input after establishing a connection with the receiving electronic device (20) to perform a video call. The video input may include video captured through a camera (not shown) included in the transmitting electronic device (10).

[0029] The transmitting electronic device (10) can convert a call input (101) obtained through an analog-to-digital converter (ADC) from an analog signal to a digital signal. The call input (101) converted into a digital signal can be referred to as call data. The call data may include data in a form that can be transmitted over a call (e.g., audio data and / or video data). The transmitting electronic device (10) can encode the call data using a codec (e.g., G.711, G.729, OPUS, H.264, H.265, VP8, and / or VP9) to efficiently transmit the call data converted into a digital signal.

[0030] In one example, a transmitting electronic device (10) may transmit an uplink real-time transport protocol (RTP) packet (111) to a base station (30) over a network. The RTP packet may contain data that has been compressed by a codec and then packetized into small units. Packetization may mean dividing a large amount of data into small units so that it can be efficiently transmitted over a network. The transmitting electronic device (10) may divide the call data into multiple packets using a real-time protocol (e.g., real-time transport protocol (RTP)) for transmitting call data. RTP is a network protocol used for real-time data transmission and may refer to a protocol used in real-time voice calls. For example, each RTP packet may include a timestamp, a packet sequence number, and / or synchronization source (SSRC) information. For example, an uplink RTP packet (111) transmitted by a transmitting electronic device (10) to a base station (30) may not be delivered to the base station (30) or may be delivered later than scheduled due to network degradation. In one example, the base station (30) may relay the uplink RTP packet (111) received from the transmitting electronic device (10) to the receiving electronic device (20). For example, the base station (30) may transmit a downlink RTP packet (112) to the receiving electronic device (20). For example, the downlink RTP packet (112) may contain the same information (e.g., call data, timestamp, and / or packet sequence number) that is included in the uplink RTP packet (111). For example, at least some of the downlink RTP packets (112) may not contain the information that is included in the uplink RTP packet (111) due to network degradation.For example, a downlink RTP (112) packet transmitted by the base station (30) to the receiving electronic device (20) may not be delivered to the receiving electronic device (20) or may be delivered later than scheduled due to network degradation.

[0031] In one example, the receiving electronic device (20) can receive downlink RTP packets (112) and reassemble or correct the packets. For example, if the downlink RTP packets (112) do not arrive in order, the receiving electronic device (20) can reassemble the packets using downlink RTP packet information (e.g., timestamp and / or packet sequence number). For example, if the downlink RTP packets (112) are received at irregular time intervals, the receiving electronic device (20) can temporarily store the downlink RTP packets (112) using a jitter buffer. A jitter buffer may refer to a buffer for adjusting the output time to ensure that the call data output when packets arrive irregularly is played back without interruption.

[0032] In one example, the receiving electronic device (20) can play back the call output (102). For example, when performing a voice call, the receiving electronic device (20) can decode the received downlink RTP packet (112) to restore the original voice signal and play it back through a speaker. For example, when performing a video call, the receiving electronic device (20) can decode the received data packet to restore the compressed data back to the original data. The receiving electronic device (20) can reconstruct the restored data into a series of video frames and play them back through a speaker and / or display.

[0033] In this disclosure, the term "decoding" may be referred to as a data processing operation performed by a decoder. For example, decoding may include at least one of the operation of restoring compressed data to its original form, the operation of depacketizing packetized data, or the operation of error recovery processing for data in which errors have occurred. For convenience of explanation, decoding has been described as the operation of restoring compressed data to its original form, but the examples of this disclosure are not limited thereto. Even if decoding is described below as the operation of restoring compressed data for convenience of explanation, a person skilled in the art will understand that decoding may further include at least one of the operation of depacketizing packetized data or the operation of error recovery processing for data in which errors have occurred.

[0034] In FIG. 1, uplink RTP packets (111) and downlink RTP packets (112) are described, but the embodiments of the present disclosure are not limited thereto. For example, electronic devices (10, 20) may packetize call data using protocols other than RTP. For example, when performing a video call between a transmitting electronic device (10) and a receiving electronic device (20), the transmitting electronic device (10) may transmit at least one of RTP packets (111, 112) or packets based on a protocol for packetizing video data (e.g., RTP, RTSP (real-time streaming protocol), and / or SIP (session initiation protocol)) to the receiving electronic device (20) through a base station (30).

[0035] In one example, the receiving electronic device (20) may transmit an uplink real-time transport control protocol (RTCP) packet (121) to the base station (30) over the network. The RTCP packet may include, for example, an RTCP receiver report (RTCP RR) packet and / or an application-defined RTCP (RTCP APP) packet. The RTCP RR packet is a packet sent by the receiving electronic device (20) to the transmitting electronic device (10) and may include information for providing feedback on network status. For example, the RTCP RR packet may include packet loss rate, jitter, and / or round-trip time (RTT) information. The RTCP APP packet may be a packet for transmitting additional information defined by a specific application. For example, the application-dependent data field of the RTCP APP packet may include additional information defined by the application (e.g., call data output quality information of the receiving electronic device (20)). The structure of the RTCP APP packet may be described later with reference to FIG. 6.

[0036] In one example, the base station (30) may relay an uplink RTCP packet (121) received from the receiving electronic device (20) and transmit it to the transmitting electronic device (10). For example, the base station (30) may transmit a downlink RTCP packet (122) to the transmitting electronic device (10). For example, the downlink RTCP packet (122) may include the RTCP RR and / or RTCP APP packets included in the uplink RTCP packet (121) as they are. The transmitting electronic device (10) may utilize the information from the downlink RTCP packet (122) to provide information (e.g., UI, notification sound and / or vibration) regarding the call status of the receiving electronic device (20) to the user of the transmitting electronic device (10). A method for providing information regarding the call status of the receiving electronic device (20) may be described later in relation to FIG. 10.

[0037] The transmitting electronic device (10) and the receiving electronic device (20) illustrated in FIG. 1 are exemplary, and the embodiments of the present disclosure are not limited thereto. For example, during a call, the receiving electronic device (20) may also acquire the call input of the receiving electronic device (20) and transmit an RTP packet to the transmitting electronic device (10). In this case, the direction of the path through which the RTP packet and the RTCP packet are transmitted may differ. For example, the transmitting electronic device (10) may also transmit an RTCP packet (e.g., an RTCP SR (sender report)) containing transmitting network status information to the receiving electronic device (20).

[0038] FIG. 2 illustrates a block diagram of an electronic device according to one embodiment.

[0039] Referring to FIGS. 1 and FIGS. 2, according to one embodiment, the electronic device (200) may be a transmitting electronic device (10). According to another embodiment, the electronic device (200) may be a receiving electronic device (20). The electronic device (200) may perform both the transmission and reception of a call signal during a real-time call. For example, the electronic device (200) may include the same or similar configuration as the electronic device (1100) described below in relation to FIG. 11. For example, the memory (250) may correspond to the memory (1120) of FIG. 11. The processor (240) may correspond to the processor (1110) of FIG. 11. The communication circuit (230) may correspond to the communication circuit (1160) of FIG. 11. The configuration of the electronic device (200) shown in FIG. 2 is exemplary, and the configuration of the electronic device (200) is not limited thereto. For example, the electronic device (200) may further include a configuration not shown in FIG. 2 (e.g., at least one of the configurations of the electronic device (1100) of FIG. 11). For example, the electronic device (200) may not include at least one of the configurations shown in FIG. 2.

[0040] In one example, the speaker (210) can play multimedia including audio. For example, audio data among the call signals received by the electronic device (200) during a real-time call can be played through the speaker (210). The audio data output by the speaker (210) can be referred to as voice converted from an electrical signal. For example, the speaker (210) can be controlled to convert the electrical signal using a digital-analog converter (DAC) and to acquire and output the converted voice. In FIG. 2, the speaker (210) is shown as being included inside the electronic device (200), but the embodiments of the present disclosure are not limited thereto. For example, the speaker (210) may be located outside the electronic device (200) and may include a device configured to receive audio or audio signals (e.g., a Bluetooth receiving circuit or a USB circuit). For example, the electronic device (200) may output audio through an external electronic device (e.g., an external speaker and / or wireless earphones) connected directly or wirelessly to the electronic device (200).

[0041] In one example, a display (not shown) can play multimedia including video. For example, video data among the call signals received by the electronic device (200) during a real-time call can be displayed through the display. For example, the electronic device (200) can receive packetized video data and reconstruct the received video data into a series of video frames to display on the display.

[0042] In one example, the microphone (220) can acquire various audio data from an external source (e.g., a user). For example, an electronic device (200) can acquire audio data corresponding to a voice detected from an external source (e.g., user speech) using the microphone (220). The microphone (220) may include a dynamic microphone, a condenser microphone, and / or a piezo microphone. The audio data acquired by the microphone (220) may be referenced as data obtained by converting the voice received from an external source into an electrical signal. For example, the electronic device (200) may control the acquisition of converted audio data by converting the voice detected by the microphone (220) using an analog-to-digital converter (ADC). In FIG. 2, the microphone (220) is shown as being included inside the electronic device (200), but embodiments of the present disclosure are not limited thereto. For example, the microphone (220) may be located outside the electronic device (200) and may include a device configured to transmit audio or audio signals (e.g., a Bluetooth transmission circuit, or a USB circuit). For example, the electronic device (200) may acquire audio through an external electronic device (e.g., an external microphone and / or wireless earphones) connected directly or wirelessly to the electronic device (200).

[0043] In one example, a camera (not shown) can acquire various image data from an external source (e.g., a user). The camera may include a CMOS (complementary metal-oxide-semiconductor) sensor camera, an ultra-wide angle camera, and / or a front-facing camera. For example, an electronic device (200) can acquire image data around the camera using the camera. The image data acquired by the camera can be referenced as data obtained by converting an image captured around the camera into an electrical signal. For example, the electronic device (200) can acquire image data by converting an image captured by the camera using an analog-to-digital converter (ADC).

[0044] In one example, the communication circuit (230) may include at least one circuit (e.g., a modem, a radio frequency integrated circuit, and / or a radio frequency processing circuit) configured to support communication between an electronic device (200) (e.g., a transmitting electronic device (10)) and another electronic device (e.g., a receiving electronic device (20)). The communication circuit (140) may support wired communication and / or wireless communication. The communication circuit (140) may support short-range wireless communication and / or long-range wireless communication.

[0045] In one example, the processor (240) may be connected communically, electrically, operatively, or functionally to a speaker (210), a display (not shown), a microphone (220), a camera (not shown), a communication circuit (230), and / or a memory (250). In various embodiments of the present disclosure, when one component is connected "operatively" to another component, it may mean that the component is connected to enable the other component to operate. For example, the component may enable the other component by transmitting a control signal to the other component directly or through another component. In various embodiments of the present disclosure, when one component is connected "functionally" to another component, it may mean that the component is connected to enable the function of the other component. For example, the component may enable the function of the other component by transmitting a control signal to the other component directly or through another component.

[0046] The processor (240) may include at least one processor. For example, the processor (240) may include an application processor (AP), a central processing unit (CPU), a graphics processing unit (GPU), a neural processing unit (NPU), a tensor processing unit (TPU), and / or a communication processor (CP). The processor (240) may include a chip or a chipset. In the present disclosure, the processor (240) may be referred to as a hardware component having an architecture by at least one processing circuit. For example, the processor (240) may be placed on a substrate (e.g., a printed circuit board) located inside the electronic device (200) and may communicate with other components of the electronic device (200) through at least one conductive path formed in the substrate.

[0047] For example, the processor (240) of the receiving electronic device (20) can control the communication circuit (230) to create a connection for a call with the transmitting electronic device (10). The processor (240) of the receiving electronic device (20) can cause call data packets based on a first protocol (e.g., RTP) received through the connection to be temporarily stored in a jitter buffer. The processor (240) of the receiving electronic device (20) can send the call data stored in the jitter buffer to a decoder (e.g., 403 in FIG. 4). The processor (240) can decode the call data through the decoder (e.g., 403 in FIG. 4) to generate decoded data. The processor (240) can calculate an error rate by identifying the ratio of call data packets generated by processing errors (e.g., degradation) among the decoded call data. The processor (240) of the receiving electronic device (20) can generate output quality information using the calculated error rate information and transmit it to the transmitting electronic device (10) using a first packet (e.g., RTCP APP) defined by the second protocol. The generation of output quality information may be described later in FIGS. 3 and FIGS. 4.

[0048] For example, the processor (240) of the transmitting electronic device (10) can control the communication circuit (230) to create a connection for a call with the receiving electronic device (20). The processor (240) of the transmitting electronic device (10) can control the microphone (220) and / or camera (not shown) to obtain call data from an external source (or user). The processor (240) of the transmitting electronic device (10) can transmit the call data obtained through the connection in the form of packets based on a first protocol (e.g., RTP). The processor (240) of the transmitting electronic device (10) can receive output quality information of the call data of the receiving electronic device (20) based on a first packet (e.g., RTCP APP) defined by a second protocol through the connection, and can generate a UI, a notification sound, and / or vibration based on the received output quality information of the call data to provide information about the output quality of the call data of the receiving electronic device (20) to the user.

[0049] The memory (250) can store instructions. When the instructions are executed by the processor (240), they can cause the electronic device (200) to perform various operations. In various embodiments of the present disclosure, the operation of the electronic device (200) may be referred to as an operation performed by the processor (240) by executing the instructions stored in the memory (250). The memory (250) may be configured with hardware similar to the memory (1120) described below in relation to FIG. 11.

[0050] FIG. 3 illustrates a block diagram showing a data transfer path and an electronic device according to one embodiment.

[0051] Referring to FIGS. 1, 2 and 3, in one example, the transmitting side electronic device (10) may include a utterance detection unit (311), a call transmission / reception data processing unit (312), and / or a receiving side call quality status representation unit (313).

[0052] The speech detection unit (311) can perform the function of detecting a section containing voice among the input data (321). For example, the speech detection unit (311) can detect a speech section based on voice activity detection (VAD). For example, the speech detection unit (311) can extract features of audio data (e.g., signal energy, zero crossing rate (ZCR), and / or frequency spectrum) among the input data (321), and determine that it is a speech section if the value of the features is greater than or equal to a threshold value (e.g., signal energy is greater than or equal to a threshold value).

[0053] In one example, the speech detection unit (311) can identify time information (335) of the speech interval (e.g., time information of the speech start point and speech end point) among the input data (321). The speech detection unit (311) can determine, for example, that the speech interval has not ended even if there is a silent interval of less than a specified time (e.g., 150ms) after the start point of the speech interval. The speech detection unit (311) can determine that the speech interval has ended and identify time information of the end point if, for example, a silent interval exceeding a specified time (e.g., 150ms) appears after the start point of the speech interval. The speech detection unit (311) can transmit the identified time information (335) of the speech interval to the receiving side call quality status display unit (313). The speech detection unit (311) can store the identified time information (335) of the speech interval in memory (250).

[0054] The call transmission / reception data processing unit (312) can process data to transmit call data to a receiving electronic device (20) via a network. In one example, the call transmission / reception data processing unit (312) can compress input data (321) using a codec (e.g., G.711, G.729, OPUS, H.264, H.265, VP8, and / or VP9). The call transmission / reception data processing unit (312) can compress input data (321) using a suitable codec depending on the call data quality and network bandwidth usage. For example, if compression is performed using G.711, high data quality is maintained, but a large amount of bandwidth may be used. For example, if compression is performed using G.729, less bandwidth is used, but the data quality may be somewhat lower.

[0055] In one example, the call transmission / reception data processing unit (312) can process or handle the data to be transmitted. For example, the call transmission / reception data processing unit (312) can use a packetizer to packetize the compressed call data into a data packet. The packetizer may be referred to as a module or algorithm that performs the function of dividing call data into packet units. The data packet may include additional information (e.g., a packet header) in addition to the call data. For example, the call transmission / reception data processing unit (312) can process the call data to be suitable for the packet format defined in the RTP protocol and packetize it.

[0056] In one example, the call transmission / reception data processing unit (312) may process or handle the received data. For example, the call transmission / reception data processing unit (312) may use a depacketizer to depacketize a data packet (e.g., RTCP APP packet) transmitted by the receiving electronic device (20) to restore call data. The depacketizer may be referred to as a module or algorithm that performs the function of restoring a data packet to call data. In one example, the call transmission / reception data processing unit (312) may use a codec to decode the depacketized call data. Decoding may include data processing and / or data processing performed in a decoder (e.g., 403 in FIG. 4). For example, decoding may be referred to as converting compressed data into data in its original format for efficient transmission over a network. The call transmission / reception data processing unit (312) can restore (e.g., reverse packetization and / or decoding) the output quality information (or call quality status information) of the call data received from the receiving electronic device (20) and transmit the restored output quality information (334) of the call data to the receiving electronic device (20) call quality status representation unit (313).

[0057] The receiving side call quality status display unit (313) can receive output quality information (or, call quality status information) (334) of call data restored from the call transmission / reception data processing unit (312). The receiving side call quality status display unit (313) can process the received output quality information into information that can be recognized by the user of the transmitting side electronic device (10) and provide it to the user. For example, the receiving side call quality status display unit (313) may include a display (not shown), a speaker (not shown) (e.g., the speaker (210) of FIG. 2), and / or a vibration motor (not shown). If the received output quality information (or, call quality status information) indicates a quality lower than a specified quality, the receiving side call quality status display unit (313) can display information that the output quality is not good in the form of text, an image, and / or a UI (user interface) on the display. For example, the receiving side call quality status display unit (313) may output a notification sound indicating that the output quality is not good using a speaker when the received output quality information (or, call quality status information) indicates a quality lower than a specified quality. For example, the receiving side call quality status display unit (313) may output a vibration indicating that the output quality is not good using a vibration motor when the received output quality information (or, call quality status information) indicates a quality lower than a specified quality.

[0058] In one example, the receiving side call quality status display unit (313) can use the time information (335) of the utterance interval received from the utterance detection unit (311) to process the received output quality information (or, call quality status information) (334) into information that can be perceived by the user of the transmitting side electronic device (10) only when it corresponds to the utterance interval while indicating that the quality is below a specified quality, and provide it to the user. For example, the call quality status display unit (313) can display information that the output quality is not good in the form of text, an image, and / or a UI (user interface) on a display when the received output quality information (or, call quality status information) (334) indicates that the quality is below a specified quality and corresponds to the utterance interval. For example, the call quality status display unit (313) can output a notification sound that the output quality is not good using the speaker (210) only when the received output quality information (or, call quality status information) (334) indicates that the quality is below a specified quality and corresponds to the utterance interval. For example, the call quality status indicator (313) can output a vibration indicating that the output quality is not good using a vibration motor only when the received output quality information (or, call quality status information) (334) indicates that the quality is below a specified level and simultaneously corresponds to a firing interval.

[0059] In one example, the receiving electronic device (10) may include a call transmission / reception data processing unit (314) and an output quality status calculation unit (315).

[0060] The call transmission / reception data processing unit (314) included in the receiving electronic device (20) can perform the same function as the call transmission / reception data processing unit (312) included in the transmitting electronic device (10). For example, the call transmission / reception data processing unit (314) can packetize data using a packetizer or depacketize data packets using a depacketizer. For example, the call transmission / reception data processing unit (314) can encode call data using a codec or decode compressed call data. The transmission data (322) transmitted by the transmitting electronic device (10) can be stored in a jitter buffer (not shown) after undergoing a repacketization process in the call transmission / reception data processing unit (314).

[0061] In one example, input data (321) acquired by the transmitting electronic device (10) can be transmitted to the receiving electronic device (20) through the call transmission / reception data processing unit (312). The transmission data (322) transmitted by the transmitting electronic device (10) to the receiving electronic device (20) using the call transmission / reception data processing unit (312) may correspond to an uplink RTP packet (111) and / or a downlink RTP packet (112) when referring to FIG. 1.

[0062] The call transmission / reception data processing unit (314) can decode data temporarily stored in the jitter buffer and perform error restoration processing on data where errors occurred to generate decoded call data (323).

[0063] The decoded call data (323) may include call data generated by depacketizing and decoding the transmission data (322) transmitted by the transmitting electronic device (10) in the call transmission / reception data processing unit (314).

[0064] The decoded call data (323) may include error-recovered data generated by performing error recovery processing on the data where errors occurred during decoding. The call transmission / reception data processing unit (314) can identify errors (e.g., network degradation) that occurred before or after the transmission data (322) transmitted by the transmitting electronic device (10) is received by the receiving electronic device (20). The call transmission / reception data processing unit (314) can restore the call data by performing error recovery processing on the data where errors occurred (e.g., PLC (packet loss concealment) processing, error concealment processing and / or selective ARQ (selective automatic repeat request)). The process of restoring the call data where errors occurred through error recovery processing may be described later in relation to FIG. 4.

[0065] The decoded call data (323) may be stored in an output buffer. The decoded call data (323) stored in the output buffer is data identical to the final output form, and the decoded call data (323) stored in the output buffer may be output through a speaker (210) and / or a display (not shown). The decoded call data (323) may contain data identical to the output call data that is finally output through the speaker (210) and / or the display. The decoded call data (323) may be referred to as data identical to the pending output call data waiting to be output in the output buffer.

[0066] The call transmission / reception data processing unit (314) may transmit output processing information (324) for decoded call data to the output quality status calculation unit (315). For example, the output processing information (324) may include packet loss, packet delay, software operation error (e.g., when the processing operation of the call transmission / reception data processing unit (314) stops), and / or error recovery processing information. The output processing information (324) may be used to calculate the error rate in the output quality status calculation unit (315). The error rate may be expressed as the ratio of error-recovered data among the decoded call data (323). The error rate may be expressed as the ratio of error-recovered data among the call data scheduled for output waiting in the output buffer.

[0067] The output quality status calculation unit (315) can identify the ratio of error-recovered data among the decoded call data (323). For example, the output quality status calculation unit (315) can receive output processing information (324) from the call transmission / reception data processing unit (314) and identify whether error-recovered data exists among the decoded call data (323). Error-recovered data can be generated through error recovery processing in a decoder included in the call transmission / reception data processing unit (314). Data that has not had an error can be decoded in a decoder included in the call transmission / reception data processing unit (314). For example, the output quality status calculation unit (315) can collect the call data processed by the decoder in specified interval units (e.g., G1, G2, G3, G4, G5, G6, G7, B1, B2, and B3 of FIG. 5). The output quality status calculation unit (315) can identify error-recovered data generated by error recovery processing among the collected call data. The output quality status calculation unit (315) can calculate the error rate by identifying the ratio of error-recovered data among the collected call data. For example, the output quality status calculation unit (315) can collect call data scheduled for output waiting in the output buffer in designated interval units and calculate the error rate by identifying the ratio of error-recovered data among the collected call data.

[0068] The above error rate can be used as an indicator representing the quality of call data actually output from the receiving electronic device (20). For example, the output quality status calculation unit (315) can generate output quality information (or call quality status information) (331) indicating that the output quality (or call quality status) is not good if the calculated error rate is greater than or equal to a specified value. The process of generating output quality information (or call quality status information) (331) using output processing information (324) in the output quality status calculation unit (315) may be described later in relation to FIG. 4.

[0069] The output quality status calculation unit (315) can transmit the generated output quality information (or, call quality status information) (331) to the call transmission / reception data processing unit (314).

[0070] The call transmission / reception data processing unit (314) can packetize output quality information (or, call quality status information) (331) and transmit it to the transmitting electronic device (10). For example, the call transmission / reception data processing unit (314) can packetize the output quality information (or, call quality status information) (331) in the form of an RTCP APP packet and transmit the packetized output quality information (332) to the transmitting electronic device (10).

[0071] The call transmission / reception data processing unit (312) of the transmitting electronic device (10) receives packetized output quality information (332), restores the received data through a reverse packetization and / or decoding process, and transmits the restored decoded call data output quality information (or, call quality status information) (334) to the receiving electronic device (313). The receiving electronic device (313) can process the restored decoded call data output quality information (or, call quality status information) (334) of the receiving electronic device (20) into information (e.g., UI, notification sound, and / or vibration) that can be perceived by the user of the transmitting electronic device (10) and provide it to the user.

[0072] Embodiments of the present disclosure are not limited to those illustrated in FIG. 3. For example, the roles of the transmitting electronic device (10) and the receiving electronic device (20) may be reversed, unlike those illustrated in FIG. 3. Since a call is a two-way communication, the situation during a call may include cases where the user of the transmitting electronic device (10) is the speaker and the user of the receiving electronic device (20) is the receiver, but may also include cases where the user of the receiving electronic device (20) is the speaker and the user of the transmitting electronic device (10) is the receiver. In the case where the user of the receiving electronic device (20) is the speaker and the user of the transmitting electronic device (10) is the receiver, the roles of the transmitting electronic device (10) and the receiving electronic device (20) described in FIG. 3 may be reversed. For example, although not illustrated in FIG. 3, the transmitting electronic device (10) may include an output quality status calculation unit. For example, although not shown in FIG. 3, the receiving side electronic device (20) may include a utterance detection unit and / or a receiving side call quality status representation unit.

[0073] FIG. 4 illustrates a block diagram showing a data transfer path of a receiving electronic device and a receiving electronic device according to one embodiment.

[0074] Referring to FIGS. 1 through 4, the receiving electronic device (20) may include an RTP depacketizer (401), a jitter buffer (402), a decoder (403), a time scale modification (TSM) (404), an output buffer (405), a jitter buffer controller (JBC), and / or an output quality status calculation unit (407). Referring to FIG. 3, the output quality status calculation unit (407) may correspond to the output quality status calculation unit (315) of FIG. 3.

[0075] The RTP depacketizer (401) can receive packetized call data (421). The packetized call data (421) may include, for example, payload information (e.g., call data) and header information (e.g., timestamp, packet sequence number, and / or payload type). The RTP depacketizer (401) can depacketize the received packetized call data (421) and repacketize it into a form that can be stored in the jitter buffer (402). Depacketization can be referred to as the process of restoring packetized data to the original continuous data stream. The packetized call data (421) received by the RTP depacketizer (401) may correspond to the transmission data (322) transmitted by the transmitting electronic device (10) of FIG. 3.

[0076] The RTP depacketizer (401) can generate repacketized call data (422) by using header information (e.g., timestamp, packet sequence number, and / or payload type) and payload information (e.g., audio data) of the received packetized call data (421) to repacketize it into a structure that can be stored in a jitter buffer. Repacketization can be referred to as a process of generating a new packet with a changed format using information from the packetized call data (421). The RTP depacketizer (401) can generate the repacketized call data (422) and store it in the jitter buffer (402).

[0077] FIG. 4 illustrates a case where the receiving electronic device (20) repackets the received packet using an RTP depacketizer, but the embodiments of the present disclosure are not limited thereto. For example, when performing a video call between the transmitting electronic device (10) and the receiving electronic device (20) as described above in FIG. 1, the receiving electronic device (20) may receive not only RTP packets but also packets based on protocols for packetizing video data (e.g., RTP, RTSP (real-time streaming protocol), and / or SIP (session initiation protocol)). In this case, the receiving electronic device (20) may repacketize the received data packet using a depacketizer (not shown) corresponding to the protocol of the received packet data, depending on the type of protocol of the received packet data.

[0078] The jitter buffer (402) can temporarily store repacketized call data (422). The jitter buffer may refer to a buffer used to compensate for irregular arrival times of call data over a network. For example, the jitter buffer (402) can work in conjunction with the TSM (404) to maintain output at regular time intervals (e.g., output units of the jitter buffer (402)) to minimize interruptions in output call data due to packet loss or delay.

[0079] The size of the jitter buffer (402) can be set in time units. In one example, the size of the jitter buffer (402) can be dynamically adjusted by the jitter buffer controller (406). The jitter buffer controller (406) can dynamically adjust the size of the jitter buffer (402) through the jitter buffer control signal (436). For example, if the size of the jitter buffer (402) is too small, the jitter buffer (402) is emptied quickly, which may cause interruptions during call data output. For example, if the size of the jitter buffer (402) is too large, interruptions during call data output do not occur, but the time data packets wait within the jitter buffer (402) becomes long, causing a delay in call data output and preventing real-time output. The jitter buffer controller (406) can dynamically adjust the jitter buffer (402) so that it has an appropriate size.

[0080] The jitter buffer (402) can temporarily store repacketized call data (422) and then transmit the call data (423) before decoding to the decoder (403) at regular time intervals. The jitter buffer (402) can transmit jitter buffer status information (432) (e.g., buffer underrun, buffer overrun, and / or packet arrival interval) to the jitter buffer controller (406).

[0081] The decoder (403) can receive pre-decoding call data (423) from the jitter buffer (402). The decoder (403) can de-packetize the pre-decoding call data (423), which is re-packetized in the form of a structure, and decode the data in a compressed form. The process of de-packetizing and decoding the pre-decoding call data (423) by the decoder (403) can be referred to as the process of restoring the packetized and compressed call data to its original format. The decoded call data (424), restored to its original format, can be finally output through the speaker (210) via the TSM (404) and the output buffer (405). The decoder (403) can transmit the decoded call data (424) to the TSM (404).

[0082] The decoded call data (424) may include not only data restored by reverse packetizing and / or decoding the call data (423) prior to decoding, but also data restored through the error restoration process described below. For example, the decoded call data (424) may include restored data generated through decoding and error restoration processing.

[0083] The decoder (403) can decode the call data (423) before decoding or perform error recovery processing to generate decoded call data (424). For example, the decoder (403) can process the data by performing at least one of reverse packetization, recovery of compressed data, or error recovery processing. The decoded call data (424) may correspond to the decoded call data (323) of FIG. 3.

[0084] The decoder (403) can identify call data in which errors (e.g., degradation due to packet loss and / or degradation due to packet delay) have occurred among the received call data (423) prior to decoding and restore them through error restoration processing (e.g., PLC processing, error concealment processing, and / or optional ARQ).

[0085] For example, if a loss or delay occurs in a call data packet that is to be output in real time, the decoder (403) can generate restored data through PLC processing to ensure that the call data is played continuously without interruption. For example, the decoder (403) can perform error concealment processing by repeating previous packet call data or by predicting lost data from adjacent packet data to generate restored data. For example, if call data with an error is identified by the decoder (403), the receiving electronic device (20) can selectively request retransmission of the data with the error through the communication circuit (230). The decoder (403) can perform error restoration processing using the retransmitted data instead of the call data with the error. For example, the decoder (403) can perform error restoration processing by restoring information lost during the data compression process.

[0086] For example, among the call data (423) prior to decoding, there may be missing data due to causes such as data loss caused by network jitter, packet loss within the jitter buffer (402), and / or delay in reordering packets within the jitter buffer (402). For example, if there is missing data (e.g., packet loss) among the call data (423) prior to decoding in the jitter buffer (402) where the packet sequence numbers are not consecutive, the decoder (403) can restore the missing data through error restoration processing to regenerate the data continuously. For example, if there is insufficient data within the jitter buffer (402) and data cannot be provided to the decoder (403) for a specified time unit (e.g., packet delay), the decoder (403) can generate the missing data through error restoration processing. The jitter buffer controller (406) can dynamically adjust the size of the jitter buffer (402) so that there is no shortage of data in the jitter buffer (402), but if, even after adjusting the size of the jitter buffer (402), there is a shortage of data in the jitter buffer (402) (e.g., buffer underrun) and data cannot be provided to the decoder (403) for a specified time unit, the decoder (403) can generate and play the missing data through error recovery processing.

[0087] The decoder (403) can determine the cause of the error in the data that has undergone error restoration processing. For example, the decoder (403) can identify data in which the field values ​​of the field storing the packet sequence number among the call data (423) prior to decoding are not consecutive during the error restoration processing. In this case, the decoder (403) can generate the missing data by performing error restoration processing and determine the cause of the error in the data that has undergone error restoration processing as a first cause (e.g., packet loss). For example, the decoder (403) can determine the cause of the error as the first cause by considering it as the same as the first cause if the delay time is greater than the specified time, even if the missing data packet is received later.

[0088] For example, if the decoder (403) is unable to provide data in the unit specified to the decoder (403) due to insufficient data in the jitter buffer (402), it may generate the insufficient data by performing error restoration processing, and determine the cause of the error in the error-restored data as a second cause (e.g., packet delay). For example, if there is data among the error-restored data that does not correspond to either the first cause or the second cause, the decoder (403) may determine the cause of the error in the error-restored data as a third cause (e.g., software failure, data loss during compression or decoding, and / or data loss within the jitter buffer (402).

[0089] The decoder (403) may assign an identification number according to the determined cause of the error. For example, in the case of a first cause, an identification number 1 may be assigned, in the case of a second cause, an identification number 2 may be assigned, and in the case of a third cause, an identification number 3 may be assigned. The identification number according to the cause of the error may be included in the output quality information and transmitted to the transmitting electronic device (10).

[0090] The decoder (403) may provide error recovery processing information (434) to the jitter buffer controller (406). For example, the error recovery processing information (434) may include information regarding the decoded call data (424) generated through the decoder (403), information regarding the data that has been error-recovered among the decoded call data (424), and / or information regarding the cause of the error in the data generated through the error recovery processing. The information regarding the cause of the error included in the error recovery processing information (434) may include an identification number of the cause of the error. If there are multiple causes of the error, identification numbers of multiple causes of the error may be included. For example, if the decoder (403) determines the cause of the data error as a single cause, the error recovery processing information (434) may include a single identification number. For example, if the decoder (403) determines the cause of the data error as multiple duplicate causes, the error recovery processing information (434) may include multiple identification numbers.

[0091] The TSM (404) can adjust the playback speed of the decoded call data (424) received from the decoder (403) so that data can be output at a constant rate in a specified time unit from the jitter buffer (402). For example, the TSM (404) can adjust the playback speed of the decoded call data (424) to be faster or slower. For example, the TSM (404) can increase or decrease the playback time without altering the pitch (e.g., pitch) of the decoded call data (424) stream. The playback speed of the call data passing through the TSM (404) is changed, but the pitch or frequency characteristics can be maintained.

[0092] The TSM (404) can operate in conjunction with the jitter buffer (402). For example, if a data packet arrives at the jitter buffer (402) too late and the data in the jitter buffer becomes insufficient (e.g., buffer underrun), the TSM (404) can secure time to receive the data packet by slowing down the playback time of the decoded call data (424). The TSM (404) can prevent playback interruption by slowing down the playback time of the decoded call data (424). For example, if a data packet arrives at the jitter buffer (402) too early and the data in the jitter buffer becomes full (e.g., buffer overrun), the TSM (404) can prevent the jitter buffer overrun by fastening the playback time of the decoded call data (424).

[0093] The TSM (404) can transmit TSM status information (433) to the jitter buffer controller (406). The TSM status information (433) may include, for example, information about the playback speed of the call data (424) currently decoded in the TSM (404), the minimum playback speed playable in the TSM (404), and / or the maximum playback speed playable in the TSM (404).

[0094] The output buffer (405) receives decoded call data (424) from the TSM (404) and can output call data (425) through the speaker (210). The output call data (425) output through the output buffer (405) can correspond to the call output (102) of FIG. 1.

[0095] The jitter buffer controller (406) may include a network jitter analysis module (411), an output delay prediction module (412), and / or an adaptive jitter buffer control module (413). The jitter buffer controller (406) may dynamically adjust the size of the jitter buffer (402) and dynamically adjust the playback speed at which the call data stream is played back via the TSM (404).

[0096] The network jitter analysis module (411) can receive information associated with packetized call data (421) through the RTP depacketizer (401), e.g., information on the reception time of the RTP packet. The network jitter analysis module (411) can identify network jitter information based on the information associated with the received packetized call data (421). For example, the network jitter analysis module (411) can identify network jitter information (431) by comparing the time stamp information of the received packetized call data (421) with the time information of the packetized call data (421) received. Network jitter can be referred to as a phenomenon in which data packets transmitted through a network do not arrive on time but arrive at irregular time intervals. When network jitter occurs, data packet loss or data packet delay may occur. The network jitter analysis module (411) can transmit the network jitter information (431) to the adaptive jitter buffer control module (413).

[0097] The output delay prediction module (412) may receive jitter buffer status information (432) (e.g., buffer underrun, buffer overrun, size of jitter buffer (402), and / or packet arrival interval) from the jitter buffer (402) and receive TSM status information (433) (e.g., current playback speed, minimum playback speed, and / or maximum playback speed) from the TSM (404). Based on the jitter buffer status information (432) and / or TSM status information (433), the output delay prediction module (412) may predict the delay time taken from when the receiving electronic device (20) receives packetized call data (421) until it plays it. For example, the delay time may include waiting time within the jitter buffer (402). The output delay prediction module (412) may predict the delay time by taking into account the size of the jitter buffer (402) and the speed at which the decoder (403) processes data packets. The output delay prediction module (412) can transmit the predicted output delay prediction time information (435) to the adaptive jitter buffer control module (413).

[0098] The adaptive jitter buffer control module (413) receives network jitter information (431) and / or output delay prediction time information (435) and can adjust the size of the jitter buffer (402) based on the received information. For example, the adaptive jitter buffer control module (413) can identify that data packets received from the network arrive late based on the network jitter information (431) and can increase the size of the jitter buffer (402) to maintain sufficient data within the jitter buffer (402) until the late-arriving packets arrive. The adaptive jitter buffer control module (413) can transmit a jitter buffer control signal (436) to the jitter buffer (402) to increase the size of the jitter buffer (402).

[0099] For example, the adaptive jitter buffer control module (413) can identify whether data packets received from the network are arriving normally based on network jitter information (431) and reduce the size of the jitter buffer (402) to reduce the output delay time. The adaptive jitter buffer control module (413) can transmit a jitter buffer control signal (436) to the jitter buffer (402) to reduce the size of the jitter buffer (402). For example, the adaptive jitter buffer control module (413) can transmit a jitter buffer control signal (436) to reduce the size of the jitter buffer (402) to reduce the output delay time based on output delay prediction time information (435). When the size of the jitter buffer (402) is reduced, the time data packets wait within the jitter buffer (402) is reduced, and thus the output delay time can be reduced.

[0100] For example, the adaptive jitter buffer control module (413) can transmit a jitter buffer control signal (436) to increase the size of the jitter buffer (402) when a buffer underrun is detected. For example, the adaptive jitter buffer control module (413) can transmit a jitter buffer control signal (436) to decrease the size of the jitter buffer (402) when a buffer overrun is detected.

[0101] The adaptive jitter buffer control module (413) receives network jitter information (431) and / or output delay prediction time information (435), and can adjust the playback speed of the call data (424) decoded via the TSM (404) based on the received information. The adaptive jitter buffer control module (413) can adjust the playback speed of the decoded call data (424) within the allowed range via the TSM (404). For example, the adaptive jitter buffer control module (413) can identify that data received from the network arrives with a delay based on the network jitter information (431) and adjust the playback speed of the call data (424) decoded via the TSM (404). If the decoded call data (424) is played slowly, time can be secured until the delayed data arrives. The adaptive jitter buffer control module (413) can transmit a playback speed control signal (437) to the TSM (404) to slow down the playback speed. For example, the adaptive jitter buffer control module (413) can minimize output delay time by identifying whether data packets received from the network arrive normally based on network jitter information (431) and adjusting the playback speed of call data (424) decoded through the TSM (404). The adaptive jitter buffer control module (413) can transmit a playback speed control signal (437) to the TSM (404) to speed up the playback speed. For example, the adaptive jitter buffer control module (413) can transmit a playback speed control signal (437) to increase the playback speed when a buffer overrun is detected. When the playback speed increases, the speed at which the jitter buffer is emptied increases, thereby preventing a buffer overrun. For example, the adaptive jitter buffer control module (413) can transmit a playback speed control signal (437) to reduce the playback speed when a buffer underrun is detected.If the playback speed slows down, the speed at which data is processed in the decoder (403) is reduced, thereby preventing buffer underrun.

[0102] The adaptive jitter buffer control module (413) receives error recovery processing information (434) from the decoder (403) and can transmit output processing information (438) to the output quality status calculation unit (407). The output processing information (438) may include call data packet information recovered through the decoder (403), data packet information processed for error recovery, and / or the cause of the error in the data processed for error recovery. Referring to FIG. 3, the output processing information (438) may correspond to the output processing information (324) of FIG. 3.

[0103] The output quality status calculation unit (407) can calculate an error rate using output processing information (438) and generate output quality information. For example, the output quality status calculation unit (407) can collect decoded call data (424) processed by the decoder (403) in designated interval units (e.g., G1, G2, G3, G4, G5, G6, G7, B1, B2, and B3 of FIG. 5). The output quality status calculation unit (407) can identify data generated by error restoration processing among the collected call data as error-restored data. The output quality status calculation unit (407) can calculate an error rate by identifying the ratio of data generated through error restoration processing among the collected data. For example, if the calculated error rate is greater than or equal to a designated value, the output quality status calculation unit (407) can determine that the output quality (or call quality status) is not good. For example, the output quality status calculation unit (407) can classify the output quality status into sections according to the error rate. For example, the output quality status sections according to the error rate may include a very poor output quality section (e.g., error rate exceeding 60%), a poor output quality section (e.g., error rate exceeding 30% and 60% or less), a slightly poor output quality section (e.g., error rate exceeding 10% and 30% or less), and / or a good output quality section (e.g., error rate 10% or less).

[0104] The output quality status calculation unit (407) can generate output quality information (or, call quality status information). The output quality information (or, call quality status information) may include information regarding the error rate, whether the output quality is good, and / or the output quality status interval. Referring to FIG. 3, the output quality status calculation unit (407) may correspond to the output quality status calculation unit (315) of FIG. 3. The output quality information generated by the output quality status calculation unit (407) may be packetized in the form of an RTCP APP packet and transmitted to the transmitting electronic device (10).

[0105] FIG. 5 illustrates received data and output data according to one embodiment.

[0106] Referring to FIGS. 2, FIGS. 4 and FIGS. 5, the receiving side electronic device (20) may include a jitter buffer (501) and / or an output buffer (502). The jitter buffer (501) corresponds to the jitter buffer (402) of FIG. 4, and the output buffer (502) corresponds to the output buffer (405) of FIG. 4.

[0107] A data packet (521) received from a network is repacketized into a form suitable for storage in a jitter buffer (501), and the repacketized data (511) can be temporarily stored in the jitter buffer (501). The data packet (521) received from a network can correspond to the packetized call data (421) of FIG. 4. The process of repacketizing the data packet (521) received from a network can be referred to as described in detail in FIG. 4.

[0108] In one example, the jitter buffer (501) can sort the repacketized data (511) in order according to the packet sequence number. The jitter buffer (501) can output the sorted data at regular time intervals. For example, the regular time interval at which the jitter buffer (501) outputs data can be referenced as the output unit of the jitter buffer (501). The jitter buffer (501) can perform the role of ensuring that data is output consistently at regular time intervals even if packet loss and / or packet delay occurs due to network jitter. Among the data output by the jitter buffer (501), there may be data packets with errors. For example, data packets with errors can be processed for error restoration through a decoder. Regarding error restoration processing, the details described in FIG. 4 may be referenced.

[0109] For example, referring to FIG. 5, if there is a data packet in the output unit where the packet sequence number is not consecutive (e.g., P6 is received after P4), the receiving electronic device (20) can determine that the call data packet (531) (e.g., P5) corresponding to the non-consecutive interval is a data packet that has an error due to packet loss.

[0110] For example, referring to FIG. 5, if the receiving electronic device (20) has fewer data packets stored in the jitter buffer (501) than the output unit to be output (for example, 10 data packets are needed to output the output unit, but only data packets P1 through P8 are stored in the jitter buffer (501)), the insufficient data packets (532) can be determined to be data packets that have errors caused by packet delay.

[0111] The jitter buffer (501) can transmit data packets of output units to the decoder (403) at regular time intervals so that data packets stored in the jitter buffer (501) can be replayed at regular time intervals. The data transmitted to the decoder (403) can be restored in the form of the original continuous data stream. The restored data can be referred to as decoded call data (522). The decoded call data (522) may include data restored through reverse packetization and decoding, and / or data generated through error recovery processing. Reverse packetization, decoding, and / or error recovery processing may be referred to as described in detail in FIG. 4. The decoded call data (522) may correspond to the decoded call data (323) of FIG. 3 and / or the decoded call data (424) of FIG. 4.

[0112] The output buffer (502) may store decoded call data (522) restored through the decoder (403). For example, G1, G2, G3, G4, G5, G6, and G7 may be data restored by depacketizing and decoding P1, P2, P3, P4, P6, P7, and P8, respectively, which are data packets in the jitter buffer (501) that have not had errors. For example, B1, B2, and B3 may be data generated by performing error restoration processing on data packets (e.g., 531 and 532) that have had errors in the jitter buffer (501).

[0113] In FIG. 5, data (e.g., B2 and B3) generated by error recovery processing of a data packet that has an error due to packet delay is shown as being output after a delayed received packet (e.g., P8), but the embodiments of the present disclosure are not limited thereto. For example, data (e.g., B2 and B3) generated by error recovery processing of a data packet that has an error due to packet delay may be output before data (e.g., G7) restored by reverse packetizing and decoding the delayed received packet (e.g., P8).

[0114] The decoded call data (522) may be in the same form as the output call data (523) that is actually output through the output buffer (502). Since the error rate in the receiving electronic device (20) is calculated based on the decoded call data (522), the output quality status of the output call data (523) that is actually output can be quantified by the calculated error rate. The error rate may be calculated as the ratio of data generated by error recovery processing (e.g., B1, B2, and B3) among the decoded call data (e.g., G1, G2, G3, G4, G5, G6, G7, B1, B2, and B3) within a specified interval. For example, if 3 packets are generated by error recovery processing out of the decoded call data (522) generated by recovering a total of 10 packets, the error rate may be calculated as 30%. Since the above-described error rate is calculated based on the same data as the actual output call data (523), the error rate allows for the identification of quality information of the final output state that cannot be determined from the field information of RTCP RR and / or RTCP SR. Referring to FIG. 3, the error rate calculation can be performed in the output quality state calculation unit (315).

[0115] FIG. 6 illustrates an RTCP APP packet message format according to one embodiment.

[0116] Referring to FIG. 6 (see RFC 3550 defined by the Internet Engineering Task Force (IETF)), the RTCP APP packet may include a version field (V) (605). The version field (605) may include information indicating the version of RTP (e.g., version = 2).

[0117] As the next bit field, the RTCP APP packet may include a padding bit field (P) (610). If a padding bit is provided, it may indicate that one or two padding octets that do not belong to the payload have been added.

[0118] As the next bit field, the RTCP APP packet may include a subtype field (615). (See OMA PoC Userplane specification document), the subtype may include information indicating which TBCP (talk burst control protocol) the RTCP APP packet is performing.

[0119] As the next bit field, the RTCP APP packet may include a packet type field (PT) (620). For example, if the packet type is 204, it may include information indicating that the message is an RTCP APP packet (e.g., PT=APP=204).

[0120] As the next bit field, the RTCP APP packet may include a length field (625). The length field (625) may include length information of the entire packet including the header.

[0121] As the next bit field, the RTCP APP packet may include an SSRC / CSRC field (630). The SSRC / CSRC field (630) may include the SSRC (synchronization source) identifier information of the source that generated the packet.

[0122] As the next bit field, the RTCP APP packet may include a Name field (635). The Name field (635) may contain information about a 4-byte application name expressed in ASCII characters. The Name field (635) may be used to clarify the purpose of the packet.

[0123] As the next bit field, the RTCP APP packet may include an Application dependent data field (640). The Application dependent data field (640) may be a field in which information necessary for operating the Application can be freely defined and inserted. For example, the Application dependent data field (640) may include output quality information (or call quality status information). For example, the Application dependent data field (640) may include at least one of a timestamp, an error rate, output quality status (or call quality status), and / or error cause information.

[0124] FIG. 7 is a flowchart of the operation of a receiving electronic device for providing call quality information according to one embodiment.

[0125] The operations described below in relation to FIG. 7 may be referred to as operations of the receiving electronic device (20) of FIG. 1. The order of the operations described below in relation to FIG. 7 is an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 7 or may be executed substantially simultaneously with other operations of FIG. 7. At least some of the operations described below in relation to FIG. 7 may be omitted.

[0126] Referring to FIGS. 1, FIGS. 3, FIGS. 4, and FIGS. 7, in operation 705, the receiving electronic device (20) may receive call data based on a first protocol. For example, the first protocol may include RTP. For example, the receiving electronic device (20) may receive call data in the form of an RTP packet transmitted by the transmitting electronic device (10). The operation of the receiving electronic device (20) receiving packetized call data may be referred to as described in detail in FIGS. 3 and FIGS. 4.

[0127] In operation 710, the receiving electronic device (20) can calculate an error rate and generate output quality information based on the calculated error rate. For example, the error rate can be calculated based on decoded call data recovered from the decoder. For example, the output quality information (or call quality status information) may include information regarding the error rate, whether the output quality is good, and / or the output quality status interval. The error rate can be calculated based on data identical to the actual output call data, thereby providing information about the actual output status that cannot be known from the field information of RTCP. The operation of calculating the error rate may be referenced to the details described in FIG. 5. The operation of generating output quality information based on the calculated error rate may be referenced to the details described in FIG. 3 and FIG. 4.

[0128] In operation 715, the receiving electronic device (20) can determine whether the calculated error rate is greater than or equal to a specified value. For example, if the calculated error rate indicates greater than or equal to a specified value, the output quality state (or, call quality state) may be poor, and the content of the speaker's speech may not be transmitted to the receiver. Although FIG. 7 illustrates determining whether the error rate is greater than or equal to a specified value, the embodiments of the present disclosure are not limited thereto. For example, the receiving electronic device (20) may operate in the same manner as when the error rate indicates greater than or equal to a specified value if the output quality range (e.g., very poor output quality, poor output quality, slightly poor output quality, and / or good output quality) among the output quality information generated based on the error rate corresponds to a specified range.

[0129] If the receiving electronic device (20) determines that the error rate is greater than or equal to a specified value (e.g., operation 715-Yes), the receiving electronic device (20) may perform operation 720. If the receiving electronic device (20) determines that the error rate is not greater than or equal to a specified value (e.g., operation 715-No), the receiving electronic device (20) may terminate the operation.

[0130] In operation 720, the receiving electronic device (20) can determine whether it is a speech interval. For example, the receiving electronic device (20) can determine whether the decoded call data is included in a speech interval. For example, the receiving electronic device (20) can detect a speech interval among the received call data based on VAD using a speech detection unit (not shown) included in the receiving electronic device. The operation of the receiving electronic device (20) detecting a speech interval using a speech detection unit (not shown) can be referenced to the details described in FIG. 3.

[0131] For example, the receiving electronic device (20) can receive ignition interval information detected by the ignition detection unit (311) included in the transmitting electronic device (10) from the transmitting electronic device (10). The operation of the transmitting electronic device (10) detecting the ignition interval using the ignition detection unit (311) can be referenced from the content described in detail in FIG. 3.

[0132] If the receiving electronic device (20) determines that it is included in the utterance interval (e.g., operation 720-Yes), the receiving electronic device (20) may perform operation 725. If the receiving electronic device (20) determines that it is not included in the utterance interval (e.g., operation 720-No), the receiving electronic device (20) may terminate the operation.

[0133] In operation 725, the receiving electronic device (20) can transmit output quality information generated in operation 710 to the transmitting electronic device (10) using a first packet defined by a second protocol. The second protocol may include RTCP. The first packet defined by the second protocol may include an RTCP APP packet. Referring to FIG. 6, the output quality information transmitted by the receiving electronic device (20) may be packetized and transmitted in the form of an RTCP APP packet. The message structure of the RTCP APP packet and the information that may be included in each field may be referred to as described in detail in FIG. 6. The operation of the receiving electronic device (20) transmitting output quality information using an RTCP APP packet may be referred to as described in detail in FIG. 3.

[0134] FIG. 8 is a flowchart of the operation of a transmitting electronic device for providing call quality information according to one embodiment.

[0135] The operations described below in relation to FIG. 8 may be referred to as operations of the transmitting electronic device (10) of FIG. 1. The order of the operations described below in relation to FIG. 8 is an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 8 or may be executed substantially simultaneously with other operations of FIG. 8. At least some of the operations described below in relation to FIG. 8 may be omitted.

[0136] Referring to FIGS. 1, FIGS. 2, FIGS. 3, FIGS. 4, and FIGS. 8, in operation 805, a transmitting electronic device (10) may acquire call data using a microphone (220) and / or a camera (not shown), and may packetize the acquired call data based on a first protocol and transmit it to a receiving electronic device (20). The first protocol may include RTP. The operation of the transmitting electronic device (10) acquiring call data using a microphone (220) and / or a camera (not shown) may be referred to as described in detail in FIG. 2. The operation of the transmitting electronic device (10) packetizing the acquired call data based on RTP and transmitting it to a receiving electronic device (20) may be referred to as described in detail in FIGS. 3 and FIGS. 4.

[0137] In operation 810, the transmitting electronic device (10) can receive output quality information transmitted based on a first packet defined by a second protocol transmitted by the receiving electronic device (20). The second protocol may be referred to as RTCP. The first packet defined by the second protocol may be referred to as an RTCP APP packet. The received output quality information may be reverse-packetized and decoded in the call transmission / reception data processing unit (312). For example, the output quality information (or call quality status information) may include information regarding the error rate, whether the output quality is good, and / or the output quality status interval. The operation of the transmitting electronic device (10) receiving output quality information based on the RTCP APP may be referred to as described in detail in FIG. 3.

[0138] In operation 815, the transmitting electronic device (10) can determine whether the error rate among the received output quality information indicates an error rate greater than or equal to a specified value. Although FIG. 8 illustrates determining whether the error rate is greater than or equal to a specified value, the embodiments of the present disclosure are not limited thereto. For example, the transmitting electronic device (10) may operate in the same manner as when the error rate indicates an error rate greater than or equal to a specified value when the output quality range (e.g., very poor output quality, poor output quality, slightly poor output quality, and / or good output quality) among the received output quality information corresponds to a specified range.

[0139] If the transmitting electronic device (10) determines that the error rate is greater than or equal to a specified value (e.g., operation 815-Yes), the transmitting electronic device (10) may perform operation 820. If the transmitting electronic device (10) determines that the error rate is not greater than or equal to a specified value (e.g., operation 815-No), the transmitting electronic device (20) may terminate the operation.

[0140] In operation 820, the transmitting electronic device (10) can determine whether it is a firing section. For example, the transmitting electronic device (10) can detect a firing section based on VAD using a firing detection unit (311) and store time information of the detected firing section in memory (250). For example, if the time information included in the received output quality information is included in the time information of the firing section where it is stored, the transmitting electronic device (10) can determine that the received output quality information is for the output quality information of the firing section. The operation of the transmitting electronic device (10) detecting a firing section based on VAD using a firing detection unit (311) can be referenced to the content described in detail in FIG. 3.

[0141] If the transmitting electronic device (10) determines that it is included in the ignition interval (e.g., operation 820-Yes), the transmitting electronic device (10) may perform operation 825. If the transmitting electronic device (10) determines that it is not included in the ignition interval (e.g., operation 820-No), the transmitting electronic device (10) may terminate the operation.

[0142] In operation 825, the transmitting electronic device (10) may provide information regarding the call data output quality to the user. For example, the transmitting electronic device (10) may use the receiving electronic device (313) to process the call data output quality information (or call quality status information) into a form of information that the user of the transmitting electronic device (10) can recognize and provide it to the user. The operation of the transmitting electronic device (10) providing information regarding the call data output quality to the user may be referenced to the details described in FIG. 3. The method of the transmitting electronic device (10) providing information regarding the call data output quality to the user is not limited to the details described in FIG. 3. For example, if the received output quality information (or call quality status information) indicates a quality lower than a specified quality, the transmitting electronic device (10) may display information indicating that the output quality is not good in the form of a flash signal.

[0143] FIG. 9 is a flowchart of a method for providing call quality information according to one embodiment.

[0144] The operations described below in relation to FIG. 9 may be referred to as the operations of the transmitting electronic device (10) and the receiving electronic device (20) of FIG. 1. The order of the operations described below in relation to FIG. 9 is an example, and embodiments of the present disclosure are not limited thereto. For example, at least some of the operations may be executed differently from the order of FIG. 9 or may be executed substantially simultaneously with other operations of FIG. 9. At least some of the operations described below in relation to FIG. 9 may be omitted.

[0145] Referring to FIGS. 1 through 4, FIGS. 7, FIGS. 8, and FIGS. 9, in operation 905, the transmitting electronic device (10) and the receiving electronic device (20) can create a connection for a call. For example, the transmitting electronic device (10) and the receiving electronic device (20) can be configured to perform a call based on packet data. The operation of the transmitting electronic device (10) and the receiving electronic device (20) creating a connection for a call can be referred to as described in detail in FIG. 1.

[0146] In operation 910, the transmitting electronic device (10) can acquire call data. For example, the transmitting electronic device (10) can acquire call data from an external source (or user) using a microphone (220) and / or a camera (not shown). The operation of the transmitting electronic device (10) acquiring call data can be referred to as the one described in operation 805 with reference to FIG. 8.

[0147] In operation 915, the transmitting electronic device (10) can packetize the acquired call data into a first protocol and transmit it to the receiving electronic device (20). The first protocol may include RTP. The operation of the transmitting electronic device (10) packetizing the acquired call data into a first protocol and transmitting it to the receiving electronic device (20) can be referred to as the content described in detail in operation 805 with reference to FIG. 8.

[0148] In operation 920, the receiving electronic device (20) may store the received call data in a buffer. In FIG. 9, the buffer may be referred to as a term meaning jitter buffer. The buffer may correspond to the jitter buffer (402) described in FIG. 4. The data stored in the buffer (or jitter buffer (402)) may be data in the form of repackaged received call data. The operation of storing the received call data in the buffer (or jitter buffer (402)) may be referred to as described in FIG. 4.

[0149] In operation 925, the receiving electronic device (20) can transmit data stored in a buffer (or jitter buffer (402)) to a decoder (403) at regular time intervals. The buffer can transmit data stored in the buffer to the decoder (403) at fixed time units so that call data stored in the buffer can be played back at regular time intervals. The operation of the receiving electronic device (20) transmitting data stored in the buffer to the decoder (403) at regular time intervals can be referenced as described in detail in FIG. 4.

[0150] In operation 930, the receiving electronic device (20) can decode data stored in a buffer using a decoder (403), and if data with an error is identified during the decoding process, it can perform error recovery processing to generate decoded call data. The error recovery processing may include at least one of PLC processing, error concealment processing, and / or optional ARQ. The operation of the receiving electronic device (20) performing decoding and error recovery processing may be referred to as described in detail in FIGS. 3 and FIGS. 4. The decoded call data generated by the receiving electronic device (20) performing decoding and recovery processing may correspond to the decoded call data (323) of FIG. 3 and the decoded call data (424) of FIG. 4.

[0151] In operation 935, the receiving electronic device (20) can calculate an error rate based on decoded call data. For example, the receiving electronic device (20) can calculate an error rate by identifying the ratio of data generated through error recovery processing among the decoded call data within a specified interval. The operation of the receiving electronic device (20) calculating the error rate can be referenced as described in detail in FIG. 3, FIG. 4, and FIG. 7 (operation 710).

[0152] In operation 940, the receiving electronic device (20) may generate output quality information. For example, the output quality information (or call quality status information) may include information regarding the error rate, whether the output quality is good, and / or the output quality status interval. The operation of the receiving electronic device (20) generating output quality information may be referred to as described in detail in FIGS. 3, FIGS. 4 and FIGS. 7 (operation 710).

[0153] In operation 945, the receiving electronic device (20) can transmit the generated output quality information to the transmitting electronic device (10) using a first packet defined by the second protocol. The second protocol may include RTCP. The first packet defined by the second protocol may include an RTCP APP packet. The operation of transmitting the generated output quality information to the transmitting electronic device (10) using the first packet defined by the second protocol can be referenced as described in detail in FIG. 3 and FIG. 7 (operation 725).

[0154] In operation 950, the transmitting electronic device (10) may provide received output quality information to the user. For example, the output quality information received by the transmitting electronic device (10) may include an error rate. The transmitting electronic device (10) may provide the received output quality information to the user if the error rate received indicates a value greater than or equal to a specified value. For example, the transmitting electronic device (10) may provide the received output quality information to the user if the error rate received indicates a value greater than or equal to a specified value and the received output quality information corresponds to an utterance interval identified by the utterance detection unit (311). The operation of the transmitting electronic device (10) providing the received output quality information to the user may be referred to as described in detail in FIG. 3 and FIG. 8 (operation 825).

[0155] In operation 955, the receiving electronic device (20) can output decoded call data. Referring to FIGS. 2 and FIGS. 4, the decoded call data can be stored in an output buffer (405) and then played back through a speaker (210) and / or a display (not shown). The operation of the receiving electronic device (20) outputting the decoded call data can be referred to as described in detail in FIGS. 2 and FIGS. 4.

[0156] FIG. 10 illustrates a user interface (UI) providing call quality information according to one embodiment.

[0157] Referring to FIG. 3 and FIG. 10, the transmitting electronic device (10) may generate and display a UI (e.g., 1010, 1020, and / or 1030) that represents the output quality status (or call quality status) of the receiving electronic device (20). For example, the transmitting electronic device (10) may generate a UI that represents the output quality status (or call quality status) using output quality information (or call quality status information) received from the receiving electronic device (20) by using the receiving electronic device (313). For example, the transmitting electronic device (10) may generate a UI that represents the output quality status (or call quality status) based on the received output quality information and provide information to the user. For example, the transmitting electronic device (10) may generate a UI that represents the output quality status (or call quality status) and provide information to the user when the error rate among the received output quality information indicates a value greater than or equal to a specified value. The operation of the transmitting electronic device (10) creating and displaying a UI that provides information expressing the output quality status (or, call quality status) can be referenced as described in detail in FIG. 3 and FIG. 9 (operation 950).

[0158] Referring to FIG. 10, according to one embodiment, a transmitting electronic device (10) may generate and display a UI (1010) that represents the output quality status (or call quality status) in the form of bars. For example, as the output quality status (or call quality status) is good, the number of bars displayed in bold may increase. For example, as the output quality status (or call quality status) is poor, the number of bars displayed in light may increase. For example, the UI (1010) that represents the output quality status (or call quality status) in the form of bars may have multiple bars arranged such that their length increases from left to right, and may provide information about the output quality status (or call quality status) to the user by displaying the bars in bold in order from the shortest bar on the left to the longest bar on the right, depending on the degree of goodness of the output quality status (or call quality status).

[0159] Referring to FIG. 10, according to one embodiment, a transmitting electronic device (10) may generate and display a UI (1020) that expresses an output quality status (or, call quality status) in text form. For example, the transmitting electronic device (10) may generate and display a text message such as "It is highly likely that the other party did not hear what was said a moment ago because the reception status of the other party is poor" and / or "The other party's reception status is currently poor. Please wait a moment."

[0160] Referring to FIG. 10, according to one embodiment, a transmitting electronic device (10) may generate and display a UI (1030) that expresses an output quality state (or, call quality state) in color. For example, depending on the output quality state (or, call quality state), the transmitting electronic device (10) may generate and display a UI that has a brighter color as the output quality state (or, call quality state) is better. The UI (1030) that expresses the output quality state (or, call quality state) in color by the transmitting electronic device (10) is not limited by the content illustrated in FIG. 10. For example, depending on the output quality state (or, call quality state), the transmitting electronic device (10) may generate and display a UI that has a darker color as the output quality state (or, call quality state) is better. For example, depending on the output quality state (or, call quality state), the transmitting electronic device (10) may express the output quality state (or, call quality state) in various colors. For example, the transmitting electronic device (10) can generate and display a green UI when the output quality status (or call quality status) is good. For example, the transmitting electronic device (10) can generate and display a yellow UI when the output quality status (or call quality status) is slightly poor. For example, the transmitting electronic device (10) can generate and display an orange UI when the output quality status (or call quality status) is poor. For example, the transmitting electronic device (10) can generate and display a red UI when the output quality status (or call quality status) is very poor.

[0161] The UI (e.g., 1010, 1020, and / or 1030) representing the output quality status (or, call quality status) that the transmitting electronic device (10) can generate is not limited to that illustrated in FIG. 10. For example, the transmitting electronic device (10) may generate and simultaneously display at least one of the UIs 1010, 1020, and / or 1030. For example, the transmitting electronic device (10) may generate various UIs in addition to the UI (e.g., 1010, 1020, and / or 1030) illustrated in FIG. 10 to represent the output quality status (or, call quality status).

[0162] Although not illustrated in FIG. 10, the method by which the transmitting electronic device (10) provides information about the call status of the receiving electronic device (20) to the user of the transmitting electronic device (10) is not limited to a visual method (e.g., UI and / or text). For example, the transmitting electronic device (10) may provide information to the user of the transmitting electronic device (10) using at least one of a notification sound and / or vibration indicating that the call status of the receiving electronic device (20) is poor. Regarding the transmitting electronic device (10) providing information about the call quality of the receiving electronic device (20) through a notification sound and / or vibration, reference may be made to the details described in FIG. 3.

[0163] FIG. 11 is a block diagram of an exemplary electronic device (1100) capable of performing the operations described in this document.

[0164] Referring to FIG. 11, the electronic device (1100) may be one of various forms of electronic devices, such as a notebook (1190), smartphones (1191) having various form factors (e.g., a bar-type smartphone (1191-1), a foldable-type smartphone (1191-2), or a sliderable (or rollable)-type smartphone (1191-3)), a tablet (1192), a cellular phone (not shown), and other similar computing devices (not shown). The components, their relationships, and their functions illustrated in FIG. 11 are illustrative only and are not intended to limit the implementations described or claimed herein. The electronic device (1100) may be referred to as a mobile device, a user device, a multifunction device, a portable device, or a server.

[0165] The electronic device (1100) may include components comprising at least one processor (1110) (hereinafter referred to as processor (1110)), at least one memory (1120) (hereinafter referred to as memory (1120)), at least one display (1140) (hereinafter referred to as display (1140)), at least one image sensor (1150) (hereinafter referred to as image sensor (1150)), at least one communication circuit (1160) (hereinafter referred to as communication circuit (1160)), and / or at least one sensor (1170) (hereinafter referred to as sensor (1170)). The components are merely exemplary. For example, the electronic device (1100) may include other components (e.g., power management integrated circuitry (PMIC), audio processing circuit, antenna, rechargeable battery, or input / output interface). For example, some components may be omitted from the electronic device (1100). For example, some components can be integrated into a single component.

[0166] The processor (1110) may be implemented as one or more IC (integrated circuit (or circuitry)) chips and may perform various data processing operations. The processor (1110) may include at least one electrical circuit and may process instructions (or programs, data, etc.) stored in memory (1120) individually or collectively in a distributed manner. The processor (1110) may include a processor assembly comprising one or more processing circuits. The processor (1110) may include any processing circuit that is operative to control the performance and operation of one or more components of the electronic device (1100) (e.g., memory (1120), display (1140), image sensor (1150), communication circuit (1160), and / or sensor (1170)). For example, a processor (1110) (e.g., an application processor (AP)) may be implemented as a system on chip (SoC) (e.g., a single chip or a chipset). For example, the processor (1110) may be implemented as a plurality of cores (or at least one core circuit), a plurality of chips, or a plurality of chipsets. For example, the processor (1110) may include one or more processing circuits. For example, the processor (1110) may include one or more processing circuits configured to perform the various functions of the present disclosure individually and / or collectively. As an example without limitation, at least a portion of the processor (1110) may be included in a first chip of the electronic device (1100), and at least another portion of the processor (1110) may be included in a second chip of the electronic device (1100) different from the first chip of the electronic device (1100).

[0167] For example, the processor (1110) may include a central processing unit (1111), a graphics processing unit (1112), a neural processing unit (1113), an image signal processor (1114), a display controller (1115), a memory controller (1116), a storage controller (1117), a communication processor (1118), and / or a sensor interface (1119). These components of the processor (1110) are merely exemplary. For example, the processor (1110) may include other components. For example, some components of the processor (1110) may be omitted from the processor (1110). For example, some components of the processor (1110) may be included as separate components of the electronic device (1100) outside of the processor (1110). For example, some components of the processor (1110) (e.g., memory controller (1116)) may be included in other components (e.g., at least part of memory (1120), an interface (e.g. available for connection to at least one component of the electronic device (100)), a display (1140) and / or an image sensor (1150)).

[0168] The processor (1110) may cause other components of the electronic device (1100) to perform various operations by executing instructions stored in memory (1120). The CPU (1111) (or central processing circuit) may be configured to control the components of the processor (1110) based on the execution of instructions stored in memory (1120) (e.g., volatile memory (1121) and / or non-volatile memory (1122)). The GPU (1112) (or graphics processing circuit) may be configured to execute parallel operations (e.g., rendering). The NPU (1113) (or neural processing circuit, or AI (artificial intelligence) chip) may be configured to execute operations for an artificial intelligence model (e.g., convolution computation). An ISP (1114) (or image signal processing circuit) may be configured to process a raw image acquired through an image sensor (1150) into a format suitable for a component within an electronic device (1100) or a component of a processor (1110). A display controller (1115) (or display control circuit, or DPU (display processing unit)) may be configured to process an image acquired from a CPU (1111), GPU (1112), ISP (1114), or memory (1120) (e.g., volatile memory (1121)) into a format suitable for a display (1140). A memory controller (1116) (or memory control circuit) may be configured to control reading data from volatile memory (1121) and writing data to volatile memory (1121). The storage controller (1117) (or storage control circuit) may be configured to control reading data from non-volatile memory (1122) and writing data to non-volatile memory (1122).The CP (1118) (communication processing circuit) may be configured to process data obtained from a component of the processor (1110) into a format suitable for transmitting to another electronic device via the communication circuit (1160), or to process data obtained from another electronic device via the communication circuit (1160) into a format suitable for processing by the component of the processor (1110). For example, the communication circuit (1160) may include one or more communication circuits. The sensor interface (1119) (or sensing data processing circuit, sensor hub) may be configured to process data regarding the state of the electronic device (1100) and / or the state around the electronic device (1100), obtained through the sensor (1170), into a format suitable for the component of the processor (1110).

[0169] Memory (1120) may include one or more storage media (or one or more storage devices). For example, memory (1120) may include a memory assembly comprising one or more storage media. For example, the one or more storage media may include a hard drive, a flash memory, a permanent memory such as ROM (read-only memory) (e.g., non-volatile memory (1122)), a semi-permanent memory such as RAM (random access memory) (e.g., volatile memory (1121)), any other suitable type of storage (or storage assembly), or any combination thereof. Memory (1120) may include a cache memory, which is one or more different types of memory used to temporarily store data for a function or feature of the electronic device (1100). As an example not limited to, the cache memory may be included within the processor (1110). The memory (1120) may be fixedly embedded within the electronic device (1100) or incorporated into one or more suitable types of components (e.g., a SIM (subscriber identity module) card and / or an SD (secure digital) card) that can be repeatedly inserted into and removed from the electronic device (1100).

[0170] For example, memory (1120) may store one or more software applications, such as operating system (or system) software applications, firmware software applications, driver software applications, plugin (e.g., add-in, add-on, and / or applet) software applications, and / or any other suitable software applications. For example, the one or more software applications may include instructions executable by the processor (1110). For example, memory (1120) may store instructions that can be called by an application programming interface (API). For example, memory (1120) may store instructions within a library.

[0171] According to one embodiment of the present document, an electronic device comprises: a display; at least one speaker; at least one communication circuit; at least one processor electrically connected to the display, the at least one speaker, and the at least one communication circuit, and including a processing circuit; and a memory electrically connected to the at least one processor, wherein the memory may store one or more instructions such that, when executed individually or collectively by the at least one processor, the electronic device uses the at least one communication circuit to create a connection for a call with an external electronic device, through the connection, receive call data from the external electronic device, decode or perform error recovery processing on the received call data to generate decoded call data from the received call data, calculate an error rate of the decoded call data based on the amount of error-recovered data among the decoded call data, and provide output quality information based on the error rate to the external electronic device through the connection so as to know the output quality information of the decoded call data output from the external electronic device through the electronic device.

[0172] According to one embodiment of the present document, the received call data may include packet data packetized based on a first protocol, and the output quality information may include packet data packetized based on a first packet defined by a second protocol.

[0173] According to one embodiment of the present document, when the instructions are executed individually or in combination by the at least one processor, the electronic device may identify the error-recovered data among the decoded call data and calculate an error rate based on the ratio of the error-recovered data among the decoded call data within a specified interval.

[0174] According to one embodiment of the present document, when the instructions are executed individually or in combination by the at least one processor, the cause of the error is determined when the electronic device performs the error restoration process, and the determined cause of the error is further transmitted to the external electronic device through the connection. In determining the cause of the error, if the value of a designated field of the received call data is not continuous and the error restoration process is performed, the cause of the error is determined as a first cause; in determining the cause of the error, if the value of the designated field is continuous but the error restoration process is performed because there is insufficient call data to be output, the cause of the error is determined as a second cause; and in determining the cause of the error, if the remaining cases excluding the first cause and the second cause are not determined, the cause of the error is determined as a third cause.

[0175] According to one embodiment of the present document, when the instructions are executed individually or in combination by the at least one processor, the electronic device stores the received call data in a buffer configured to store the received call data for a specified time unit, decodes the call data stored in the buffer or performs error recovery processing to generate the decoded call data, and outputs the decoded call data through the display or at least one of the at least one speaker. When the instructions are executed individually or in combination by the at least one processor, the electronic device may adjust the size of the buffer and the speed at which the decoded call data is output based on the time information of the received call data.

[0176] According to one embodiment of the present document, when the instructions are executed individually or in combination by the at least one processor, the electronic device may additionally transmit time information associated with the output quality information through the connection.

[0177] According to one embodiment of the present document, the electronic device comprises at least one camera;

[0178] The electronic device may include at least one microphone; at least one communication circuit; at least one processor electrically connected to the at least one camera, at least one microphone, and the at least one communication circuit, and comprising a processing circuit; and a memory electrically connected to the at least one processor, wherein the memory may enable the electronic device to use the at least one communication circuit to create a connection for a call with an external electronic device when executed individually or collectively by the at least one processor, to acquire call data from at least one of the at least one camera, the at least one microphone, or a wireless electronic device capable of communicating through the at least one communication circuit, to transmit the acquired call data to the external electronic device through the connection, to receive output quality information associated with the quality when the call data is output through the external electronic device through the connection, and to provide information regarding the quality when the call data is output through the external electronic device based on the output quality information.

[0179] According to one embodiment of the present document, when the instructions are executed individually or in combination by the at least one processor, the electronic device may identify a speech interval in which voice exists and time information of said speech interval among the acquired call data through voice activity detection, and if the received output quality information corresponds to the identified speech interval and the received output quality information indicates a quality lower than a specified quality, provide information regarding said output quality.

[0180] According to one embodiment of the present document, when the instructions are executed individually or in combination by the at least one processor, the electronic device may additionally transmit time information associated with the acquired call data through the connection.

[0181] According to one embodiment of the present document, the electronic device further comprises at least one of a display, at least one speaker, or at least one vibration motor, and when the instructions are executed individually or in combination by the at least one processor, if the electronic device comprises the display, a notification visually indicating the output quality information is displayed on the display to provide information about the output quality; if the electronic device comprises the at least one speaker, a notification sound indicating that the output quality is below a specified quality is output; and if the electronic device comprises the at least one vibration motor, a vibration indicating that the output quality is below a specified quality is output.

[0182] According to one embodiment of the present document, a method for generating output quality information for an electronic device to transmit to an external electronic device may include: an operation of creating a connection for a call with the external electronic device using at least one communication circuit of the electronic device; an operation of receiving call data from the external electronic device through the connection; an operation of decoding the received call data or performing error recovery processing to generate decoded call data from the received call data; an operation of calculating an error rate of the decoded call data based on the amount of error-recovered data among the decoded call data; and an operation of providing output quality information based on the error rate to the external electronic device through the connection so that the external electronic device can know the quality information of the decoded call data output through the electronic device.

[0183] According to one embodiment of the present document, the received call data may include packet data packetized based on a first protocol, and the output quality information may include packet data packetized based on a first packet defined by a second protocol.

[0184] According to one embodiment of the present document, a method for generating output quality information for an electronic device to transmit to an external electronic device may further include: identifying error-recovered data among the decoded call data; and calculating an error rate based on the ratio of error-recovered data among the decoded call data within a specified interval.

[0185] According to one embodiment of the present document, a method for generating output quality information for an electronic device to transmit to an external electronic device may further include: an operation of determining the cause of the error when performing error restoration processing; and an operation of further transmitting the determined cause of the error to the external electronic device through the connection. The operation of determining the cause of the error may include: an operation of determining the cause of the error as a first cause when error restoration processing is performed because the value of a designated field of the received call data is not continuous; an operation of determining the cause of the error as a second cause when error restoration processing is performed because the value of the designated field is continuous but there is insufficient call data to be output; and an operation of determining the cause of the error as a third cause in the remaining cases excluding the first cause and the second cause.

[0186] According to one embodiment of the present document, the operation of receiving call data further includes the operation of storing the received call data in a buffer configured to store the received call data for a specified time unit, and the operation of decoding the received call data or performing error recovery processing to generate the decoded call data further includes the operation of outputting the decoded call data through at least one of the display of the electronic device or at least one speaker of the electronic device after generating the decoded call data, and the method of generating output quality information for the electronic device to transmit to an external electronic device may further include the operation of adjusting the size of the buffer and the speed at which the decoded call data is output through the at least one speaker based on the time information of the received call data.

[0187] According to one embodiment of the present document, a method for generating output quality information for an electronic device to transmit to an external electronic device may further include the operation of additionally transmitting time information associated with the output quality information through the connection.

[0188] According to one embodiment of the present document, a method for an electronic device to receive and provide output quality information from an external electronic device may include: an operation of creating a connection for a call with the external electronic device using at least one communication circuit; an operation of acquiring call data from at least one camera of the electronic device, at least one microphone of the electronic device, or at least one wireless electronic device capable of communicating through the at least one communication circuit; an operation of transmitting the acquired call data to the external electronic device through the connection; an operation of receiving output quality information associated with the quality when the call data is output through the external electronic device through the connection; and an operation of providing information about the quality when the call data is output through the external electronic device based on the output quality information.

[0189] According to one embodiment of the present document, the operation of providing information on the output quality may include: an operation of identifying a speech interval in which voice exists among the acquired call data through voice activity detection and time information of said speech interval; and an operation of providing information on the output quality if the received output quality information corresponds to the identified speech interval and the received output quality information indicates a quality lower than or equal to a specified quality.

[0190] According to one embodiment of the present document, a method for an electronic device to receive and provide output quality information from an external electronic device may further include the operation of additionally transmitting time information associated with the acquired call data through the connection.

[0191] According to one embodiment of the present document, the operation of providing information about the output quality may include at least one of: an operation of displaying a notification that visually indicates the output quality information on a display; an operation of outputting a notification sound indicating that the output quality is below a specified quality; or an operation of outputting a vibration indicating that the output quality is below a specified quality.

Claims

In electronic devices, display; At least one speaker; At least one communication circuit; At least one processor electrically connected to the display, the at least one speaker, and the at least one communication circuit, and including a processing circuit; and The electronic device includes a memory electrically connected to at least one processor, wherein the memory is executed individually or collectively by the at least one processor: Using the above-mentioned at least one communication circuit, a connection for a call with an external electronic device is established, and Through the above connection, call data is received from the external electronic device, and Decoding the received call data or performing error restoration processing to generate decoded call data from the received call data, and Calculate the error rate of the decoded call data based on the amount of error-recovered data among the decoded call data, and An electronic device storing one or more instructions that provide output quality information based on the error rate to the external electronic device through the connection, so as to be able to know output quality information of the decoded call data output from the external electronic device through the electronic device. In Article 1, The received call data above includes packet data that has been packetized based on the first protocol, and An electronic device comprising packet data that is packetized based on a first packet defined by a second protocol, wherein the output quality information above includes packet data. In Article 1, When the above instructions are executed individually or in combination by the at least one processor, the electronic device: Identify the error-recovered data among the decoded call data above, and An electronic device that calculates an error rate based on the ratio of the error-recovered data among the decoded call data within a specified interval. In Paragraph 3, When the above instructions are executed individually or in combination by the at least one processor, the electronic device: When performing the above error restoration process, determine the cause of the above error, and Further transmitting the cause of the determined error to the external electronic device through the above connection, and In determining the cause of the above error, if error restoration processing is performed because the value of a designated field of the received call data is not consecutive, the cause of the above error is determined as the first cause, and In determining the cause of the above error, if the value of the specified field is continuous but there is insufficient call data to output and error restoration processing is performed, the cause of the above error is determined as a second cause, and An electronic device that, in determining the cause of the above error, determines the cause of the error as a third cause in cases other than the first cause and the second cause. In Article 1, When the above instructions are executed individually or in combination by the at least one processor, the electronic device: The received call data is stored in a buffer configured to store the received call data for a specified time unit, and Decoding the call data stored in the above buffer or performing error recovery processing to generate the decoded call data, and The above decoded call data is output through the display or at least one of the at least one speaker, and When the above instructions are executed individually or in combination by the at least one processor, the electronic device: An electronic device that adjusts the size of the buffer and the speed at which the decoded call data is output based on the time information of the received call data. In Article 1 When the above instructions are executed individually or in combination by the at least one processor, the electronic device: An electronic device that further transmits time information associated with the output quality information through the above connection. In electronic devices, At least one camera; At least one microphone; At least one communication circuit; At least one processor electrically connected to the at least one camera, at least one microphone, and at least one communication circuit, and including a processing circuit; and The electronic device includes a memory electrically connected to at least one processor, wherein the memory is executed individually or collectively by the at least one processor: Using the above-mentioned at least one communication circuit, a connection for a call with an external electronic device is established, and Acquiring call data from at least one of a wireless electronic device capable of communicating through the at least one camera, the at least one microphone, or the at least one communication circuit, and Through the above connection, the acquired call data is transmitted to the external electronic device, and Through the above connection, output quality information related to the quality when the call data is output through the external electronic device is received, and An electronic device storing one or more instructions that provide information about the quality when the call data is output through the external electronic device based on the output quality information. In Article 7, When the above instructions are executed individually or in combination by the at least one processor, the electronic device: Identifying speech segments containing voice and time information of said speech segments among the acquired call data through voice activity detection, and An electronic device that provides information on the output quality when the received output quality information corresponds to the identified utterance interval and the received output quality information indicates a specified quality or lower. In Article 7, When the above instructions are executed individually or in combination by the at least one processor, the electronic device: An electronic device that, through the above connection, further transmits time information associated with the acquired call data. In Article 7, It further includes at least one of a display, at least one speaker, or at least one vibration motor, When the above instructions are executed individually or in combination by the at least one processor, the electronic device: In the case of including the above display, a notification that visually informs of the output quality information is displayed on the display to provide information about the output quality, and If the above includes at least one speaker, an alert sound indicating that the output quality is lower than a specified quality is output, and An electronic device comprising at least one vibration motor, wherein the device outputs a vibration indicating that the output quality is lower than or equal to a specified quality. A method for generating output quality information for an electronic device to transmit to an external electronic device, The operation of creating a connection for a call with the external electronic device using at least one communication circuit of the electronic device; The operation of receiving call data from the external electronic device through the above connection; An operation to decode the received call data or perform error restoration processing to generate decoded call data from the received call data; The operation of calculating the error rate of the decoded call data based on the amount of error-recovered data among the decoded call data; and A method comprising the operation of providing output quality information based on the error rate to the external electronic device through the connection, so as to be able to know the quality information of the decoded call data output through the electronic device from the external electronic device. In Article 11, The received call data above includes packet data that has been packetized based on the first protocol, and A method comprising the above output quality information including packet data packetized based on a first packet defined by a second protocol. In Article 11, An operation to identify the error-restored data among the decoded call data; and A method further comprising the operation of calculating an error rate based on the ratio of the error-recovered data among the decoded call data within a specified interval. In Article 11, The operation of receiving the above call data further includes the operation of storing the received call data in a buffer configured to store the received call data for a specified time unit. The operation of decoding the received call data or performing error restoration processing to generate the decoded call data further includes, after generating the decoded call data, an operation of outputting the decoded call data through at least one of the display of the electronic device or at least one speaker of the electronic device. A method further comprising, based on the time information of the received call data, adjusting the size of the buffer and the speed at which the decoded call data is output through the at least one speaker. In Article 11, A method further comprising the operation of additionally transmitting time information associated with the output quality information through the above connection.

Images