Image processing device and method

Abstract

A transmission device of an image processing system is disclosed. The transmission device may transmit, to a reception device, first FEC encoded data including first source data encoded at a first bitrate and first repair data used for error correction for the first source data, receive, from the reception device, information related to a reception error for the first FEC encoded data, and determine, on the basis of the information related to the reception error, whether to change the bitrate of second source data to a second bitrate that is higher than the first bitrate. The second source data can be included in second FEC encoded data transmitted after the first FEC encoded data is transmitted.

Description

Image processing device and method

[0001] The present disclosure relates to an image processing device and method.

[0002] The video processing device can adaptively adjust the video transmission rate depending on the network environment. For example, the video processing device can lower the video transmission rate in a congested network environment and increase it in a clear network environment.

[0003] When an image processing device provides video, a target bandwidth can be set to ensure the desired image quality. This target bandwidth may differ from the current video transmission rate. Therefore, the image processing device must adjust the video transmission rate to the target bandwidth. At this time, a method for adjusting the video transmission rate to the target bandwidth without causing video breakup or deterioration in image quality must be considered.

[0004] According to one embodiment of the present disclosure, a transmitting device in an image processing system includes: a communication circuit; a memory storing a program including at least one command; and at least one processor connected to the communication circuit and the memory and executing at least one command of the program stored in the memory, wherein the at least one processor is configured to: transmit, to a receiving device, first forward error correction (FEC) encoded data including first source data encoded at a first bit rate and first repair data used for error correction for the first source data; receive, from the receiving device, information related to a reception error for the first FEC encoded data; and determine, based on the information related to the reception error, whether to change a bit rate of second source data to a second bit rate higher than the first bit rate, wherein the second source data may be included in second FEC encoded data transmitted after the first FEC encoded data is transmitted.

[0005] According to one embodiment of the present disclosure, a receiving device in an image processing system includes: a communication circuit; a memory storing a program including at least one command; and at least one processor connected to the communication circuit and the memory and executing at least one command of the program stored in the memory, wherein the at least one processor is configured to: receive, from a transmitting device, first forward error correction (FEC) encoded data including first source data encoded at a first bit rate and first repair data used for error correction for the first source data, generate information related to a reception error for the first FEC encoded data, transmit the information related to the reception error for the first FEC encoded data to the transmitting device, and receive, from the transmitting device, second FEC encoded data including second source data encoded at a second bit rate and second repair data used for error correction for the second source data, wherein the second bit rate of the second source data can be determined based on the information related to the reception error.

[0006] According to one embodiment of the present disclosure, a method of a transmitting device in an image processing system includes the steps of transmitting, to a receiving device, first forward error correction (FEC) encoded data including first source data encoded at a first bit rate and first repair data used for error correction for the first source data; receiving, from the receiving device, information related to a reception error for the first FEC encoded data; and determining, based on the information related to the reception error, whether to change a bit rate of second source data to a second bit rate higher than the first bit rate, wherein the second source data may be included in second FEC encoded data transmitted after the first FEC encoded data is transmitted.

[0007] FIG. 1 illustrates an image processing system according to one embodiment of the present disclosure.

[0008] FIG. 2 illustrates a transmitting device according to one embodiment of the present disclosure.

[0009] FIG. 3 illustrates a receiving device according to one embodiment of the present disclosure.

[0010] FIG. 4A illustrates a method for a transmitting device to perform encoding and FEC encoding according to one embodiment of the present disclosure.

[0011] FIG. 4b illustrates an example of a packet structure for transmitting FEC encoded data according to one embodiment of the present disclosure.

[0012] FIG. 5 illustrates a method by which a receiving device performs FEC decoding and decoding according to one embodiment of the present disclosure.

[0013] FIG. 6A illustrates examples of images displayed based on data decoded by a receiving device according to one embodiment of the present disclosure.

[0014] FIG. 6b illustrates an example of a method by which a receiving device performs FEC decoding to restore a data packet, according to one embodiment of the present disclosure.

[0015] FIG. 7 illustrates an exemplary relationship between target bandwidth, network bandwidth, and packet transmission rate according to one embodiment of the present disclosure.

[0016] FIG. 8A illustrates an exemplary packet transmission and reception situation according to a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0017] FIG. 8b illustrates a change in the transmission rate of a data packet according to a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0018] FIG. 9 illustrates a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0019] FIG. 10 illustrates operations of a transmitting device in a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0020] FIG. 11 illustrates operations of a receiving device in a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0021] FIG. 12 illustrates a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0022] FIG. 13 illustrates operations of a transmitting device in a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0023] FIG. 14 illustrates operations of a receiving device in a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0024] FIG. 15a illustrates an exemplary packet transmission and reception situation according to a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0025] FIG. 15b illustrates an exemplary change in the transmission rate of a data packet according to a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0026] FIG. 16 is a flowchart of a procedure for setting a transmission rate adjustment mode according to one embodiment of the present disclosure.

[0027] FIG. 17 illustrates a procedure for setting a transmission rate adjustment mode according to one embodiment of the present disclosure.

[0028] FIG. 18 illustrates an example of a user interface provided in a receiving device to set a transmission rate adjustment mode according to one embodiment of the present disclosure.

[0029] FIG. 19 illustrates a method of a transmitting device according to one embodiment of the present disclosure.

[0030] FIG. 20 illustrates a method of a receiving device according to one embodiment of the present disclosure.

[0031] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings so that those skilled in the art can easily implement the present disclosure. However, the present disclosure may be implemented in various different forms and is not limited to the embodiments described herein. In connection with the description of the drawings, the same or similar reference numerals may be used for identical or similar components. Furthermore, in the drawings and related descriptions, descriptions of well-known functions and configurations may be omitted for clarity and conciseness.

[0032] At this time, it will be understood that each block of the processing flow diagrams and combinations of the flow diagrams can be performed by computer program instructions.

[0033] Additionally, each block may represent a module, segment, or portion of code that contains one or more executable instructions for performing a specific logical function(s). It should also be noted that in some alternative implementation examples, the functions described in the blocks may occur out of order. For example, two blocks depicted in succession may actually be executed substantially concurrently, or the blocks may sometimes be executed in reverse order, depending on their respective functions.

[0034] The terminology used in this disclosure is provided solely to describe specific embodiments and is not intended to limit the scope of other embodiments. The singular form "a," "an," and "the" include plural referents unless the context clearly dictates otherwise. Terms and words used in this disclosure, including technical or scientific terms, may have the same meaning as commonly understood by those skilled in the art. Terms commonly defined in dictionaries may be interpreted as having the same or similar meaning in the context of the relevant technical field. Unless otherwise defined, such terms should not be construed in an ideal or overly formal sense. Even if a term is defined in this disclosure, such term should not be construed to exclude embodiments of the present disclosure depending on a specific situation.

[0035] Before beginning the detailed description below, it may be helpful to set forth definitions of certain words and phrases used throughout this disclosure. The term "couple" and its derivatives refer to direct or indirect communication between two or more elements, regardless of whether the two or more elements are in physical contact with each other. The terms "transmit," "receive," and "communicate," and their derivatives, encompass both direct and indirect communication. The terms "include" and "comprise," and their derivatives, mean inclusion without limitation. The term "or" is an inclusive term meaning "and / or." The phrase "associated" and its derivatives include, be included within, interconnect with, contain, be contained within, connect to or with, couple to or with, be communicable with, cooperate with, interleave, juxtapose, be proximate to, bound to or with, have, have a property of, have a relationship to or with, etc. The term "controller" means any device, system, or part thereof that controls at least one operation. The functionality associated with any particular controller may be centralized or distributed, locally or remotely.When used with a list of items, the phrase "at least one of" means that various combinations of one or more of the listed items may be used, and only one item from the list may be required. For example, "at least one of A, B, and C" includes A, B, C, A and B, A and C, B and C, A, B, and C, and all combinations thereof. As a further example, the phrase "at least one of a, b, or c" can refer to a only, b only, c only, both a and b, both a and c, both b and c, all a, b, and c, and variations thereof. Similarly, the term "set" means one or more. Thus, a set of items may be a single item or a collection of two or more items. The term "glitch" used in connection with images can mean "defect," "flaw," "fault," "error," etc.

[0036] Here, the term '~ part' used in this embodiment means software or hardware components such as FPGA or ASIC, and the '~ part' performs certain roles. However, the '~ part' is not limited to software or hardware. The '~ part' may be configured to be on an addressable storage medium and may be configured to reproduce one or more packet processing devices. Therefore, as an example, the '~ part' includes components such as software components, object-oriented software components, class components, and task components, processes, functions, properties, procedures, subroutines, segments of program code, drivers, firmware, microcode, circuits, data, databases, data structures, tables, arrays, and variables. The functions provided within the components and '~ parts' may be combined into a smaller number of components and '~ parts' or further separated into additional components and '~ parts'. In addition, the components and '~parts' may be implemented to play one or more central processing units (CPUs) within the device or secure multimedia card. In addition, in the embodiment, the '~parts' may include one or more packet processing devices.

[0037] FIG. 1 schematically illustrates an image processing system according to one embodiment of the present disclosure.

[0038] Referring to FIG. 1, the image processing system (100) may include a transmitting device (110) and a receiving device (120). In the present disclosure, the transmitting device (110) may be referred to as an image transmitting device or an image providing device, and the receiving device (120) may be referred to as an image receiving device or an image reproducing device.

[0039] According to one embodiment, the transmitting device (110) can transmit media data (e.g., video and images) to the receiving device (120). For example, the transmitting device (110) can transmit a video signal including the media data to the receiving device (120) via a network, and the receiving device (120) can receive and play the video signal from the receiving device (110). The transmitting device (110) can encode the video signal by compressing the video signal by removing redundant information in the video signal to a level that is not visually detectable. The transmitting device (110) can efficiently store, transmit, and manage the video signal by encoding the video signal within a set compression ratio range. The video signal may include, but is not limited to, real-time encoded video including, for example, screen mirroring video, video conferencing video, and game video.

[0040] According to one embodiment, a transmitting device (110) and a receiving device (120) may exchange signaling information (or signaling messages) for delivery and / or consumption of media data (or video signals including media data). As an example, the signaling messages may include FEC-related information.

[0041] According to one embodiment, the transmitting device (110) may include various video source devices such as a television (TV), a personal computer (PC), a smartphone, a tablet, a set-top box, a game console, a server, etc., and the receiving device (120) may include various video playback devices such as a TV, a smartphone, a tablet, a PC, etc. The transmitting device (110) and the receiving device (120) are not limited to a specific type of device.

[0042] According to one embodiment, the transmitting device (110) and the receiving device (120) can transmit and receive video signals through a network. According to one or more embodiments, the network (130) connecting the transmitting device (110) and the receiving device (120) can include a short-range communication network such as Wi-Fi (wireless fidelity), a long-range communication network such as a cellular network, a next-generation communication network, the Internet, or a computer network (e.g., a local area network (LAN) or a wide area network (WAN)), and can communicate based on an IP communication protocol. The cellular network can include GSM (global system for mobile communications), EDGE (enhanced data GSM environment), CDMA (code division multiple access), TDMA (time division multiplexing access), 5G, LTE (long term evolution), and LTE-A (LTE advance). The network can include connections of network elements such as hubs, bridges, routers, switches, and gateways. A network may include one or more connected networks, such as a multi-network environment, including public networks like the Internet and private networks like corporate private networks. Access to the network may be provided through one or more wired or wireless access networks. The network may support an Internet of Things (IoT) network, which exchanges and processes information between distributed components, such as objects.

[0043] FIG. 2 is a schematic block diagram of a transmitting device according to one embodiment of the present disclosure.

[0044] Referring to FIG. 2, the transmitting device (110) may include a display (111), a memory (112), a user interface (113), an input / output interface (114), a control unit (115), a sensor (116), a communication circuit (117), and / or a power supply (118). In some embodiments, the transmitting device (110) may omit at least one of these components (e.g., the display (111)), or may have one or more other components added. In some embodiments, some of these components may be integrated into one component. In the present disclosure, the control unit (115) may be defined as a circuit or an application-specific integrated circuit, or at least one processor.

[0045] In one embodiment, the display (111) can visually provide information to an external device (e.g., a user) of the transmitting device (110). The display (111) may include, for example, a display, a holographic device, or a projector and a control circuit for controlling the device. In one embodiment, the display (111) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the intensity of a force generated by the touch.

[0046] According to one embodiment, the memory (112) can store various data used by at least one component (e.g., the control unit (115) or the sensor (116)) of the transmitting device (110). The data can include, for example, input data or output data for software (e.g., a program) and commands related thereto. The memory (112) can include volatile memory or non-volatile memory.

[0047] In one embodiment, the user interface (113) may provide an interface for facilitating interaction between a user and the transmitting device (110). The user interface (113) may, for example, provide a collection of methods, means, and tools for a user to use in operating the transmitting device (110) and obtaining information. The user interface (113) may include, but is not limited to, a graphical user interface, a text-based user interface, a touch interface, and a voice recognition interface.

[0048] According to one embodiment, the input / output interface (114) may receive input signals (e.g., commands or data) to be used by a component of the transmitter (110) (e.g., a control unit (115)) from an external source (e.g., a user) of the transmitter (110), or output output signals to the external source. The input / output interface (114) may include, for example, a microphone, a speaker, audio, a mouse, a keyboard, keys (e.g., buttons), or a digital pen (e.g., a stylus pen).

[0049] According to one embodiment, the control unit (115) can control the overall operation of the transmitting device (110). The control unit (115) can, for example, execute software (e.g., a program) to control at least one other component (e.g., a hardware or software component) of the transmitting device (110) connected to the processor, and can perform various data processing or calculations. According to one embodiment, as at least a part of the data processing or calculation, the control unit (115) can store a command or data received from another component (e.g., a sensor (116) or a communication circuit (117)) in the memory (112), process the command or data stored in the memory (112), and store result data in the memory (112).

[0050] According to one embodiment, the control unit (115) may include circuitry such as a CPU, a GPU (graphics processing unit), an NPU (neural processing unit), an ISP (image signal processor), a display controller, a memory controller, an MPU (microprocessor unit), an AP (application processor), a coprocessor, a SoC (system-on-chip), an IC (integrated circuit), and / or a CP (communication processor), or may refer to at least one processor corresponding to the circuitry.

[0051] The above sub-components are merely exemplary. For example, the control unit (115) may further include other sub-components. For example, some sub-components may be omitted from the control unit (115).

[0052] According to one embodiment, the sensor (116) can detect the operating status (e.g., power or temperature) of the transmitting device (110) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. The sensor (116) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0053] According to one embodiment, the communication circuit (117) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the transmitting device (110) and an external electronic device (e.g., the receiving device (120), another electronic device, or a server), and the performance of communication through the established communication channel. The communication circuit (117) may operate independently from the control unit (115) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication circuit (117) may include a communication circuit (117) (e.g., a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a LAN communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device via a first network (e.g., a short-range communication network such as Bluetooth, Wi-Fi direct, or IrDA (infrared data association)) or a second network (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a local area network or a wide area network)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips).

[0054] In one embodiment, the power supply (118) can supply power to the transmitting device (110) and manage the power supplied to the transmitting device (110).

[0055] FIG. 3 is a schematic block diagram of a receiving device according to one embodiment of the present disclosure.

[0056] Referring to FIG. 3, the receiving device (120) may include a display (121), a memory (122), a user interface (123), an input / output interface (124), a control unit (125), a sensor (126), a communication circuit (127), and / or a power supply (128). In some embodiments, the transmitting device (110) may omit at least one of these components (e.g., the sensor (126)), or may have one or more other components added. In some embodiments, some of these components may be integrated into a single component. In the present disclosure, the control unit (125) may be defined as a circuit or an application-specific integrated circuit, or at least one processor.

[0057] In one embodiment, the display (121) can visually present information to an external device (e.g., a user) of the receiving device (120). The display (121) may include, for example, a display, a holographic device, or a projector and control circuitry for controlling the device. In one embodiment, the display (121) may include a touch sensor configured to detect a touch, or a pressure sensor configured to measure the strength of a force generated by the touch.

[0058] According to one embodiment, the memory (122) can store various data used by at least one component (e.g., the control unit (125) or the sensor (126)) of the receiving device (120). The data can include, for example, input data or output data for software (e.g., a program) and commands related thereto. The memory (122) can include volatile memory or non-volatile memory.

[0059] In one embodiment, the user interface (123) may provide an interface for facilitating interaction between a user and the receiving device (120). The user interface (123) may, for example, provide a collection of methods, means, and tools for a user to use in operating the receiving device (120) and obtaining information. The user interface (123) may include, but is not limited to, a graphical user interface, a text-based user interface, a touch interface, and a voice recognition interface.

[0060] According to one embodiment, the input / output interface (124) may receive input signals (e.g., commands or data) to be used by a component (e.g., a control unit (125)) of the receiving device (120) from an external source (e.g., a user) of the receiving device (120), or output output signals to the external source. The input / output interface (124) may include, for example, a microphone, a speaker, audio, a mouse, a keyboard, keys (e.g., buttons), or a digital pen (e.g., a stylus pen).

[0061] According to one embodiment, the control unit (125) can control the overall operation of the receiving device (120). The control unit (125) can, for example, execute software (e.g., a program) to control at least one other component (e.g., a hardware or software component) of the receiving device (120) connected to the processor, and can perform various data processing or operations. According to one embodiment, as at least a part of the data processing or operations, the control unit (125) can store commands or data received from other components (e.g., a sensor (126) or a communication circuit (127)) in the memory (122), process the commands or data stored in the memory (122), and store result data in the memory (122).

[0062] According to one embodiment, the control unit (125) may include subcomponents (e.g., processors) including, for example, a CPU, a GPU, an NPU, an ISP, a display controller, a memory controller, and / or a CP. The above subcomponents are merely exemplary. For example, the control unit (125) may further include other subcomponents. For example, some subcomponents may be omitted from the control unit (125).

[0063] According to one embodiment, the sensor (126) can detect the operating status (e.g., power or temperature) of the receiving device (120) or the external environmental status (e.g., user status) and generate an electrical signal or data value corresponding to the detected status. The sensor (126) can include, for example, a gesture sensor, a gyro sensor, a barometric pressure sensor, a magnetic sensor, an acceleration sensor, a grip sensor, a proximity sensor, a color sensor, an IR (infrared) sensor, a biometric sensor, a temperature sensor, a humidity sensor, or an illuminance sensor.

[0064] According to one embodiment, the communication circuit (127) may support the establishment of a direct (e.g., wired) communication channel or a wireless communication channel between the receiving device (120) and an external electronic device (e.g., the transmitting device (110), another electronic device, or a server), and the performance of communication through the established communication channel. The communication circuit (127) may operate independently from the control unit (125) and may include one or more communication processors that support direct (e.g., wired) communication or wireless communication. According to one embodiment, the communication circuit (127) may include a communication circuit (127) (e.g., a cellular communication module, a short-range communication module, or a global navigation satellite system (GNSS) communication module) or a wired communication module (e.g., a LAN communication module, or a power line communication module). Any of these communication modules may communicate with an external electronic device via a first network (e.g., a short-range communication network such as Bluetooth, Wi-Fi direct, or IrDA) or a second network (e.g., a long-range communication network such as a legacy cellular network, a 5G network, a next-generation communication network, the Internet, or a computer network (e.g., a LAN or WAN)). These various types of communication modules may be integrated into a single component (e.g., a single chip) or implemented as multiple separate components (e.g., multiple chips).

[0065] In one embodiment, the power supply (128) can supply power to the receiving device (120) and manage the power supplied to the receiving device (120).

[0066] FIG. 4A schematically illustrates a method for a transmitting device to perform encoding and FEC encoding according to one embodiment of the present disclosure.

[0067] According to one embodiment, encoding and / or FEC encoding may be performed by a transmitting device (e.g., transmitting device (110) of FIGS. 1 and 2) or a control unit of the transmitting device (e.g., control unit (115) of FIG. 2).

[0068] Referring to FIG. 4A, the transmitting device can perform an encoding operation (410) and an FEC encoding operation (420). According to one embodiment, the transmitting device can perform the FEC encoding operation (420) after performing the encoding operation (410).

[0069] According to one embodiment, in the encoding operation (410), the transmitting device may encode input data using encoding configuration information to generate encoded data. The encoding configuration information may include, but is not limited to, information about an encoding / decoding scheme (e.g., codec information) and information about a bitrate used to encode media data (e.g., 3.8 megabits / second for 720p video quality, 6.8 megabits / second for 1080p video quality). As the bitrate increases, the quality of the video may improve. The encoding / decoding scheme may include, but is not limited to, advanced video coding (AVC) / H.264, high efficiency video coding (HEVC) / H.265, versatile video coding (VVC) / H.266, for example. The input data may include, but is not limited to, media data (e.g., video data, image data, frame data). Encoded data generated through the encoding operation (410) can be used as source data protected by FEC encoding in the FEC encoding operation (420). In the present disclosure, the encoded data may be referred to as source data. In the present disclosure, the bit rate used to encode media data may also be referred to as an encoding bit rate.

[0070] According to one embodiment, in the FEC encoding operation (420), the transmitting device may FEC encode encoded data using FEC encoding configuration information to generate FEC encoded data or repair data. In the present disclosure, FEC encoded data may be referred to as FEC encoded data. The FEC encoding configuration information may include, for example, information on an FEC scheme, information on an FEC rate, and information on an FEC code rate, but is not limited thereto. The FEC scheme may be, for example, a Reed Solomon code scheme, an LDPC (low density parity check) scheme, or a polar code scheme, but is not limited thereto. The FEC code rate is, for example, information used to FEC encode encoded data to generate repair data, and may correspond to a value obtained by dividing the size of the source data by the size of the FEC encoded data (e.g., source data (K) / FEC encoded data (N)). The FEC encoding data generated through the FEC encoding operation (420) may include source data and restoration data for forward error correction of the source data. In the present disclosure, the restoration data may be referred to as parity data.

[0071] According to one embodiment, the FEC encoding operation (420) may be an operation performed in the application layer. The protocol for processing in the application layer may be, but is not limited to, the MPEG-2 TS (transport stream) protocol, the MPEG media transport (MMT) protocol, the DASH (dynamic adaptive streaming over HTTP) protocol, or the ROUTE (real-rime object delivery over unidirectional transport) protocol.

[0072] According to one embodiment, the encoding operation (410) and the FEC encoding operation (420) may be performed on a frame (image frame) basis. In this case, the encoding setting information and / or the FEC setting information may be set for each frame. For example, the transmitting device may individually set the encoding setting information and the FEC setting information for each frame. Through this, the transmitting device may adaptively adjust the bit rate of the source data (or media data) on a frame-by-frame basis according to the network congestion situation.

[0073] In one embodiment, the bit rate of the source data may be the number of bits of the source data transmitted over a given period of time. For example, if the source data is encoded at a bit rate of 6 megabits per second, 6 megabits of source data may be transmitted per second.

[0074] In one embodiment, the bit rate of the restored data may be the number of bits of restored data transmitted over a given period of time. For example, if the restored data is FEC-encoded restored data at a bit rate of 1 megabit / second, source data may be transmitted at a rate of 1 megabit per second. In the present disclosure, the bit rate of the restored data may be referred to as the FEC rate.

[0075] According to one embodiment, the transmitting device can obtain first FEC encoded data including the first source data and the first restored data by encoding data of a first frame at a first bit rate to generate first source data (i.e., generating first source data encoded at the first bit rate), and FEC encoding the first source data at a first FEC code rate (or, the first FEC rate) to generate first restored data. According to one embodiment, the transmitting device can obtain second FEC encoded data including the first source data and the first restored data by encoding data of a second frame following the first frame at a second bit rate to generate second source data (i.e., generating second source data encoded at the second bit rate), and FEC encoding the second source data at a second FEC code rate (or, the second FEC rate) to generate second restored data. According to one embodiment, the first bit rate for the first frame may be the same as or different from the second bit rate for the second frame. In one embodiment, the first FEC code rate for the first frame may be the same as or different from the first FEC code rate for the second frame. For example, the bit rate and code rate may be set independently for each frame.

[0076] Meanwhile, depending on the embodiment, the encoding operation (410) and the FEC encoding operation (420) may be performed in units of groups of frames, or in units of blocks or groups of blocks included in a frame.

[0077] According to one embodiment, the transmitting device can transmit FEC encoded data via at least one packet. For example, the transmitting device can transmit source data via at least one data packet and transmit restoration data via at least one FEC packet. In the present disclosure, a data packet may be referred to as an FEC data packet, a source packet, or an FEC source packet, an FEC packet may be referred to as a restoration packet, a parity packet, an FEC restoration packet, or an FEC parity packet, and a packet group including data packet(s) and associated FEC packet(s) may be referred to as an FEC packet group, a transport packet group, or an FEC encoded packet group.

[0078] According to one embodiment, the transmitting device can transmit at least one data packet belonging to an FEC packet group based on a transmission rate of the data packet, and can transmit at least one FEC packet belonging to the FEC packet group based on a transmission rate of the FEC packet. In the present disclosure, the transmission rate of the data packet may be referred to as a data transmission rate, a data packet transmission rate, or a data packet transmission rate. In the present disclosure, the transmission rate of the FEC packet may be referred to as an FEC transmission rate, an FEC packet transmission rate, or an FEC packet transmission rate.

[0079] In one embodiment, the transmission rate of the data packets may be, for example, the number of data packets transmitted over a given period of time. For example, if encoding and FEC encoding are performed on a frame-by-frame basis, the transmission rate of the data packets for the corresponding frame may be the number of data packets transmitted over a specified period of time (e.g., 1 / 60 of a second when the frame rate is 60 fps) while the corresponding frame is being viewed (or displayed).

[0080] In one embodiment, the transmission rate of a data packet may be determined based on the bit rate of the source data. For example, as the bit rate of the source data increases, the transmission rate of the data packet may increase (proportional relationship). In the present disclosure, the transmission rate of the data packet and the bit rate of the source data transmitted through the data packet(s) may be used interchangeably, and the description of an embodiment based on the transmission rate of the data packet may be applied substantially identically or similarly to the description of an embodiment based on the bit rate of the source data. In one embodiment, the transmission rate of an FEC packet may be, for example, the number of FEC packets transmitted during a certain period of time. For example, when encoding and FEC encoding are performed on a frame-by-frame basis, the transmission rate of the FEC packet for the corresponding frame may be the number of FEC packets transmitted during a specified period of time (e.g., 1 / 60 second when the frame rate is 60 fps) during which the corresponding frame is shown (or displayed).

[0081] According to one embodiment, the transmission rate of the FEC packet may be determined based on the FEC rate (bit rate) or the FEC code rate of the restored data. For example, as the FEC rate increases, the transmission rate of the FEC packet may increase (proportional relationship). For example, as the FEC code rate increases, the transmission rate of the FEC packet may decrease (inversely proportional relationship). In the present disclosure, the transmission rate of the FEC packet and the FEC rate of the restored data transmitted via the FEC packet(s) may be used interchangeably, and the description of the embodiment based on the transmission rate of the FEC packet may be substantially identically or similarly applied to the description of the embodiment based on the FEC rate of the restored data.

[0082] In one or more embodiments of the present disclosure, the transmission rate of the data packets and the transmission rate of the FEC packets may be, but are not limited to, the number of data packets and the number of FEC packets transmitted during the same period of time (e.g., during the time that the frame is displayed).

[0083] FIG. 4b illustrates an example of a packet structure for transmitting FEC encoded data according to one embodiment of the present disclosure.

[0084] According to one embodiment, a transmitting device (e.g., transmitting device (110) of FIGS. 1 and 2) may perform application layer processing (e.g., packetization processing) to generate at least one data packet (401a) containing source data and at least one FEC packet (401b) containing restoration data.

[0085] In one embodiment, packetization processing of the application layer may be performed after, but is not limited to, an FEC encoding operation (e.g., the FEC encoding operation (420) of FIG. 4A). For example, packetization processing of source data may be performed before, or in parallel with, the FEC encoding operation.

[0086] In one embodiment, a data packet (401a) may include a header (e.g., header #1) and a payload including at least a portion of the source data. For example, the payload of the data packet (401a) may include a portion of the source data for a single frame. In this case, the entire source data for a single frame may be transmitted via multiple data packets (401a).

[0087] In one embodiment, the FEC packet (401b) may include a header (e.g., header #1) and a payload including at least a portion of the restoration data. For example, the payload of the FEC packet (401b) may include a portion of the restoration data for one frame. In this case, the entire restoration data for one frame may be transmitted through multiple FEC packets (401b).

[0088] According to one embodiment, the header of the data packet (401a) and the header of the FEC packet (401b) may include packet sequence information and / or FEC related information.

[0089] In one embodiment, FEC related information may be generated on a per-frame basis.

[0090] In one embodiment, the packet sequence number information may indicate the sequence number of the corresponding packet. The receiving device can verify the packet sequence based on the packet sequence number information. Through this, the receiving device can identify which packet was dropped, which packet number it is, etc.

[0091] According to one embodiment, the FEC-related information may include, but is not limited to, FEC type information, FEC packet group identification information, total packet count information of an FEC packet group, data packet count information, and / or FEC packet count information.

[0092] In one embodiment, the FEC type information may indicate whether the packet is a data packet or an FEC packet. Based on the FEC type information, the receiving device can identify the FEC type of the packet.

[0093] According to one embodiment, the FEC packet group identification information may include information for identifying the FEC packet group to which the packet belongs. Based on the FEC packet group identification information, the receiving device may identify the FEC packet group to which the packet belongs. The FEC packet group may include data packets and FEC packets associated with one FEC encoded data. The FEC packets belonging to the FEC packet group may be used for forward error correction for data packets belonging to the same FEC packet group. For example, a separate FEC packet group may be generated for each frame.

[0094] According to one embodiment, the total packet count information of the FEC packet group may indicate the total number of packets belonging to the corresponding FEC packet group. For example, the total packet count information may be the sum of the total number of data packets belonging to the corresponding FEC packet group and the total number of FEC packets. Based on the total packet count information of the FEC packet group, the receiving device can determine how many packets within the corresponding FEC packet group were dropped (or lost) during transmission. For example, if the total number of packets indicated by the total packet count information of the FEC packet group is 10, and the total number of data packets and FEC packets counted by the receiving device is 9, the receiving device can determine that one packet was dropped during transmission. The receiving device can identify the sequence of the dropped packet, for example, based on packet sequence number information.

[0095] In one embodiment, the data packet count information may indicate the number of data packets belonging to the corresponding FEC packet group. Based on the data packet count information, the receiving device can determine how many data packets within the corresponding FEC packet group were dropped during transmission. The receiving device can identify the sequence of dropped data packets, for example, based on packet sequence number information.

[0096] According to one embodiment, the FEC packet count information may indicate the number of FEC packets belonging to a corresponding FEC packet group. Based on the FEC packet count information, the receiving device can determine how many FEC packets within the corresponding FEC packet group were dropped during transmission. The receiving device can identify the sequence of dropped FEC packets, for example, based on packet sequence number information.

[0097] In one embodiment, the transmitting device may transmit at least a portion of the FEC-related information (e.g., total packet count information, data packet count information, and / or FEC packet count information) to the receiving device via a separate signaling message. For example, the transmitting device may transmit a signaling message including at least a portion of the FEC-related information to the receiving device while establishing a wireless connection with the receiving device.

[0098] According to one embodiment, the size of each data packet (401a) may be variable or fixed. According to one embodiment, the size of each FEC packet (401b) may be variable or fixed.

[0099] According to one embodiment, the transmitting device may perform transport layer processing to generate at least one transport layer packet (402a) including data of a data packet (401a) and at least one transport layer packet (402b) including data of an FEC packet (401b).

[0100] According to one embodiment, a transport layer packet (402a, 402b) may include a header (e.g., header #2) and a payload including at least a portion of data of a related application layer packet (401a, 402b). For example, as illustrated in FIG. 4b, one transport layer packet (402a, 402b) may include data of one related application layer packet (401a, 401b), but is not limited thereto. For example, data of one application layer packet (401a, 401b) may be included in multiple transport layer packets (402a, 402b) (segmentation case), or data of multiple application layer packets (401a, 401b) may be included in one transport layer packet (402a, 402b) (concatenation case).

[0101] According to one embodiment, the header of the transport layer packet (402a, 402b) may include packet sequence information and / or information related to transport layer processing. The protocol for transport layer processing may be, for example, but is not limited to, the transmission control protocol (TCP) protocol or the user datagram protocol (UDP) protocol.

[0102] According to one embodiment, the transmitting device may perform network layer processing to generate at least one network layer packet (403a, 403b) containing data of a transport layer packet (402a, 402b).

[0103] According to one embodiment, a network layer packet (403a, 403b) may include a header (e.g., header #3) and a payload including at least a portion of data of a related transport layer packet (402a, 403b). As an example, as illustrated in FIG. 4b, one network layer packet (403a, 403b) may include data of one related transport layer packet (402a, 402b), but is not limited thereto. For example, data of one transport layer packet (402a, 402b) may be included in multiple network layer packets (403a, 403b) (segmentation case), or data of multiple transport layer packets (402a, 402b) may be included in one network layer packet (403a, 403b) (concatenation case).

[0104] According to one embodiment, the header of a network layer packet (403a, 403b) may include packet sequence information and / or information related to network layer processing. The protocol for network layer processing may be, for example, the Internet Protocol (IP) protocol, but is not limited thereto.

[0105] According to one embodiment, the transmitting device can process at least one generated network layer packet (403a, 403b) at the physical layer and transmit it to the receiving device.

[0106] In one embodiment, the receiving device can perform the reverse operation of the transmitting device (e.g., in reverse order). For example, the receiving device can obtain at least one network layer packet (403a, 403b) through physical layer processing, at least one transport layer packet (402a, 402b) through network layer processing, and at least one application layer packet (e.g., data packet / FEC packet) (401a, 401b) through transport layer processing.

[0107] FIG. 5 illustrates a method by which a receiving device performs FEC decoding and decoding according to an embodiment of the present disclosure. FIG. 6a illustrates examples of images displayed based on data decoded by a receiving device according to an embodiment of the present disclosure. FIG. 6b illustrates an example of a method by which a receiving device performs FEC decoding to restore a data packet according to an embodiment of the present disclosure.

[0108] According to one embodiment, decoding and / or FEC decoding may be performed by a receiving device (e.g., receiving device (120) of FIGS. 1 and 3) or a control unit of the receiving device (e.g., control unit (125) of FIG. 3).

[0109] According to one embodiment, a receiving device can receive FEC encoded data (or a group of FEC packets). For example, the receiving device can receive at least one data packet and at least one FEC packet belonging to an FEC packet group. For example, the receiving device can receive at least one data packet transmitted (or processed) based on a transmission rate of a data packet, and can receive at least one FEC packet transmitted (or processed) based on a transmission rate of an FEC packet. For a description of the transmission rate of the data packet and the transmission rate of the FEC packet, refer to the description of FIG. 4A. Therefore, a duplicate description is omitted.

[0110] In one embodiment, the FEC decoding and decoding operations of the receiving device may be the reverse operations (e.g., in reverse order) of the encoding and FEC encoding operations of the transmitting device.

[0111] Referring to FIG. 5, the receiving device can perform an FEC decoding operation (510) and a decoding operation (520). According to one embodiment, the receiving device can perform a decoding operation (520) after performing the FEC decoding operation (510).

[0112] According to one embodiment, the FEC decoding operation (510) may be an operation performed in the application layer. According to one embodiment, the receiving device may perform packet decapsulation processing in the application layer before the FEC decoding operation (510) is performed, or after the FEC decoding operation (510) is performed and before the decoding operation (520) is performed.

[0113] According to one embodiment, in the FEC decoding operation (520), the receiving device may perform FEC decoding on the received data using FEC decoding configuration information to generate FEC decoded data. For example, the receiving device may restore source data dropped during transmission through forward error correction using the received restored data.

[0114] According to one embodiment, the receiving device can recover at least one data packet dropped during transmission through forward error correction using at least one received FEC packet.

[0115] Hereinafter, with reference to FIG. 6b, a process for restoring a data packet dropped during transmission using an FEC packet will be described as an example. In the examples of the present disclosure, one FEC packet is used to restore one data packet, but the embodiments are not limited thereto. For example, depending on FEC settings (e.g., FEC code rate, FEC restoration rate, and / or packet size, etc.), multiple FEC packets may be used to restore one data packet, or one FEC packet may be used to restore multiple data packets.

[0116] The packet group illustrated in (a) to (e) of FIG. 6b may be, for example, a packet group for one frame among frames having a frame rate of 60 fps, but is not limited thereto.

[0117] For example, as illustrated in parts (a) to (c) of FIG. 6b, the transmitting device can transmit 10 data packets based on a first data transmission rate (e.g., a transmission rate that transmits 10 data packets in a given period of time (e.g., per 1 / 60 second)) and transmit 2 FEC packets based on a first FEC transmission rate (e.g., a transmission rate that transmits 2 FEC packets in a given period of time (e.g., per 1 / 60 second)). In this case, for example, when two packets are dropped during the transmission process (e.g., when data packet 1 and FEC packet 1 are dropped as illustrated in part (a) of FIG. 6b, when data packets 1 and 2 are dropped as illustrated in part (b) of FIG. 6b, or when data packets 1 and 10 are dropped as illustrated in part (c) of FIG. 6b), the receiving device can successfully restore the dropped data packet(s) using the received FEC packet(s). That is, the receiving device can successfully perform FEC decoding. In this case, the receiving device does not request the transmitting device to retransmit the dropped data packets.

[0118] For example, as illustrated in part (d) of FIG. 6b, the transmitting device can transmit 10 data packets based on a first data rate (e.g., a rate of transmitting 10 data packets for a certain period of time (e.g., per 1 / 60 second)) and transmit 3 FEC packets based on a second FEC rate (e.g., a rate of transmitting 3 FEC packets for a certain period of time (e.g., per 1 / 60 second)) that is higher than the first FEC rate (e.g., a rate of transmitting 2 FEC packets for a certain period of time (e.g., per 1 / 60 second)) of parts (a) to (c) of FIG. 6b. In this case, the receiving device can receive a larger number of FEC packets compared to the examples of parts (a) to (c) of FIG. 6b, and thus can restore a larger number of dropped packets. For example, if three packets are dropped (e.g., data packets 1, 3, and 10 are dropped, as illustrated in part (d) of FIG. 6b), the receiving device can successfully recover the three dropped data packets using the three received FEC packets. That is, the receiving device can successfully perform FEC decoding. In this case, the receiving device does not request the transmitting device to retransmit the dropped data packets.

[0119] For example, as illustrated in part (e) of FIG. 6B, the transmitting device may transmit 10 data packets based on a first data rate (e.g., a rate of transmitting 10 data packets per a certain period of time (e.g., per 1 / 60 second)) and transmit 2 FEC packets based on a first FEC rate (e.g., a rate of transmitting 2 FEC packets per a certain period of time (e.g., per 1 / 60 second)) of parts (a) to (c) of FIG. 6B. In this case, for example, if 3 packets are dropped (e.g., if data packets 1, 3, and 10 are dropped, as illustrated in part (e) of FIG. 6B), the receiving device may not be able to successfully restore the 1 dropped data packet using only the 2 received FEC packets, but may be able to successfully perform FEC decoding. In this case, image breakage may occur. The receiving device can request retransmission of dropped data packets from the transmitting device. Alternatively, if latency is critical, the receiving device can display the image as is, even if it contains a defect or flaw, without retransmission.

[0120] The FEC decoding configuration information according to one embodiment may include information corresponding to the FEC encoding configuration information. For example, the FEC decoding configuration information may include, but is not limited to, information on the FEC method, information on the FEC rate, information on the FEC code rate, and / or information on FEC-related information. For a description of the FEC method, FEC rate, FEC code rate, and FEC-related information, refer to the description given above in FIGS. 4a / b. Therefore, any duplicate description will be omitted.

[0121] According to one embodiment, in the decoding operation (520), the receiving device may decode the FEC decoded data using the decoding configuration information to generate decoded data. The decoding configuration information may include, but is not limited to, information on the encoding / decoding method (e.g., codec information) and information on the bit rate. For a description of the information on the encoding / decoding method and information on the bit rate, refer to the description given above in FIGS. 4a / b. Therefore, a duplicate description is omitted.

[0122] Referring to FIG. 6a, the receiving device can display an image based on the decoded data.

[0123] For example, when data packets are transmitted based on a first data rate (or a first bit rate) and FEC decoding is successfully performed (e.g., cases (a) to (c) of FIG. 6B), the receiving device can display a first image (610) having a first image quality (or a first image quality) based on the decoded data. According to one embodiment, the first image quality may be an image quality corresponding to the first transmission rate (or the first bit rate).

[0124] For example, if data packets are transmitted based on a second data rate that is higher than the first data rate, and FEC decoding is successfully performed (e.g., case (d) of FIG. 6b), the receiving device can display a second image (620) having a second image quality (or first image quality) based on the decoded data. According to one embodiment, the second image quality may be an image quality corresponding to the second transmission rate. For example, if other factors that determine the image quality (e.g., resolution, frame rate, compression method (e.g., HEVC), etc.) are all the same, the second image quality corresponding to the second data rate (or second bit rate) that is higher than the first data rate (or first bit rate) may have a better image quality than the first image quality corresponding to the first transmission rate. That is, under the condition that other factors that determine the image quality are the same, the image quality may improve as the transmission rate (or bit rate) of the data packets increases.

[0125] For example, when data packets are transmitted based on a first data rate and FEC decoding is not successfully performed (e.g., case (e) of FIG. 6b), the receiving device may display a third image (630) having a portion (631) of an image corresponding to a dropped packet that cannot be restored and has a breakage (e.g., a defect or flaw). In one embodiment, the portion (631) of the broken image may be an area corresponding to at least one block within a frame included in the unrestored packet. In one embodiment, the image quality of the third image (630) may be substantially the same as the first image quality corresponding to the first data rate, or may be a lower image quality than the first image quality.

[0126] FIG. 7 illustrates an exemplary relationship between target bandwidth, network bandwidth, and packet transmission rate according to one embodiment of the present disclosure.

[0127] In the embodiment of FIG. 7, the transmitting device may utilize a variable bit rate method or an adaptive bit rate method. In this case, the transmitting device may adaptively adjust the packet transmission rate (or data bit rate) depending on the network conditions (e.g., status). For example, in a congested network, the packet transmission rate may be lowered, and in a good network environment (e.g., no or low latency, low or no congestion), the packet transmission rate may be increased to a target bandwidth.

[0128] According to one embodiment, the transmitting device can adjust the bit rate of the source data on a frame-by-frame basis.

[0129] In one embodiment, when a target bandwidth is set, the transmitting device may increase the transmission rate of a packet (or the bit rate of data) up to the target bandwidth. For example, the transmitting device may increase the transmission rate of a packet (e.g., a data packet) up to the target bandwidth.

[0130] In one embodiment, as illustrated in FIG. 7, the target bandwidth may be higher than the current network bandwidth. For example, the target bandwidth may have a bandwidth capable of processing N packets in a given period of time (e.g., a bandwidth capable of processing 20 packets per 1 / 60 second) (or 20 Mbps), and the current network bandwidth may have a bandwidth capable of processing M packets that are smaller than N in a given period of time (e.g., a bandwidth capable of processing 10 packets per 1 / 60 second) (or 10 Mbps). If the target bandwidth is higher than the current network bandwidth, the transmission rate of the uplink packets (or the bit rate of the data) to reach the target bandwidth may be higher than the current network bandwidth. In this case, some packets may be dropped during the transmission process due to limitations in the network bandwidth. For example, in the section (710) of FIG. 7 where the packet transmission rate (or bit rate of data) is higher than the current network bandwidth, some of the data packets (or source data) transmitted by the transmitting device may be dropped.

[0131] According to one embodiment, if the receiving device cannot successfully restore dropped data packets (or source data) using FEC packets (or restoration data), image corruption occurs in the displayed image on the receiving device. For example, image corruption occurs, such as in the third image (630) of FIG. 6A. Meanwhile, if image corruption continues to occur, the transmitting device can lower the packet transmission rate (or data bit rate) below the network bandwidth, as illustrated in FIG. 7, to prevent image corruption. In this case, the image quality deteriorates again.

[0132] Hereinafter, one or more embodiments for adjusting the transmission rate of a data packet (or the bit rate of source data) to a target bandwidth are described. According to one embodiment, adjusting the transmission rate of the data packet (or the bit rate of the source data) to the target bandwidth may include upwardly adjusting the transmission rate of the data packet (or the bit rate of the source data) to a maximum settable transmission rate (or bit rate) within a set target bandwidth.

[0133] According to one embodiment, the transmitting device can adjust the transmission rate of a data packet (or the bit rate of source data) to a target bandwidth based on a transmission rate adjustment mode. The transmission rate adjustment mode may include, for example, at least one of a first transmission rate adjustment mode or a second transmission rate adjustment mode.

[0134] According to one embodiment, the first rate adjustment mode may be a rate adjustment mode including an operation in which the transmitting device uses information related to a reception error (hereinafter, reception error information) for the FEC packet group (or FEC encoded data) of the receiving device to determine whether to increase the transmission rate of a data packet (or a bit rate of source data) for the next FEC packet group (or the next FEC encoded data). For example, the first rate adjustment mode may be a rate adjustment mode including an operation in which the receiving device determines whether to increase the transmission rate of a data packet (or a bit rate of source data) for the next FEC packet group (or the next FEC encoded data) based on reception error information due to the transmitting device first increasing the transmission rate of an FEC packet (or the FEC rate of restored data, or the FEC size of restored data) for the FEC packet group (or the FEC encoded data) and transmitting it. Embodiments associated with the first rate adjustment mode are described below with reference to, for example, FIGS. 9 to 15B.

[0135] According to one embodiment, the second rate adjustment mode may be a rate adjustment mode including an operation in which the transmitting device preferentially increases the rate of a data packet (or the bit rate of the source data) for a next FEC packet group (or the next FEC encoded data) without using reception error information about the FEC packet group (or the FEC encoded data) of the receiving device. For example, the rate adjustment mode may include an operation in which the transmitting device preferentially increases the rate of a data packet (or the bit rate of the source data) for a next FEC packet group (or the next FEC encoded data) without using reception error information due to the transmitting device preferentially transmitting an FEC packet (or the FEC rate of the restored data, or the FEC size of the restored data) for the FEC packet group (or the FEC encoded data). Embodiments associated with the second rate adjustment mode are described below with reference to, for example, FIGS. 8A and 8B .

[0136] In one embodiment, the transmitting device can set the first transmission rate adjustment mode to the default transmission rate adjustment mode. In one embodiment, the transmitting device can selectively set either the first transmission rate adjustment mode or the second transmission rate adjustment mode. For example, the transmitting device can selectively set either the first transmission rate adjustment mode or the second transmission rate adjustment mode based on a user input of the receiving device. Embodiments related to selective setting of the transmission rate adjustment mode are described below with reference to FIGS. 16 to 18.

[0137] FIG. 8A illustrates an exemplary packet transmission and reception situation according to a transmission rate adjustment procedure according to an embodiment of the present disclosure. FIG. 8B illustrates an exemplary change in the transmission rate of a data packet according to a transmission rate adjustment procedure according to an embodiment of the present disclosure.

[0138] The transmission rate adjustment procedure of the embodiments of FIGS. 8A and 8B may be, for example, a procedure for adjusting the transmission rate of a data packet to a target bandwidth according to a second transmission rate adjustment mode.

[0139] According to one embodiment, a transmitting device (e.g., a transmitting device (110) of FIGS. 1 and 2) may transmit at least one data packet based on a first data transmission rate and at least one FEC packet based on a second FEC transmission rate to a receiving device (e.g., a receiving device (120) of FIGS. 1 and 3). For a description of the transmission of data packets and FEC packets, reference may be made to the descriptions of FIGS. 4A and 4B. Therefore, any duplicate description will be omitted.

[0140] According to one embodiment, a receiving device can perform FEC decoding and decoding on a group of FEC packets (transmission packet group) including at least one data packet transmitted (or processed) based on a transmission rate of a data packet and at least one FEC packet transmitted (or processed) based on a transmission rate of an FEC packet. One FEC packet group can correspond to, for example, one frame. In addition, the receiving device can display an image based on the decoded data. For a description of FEC decoding, decoding, and image display, refer to the descriptions of FIGS. 5 to 6B. Duplicate descriptions thereof will be omitted.

[0141] According to one embodiment, the receiving device can check (or identify) reception errors (or abnormalities) in received packets.

[0142] According to one embodiment, the receiving device can identify the number of packets in which a reception error (e.g., a packet drop) occurs within an FEC packet group including at least one data packet transmitted based on a transmission rate of the data packet and at least one FEC packet transmitted based on a transmission rate of the FEC packet. For example, the receiving device (120) can identify the number of packets in which a reception error occurs within the corresponding FEC packet group. The number of packets in which a reception error occurs may be, for example, the sum of the number of dropped data packets and the number of dropped FEC packets within the corresponding FEC packet group. Alternatively, the number of packets in which a reception error occurs may be, for example, the number of dropped data packets within the corresponding FEC packet group.

[0143] According to one embodiment, the receiving device can identify the number of packets in which reception errors occurred (e.g., the number of dropped packets) based on FEC-related information for the corresponding FEC packet group transmitted from the transmitting device (e.g., information on the total number of packets for the corresponding FEC packet group, information on the number of data packets, and / or information on the number of FEC packets). For the FEC-related information and the operation of the receiving device identifying the number of packets in which reception errors occurred based on the FEC-related information, reference may be made to the descriptions in the above descriptions (e.g., FIGS. 4A to 6B). Therefore, duplicate descriptions are omitted.

[0144] According to one embodiment, the receiving device can identify whether the number of packets in which reception errors occur is less than or equal to a preset error threshold value (error threshold value) (e.g., 2). If the number of packets in which reception errors occur is less than or equal to the error threshold value, the receiving device (120) can identify that no reception errors (or abnormalities) are detected for the corresponding FEC packet group. If the number of packets in which reception errors occur exceeds the error threshold value, the receiving device can identify that reception errors (or abnormalities) are detected for the corresponding FEC packet group.

[0145] Hereinafter, with reference to FIGS. 8A and 8B, examples of packet transmission and reception according to a rate adjustment procedure based on, for example, a second rate adjustment mode will be described. In the embodiments of FIGS. 8A and 8B, it is assumed that the rate is the number of packets transmitted over a given period of time, and the network bandwidth and target bandwidth are bandwidths capable of processing, for example, 10 packets and 20 packets over a given period of time, respectively. The restoration operation of a data packet based on an FEC packet may be described, for example, with reference to the description of FIG. 6B.

[0146] According to one embodiment, each FEC packet group of FIG. 8A may correspond to a respective frame. For example, FEC packet group 1 may correspond to a first frame, and FEC packet group 2 transmitted after FEC packet group 1 may correspond to a second frame following the first frame.

[0147] First, the packet transmission and reception process of the first FEC packet group of FIG. 8A (e.g., the FEC packet group corresponding to the first frame) will be described. A transmitting device can transmit 11 FEC packet groups 1 including 9 data packets and 2 associated FEC packets for a certain period of time (e.g., 1 / 60 second of 60 fps), and a receiving device can receive FEC packet group 1. In this case, due to the limitation of network bandwidth, one packet may be dropped during the transmission process. However, since the number of dropped packets is within the number that can be restored by the FEC packets, decoding of the data packets can be successfully performed. Accordingly, the receiving device can identify that the number of packets in which a reception error occurred is 1 and that the decoding (decoding) result is successful. The receiving device can display an image based on the decoded data for FEC packet group 1. Since decoding is successful, the displayed image does not exhibit image breakage or image quality degradation, and can have a first image quality corresponding to the transmission rate of the data packet of FEC packet group 1. For example, the displayed image may be the first image (610) of FIG. 6A.

[0148] When explaining the packet transmission and reception process of the second FEC packet group of FIG. 8A (e.g., the FEC packet group corresponding to the second frame following the first frame), since the target bandwidth has not yet been reached, the transmitting device increases the transmission rate of the data packets. For example, as illustrated in FIG. 8A, the transmitting device increases the number of data packets transmitted for a certain period of time (e.g., for 1 / 60 second of 60 fps) by 1 compared to the previous FEC packet group 1 in order to increase the transmission rate of the data packets. The transmitting device can transmit 12 FEC packet groups 2, each including 10 data packets and 2 associated FEC packets, during the certain period of time, and the receiving device can receive the FEC packet group 2. In this case, due to the limitation of the network bandwidth, 2 packets may be dropped during the transmission process. However, since the number of dropped packets is within the number that can be restored by the FEC packets, decoding of the data packets can be successfully performed. Accordingly, the receiving device can identify that the number of packets in which reception errors occurred is 2 and that the decoding (decoding) result is successful. The receiving device can display an image based on the decoded data for FEC packet group 2. Since the decoding was successful, the displayed image does not have image breakage or image quality deterioration, and can have a second image quality corresponding to the transmission rate of the data packets of FEC packet group 2. For example, the displayed image can be the second image (620) of FIG. 6A.

[0149] When explaining the transmission and reception process of the packets of the third FEC packet group of FIG. 8A (e.g., the FEC packet group corresponding to the third frame following the second frame), since the target bandwidth has not yet been reached, the transmitting device increases the transmission rate of the data packets again. For example, as illustrated in FIG. 8A, the transmitting device increases the number of data packets transmitted for a certain period of time (e.g., for 1 / 60 second of 60 fps) by 1 compared to the previous FEC packet group 2 in order to increase the transmission rate of the data packets. The transmitting device can transmit 13 FEC packet groups 3, which include 11 data packets and 2 associated FEC packets, for a certain period of time, and the receiving device can receive FEC packet group 3. In this case, due to the limitation of the network bandwidth, 3 packets may be dropped during the transmission process. However, since the number of dropped packets exceeds the number that can be restored by the FEC packets, decoding of the data packets cannot be successfully performed. Accordingly, the receiving device can identify that the number of packets with reception errors is 3 and that the decoding (decoding) result is a failure. The receiving device can display an image based on the decoded data for FEC packet group 3. Since the decoding failed, the displayed image may have image breakage or deteriorated image quality. For example, the displayed image may have image breakage, as in the third image (630) of FIG. 6A.

[0150] When explaining the transmission and reception process of the packets of the fourth FEC packet group of FIG. 8A (e.g., the FEC packet group corresponding to the fourth frame following the third frame), since the target bandwidth has not yet been reached, the transmitting device increases the transmission rate of the data packets again. For example, as illustrated in FIG. 8A, the transmitting device increases the number of data packets transmitted for a certain period of time (e.g., 1 / 60 second of 60 fps) by 1 compared to the previous FEC packet group 3 in order to increase the transmission rate of the data packets. The transmitting device can transmit 14 FEC packet groups 4, which include 12 data packets and 2 associated FEC packets, for a certain period of time, and the receiving device can receive FEC packet group 4. In this case, due to the limitation of the network bandwidth, 4 packets may be dropped during the transmission process. However, since the number of dropped packets exceeds the number that can be restored by the FEC packets, decoding of the data packets cannot be successfully performed. Accordingly, the receiving device can identify that the number of packets with reception errors is 4 and that the decoding (decoding) result is a failure. The receiving device can display an image based on the decoded data for FEC packet group 4. Since the decoding failed, the displayed image may have image breakage or deteriorated image quality. For example, the displayed image may have image breakage, as in the third image (630) of FIG. 6A.

[0151] When explaining the transmission and reception process of the packets of the fifth FEC packet group of FIG. 8A (e.g., the FEC packet group corresponding to the fifth frame following the fourth frame), if the target bandwidth has not yet been reached, but decoding failures continue to occur (e.g., if a preset number (e.g., 3) or more occur), the transmitting device can lower the transmission rate of the data packets. For example, as illustrated in FIG. 8A, the transmitting device can adjust the transmission rate of the data packets to a basic transmission rate (e.g., the transmission rate of the previous FEC packet group 1). The transmitting device can transmit 11 FEC packet groups 5, which include 9 data packets and 2 associated FEC packets, for a certain period of time, and the receiving device can receive the FEC packet group 5. In this case, due to the limitation of the network bandwidth, one packet may be dropped during the transmission process. However, since the number of dropped packets is within the number that can be restored by the FEC packets, decoding of the data packets cannot be successfully performed. Accordingly, the receiving device can identify that the number of packets in which reception errors occurred is 1 and that the decoding (decoding) result is successful. The receiving device can display an image based on the decoded data for FEC packet group 5. Since the decoding was successful, the displayed image does not have image breakage or image quality deterioration, and can have a first image quality corresponding to the transmission rate of the data packet of FEC packet group 5. For example, the displayed image can be the first image (610) of FIG. 6A.

[0152] As in the example described above, when adjusting the transmission rate according to the second transmission rate adjustment mode, if it is determined that the target bandwidth is not reached, the transmitting device preferentially increases the transmission rate of the data packets for the FEC packet group, regardless of the number of packets in which reception errors occurred for the previous FEC packet group and the decoding result (e.g., whether decoding was successfully performed). In this case, jitter (or packet drop) may occur for the data packets in a section where the transmission rate of the data packets is higher than the network bandwidth (e.g., section 710 in FIG. 7 or the section indicated by the solid box in FIG. 8b). This causes a decrease in the transmission rate (e.g., bit rate) of the data packets, as in the section indicated by the dotted line in FIG. 8b, thereby causing image breakup or image quality deterioration displayed on the receiving device. Therefore, to solve this problem, it is necessary to consider a method for adjusting the transmission rate of the data packets to the target bandwidth, for example, without image breakup or image quality deterioration.

[0153] FIG. 9 illustrates a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0154] The transmission rate adjustment procedure of the embodiment of FIG. 9 may be a procedure for adjusting (e.g., upward adjustment) the transmission rate of a data packet (or bit rate of source data) to a target bandwidth according to, for example, the first transmission rate adjustment mode.

[0155] One operation cycle of operations 9020 to 9080 of FIG. 9 may be, for example, an operation cycle corresponding to one frame.

[0156] Referring to FIG. 9, in operation 9010, the transmitting device (110) and the receiving device (120) may establish a wireless connection. According to one embodiment, the transmitting device (110) and the receiving device (120) may negotiate parameter(s) for establishing the wireless connection. According to one embodiment, the transmitting device (110) and the receiving device (120) may transmit / receive FEC-related information for utilizing FEC. Depending on the embodiment, operation 9010 may be omitted.

[0157] In operation 9020, the transmitting device (110) may transmit, to the receiving device (120), a group of FEC packets including at least one data packet and at least one FEC packet for error correction for the at least one data packet. According to one embodiment, the transmitting device (110) may transmit, to the receiving device (120), at least one data packet at a transmission rate of the data packet and at least one FEC packet at a transmission rate of the FEC packet. For a description of the transmission of the data packet and the transmission of the FEC packet, reference may be made to the descriptions of FIGS. 4A and 4B . Therefore, a duplicate description will be omitted.

[0158] In operation 9030, the receiving device (120) can check (or identify) an error condition of the received packet.

[0159] According to one embodiment, the receiving device (120) can identify the number of packets in which a reception error (e.g., packet drop) occurred within an FEC packet group including at least one data packet transmitted (or processed) based on a transmission rate of the data packet and at least one FEC packet transmitted (or processed) based on a transmission rate of the FEC packet. For example, the receiving device (120) can identify the number of packets in which a reception error occurred within the corresponding FEC packet group. The number of packets in which a reception error occurred may be, for example, the sum of the number of dropped data packets and the number of dropped FEC packets within the corresponding FEC packet group.

[0160] According to one embodiment, the receiving device (120) can identify the number of packets in which reception errors occurred (e.g., the number of dropped packets) based on FEC-related information for the corresponding FEC packet group transmitted from the transmitting device (110) (e.g., total packet count information for the corresponding FEC packet group, data packet count information, and / or FEC packet count information). For the operation of transmitting and receiving FEC-related information and identifying the number of packets in which reception errors occurred based on the FEC-related information, reference may be made to the above-described description (e.g., the description of FIGS. 4A to 6B). Therefore, duplicate descriptions are omitted.

[0161] According to one embodiment, the receiving device (120) can identify whether the number of packets in which reception errors occur is less than or equal to a preset error threshold value (error threshold value). If the number of packets in which reception errors occur is less than or equal to the error threshold value, the receiving device (120) can identify that no reception errors (or abnormalities) are detected for the corresponding FEC packet group. If the number of packets in which reception errors occur exceeds the error threshold value, the receiving device (120) can identify that reception errors (or abnormalities) are detected for the corresponding FEC packet group. According to one embodiment, when the receiving device (120) determines whether reception errors (or abnormalities) are detected for the corresponding FEC packet group, the receiving device (120) can directly transmit information indicating whether reception errors are detected to the transmitting device (110). Otherwise, the receiving device (120) can only transmit information indicating the number of packets in which reception errors occur to the transmitting device (110).

[0162] In operation 9040, the receiving device (120) may transmit information regarding the result of confirmation regarding an error condition (reception error condition) of the received packet. According to one embodiment, the information regarding the result of confirmation regarding an error condition of the received packet may include reception error information.

[0163] According to one embodiment, the reception error information may include information indicating the number of packets in which reception errors occurred within an FEC packet group (hereinafter, error packet count information) or information indicating whether the number of packets in which reception errors occurred is less than or equal to a preset error threshold value (or information indicating whether a reception error is detected) (hereinafter, error situation information).

[0164] In operation 9050, the transmitting device (110) can identify whether the target bandwidth has been reached. According to one embodiment, the transmitting device (110) can identify whether the transmission rate of the data packet (or the transmission rate of the FEC packet group) is less than the target bandwidth, thereby identifying whether the target bandwidth has been reached. For example, the transmitting device (110) can identify whether the transmission rate of the data packet has reached the maximum settable transmission rate within the target bandwidth, thereby identifying whether the target bandwidth has been reached.

[0165] For example, if the transmission rate of the data packet is less than the target bandwidth (or less than the maximum configurable transmission rate of the data packet within the target bandwidth), the transmitting device can identify that the target bandwidth is not reached. If the transmission rate of the data packet (or the transmission rate of the FEC packet group) is greater than or equal to the target bandwidth (or greater than or equal to the maximum configurable transmission rate within the target bandwidth), the transmitting device can identify that the target bandwidth is reached. Operation 9050 may be omitted depending on the embodiment. Alternatively, operation 9050 may be performed before operation 9040.

[0166] If it is identified that the target bandwidth has not been reached, action 9060 may be performed.

[0167] In operation 9060, the transmitting device (110) can check the error status of the received packets and identify whether the number of errors in the received packets is greater than an error threshold (error threshold value). The number of errors in the received packets may correspond to, for example, the number of packets in which a reception error occurred.

[0168] According to one embodiment, the transmitting device (110) can identify whether the number of packets in which reception errors occurred is greater than a preset error threshold value (error threshold value) based on reception error information. If the number of packets in which reception errors occurred is identified as being greater than the error threshold value (e.g., if the number of packets in which reception errors occurred is identified as exceeding the error threshold value), operation 9070 may be performed. If the number of packets in which reception errors occurred is identified as being equal to or less than the error threshold value (e.g., if the number of packets in which reception errors occurred is identified as being less than or equal to the error threshold value), operation 9080 may be performed.

[0169] In operation 9070, the transmitting device (110) may increase the transmission rate of the FEC packet. For example, if the number of packets in which reception errors occurred is identified as exceeding an error threshold value, the transmitting device (110) may set the transmission rate of the FEC packet for the corresponding transmission packet group or the FEC code rate associated with the transmission rate of the FEC packet to a value such that all of at least one data packet dropped within the corresponding transmission packet group can be restored using at least one of a plurality of FEC packets transmitted based on the transmission rate of the FEC packet. For example, if the number of packets in which reception errors occurred is identified as exceeding an error threshold value, the transmitting device (110) may increase the number of FEC packets transmitted for a certain period of time by a preset number (e.g., 1). In this case, the transmitting device (110) may perform operation 9020 again based on the increased transmission rate of the FEC packet. At this time, the transmission rate of the data packet may be maintained. For example, the transmitting device (110) may transmit at least one data packet for the FEC packet group based on a transmission rate of the data packet that is the same as the transmission rate of the data packet for the previous FEC packet group, and may transmit at least one FEC packet for the FEC packet group based on a transmission rate of the FEC packet that is higher than the transmission rate of the FEC packet for the previous FEC packet group.

[0170] In operation 9080, the transmitting device (110) may increase the transmission rate of the data packet. For example, if the number of packets in which a reception error occurred is identified as being less than or equal to an error threshold value, the transmitting device (110) may set the transmission rate of the data packet for the corresponding transmission packet group or the bit rate of the source data associated with the transmission rate of the data packet to a value such that all of at least one data packet dropped within the corresponding transmission packet group can be restored using at least one of a plurality of FEC packets transmitted based on a basic FEC transmission rate (e.g., a transmission rate at which two FEC packets are transmitted during a predetermined time in FIG. 15A).

[0171] For example, if the number of packets in which a reception error occurred is identified as being less than or equal to an error threshold value, the transmitting device (110) may increase the number of data packets transmitted for a certain period of time by a preset number (e.g., 1). In this case, the transmitting device (110) may perform (or repeat) operation 9020 again based on the transmission rate of the uplink data packets. At this time, the transmission rate of the FEC packets may be maintained. For example, the transmitting device (110) may transmit at least one data packet for the corresponding FEC packet group based on a transmission rate of the data packets that is higher than the transmission rate of the data packets for the previous FEC packet group, and may transmit at least one FEC packet for the corresponding FEC packet group based on a transmission rate of the FEC packets that is the same as the transmission rate of the FEC packets for the previous FEC packet group (e.g., the transmission rate of the basic FEC packet). Thereafter, operations 9030 to 9080 may be performed again. These operations 9020 to 9080 may be performed repeatedly until the transmission rate of the data packets reaches the target bandwidth.

[0172] If it is determined that the target bandwidth has been reached, operation 9020 may be performed (or repeated) again without performing operation 9060. In this case, the transmitting device (110) does not need to adjust the transmission rate of the data packet to reach the target bandwidth. Accordingly, the transmitting device (110) may transmit at least one data packet for the corresponding FEC packet group based on a transmission rate of the data packet that is the same as the transmission rate of the data packet for the previous FEC packet group, and may transmit at least one FEC packet for the corresponding FEC packet group based on a transmission rate of the FEC packet that is the same as the transmission rate of the FEC packet for the previous FEC packet group.

[0173] As described above, in the rate adjustment procedure of FIG. 9 using the first rate adjustment mode, unlike the rate adjustment procedure using the second rate mode, if the target bandwidth is not reached, instead of immediately increasing the rate of the data packet, the transmitting device (110) uses the reception error information received from the receiving device (120) for the previous FEC packet group to determine whether to increase the rate of the data packet for the corresponding FEC packet group. Through this, the rate of the data packet can be stably (e.g., steadily or incrementally) increased to the target bandwidth without image breakage or deterioration of image quality.

[0174] FIG. 10 illustrates operations of a transmitting device in a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0175] The transmission rate adjustment procedure of the embodiment of FIG. 10 may be a procedure for adjusting (e.g., upward adjustment) the transmission rate of a data packet to a target bandwidth according to, for example, the first transmission rate adjustment mode.

[0176] Referring to FIG. 10, in operation 10010, a transmitting device (e.g., the transmitting device (110) of FIGS. 1 and 2) may establish a wireless connection with a receiving device (e.g., the receiving device (120) of FIGS. 1 and 3). The description of operation 10010 may refer to the description of operation 9010. Therefore, any duplicate description will be omitted. Depending on the embodiment, operation 10010 may be omitted.

[0177] In operation 10020, the transmitting device (110) may transmit to the receiving device (120) an FEC packet group including at least one data packet and at least one FEC packet for error correction for the at least one data packet. The description of operation 10020 may refer to the description of operation 9020. Therefore, any duplicate description will be omitted.

[0178] According to one embodiment, the transmitting device (110) may transmit at least one data packet to the receiving device (120) based on a transmission rate of the data packet, and may transmit at least one FEC packet based on a transmission rate of the FEC packet. For example, as illustrated in the transmitting and receiving process of the second FEC packet group of FIG. 15A, the transmitting device (110) may transmit 12 FEC packet groups 2, which include 9 data packets and 3 associated FEC packets, for a certain period of time. For example, as illustrated in the transmitting and receiving process of the fourth FEC packet group of FIG. 15A, the transmitting device (110) may transmit 13 FEC packet groups 4, which include 10 data packets and 3 associated FEC packets, for a certain period of time.

[0179] In operation 10030, the transmitting device (110) can determine whether the target bandwidth has been reached. The description of operation 10030 may refer to the description of operation 9050. Therefore, any duplicate description will be omitted. Operation 10030 may be omitted depending on the embodiment. Alternatively, operation 10030 may be performed prior to operation 10020.

[0180] If it is identified that the target bandwidth has not been reached, action 10040 may be performed.

[0181] In operation 10040, the transmitting device (110) can check the error status of the received packet and identify whether the number of errors in the received packet is greater than the error threshold (error threshold value). The description of operation 10040 may refer to the description of operation 9060. Therefore, any duplicate description will be omitted.

[0182] According to one embodiment, when the number of packets in which reception errors occurred is identified as being greater than the error threshold value (e.g., when the number of packets in which reception errors occurred is identified as exceeding the error threshold value), operation 10050 may be performed. For example, as illustrated in the transmission and reception process of the fourth FEC packet group of FIG. 15A, the transmitting device (110) may identify, based on reception error information for FEC packet group 4 received from the receiving device (120), that the number of packets in which reception errors occurred (e.g., 3) is greater than the error threshold value (e.g., 2).

[0183] According to one embodiment, when the number of packets in which reception errors occurred is identified as being equal to or less than an error threshold value (e.g., when the number of packets in which reception errors occurred is identified as being less than or equal to the error threshold value), operation 10060 may be performed. For example, as illustrated in the transmission and reception process of the second FEC packet group of FIG. 15A, the transmitting device (110) may identify, based on reception error information for FEC packet group 2 received from the receiving device (120), that the number of packets in which reception errors occurred (e.g., 2) is less than or equal to the error threshold value (e.g., 2).

[0184] In operation 10050, the transmitting device (110) can increase the transmission rate of the FEC packet. The description of operation 10050 may refer to the description of operation 9070. Therefore, any duplicate description is omitted.

[0185] According to one embodiment, if the number of packets in which reception errors have occurred is identified as exceeding an error threshold value, the transmitting device (110) may increase the number of FEC packets transmitted for a certain period of time by a preset number (e.g., 1). In this case, the transmitting device (110) may perform operation 10020 again based on the increased transmission rate of the FEC packets. At this time, the transmission rate of the data packets may be maintained. For example, the transmitting device (110) may transmit at least one data packet for the corresponding FEC packet group based on a transmission rate of the data packets that is the same as the transmission rate of the data packets for the previous FEC packet group, and may transmit at least one FEC packet for the corresponding FEC packet group based on a transmission rate of the FEC packets that is increased compared to the transmission rate of the FEC packets for the previous FEC packet group. For example, since the number of packets in which reception errors occurred in the previous FEC packet group 4 (e.g., 3) is identified as exceeding the error threshold value (e.g., 2), the transmitting device (110) can transmit 14 FEC packet groups 5, each including 10 data packets and 4 associated FEC packets, for a certain period of time, as illustrated in the transmission and reception process of the fifth FEC packet group in FIG. 15a.

[0186] In operation 10060, the transmitting device (110) can increase the transmission rate of the data packet. The description of operation 10060 may refer to the description of operation 9080. Therefore, any duplicate description will be omitted.

[0187] According to one embodiment, if the number of packets in which reception errors have occurred is identified as being less than or equal to an error threshold value, the transmitting device (110) may increase the number of data packets transmitted for a certain period of time by a preset number (e.g., 1). In this case, the transmitting device (110) may perform operation 10020 again based on the transmission rate of the uplink data packets. At this time, the transmission rate of the FEC packets may be maintained (e.g., maintained at the default transmission rate). For example, the transmitting device (110) may transmit at least one data packet based on a transmission rate of the data packets that is higher than the transmission rate of the data packets for the previous FEC packet group, and may transmit at least one FEC packet based on a transmission rate of the FEC packets that is the same as the transmission rate of the FEC packets for the previous FEC packet group. For example, since the number of packets in which reception errors occurred in the previous FEC packet group 2 (e.g., 2) is identified as being less than or equal to the error threshold value (e.g., 2), the transmitting device (110) can transmit 12 FEC packet groups 3 including 10 data packets and 2 associated FEC packets for a certain period of time, as illustrated in the transmission and reception process of the third FEC packet group in FIG. 15a.

[0188] These operations 10020 to 10060 may be performed repeatedly until the transmission rate of the data packets reaches the target bandwidth.

[0189] If it is determined that the target bandwidth has been reached, operation 10020 may be performed again without performing operation 10040. In this case, the transmitting device (110) may transmit at least one data packet based on a transmission rate of the data packet that is the same as the transmission rate of the data packet for the previous FEC packet group, and may transmit at least one FEC packet based on a transmission rate of the FEC packet that is the same as the transmission rate of the FEC packet for the previous FEC packet group. Thereafter, operations 10030 to 10060 may be performed again.

[0190] FIG. 11 illustrates operations of a receiving device in a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0191] The transmission rate adjustment procedure of the embodiment of FIG. 11 may be, for example, a procedure for adjusting (e.g., upward adjustment) the transmission rate of a data packet to a target bandwidth according to a first transmission rate adjustment mode.

[0192] Referring to FIG. 11, in operation 11010, a receiving device (e.g., the receiving device (120) of FIGS. 1 and 3) may establish a wireless connection with a transmitting device (e.g., the transmitting device (110) of FIGS. 1 and 2). The description of operation 11010 may refer to the description of operation 9010. Therefore, any duplicate description will be omitted. Depending on the embodiment, operation 11010 may be omitted.

[0193] In operation 11020, the receiving device (120) may receive, from the transmitting device (110), an FEC packet group including at least one data packet and at least one FEC packet for error correction for the at least one data packet. The description of operation 11020 may refer to the description of operation 9020. Therefore, any duplicate description will be omitted.

[0194] According to one embodiment, the receiving device (120) may receive at least one data packet transmitted from the transmitting device (110) based on a transmission rate of the data packet, and at least one FEC packet transmitted based on a transmission rate of the FEC packet. For example, as illustrated in the transmitting and receiving process of the second FEC packet group of FIG. 15A, the receiving device (120) may receive 12 FEC packet groups 2 including 9 data packets and 2 associated FEC packets during a certain period of time. For example, as illustrated in the transmitting and receiving process of the fourth FEC packet group of FIG. 15A, the receiving device (120) may receive 13 FEC packet groups 4 including 10 data packets and 3 associated FEC packets during a certain period of time.

[0195] In operation 11030, the receiving device (120) can check the error status of the received packet. The description of operation 11030 may refer to the description of operation 9030. Therefore, duplicate description is omitted.

[0196] For example, as illustrated in the transmission and reception process of the second FEC packet group of FIG. 15a, due to the limitation of the network bandwidth, two packets are dropped during the transmission process, but since the number of dropped packets is within the number that can be restored by the FEC packets, decoding of the data packets can be successfully performed. The receiving device (120) can identify that the number of packets in which reception errors occurred is 2 and that the decoding (decryption) result is successful. The receiving device (120) can generate reception error information for the FEC packet group 2 based on the confirmation of the error status of the reception packet. The reception error information for the FEC packet group 2 can include information indicating the number of packets in which reception errors occurred (e.g., 2) within the FEC packet group 2 or information indicating that the number of packets in which reception errors occurred (e.g., 2) is less than or equal to a preset error threshold value (e.g., 2) (or information indicating that no reception errors were detected).

[0197] For example, as illustrated in the transmission and reception process of the fourth FEC packet group in FIG. 15a, due to the limitation of the network bandwidth, three packets are dropped during the transmission process, but since the number of dropped packets is within the number that can be restored by the FEC packets, decoding of the data packets can be successfully performed. The receiving device (120) can identify that the number of packets in which reception errors occurred is 3 and that the decoding (decryption) result is successful. The receiving device (120) can generate reception error information for the FEC packet group 4 based on the confirmation of the error status of the reception packet. The reception error information for the FEC packet group 4 can include information indicating the number of packets in which reception errors occurred (e.g., 3) within the FEC packet group 4 or information indicating that the number of packets in which reception errors occurred (e.g., 3) exceeds a preset error threshold value (e.g., 2) (or information indicating that a reception error is detected).

[0198] In operation 11040, the receiving device (120) may transmit information (reception error information) regarding the result of checking the error status of the received packet. The description of operation 11030 may refer to the description of operation 9030. Therefore, any duplicate description will be omitted.

[0199] For example, as illustrated in the transmission and reception process of the second FEC packet group of FIG. 15a, the receiving device (120) can transmit reception error information for FEC packet group 2.

[0200] For example, as illustrated in the transmission and reception process of the fourth FEC packet group of FIG. 15a, the receiving device (120) can transmit reception error information for FEC packet group 4.

[0201] According to one embodiment, before or after operation 11040, the receiving device (120) can display an image based on the decoded data.

[0202] For example, as illustrated in the transmission and reception process of the second FEC packet group of FIG. 15A, the receiving device (120) can display an image based on the decoded data for FEC packet group 2. Since the decoding is successful, the displayed image does not have image breakage or image quality deterioration, and can have a first image quality corresponding to the transmission rate of the data packet of FEC packet group 2. For example, the displayed image can be the first image (610) of FIG. 6A.

[0203] For example, as illustrated in the transmission and reception process of the fourth FEC packet group of FIG. 15A, the receiving device (120) can display an image based on the decoded data for FEC packet group 4. Since the decoding is successful, the displayed image does not have image breakup and can have a second image quality corresponding to the transmission rate of the data packet of FEC packet group 4. For example, since the transmission rate of the data packet of FEC packet group 4 is higher than the transmission rate of the data packet of FEC packet group 2, the displayed image can be a second image (620) of FIG. 6B having a higher image quality than the first image (610) of FIG. 6A.

[0204] FIG. 12 is a schematic flowchart of a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0205] The transmission rate adjustment procedure of the embodiment of FIG. 12 may be, for example, a procedure for adjusting (e.g., upwardly adjusting) the bit rate of source data to a target bandwidth according to a first transmission rate adjustment mode. According to one embodiment, the transmission rate adjustment procedure of the embodiment of FIG. 12 may be an example of the transmission rate adjustment procedure of FIG. 9. Accordingly, the description of the transmission rate adjustment procedure of the embodiment of FIG. 9 may be referred to for the description of the transmission rate adjustment procedure of the embodiment of FIG. 12, unless otherwise contradictory.

[0206] Referring to FIG. 12, in operation 12010, a transmitting device (110) may transmit first FEC encoded data including first source data encoded at a first bit rate and first restoration data for error correction for the first source data to a receiving device (120). According to one embodiment, the first FEC encoded data may be FEC encoded data for a first frame. For a description of the transmission of the FEC encoded data, reference may be made to the descriptions of FIGS. 4A and 4B. Therefore, a duplicate description will be omitted.

[0207] In operation 12020, the transmitting device (110) may transmit second FEC encoded data including second source data encoded at a first bit rate and second restoration data for error correction for the second source data to the receiving device (120). According to one embodiment, the size (or FEC rate) of the second restoration data may be greater than the size (or FEC rate) of the first restoration data. In other words, the transmitting device (110) may transmit source data for the current second FEC encoded data at a bit rate that is the same as the bit rate of the first source data of the first FEC encoded data, which is previous FEC encoded data, and may transmit restoration data for the current second FEC encoded data at a FEC rate that is the same as the FEC rate of the first restoration data of the previous first encoded data. That is, the transmitting device (110) may maintain the transmission rate of the source data and only increase the transmission rate of the restoration data. According to one embodiment, the second FEC encoded data may be FEC encoded data for a second frame following the first frame.

[0208] According to one embodiment, the transmitting device (110) may set the size of the second restoration data to a value such that a portion of the second source data in which a reception error occurred can be restored using at least a portion of the second restoration data.

[0209] In operation 12030, the receiving device (120) may generate reception error information for the second FEC encoded data. According to one embodiment, the receiving device (120) may identify the number of packets in which reception errors occurred within a second FEC packet group for transmitting the second FEC encoded data, and generate reception error information for the second FEC encoded data based on the identified number of packets in which reception errors occurred. The number of packets in which reception errors occurred may be, for example, the sum of the number of dropped data packets and the number of dropped FEC packets within the second FEC packet group. Operation 12030 of FIG. 12 may be an example of operation 9030 of FIG. 9.

[0210] According to one embodiment, the receiving device (120) can identify the number of packets in which reception errors occurred (e.g., the number of dropped packets) based on FEC-related information for the second FEC packet group transmitted from the transmitting device (110) (e.g., total packet count information, data packet count information, and / or FEC packet count information for the second FEC packet group). For the operation of the transmitting and receiving device (120) of the FEC-related information to identify the number of packets in which reception errors occurred based on the FEC-related information, reference may be made to the above-described description (e.g., the description of FIGS. 4A to 6B). Therefore, duplicate descriptions are omitted.

[0211] According to one embodiment, the receiving device (120) may generate reception error information for the second FEC encoded data based on the number of packets in which the identified reception errors occurred. According to one embodiment, the reception error information for the second FEC encoded data may include information indicating the number of packets in which reception errors occurred within the second FEC packet group for transmitting the second FEC encoded data, or information indicating whether the number of packets in which reception errors occurred is less than or equal to a preset error threshold value (or information indicating whether a reception error is detected).

[0212] In operation 12040, the receiving device (120) may transmit reception error information for the second FEC encoded data to the transmitting device (110).

[0213] In operation 12050, the transmitting device (110) may determine, based on reception error information for the second FEC encoded data, whether to change the bit rate of third source data included in third FEC encoded data transmitted after the second FEC encoded data to a second bit rate higher than the first bit rate. According to one embodiment, the transmitting device (110) may perform an operation (e.g., operation 9050 of FIG. 9) for identifying whether a target bandwidth has been reached before operation 12050, and may perform operation 12050 only when the target bandwidth has been reached.

[0214] According to one embodiment, the transmitting device (110) can identify whether the number of errors in the received packets is greater than an error threshold (error threshold value) based on the reception error information for the second FEC encoded data. The number of errors in the received packets may correspond to, for example, the number of packets in which reception errors occurred.

[0215] According to one embodiment, when the number of packets in which reception errors occur for the second FEC encoded data is identified as being greater than the error threshold value (e.g., when the number of packets in which reception errors occur is identified as exceeding the error threshold value), the transmitting device (110) may determine to change the bit rate of third source data included in third FEC encoded data transmitted after the second FEC encoded data to a second transmission rate higher than the first transmission rate of the second source data included in the second FEC encoded data. According to one embodiment, the transmitting device (110) may set the bit rate of the third source data to a value that enables the third restoration data in which an error occurs in the third source data to have a size of basic restoration data (e.g., a size or an FEC rate of the first restoration data) to be restored. When it is determined to change the bit rate of the third source data to a second transmission rate higher than the first transmission rate, operation 12060 may be performed.

[0216] According to one embodiment, if the number of packets in which reception errors occurred for the second FEC encoded data is identified as being equal to or less than an error threshold value (e.g., if the number of packets in which reception errors occurred is identified as being less than or equal to the error threshold value), the transmitting device (110) may determine not to change the bit rate of third source data included in third FEC encoded data transmitted after the second FEC encoded data to a second bit rate higher than the first bit rate of the second source data included in the second FEC encoded data. If it is determined not to change the bit rate of the third source data to a second bit rate higher than the first bit rate, operation 12070 may be performed.

[0217] In operation 12060, the transmitting device (110) may transmit third source data encoded at a second bit rate to the receiving device (120), and may transmit third FEC encoded data including third restored data having the same size (or the same FEC rate) as the first restored data. In other words, the transmitting device (110) may transmit third source data for the third FEC encoded data at a second bit rate that is higher than the first bit rate, which is the bit rate of the second source data of the second FEC encoded data, which is previous FEC encoded data, and may transmit third restored data for the third FEC encoded data at a size (or FEC rate) that is the same as the size (or FEC rate) of the primary restored data. In one embodiment, the third FEC encoded data may be FEC encoded data for a third frame following the second frame.

[0218] In operation 12070, the transmitting device (110) may transmit third source data encoded at a first bit rate to the receiving device (120), and transmit third FEC encoded data including third restored data having a size equal to or larger than that of the second restored data. In other words, the transmitting device (110) may transmit third source data for the third FEC encoded data at a first bit rate equal to a bit rate of source data of the second FEC encoded data, which is previous FEC encoded data, and transmit third restored data for the third FEC encoded data at an FEC rate equal to or higher than an FEC rate of restored data of the second FEC encoded data.

[0219] As described above, in the rate adjustment procedure of FIG. 12 using the first rate adjustment mode, unlike the rate adjustment procedure using the second rate mode, if the target bandwidth is not reached, instead of immediately increasing the bit rate of the source data, the transmitting device (110) first increases the size (or FEC rate) of the restored data and transmits it, thereby using reception error information on the previous FEC encoded data received from the receiving device (120) to determine whether to increase the bit rate of the source data for the corresponding FEC encoded data. Through this, the bit rate of the source data can be increased to the target bandwidth without image breakage or deterioration of image quality.

[0220] FIG. 13 is a flowchart illustrating the operation of a transmitting device in a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0221] The transmission rate adjustment procedure of the embodiment of FIG. 13 may be, for example, a procedure for adjusting (e.g., upwardly adjusting) the bit rate of source data to a target bandwidth according to the first transmission rate adjustment mode. According to one embodiment, the transmission rate adjustment procedure of the embodiment of FIG. 13 may be an example of the transmission rate adjustment procedure of FIG. 12.

[0222] Referring to FIG. 13, in operation 13010, a transmitting device (e.g., a transmitting device (110) of FIGS. 1 and 2) may transmit first FEC encoded data including first source data encoded at a first bit rate and first restoration data for error correction for the first source data to a receiving device (e.g., a receiving device (120) of FIGS. 1 and 3). The description of operation 13010 may refer to the description of operation 12010. Therefore, any duplicate description will be omitted.

[0223] For example, as illustrated in the transmission and reception process of the first FEC packet group of FIG. 15A, the transmitting device (110) may transmit 11 FEC packet groups 1 including 9 data packets and 2 associated FEC packets for a certain period of time (e.g., 1 / 60 second of 60 fps). For example, as illustrated in the transmission and reception process of the third FEC packet group of FIG. 15A, the transmitting device (110) may transmit 12 FEC packet groups 3 including 10 data packets and 2 associated FEC packets for a certain period of time.

[0224] In operation 13020, the transmitting device (110) may transmit, to the receiving device (120), first FEC encoded data including second source data encoded at a first bit rate and second restoration data for error correction of the second source data. According to one embodiment, the size (or FEC rate) of the second restoration data may be larger than the size (or FEC rate) of the first restoration data. In other words, the transmitting device (110) may transmit the first source data at a bit rate that is the same as the bit rate of the first source data of the first FEC encoded data, which is previous FEC encoded data, and transmit the second restoration data at a higher FEC rate than the FEC rate of the first restoration data of the previous first FEC encoded data. In other words, the transmitting device (110) may maintain the transmission rate of the data packet and only increase the transmission rate of the FEC packet. The description of operation 13020 may refer to the description of operation 12020. Therefore, redundant descriptions are omitted.

[0225] For example, as illustrated in the transmission and reception process of the second FEC packet group of FIG. 15A, the transmitting device (110) can transmit 12 FEC packet groups 2 including 9 data packets and 3 associated FEC packets for a certain period of time. For example, as illustrated in the transmission and reception process of the fourth FEC packet group of FIG. 15A, the transmitting device (110) can transmit 13 FEC packet groups 4 including 10 data packets and 3 associated FEC packets for a certain period of time.

[0226] In operation 13030, the transmitting device (110) may receive reception error information for the second FEC encoded data from the receiving device (120). The description of operation 13030 may refer to the description of operation 12040. Therefore, any duplicate description will be omitted.

[0227] For example, as illustrated in the transmission and reception process of the second FEC packet group of FIG. 15A, the transmitting device (110) may receive reception error information for FEC packet group 2. The reception error information for FEC packet group 2 may include information indicating the number of packets in which reception errors occurred (e.g., 2) within the FEC packet group 2, or information indicating that the number of packets in which reception errors occurred (e.g., 2) is less than or equal to a preset error threshold value (e.g., 2) (or information indicating that no reception errors were detected).

[0228] For example, as illustrated in the transmission and reception process of the fourth FEC packet group of FIG. 15A, the transmitting device (110) may receive reception error information for FEC packet group 4. The reception error information for FEC packet group 4 may include information indicating the number of packets in which reception errors occurred (e.g., 3) within the FEC packet group 4, or information indicating that the number of packets in which reception errors occurred (e.g., 3) exceeds a preset error threshold value (e.g., 2) (or information indicating that a reception error is detected).

[0229] In operation 13040, the transmitting device (110) may determine, based on reception error information for the second encoded data, whether to change the bit rate of third source data included in third FEC encoded data transmitted after the second FEC encoded data to a second bit rate higher than the first bit rate. The description of operation 13040 may refer to the description of operation 12050. Therefore, any duplicate description will be omitted.

[0230] For example, since the number of packets in which reception errors occurred in the previous second FEC encoding data (e.g., 2) is identified as being less than or equal to an error threshold value (e.g., 2), the transmitting device (110) may determine to increase the bit rate of source data included in the third FEC encoding data to a second bit rate higher than the first bit rate of source data included in the previous second FEC encoding data. For example, since the number of packets in which reception errors occurred in the previous fourth FEC encoding data (e.g., 3) is identified as exceeding an error threshold value (e.g., 2), the transmitting device (110) may determine not to increase the bit rate of source data included in the third FEC encoding data to a second full bit rate higher than the first bit rate of source data included in the second FEC encoding data.

[0231] In operation 13050, if it is determined to change the bit rate of the third source data to a second bit rate higher than the first bit rate, the transmitting device (110) may transmit third FEC encoded data including third source data encoded at the second bit rate and third restoration data for error correction for the third source data to the receiving device (120). According to one embodiment, the size (or FEC rate) of the third restoration data may be the same as the size (or FEC rate) of the basic restoration data (e.g., the size (or FEC rate) of the first restoration data). For example, as illustrated in the transmitting and receiving process of the third FEC packet group of FIG. 15A, the transmitting device (110) may transmit 12 FEC packet groups 3 including 10 data packets and 2 associated FEC packets for a certain period of time. The description of operation 13050 may refer to the description of operation 12060. Therefore, any duplicate description will be omitted.

[0232] In operation 13060, if it is determined not to change the bit rate of the third source data to a second bit rate higher than the first bit rate, the transmitting device (110) may transmit third FEC encoded data including third source data encoded at the first bit rate and third restoration data for error correction for the third source data to the receiving device (120). According to one embodiment, the size (or FEC rate) of the third restoration data may be equal to or greater than the size (or FEC rate) of the second restoration data. For example, as illustrated in the transmitting and receiving process of the fifth FEC packet group of FIG. 15A, the transmitting device (110) may transmit 14 FEC packet groups 5 including 10 data packets and 4 associated FEC packets for a certain period of time. The description of operation 13060 may refer to the description of operation 12070. Therefore, any duplicate description will be omitted.

[0233] FIG. 14 is a flowchart illustrating the operation of a receiving device in a transmission rate adjustment procedure according to one embodiment of the present disclosure.

[0234] The transmission rate adjustment procedure of the embodiment of FIG. 14 may be, for example, a procedure for adjusting (e.g., upwardly adjusting) the bit rate of source data to a target bandwidth according to the first transmission rate adjustment mode. According to one embodiment, the transmission rate adjustment procedure of the embodiment of FIG. 14 may be an example of the transmission rate adjustment procedure of FIG. 12.

[0235] Referring to FIG. 14, in operation 14010, a receiving device (e.g., the receiving device (120) of FIGS. 1 and 3) may receive, from a transmitting device (e.g., the transmitting device (110) of FIGS. 1 and 2), first FEC encoded data including first source data encoded at a first bit rate and first restoration data for error correction for the first source data. For example, as illustrated in the transmitting and receiving process of the first FEC packet group of FIG. 15A, the receiving device (120) may receive 11 FEC packet groups 1 including 9 data packets and 2 associated FEC packets during a certain period of time. As an example, as illustrated in the transmitting and receiving process of the third FEC packet group of FIG. 15A, the receiving device (120) may receive 12 FEC packet groups 3 including 10 data packets and 2 associated FEC packets during a certain period of time. The description of operation 14010 can be referenced from the description of operation 12010. Therefore, any duplicate description is omitted.

[0236] In operation 14020, the receiving device (120) may receive first FEC encoded data including second source data encoded at a first bit rate and second restoration data for error correction for the second source data from the transmitting device (110). According to one embodiment, the size (or FEC rate) of the second restoration data may be larger than the size (or FEC rate) of the first restoration data. For example, as illustrated in the process of transmitting and receiving the second FEC packet group of FIG. 15A, the receiving device (120) may receive twelve FEC packet groups 2 including nine data packets and three associated FEC packets during a certain period of time. As an example, as illustrated in the process of transmitting and receiving the fourth FEC packet group of FIG. 15A, the receiving device (120) may receive thirteen FEC packet groups 4 including ten data packets and three associated FEC packets during a certain period of time. The description of operation 14020 can be referenced from the description of operation 12020. Therefore, any duplicate description is omitted.

[0237] In operation 14030, the receiving device (120) may generate reception error information for the second FEC encoded data. According to one embodiment, the receiving device (120) may identify the number of packets in which reception errors occurred within the second FEC packet group for transmitting the second FEC encoded data, and generate reception error information for the second FEC encoded data based on the number of identified packets in which reception errors occurred. The description of operation 14030 may refer to the description of operation 12030. Therefore, duplicate descriptions are omitted.

[0238] For example, as illustrated in the transmission and reception process of the second FEC packet group of FIG. 15a, due to the limitation of the network bandwidth, two packets are dropped during the transmission process, but since the number of dropped packets is within the number that can be restored by the FEC packets, decoding of the data packets can be successfully performed. The receiving device (120) can identify that the number of packets in which reception errors occurred is 2 and that the decoding (decryption) result is successful. The receiving device (120) can identify the number of packets in which reception errors occurred within the FEC packet group 2 and generate reception error information for the FEC packet group 2 based on the identified number of packets in which reception errors occurred. The reception error information for FEC packet group 2 may include information indicating the number of packets in which reception errors occurred (e.g., 2) within FEC packet group 2, or information indicating that the number of packets in which reception errors occurred (e.g., 2) is less than or equal to a preset error threshold value (e.g., 2) (or information indicating that no reception errors were detected).

[0239] For example, as illustrated in the transmission and reception process of the fourth FEC packet group of FIG. 15a, due to the limitation of the network bandwidth, three packets are dropped during the transmission process, but since the number of dropped packets is within the number that can be restored by the FEC packets, decoding of the data packets can be successfully performed. The receiving device (120) can identify that the number of packets in which reception errors occurred is 3 and that the decoding (decryption) result is successful. The receiving device (120) can identify the number of packets in which reception errors occurred within the FEC packet group 4 and generate reception error information for the FEC packet group 4 based on the identified number of packets in which reception errors occurred. The reception error information for FEC packet group 4 may include information indicating the number of packets in which reception errors occurred (e.g., 3) within FEC packet group 4, or information indicating that the number of packets in which reception errors occurred (e.g., 3) exceeds a preset error threshold value (e.g., 2) (or information indicating that a reception error was detected).

[0240] In operation 14040, the receiving device (120) may transmit reception error information for the second FEC encoded data to the transmitting device (110). The description of operation 14040 may refer to the description of operation 12040. Therefore, any duplicate description will be omitted.

[0241] For example, as illustrated in the transmission and reception process of the second FEC packet group of FIG. 15a, the receiving device (120) can transmit reception error information for FEC packet group 2.

[0242] For example, as illustrated in the transmission and reception process of the fourth FEC packet group of FIG. 15a, the receiving device (120) can transmit reception error information for FEC packet group 4.

[0243] According to one embodiment, before or after operation 14040, the receiving device (120) can display an image based on the decoded data.

[0244] For example, as illustrated in the transmission and reception process of the second FEC packet group of FIG. 15A, the receiving device (120) can display an image based on the decoded data for FEC packet group 2. Since the decoding is successful, the displayed image does not have image breakage or image quality deterioration, and can have a first image quality corresponding to the transmission rate of the data packet of FEC packet group 2. For example, the displayed image can be the first image (610) of FIG. 6A.

[0245] For example, as illustrated in the transmission and reception process of the fourth FEC packet group of FIG. 15A, the receiving device (120) can display an image based on the decoded data for FEC packet group 4. Since the decoding is successful, the displayed image does not have image breakup and can have a second image quality corresponding to the transmission rate of the data packet of FEC packet group 4. For example, since the transmission rate of the data packet of FEC packet group 4 is higher than the transmission rate of the data packet of FEC packet group 2, the displayed image can be the second image (620) of FIG. 6B having a higher image quality than the first image (610) of FIG. 6A. In this way, compared to the case where the second transmission rate adjustment mode is used, when the first transmission rate adjustment mode is used, image breakup or image quality deterioration does not occur even if a reception error (or abnormality) is detected.

[0246] FIG. 15a illustrates an exemplary packet transmission and reception situation according to a transmission rate adjustment procedure according to an embodiment of the present disclosure. FIG. 15b illustrates an exemplary change in the transmission rate of a data packet according to a transmission rate adjustment procedure according to an embodiment of the present disclosure.

[0247] The transmission rate adjustment procedure of the embodiments of FIGS. 15a and 15b may be, for example, a procedure for adjusting the transmission rate of a data packet to a target bandwidth according to a first transmission rate adjustment mode (e.g., the transmission rate adjustment procedure of FIG. 9 or FIG. 12).

[0248] In the embodiments of FIGS. 15a and 15b, it is assumed that the transmission rate is the number of packets transmitted over a given period of time, and the network bandwidth and the target bandwidth are, for example, a bandwidth capable of processing 10 packets over a given period of time (e.g., 10 Mbps) and a bandwidth capable of processing 20 packets over a given period of time (e.g., 20 Mbps), respectively.

[0249] According to one embodiment, each FEC packet group of FIG. 15A may correspond to a respective frame. For example, FEC packet group 1 may correspond to the first frame, and FEC packet group 2 transmitted after FEC packet group 1 may correspond to the second frame following the first frame.

[0250] When explaining the packet transmission and reception process of the first FEC packet group of FIG. 15a (e.g., the FEC packet group corresponding to the first frame), a transmitting device (e.g., the transmitting device (110) of FIGS. 1 and 2) can transmit 11 FEC packet groups 1 including 9 data packets and 2 associated FEC packets for a certain period of time (e.g., for 1 / 60 second of 60 fps), and a receiving device (e.g., the receiving device (120) of FIGS. 1 and 3) can receive FEC packet group 1. In this case, due to the limitation of network bandwidth, one packet may be dropped during the transmission process. However, since the number of dropped packets is within the number that can be restored by the FEC packets, decoding of the data packets can be successfully performed. Therefore, the receiving device (120) can identify that the number of packets in which a reception error occurred is 1 and that the decoding (decoding) result is successful. The receiving device (120) may transmit reception error information for FEC packet group 1 to the transmitting device. The reception error information for FEC packet group 1 may include information indicating that the number of packets in which reception errors occurred within FEC packet group 1 is 1 or information indicating that the number of packets in which reception errors occurred is less than or equal to a preset error threshold value (or information indicating that no reception errors were detected). In addition, the receiving device (120) may display an image based on data decoded before or after transmitting the reception error information. For example, the receiving device (120) may display an image based on decoded data for FEC packet group 1. Since the decoding is successful, the displayed image does not have image breakage or image quality deterioration, and may have a first image quality corresponding to the transmission rate of the data packets of FEC packet group 1. For example, the displayed image may be the first image (610) of FIG. 6A.

[0251] When explaining the packet transmission and reception process of the second FEC packet group of FIG. 15a (e.g., the FEC packet group corresponding to the second frame following the first frame), since the target bandwidth has not yet been reached, the transmitting device (110) first increases the transmission rate of the FEC packets in order to determine whether to increase the transmission rate of the data packets. For example, as illustrated in FIG. 15a, the transmitting device (110) increases the number of FEC packets transmitted for a certain period of time (e.g., for 1 / 60 second of 60 fps) by 1 compared to the previous FEC packet group 1 in order to increase the transmission rate of the FEC packets. At this time, the transmitting device can maintain the transmission rate of the data packets. Through this, the FEC recovery rate can be increased as much as the transmission rate of the entire packets is increased. For example, a transmitting device (110) may transmit 12 FEC packet groups 2, each containing 9 data packets and 3 associated FEC packets, over a certain period of time, and a receiving device (120) may receive FEC packet groups 2. In this case, due to limitations in network bandwidth, 2 packets may be dropped during the transmission process. However, since the number of dropped packets is within the number that can be restored by FEC packets, decoding of the data packets may be successfully performed. Accordingly, the receiving device (120) may identify that the number of packets in which reception errors occurred is 2 and that the decoding (decoding) result is successful. The receiving device (120) may transmit reception error information for FEC packet group 2 to the transmitting device. The reception error information for FEC packet group 2 may include information indicating that the number of packets in which reception errors occurred within FEC packet group 2 is 2, or information indicating that the number of packets in which reception errors occurred is less than or equal to a preset error threshold value (or information indicating that no reception errors were detected). In addition, the receiving device (120) may display an image based on data decoded before or after transmitting the reception error information.For example, the receiving device (120) can display an image based on the decoded data for FEC packet group 2. Since the decoding is successful, the displayed image does not have image breakage or image quality deterioration, and can have a first image quality corresponding to the transmission rate of the data packets of FEC packet group 2. For example, since the transmission rate of the data packets of FEC packet group 2 and the transmission rate of the data packets of FEC packet group 1 are the same, the displayed image can be the first image (610) of FIG. 6A.

[0252] When explaining the transmission and reception process of the packets of the third FEC packet group of FIG. 15a (e.g., the FEC packet group corresponding to the third frame following the second frame), since the target bandwidth has not yet been reached and it is identified that the number of packets in which reception errors have occurred in the previous FEC packet group, FEC packet group 2, is less than or equal to the error threshold value, the transmitting device (110) increases the transmission rate of the data packets. For example, as illustrated in FIG. 15a, the transmitting device (110) increases the number of data packets transmitted for a certain period of time (e.g., for 1 / 60 second of 60 fps) by 1 compared to the FEC packet group 2 in order to increase the transmission rate of the data packets. At this time, the transmission rate of the FEC packets can be set to a basic transmission rate (e.g., a transmission rate corresponding to 2 of the number of data packets transmitted for a certain period of time). Through this, the transmission rate of the data packets can be increased only by the amount of spare bandwidth identified based on the reception error information for the previous FEC packet group 2. For example, a transmitting device (110) can transmit 12 FEC packet groups 3, each containing 10 data packets and 2 associated FEC packets, over a certain period of time, and a receiving device (120) can receive FEC packet groups 3. In this case, due to limitations in network bandwidth, 2 packets may be dropped during the transmission process. However, since the number of dropped packets is within the number that can be restored by FEC packets, decoding of the data packets can be successfully performed. Accordingly, the receiving device (120) can identify that the number of packets in which reception errors occurred is 2 and that the decoding (decoding) result is successful. The receiving device (120) can transmit reception error information for FEC packet group 3 to the transmitting device (110).The reception error information for FEC packet group 3 may include information indicating that the number of packets in which reception errors occurred within FEC packet group 3 is 2, or information indicating that the number of packets in which reception errors occurred is less than or equal to a preset error threshold value (or information indicating that no reception errors were detected). In addition, the receiving device (120) may display an image based on data decoded before or after transmitting the reception error information. As an example, the receiving device (120) may display an image based on decoded data for FEC packet group 3. Since the decoding is successful, the displayed image does not have image breakage or image quality deterioration, and may have a second image quality corresponding to the transmission rate of the data packets of FEC packet group 3. Since the transmission rate of the data packets of FEC packet group 3 is higher than the transmission rate of the data packets of FEC packet group 2 (or FEC packet group 1), the displayed image may be a second image (620) with a higher image quality than, for example, the first image (610) of FIG. 6A.

[0253] When explaining the transmission and reception process of the packets of the fourth FEC packet group of FIG. 15a (e.g., the FEC packet group corresponding to the fourth frame following the third frame), since the target bandwidth has not yet been reached, the transmitting device (110) first increases the transmission rate of the FEC packets in order to determine whether to increase the transmission rate of the data packets. For example, as illustrated in FIG. 15a, the transmitting device (110) increases the number of FEC packets transmitted for a certain period of time (e.g., for 1 / 60 second of 60 fps) by 1 compared to the previous FEC packet group 3 in order to increase the transmission rate of the FEC packets. At this time, the transmitting device (110) can maintain the transmission rate of the data packets. Through this, the FEC recovery rate can be increased as much as the transmission rate of the entire packets is increased. For example, a transmitting device may transmit 13 FEC packet groups 4, each containing 10 data packets and 3 associated FEC packets, over a certain period of time, and a receiving device (120) may receive FEC packet groups 4. In this case, due to limitations in network bandwidth, 3 packets may be dropped during the transmission process. However, since the number of dropped packets is within the number that can be restored by the FEC packets, decoding of the data packets may be successfully performed. Accordingly, the receiving device (120) may identify that the number of packets in which reception errors occurred is 3 and that the decoding (decoding) result is successful. However, the receiving device (120) may identify that a reception error (or anomaly) is detected because the number of packets in which reception errors occurred exceeds a preset error threshold value. The receiving device (120) may transmit reception error information for FEC packet groups 4 to the transmitting device (110).The reception error information for FEC packet group 4 may include information indicating that the number of packets in which reception errors occurred within FEC packet group 4 is 3, or information indicating that the number of packets in which reception errors occurred exceeds a preset error threshold value (or information indicating that a reception error is detected). In addition, the receiving device (120) may display an image based on data decoded before or after transmitting the reception error information. As an example, the receiving device (120) may display an image based on decoded data for FEC packet group 4. Since the decoding is successful, the displayed image does not have image breakage or image quality deterioration, and may have a second image quality corresponding to the transmission rate of the data packets of FEC packet group 4. For example, the displayed image may be the second image (620) of FIG. 6A.

[0254] When explaining the transmission and reception process of the packets of the fifth FEC packet group of FIG. 15a (e.g., the FEC packet group corresponding to the fifth frame following the fourth frame), although the target bandwidth has not yet been reached, since it is identified that the number of packets in which reception errors have occurred in the previous FEC packet group, FEC packet group 4, exceeds the error threshold value, the transmitting device (110) can maintain the transmission rate of the data packet, increase the transmission rate of the FEC packet, or keep it the same. For example, as illustrated in FIG. 15a, the transmitting device (110) can adjust the transmission rate of the data packet to a transmission rate that is higher than the transmission rate of the previous FEC packet group 4. For example, as illustrated in FIG. 15a, the transmitting device increases the number of FEC packets transmitted for a certain period of time (e.g., for 1 / 60 seconds of 60 fps) by 1 compared to the previous FEC packet group 4 in order to increase the transmission rate of the FEC packet. At this time, the transmission rate of the data packet can be maintained. Through this, the FEC recovery rate can be increased as much as the transmission rate of the entire packet is increased. For example, the transmitting device (110) can transmit 14 FEC packet groups 5 including 10 data packets and 4 associated FEC packets for a certain period of time, and the receiving device (120) can receive the FEC packet group 5. In this case, due to the limitation of the network bandwidth, 4 packets may be dropped during the transmission process. However, since the number of dropped packets is within the number that can be restored by the FEC packets, decoding of the data packets can be successfully performed. Accordingly, the receiving device (120) can identify that the number of packets in which reception errors occurred is 4 and that the decoding (decoding) result is successful. The receiving device (120) can transmit reception error information for the FEC packet group 5 to the transmitting device.The reception error information for FEC packet group 5 may include information indicating that the number of packets in which reception errors occurred within FEC packet group 5 is 4, or information indicating that the number of packets in which reception errors occurred exceeds a preset error threshold value (or information indicating that a reception error is detected). In addition, the receiving device (120) may display an image based on data decoded before or after transmitting the reception error information. For example, the receiving device (120) may display an image based on decoded data for FEC packet group 5. Since the decoding is successful, the displayed image does not have image breakage or image quality degradation, and may have a second image quality corresponding to the transmission rate of the data packets of FEC packet group 5. For example, the displayed image may be the second image (620) of FIG. 6A.

[0255] When explaining the transmission and reception process of the packets of the sixth FEC packet group of FIG. 15a (e.g., the FEC packet group corresponding to the sixth frame following the fifth frame), if the target bandwidth has not yet been reached, but decoding failures continue to occur (e.g., if a preset number (e.g., 3) or more occur), the transmitting device (110) may lower the transmission rate of the data packets. For example, as illustrated in FIG. 15a, the transmitting device (110) may adjust the transmission rate of the data packets to a basic data transmission rate (e.g., the transmission rate of the previous FEC packet group 1). As an example, the transmitting device (110) may transmit 11 FEC packet groups 6 including 9 data packets and 2 associated FEC packets for a certain period of time, and the receiving device (120) may receive the FEC packet group 6. In this case, due to the limitation of the network bandwidth, one packet may be dropped during the transmission process. However, since the number of dropped packets is within the number that can be restored by FEC packets, decoding of data packets cannot be successfully performed. Therefore, the receiving device (120) can identify that the number of packets in which reception errors occurred is 1 and that the decoding (decoding) result is successful. The receiving device (120) can transmit reception error information for FEC packet group 6 to the transmitting device (110). The reception error information for FEC packet group 4 can include information indicating that the number of packets in which reception errors occurred within FEC packet group 6 is 1 or information indicating that the number of packets in which reception errors occurred is less than a preset error threshold value (or information indicating that no reception errors were detected). In addition, the receiving device (120) can display an image based on decoded data before or after transmitting the reception error information. As an example, the receiving device (120) can display an image based on decoded data for FEC packet group 6.Since decoding is successful, the displayed image does not have image breakage or image quality degradation, and can have the first image quality corresponding to the transmission rate of the data packet of FEC packet group 6. For example, the displayed image can be the first image (610) of FIG. 6A.

[0256] As in the example described above, when adjusting the transmission rate according to the first transmission rate adjustment mode, if it is determined that the target bandwidth is not reached, the transmitting device preferentially increases the transmission rate of the FEC packet (or the size of the restored data or the FEC rate) and maintains the transmission rate of the data packet (or the bit rate of the source data). Through this, the FEC restoration rate can increase as much as the transmission rate of the entire packet increases. In this case, even if a larger number of packets are dropped during the transmission process due to limitations in the network bandwidth, the receiving device can restore the dropped data packets using the FEC packets, so that decoding failure does not occur. In addition, image breakup due to decoding failure does not occur. For example, as illustrated in the solid boxes in FIG. 15b, the transmitting device does not generate jitter for the data packets even in a section where the transmission rate of the entire packet is higher than the network bandwidth. Therefore, since a decrease in the transmission rate of the data packet (or the bit rate of the source data) does not occur, the image quality of the image displayed on the receiving device does not deteriorate. This allows the transmission rate of data packets to be adjusted to the target bandwidth without video breakage or image quality degradation.

[0257] FIG. 16 illustrates a procedure for setting a transmission rate adjustment mode according to one embodiment of the present disclosure.

[0258] In one embodiment, the procedure for setting the transmission rate adjustment mode may be a procedure used when multiple transmission rate modes are selectively set. For example, in one embodiment, the procedure for setting the transmission rate adjustment mode may be a procedure used when multiple transmission rate modes are selectively set based on user input.

[0259] According to one embodiment, a transmitting device (e.g., transmitting device (110) of FIGS. 1 and 2) can adjust (e.g., increase) the transmission rate of a data packet to a target bandwidth based on a transmission rate adjustment mode. According to one embodiment, the transmission rate adjustment mode can be set to, for example, one of a first transmission rate adjustment mode or a second transmission rate adjustment mode.

[0260] According to one embodiment, the first transmission rate adjustment mode may be a transmission rate adjustment mode including an operation of determining whether to increase the transmission rate of a data packet (or a bit rate of source data) for a next FEC packet group (or a next FEC encoded data) by using information related to a reception error (hereinafter, reception error information) for an FEC packet group (or FEC encoded data) of a receiving device. For example, the first transmission rate adjustment mode may be a transmission rate adjustment mode including an operation of determining whether to increase the transmission rate of a data packet (or a bit rate of source data) for a next FEC packet group (or a next FEC encoded data) based on reception error information of the receiving device due to the transmitting device first increasing the transmission rate of an FEC packet (or the FEC rate of restored data, or the FEC size of restored data) for the FEC packet group (or the FEC encoded data) and transmitting it. The transmission rate adjustment procedure based on the first transmission rate adjustment mode may be, for example, a transmission rate adjustment procedure described with reference to FIGS. 9 to 15.

[0261] According to one embodiment, the second rate adjustment mode may be a rate adjustment mode including an operation in which the transmitting device preferentially increases the rate of a data packet (or the bit rate of the source data) for a next FEC packet group (or the next FEC encoded data) without using reception error information about the FEC packet group (or the FEC encoded data) of the receiving device. For example, the rate adjustment mode may include an operation in which the transmitting device preferentially increases the rate of a data packet (or the bit rate of the source data) for a next FEC packet group (or the next FEC encoded data) without using reception error information due to the transmitting device transmitting the FEC packet with preferentially increased rate (or the FEC rate of the restored data, or the FEC size of the restored data) for the FEC packet group (or the FEC encoded data). The rate adjustment procedure based on the second rate adjustment mode may be, for example, the rate adjustment procedure described with reference to FIGS. 8A and 8B.

[0262] Referring to FIG. 16, in operation 16010, the receiving device (120) may obtain information for setting a resolution associated with a target bandwidth. According to one embodiment, the receiving device (120) may obtain information for setting a resolution associated with a target bandwidth based on a user input. The resolution may be, for example, one of HD (high definition), FHD (full HD), UHD (ultra HD), 8K, or 4K, but is not limited thereto.

[0263] In operation 16020, the receiving device (120) can transmit information for setting the resolution to the transmitting device (110).

[0264] In operation 16030, the transmitting device (130) may set the resolution and target bandwidth based on the information for setting the resolution. According to one embodiment, the transmitting device (130) may set the resolution to one of HD, FHD, UHD, 8K, or 4K based on the information for setting the resolution, and may set the target bandwidth corresponding to the set resolution. For example, as the resolution increases, the target bandwidth may be set to a higher value.

[0265] Depending on the embodiment, operations 16010 to 16030 may be omitted. For example, the transmitting device (120) may independently set the resolution and the target bandwidth corresponding to the resolution, regardless of the user input of the receiving device (110).

[0266] In operation 16040, the receiving device (120) may obtain information for setting a transmission rate adjustment mode. In one embodiment, the receiving device (120) may obtain information for setting a transmission rate adjustment mode based on a user input. An example of operation 16040 is described below with reference to FIGS. 17 and 18 .

[0267] In operation 16050, the receiving device (120) can transmit information for setting a transmission rate adjustment mode to the transmitting device (110).

[0268] In operation 16060, the transmitting device (110) may set the transmission rate adjustment mode based on the information for setting the transmission rate adjustment mode. According to one embodiment, the transmitting device (110) may set the transmission rate adjustment mode to either the first transmission rate adjustment mode or the second transmission rate adjustment mode based on the information for setting the transmission rate adjustment mode.

[0269] Depending on the embodiment, operations 16040 and 16050 may be omitted. For example, the transmitting device (110) may set the transmission rate adjustment mode independently of user input, or may use only the default transmission rate adjustment mode.

[0270] In operation 16070, the transmitting device (110) may transmit a group of FEC packets (or FEC encoded data) including processed data packets and FEC packets based on the set transmission rate adjustment mode. For example, when the transmission rate adjustment mode is set to the first transmission rate adjustment mode, the transmitting device (110) may transmit data packets and FEC packets (or source data and recovery data) according to the transmission rate adjustment procedure described above in FIGS. 9 to 15, for example. For example, when the transmission rate adjustment mode is set to the second transmission rate adjustment mode, the transmitting device (110) may transmit data packets and FEC packets (or source data and recovery data) according to the transmission rate adjustment procedure described above in FIGS. 8A to 8B, for example.

[0271] FIG. 17 illustrates operations of a receiving device in a procedure for setting a transmission rate adjustment mode according to one embodiment of the present disclosure. FIG. 18 illustrates an example of a user interface provided by a receiving device for setting a transmission rate adjustment mode according to one embodiment of the present disclosure.

[0272] The operation of the receiving device of the embodiment of FIG. 17 may be an example of operation 16040 of the procedure of FIG. 16.

[0273] Referring to FIG. 17, in operation 1710, a receiving device (e.g., receiving device (120) of FIGS. 1 and 3) may display a first user interface for setting a transmission rate adjustment mode. According to one embodiment, the receiving device (120) may display a first screen (1810) including a first user interface (1812) (e.g., an icon) for setting a transmission rate adjustment mode, as illustrated in FIG. 18. The first screen (1810) may further include a user interface (1811) (e.g., an icon) for setting a resolution.

[0274] In operation 1720, a receiving device (e.g., receiving device (120) of FIGS. 1 and 3) may obtain a first user input for selecting a first user interface. In one embodiment, the receiving device (120) may receive a first user input (1813) (e.g., input via a touch input or another device (e.g., a remote control)) for selecting a first user interface (1812) (e.g., an icon) for setting a transmission rate adjustment mode within a first screen (1810), as illustrated in FIG. 18.

[0275] In operation 1730, the receiving device (120) may display a second user interface corresponding to the first transmission rate adjustment mode and a third user interface corresponding to the second transmission rate adjustment mode in response to obtaining the first user input. According to one embodiment, the receiving device (120) may display a second screen (1820) including a second user interface (1821) (e.g., an icon) corresponding to the first transmission rate adjustment mode and a third user interface (1822) (e.g., an icon) corresponding to the second transmission rate adjustment mode, as illustrated in FIG. 18.

[0276] In operation 1740, the receiving device (e.g., the receiving device (120) of FIGS. 1 and 3) may obtain a second user input selecting one of the second user interface or the third user interface. In one embodiment, the receiving device (120) may receive a second user input (1823) (e.g., an input via a touch input or another device (e.g., a remote control)) selecting a second user interface (1821) corresponding to the first transmission rate adjustment mode within the second screen (1820), as illustrated in FIG. 18.

[0277] Through this, the receiving device (120) can confirm that the first transmission rate adjustment mode has been selected by the user, and can transmit information about the selected first transmission rate adjustment mode to the transmitting device (110). The transmitting device (110) can set the transmission rate adjustment mode to the first transmission rate adjustment mode based on the received information. Thereafter, the transmitting device (110) can perform a transmission procedure of a data packet and an FEC packet (e.g., a transmission rate adjustment procedure of FIG. 9 or 12) based on the set first transmission rate adjustment mode.

[0278] FIG. 19 illustrates a method of a transmitting device according to one embodiment of the present disclosure.

[0279] In Fig. 19, any description that overlaps with the contents described above in Figs. 1 to 18 is omitted.

[0280] Referring to FIG. 19, a transmitting device (e.g., a transmitting device (110) of FIGS. 1 and 2) can transmit first FEC encoded data (first FEC encoded data) including first source data encoded at a first bit rate and first restoration data for error correction for the first source data to a receiving device (e.g., a receiving device (120) of FIGS. 1 and 3) (19010).

[0281] The transmitting device can receive information related to a reception error for the first FEC encoded data from the receiving device (19020).

[0282] The transmitting device may determine whether to change the bit rate of the second source data to a second bit rate higher than the first bit rate based on information related to the reception error (19030). According to one embodiment, the second source data may be included in second FEC encoded data transmitted after the first FEC encoded data is transmitted.

[0283] According to one embodiment, the transmitting device may transmit, to the receiving device, third FEC encoded data including third source data encoded at a bit rate equal to the first bit rate and third restoration data for error correction for the third source data, before the first FEC encoded data is transmitted. According to one embodiment, the size of the first restoration data may be larger than the size of the third restoration data. According to one embodiment, the size of the first restoration data may be set to a value such that a portion of the first source data in which a reception error occurs can be successfully restored using at least a portion of the first restoration data.

[0284] According to one embodiment, the transmitting device may determine, based on information related to the reception error for the first FEC encoded data, whether the number of packets in which reception errors occur within a first packet group for transmitting the first FEC encoded data is less than or equal to a preset error threshold value, and, if the number of packets in which reception errors occur is determined to be less than or equal to the preset error threshold value, determine to change the bit rate of the second source data to a second bit rate higher than the first bit rate. According to one embodiment, the first packet group may include at least one first data packet including the first source data and at least one first FEC packet including the first restored data.

[0285] According to one embodiment, the transmitting device may determine to maintain the bit rate of the second source data at the first bit rate if the number of packets in which reception errors occur is identified to exceed a preset error threshold value.

[0286] According to one embodiment, information related to reception errors for the first FEC encoded data may include first information indicating the number of packets in which reception errors occurred or second information indicating whether the number of packets in which reception errors occurred is less than or equal to a preset error threshold value.

[0287] According to one embodiment, when it is determined that the bit rate of the second source data is changed to a second bit rate higher than the first bit rate, the transmitting device may transmit second FEC encoded data including the second source data encoded at the second bit rate and the second restored data to the receiving device (19040). According to one embodiment, the size of the second restored data may be smaller than the size of the first restored data.

[0288] According to one embodiment, when it is determined that the bit rate of the second source data is maintained at the first bit rate, the transmitting device may transmit second FEC encoded data including second source data encoded at the first bit rate and second restored data to the receiving device (19050). According to one embodiment, the size (or FEC rate) of the second restored data may be equal to or greater than the size (or FEC rate) of the first restored data.

[0289] According to one embodiment, the first FEC encoding data may be FEC encoding data for a first frame, the second FEC encoding data may be FEC encoding data for a second frame following the first frame, and the third FEC encoding data may be FEC encoding data for a second frame preceding the first frame.

[0290] According to one embodiment, an FEC code rate used to generate restoration data by performing FEC encoding on source data is set per frame, and the size of the generated restoration data can be determined based on the FEC code rate.

[0291] According to one embodiment, the number of at least one first data packet containing the first source data is equal to the number of at least one third data packet containing the third source data, the number of at least one first FEC packet containing the first restored data is greater than the number of at least one third FEC packet containing the third restored data by a first value, and the number of at least one second data packet containing the second source data encoded at a second bit rate is greater than the number of at least one first data packet containing the first source data by the first value.

[0292] According to one embodiment, the first data packet or the header of the first data packet includes FEC-related information, and the FEC-related information may include at least one of information indicating the total number of packets included in a first packet group to which the first data packet belongs, information indicating the number of at least one first data packet, or information indicating the number of at least one first FEC packet.

[0293] In one embodiment, the transmitting device may utilize an adaptive bit rate scheme.

[0294] According to one embodiment, the transmitting device can identify, prior to transmitting the first FEC encoded data, that the first bit rate is lower than the maximum configurable bit rate within the target bandwidth.

[0295] According to one embodiment, the transmitting device may receive, from the receiving device, configuration information for setting a transmission rate adjustment mode to the first transmission rate adjustment mode among a plurality of transmission rate adjustment modes, including a first transmission rate adjustment mode and a second transmission rate adjustment mode, before transmitting the first FEC encoded data, and may set the transmission rate adjustment mode to the first transmission rate adjustment mode based on the configuration information. According to one embodiment, the first transmission rate adjustment mode may be a transmission rate adjustment mode including an operation in which the transmitting device determines whether to increase a bit rate of source data in the next FEC encoded data by using information related to a reception error for FEC encoded data received from the receiving device, and the second transmission rate adjustment mode may be a transmission rate adjustment mode including an operation in which the transmitting device first increases a bit rate of source data in the next FEC encoded data without using information related to a reception error for FEC encoded data received from the receiving device.

[0296] FIG. 20 illustrates a method of a receiving device according to one embodiment of the present disclosure.

[0297] In Fig. 20, descriptions that overlap with those described above in Figs. 1 to 19 are omitted.

[0298] Referring to FIG. 20, a receiving device (e.g., receiving device (120) of FIG. 1,3) can receive first FEC encoded data (first FEC encoded data) including first source data encoded at a first bit rate and first restoration data for error correction for the first source data from a transmitting device (e.g., transmitting device (110) of FIG. 1,3) (20010).

[0299] The receiving device can generate information related to reception errors for the first FEC encoded data (20020).

[0300] The receiving device is a transmitting device and can transmit information related to a reception error for the first FEC encoded data (20030).

[0301] The receiving device may receive, from the transmitting device, second FEC encoded data including second source data encoded at a second bit rate and second restoration data for error correction of the second source data. According to one embodiment, the second bit rate of the second source data may be determined based on information related to a reception error.

[0302] According to one embodiment, before the first FEC encoded data is received, the receiving device can receive third FEC encoded data from the transmitting device, which includes third source data encoded at a bit rate equal to the first bit rate and third restoration data for error correction for the third source data.

[0303] According to one embodiment, information related to a reception error for the first FEC encoded data may include first information indicating the number of packets in which a reception error occurred within a first packet group for transmitting the first FEC encoded data, or second information indicating whether the number of packets in which a reception error occurred is less than or equal to a preset error threshold value.

[0304] According to one embodiment, before receiving the first FEC encoded data, the receiving device may display a first user interface for setting a transmission rate adjustment mode, obtain a first user input for selecting the first user interface, and in response to obtaining the first user input, display a second user interface corresponding to the first transmission rate adjustment mode and a third user interface corresponding to the second transmission rate adjustment mode, obtain a second user input for selecting the second user interface from among the second user interface or the third user interface, and in response to obtaining the second user input, transmit information for setting the transmission rate adjustment mode to the first transmission rate adjustment mode to the transmitting device. According to one embodiment, the first transmission rate adjustment mode may be a transmission rate adjustment mode including an operation in which the transmitting device determines whether to increase a bit rate of source data in the next FEC encoded data by using information related to a reception error for FEC encoded data received from the receiving device, and the second transmission rate adjustment mode may be a transmission rate adjustment mode including an operation in which the transmitting device first increases a bit rate of source data in the next FEC encoded data without using information related to a reception error for FEC encoded data received from the receiving device.

[0305] In the specific embodiments of the present disclosure described above, components included in the present disclosure are expressed in the singular or plural form, depending on the specific embodiment presented. However, the singular or plural expressions are selected to suit the presented situation for convenience of explanation, and the present disclosure is not limited to singular or plural components. Components expressed in the plural form may be composed of singular elements, or components expressed in the singular form may be composed of plural elements.

[0306] While the detailed description of this disclosure has described specific embodiments, it should be understood that various modifications are possible without departing from the scope of this disclosure. Therefore, the scope of this disclosure should not be limited to the described embodiments, but should be defined not only by the scope of the claims described below, but also by equivalents thereof.

Claims

1. In a transmission device within an image processing system, communication circuit; A memory storing a program containing at least one instruction; and At least one processor connected to the communication circuit and the memory and executing at least one instruction of a program stored in the memory, wherein the at least one processor: Transmitting, to a receiving device, first FEC (forward error correction) encoded data including first source data encoded at a first bit rate and first repair data used for error correction of the first source data, Receive information related to a reception error for the first FEC encoded data from the receiving device, A transmitting device, wherein the transmitting device is configured to determine whether to change the bit rate of the second source data to a second bit rate higher than the first bit rate based on information related to the reception error, wherein the second source data is included in second FEC encoded data transmitted after the first FEC encoded data is transmitted.

2. In paragraph 1, at least one processor: Before the first FEC encoded data is transmitted, third FEC encoded data is transmitted to the receiving device, The third FEC encoded data includes third source data encoded at a bit rate identical to the first bit rate and third restoration data used for error correction for the third source data. A transmitting device, wherein the size of the first restoration data is larger than the size of the third restoration data.

3. In paragraph 2, A transmitting device, wherein the size of the first restoration data corresponds to a value that allows a portion of the first source data in which a reception error occurs to be successfully restored using at least a portion of the first restoration data.

4. In paragraph 1, at least one processor: Based on the information related to the reception error for the first FEC encoded data, identifying whether the number of packets in which a reception error occurs within the first packet group for transmitting the first FEC encoded data is less than or equal to a preset error reference value; Based on identifying that the number of packets in which the reception error occurred is less than or equal to the preset error threshold value, it is determined to change the bit rate of the second source data to the second bit rate higher than the first bit rate, A transmitting device, wherein the first packet group includes at least one first data packet including the first source data and at least one first FEC packet including the first restoration data.

5. In paragraph 4, at least one processor: A transmitting device, which determines to maintain the bit rate of the second source data at the first bit rate based on identifying that the number of packets in which the reception error occurred exceeds the preset error criterion value.

6. In paragraph 4, A transmitting device, wherein information related to a reception error for the first FEC encoded data includes first information indicating the number of packets in which the reception error occurred or second information indicating whether the number of packets in which the reception error occurred is less than or equal to a preset error reference value.

7. In paragraph 4, at least one processor: Based on a decision to change the bit rate of the second source data to the second bit rate higher than the first bit rate, the second FEC encoded data including the second source data encoded at the second bit rate and the second restored data is transmitted to the receiving device, A transmitting device, wherein the size of the second restoration data is smaller than the size of the first restoration data.

8. In paragraph 5, the at least one processor: A transmitting device, based on a decision to maintain the bit rate of the second source data at the first bit rate, transmits the second FEC encoded data including the second source data encoded at the first bit rate and the second restored data to the receiving device, wherein the size of the second restored data is equal to or greater than the size of the first restored data.

9. In the second paragraph, the first FEC encoding data is FEC encoding data for the first frame, The above second FEC encoding data is FEC encoding data for a second frame following the first frame, The above third FEC encoding data is FEC encoding data for the second frame preceding the first frame, A transmitting device, wherein an FEC code rate used to generate restoration data by performing FEC encoding on source data is set for each frame, and the size of the generated restoration data is determined based on the FEC code rate.

10. In paragraph 2, The number of at least one first data packet containing the first source data is equal to the number of at least one third data packet containing the third source data, The number of at least one first FEC packet containing the first restoration data is greater than the number of at least one third FEC packet containing the third restoration data by a first value, The number of at least one second data packet containing the second source data encoded at the second bit rate is greater than the number of at least one first data packet containing the first source data by the first value, The first data packet or the header of the first data packet includes FEC related information, A transmitting device, wherein the FEC-related information includes at least one of information indicating the total number of packets included in the first packet group to which the first data packet belongs, information indicating the number of the at least one first data packet, or information indicating the number of the at least one first FEC packet.

11. In paragraph 1, A transmitting device, wherein the transmitting device is set to use an adaptive bitrate method.

12. In paragraph 1, at least one processor: A transmitting device that identifies, before the first FEC encoded data is transmitted, that the first bit rate is lower than the maximum settable bit rate within the target bandwidth.

13. In paragraph 1, at least one processor: Before transmitting the first FEC encoded data, setting information for setting the transmission rate adjustment mode to the first transmission rate adjustment mode among a plurality of transmission rate adjustment modes including the first transmission rate adjustment mode and the second transmission rate adjustment mode is received from the receiving device, Based on the above setting information, the transmission rate adjustment mode is set to the first transmission rate adjustment mode, The first transmission rate adjustment mode includes an operation in which the transmitting device uses information related to a reception error for FEC encoded data received from the receiving device to determine whether to increase the bit rate of source data in the next FEC encoded data. A transmitting device, wherein the second transmission rate adjustment mode includes an operation in which the transmitting device first increases the bit rate of source data in the next FEC encoded data without using information related to a reception error for FEC encoded data received from the receiving device.

14. In a receiving device within an image processing system, communication circuit; A memory storing a program containing at least one instruction; and At least one processor connected to the communication circuit and the memory and executing at least one instruction of a program stored in the memory, wherein the at least one processor: Receive, from a transmitting device, first FEC (forward error correction) encoded data including first source data encoded at a first bit rate and first repair data used for error correction of the first source data, Generate information related to a reception error for the above first FEC encoded data, To the above transmitting device, information related to a reception error for the first FEC encoded data is transmitted, is configured to receive, from the transmitting device, second FEC encoded data including second source data encoded at a second bit rate and second restoration data used for error correction for the second source data; A receiving device, wherein the second bit rate of the second source data is determined based on information related to the reception error.

15. In a method of a transmission device in an image processing system, An operation of transmitting, to a receiving device, first FEC (forward error correction) encoded data including first source data encoded at a first bit rate and first repair data used for error correction of the first source data; An operation of receiving information related to a reception error for the first FEC encoded data from the receiving device; and A method comprising: an operation of determining whether to change a bit rate of second source data to a second bit rate higher than the first bit rate based on information related to the reception error, wherein the second source data is included in second FEC encoded data transmitted after the first FEC encoded data is transmitted.

Images