MIRACAST end-to-end (E2E) streaming

Abstract

Various embodiments include systems and methods for supporting # imgabs0 # edge-to-edge (E2E) streaming. Various embodiments provide systems and methods for # imgabs1 # E2E streaming for a partial frame region. Some embodiments include a method for supporting mirroring projection of video content in a session established between a source wireless device and a sink wireless device.

Description

Background Technology

[0001] Long Term Evolution (LTE), 5G New Radio (NR), and other newly developed communication technologies allow wireless devices to transmit information at data rates several orders of magnitude higher (e.g., in gigabits per second) than were available just a few years ago.

[0002] Today's communication networks are also more secure, resistant to multipath fading, allow for lower network traffic latency, and provide better communication efficiency (e.g., measured in bits per second per unit of bandwidth). These and other recent improvements have enabled the emergence of the Internet of Things (IoT), large-scale machine-to-machine (M2M) communication systems, autonomous vehicles, and other technologies that rely on continuous and secure communication.

[0003] It allows users to Technical standards for wirelessly sharing multimedia (including high-resolution images and high-definition (HD) video content) between devices. For example, Enabled to pass Network connection or (WFD) connections enable the display of multimedia between wireless devices. Summary of the Invention

[0004] All aspects including support Systems and methods for edge-to-edge (E2E) stream transmission. Various aspects are provided for partial frame regions. Systems and methods for E2E streaming. Some aspects include methods for supporting mirroring of video content in a session established between a source wireless device and a destination wireless device.

[0005] The methods include a method for supporting mirroring of video content in a session established between a source wireless device and a destination wireless device, the method comprising: in response to establishing a session with the destination wireless device for mirroring video content, generating and encoding a plain fill color frame by a processor of the source wireless device; transmitting the encoded plain fill color frame to the destination wireless device by the processor of the source wireless device; determining a region of interest in valid frames by the processor of the source wireless device; transmitting an indication of the region of interest in valid frames to the destination wireless device by the processor of the source wireless device; generating and encoding video content of only the region of interest in valid frames as a partial frame by the processor of the source wireless device; and transmitting the partial frame to the destination wireless device by the processor of the source wireless device.

[0006] Some aspects may also include: the processor of the source wireless device determining that a change has occurred in the valid frame region of interest; in response to determining that the change has occurred in the valid frame region of interest, the processor of the source wireless device generating and encoding a new plain fill color frame; the processor of the source wireless device transmitting the encoded new plain fill color frame to the destination wireless device; the processor of the source wireless device determining a new valid frame region of interest; the processor of the source wireless device transmitting an indication of the new valid frame region of interest to the destination wireless device; the processor of the source wireless device generating and encoding only the video content of the new valid frame region of interest as a new partial frame; and the processor of the source wireless device transmitting the new partial frame to the destination wireless device.

[0007] In some respects, the session used for mirroring video content can be End-to-end (E2E) session. In some aspects, the pure fill color frame can be a pure black frame. In some aspects, the indication of the valid frame region of interest can be an indication of the pixel height, pixel width, and coordinates of the entire frame of the valid frame region of interest. In some aspects, the indication of the valid frame region of interest can be a Supplemental Enhancement Information (SEI) message.

[0008] Another aspect includes a method for supporting mirroring of video content in a session established between a source wireless device and a destination wireless device, the method comprising: in response to establishing a session with the source wireless device for mirroring video content, receiving, by a processor of the destination wireless device, an encoded pure fill color frame; decoding, by the processor of the destination wireless device, the encoded pure fill color frame into a first full frame; receiving, by the processor of the destination wireless device, an indication from the source wireless device of a valid frame region of interest; receiving, by the processor of the destination wireless device, an encoded partial frame from the source wireless device; decoding, by the processor of the destination wireless device, the encoded partial frame to generate a partial frame update for the valid frame region of interest; applying, by the processor of the destination wireless device, the partial frame update to the first full frame to generate a new full frame; and displaying, by the processor of the destination wireless device, the new full frame.

[0009] In some respects, the session used for mirroring video content is End-to-end (E2E) session. In some aspects, the pure fill color frame is a pure black frame. In some aspects, the indication of the valid frame region of interest is an indication of the pixel height, pixel width, and coordinates of the entire frame of the valid frame region of interest. In some aspects, the indication of the valid frame region of interest is a Supplemental Enhancement Information (SEI) message.

[0010] A further aspect may include a wireless device having a processor configured to perform one or more operations of any of the methods outlined above. A further aspect may include a non-transient processor-readable storage medium having processor-executable instructions stored thereon, the processor-executable instructions being configured to cause the processor of the wireless device to perform any of the methods outlined above. Another aspect includes a wireless device having components for performing functions of any of the methods outlined above. Another aspect includes a system-on-a-chip used in a wireless device, the system-on-a-chip including a processor configured to perform one or more operations of any of the methods outlined above. A further aspect includes a system-in-package including two systems-on-a-chip for a wireless device, the system-on-a-chip including processors configured to perform one or more operations of any of the methods outlined above. Attached Figure Description

[0011] The accompanying drawings, incorporated herein and forming part of this specification, illustrate exemplary embodiments of the claims and, together with the general description given above and the detailed description given below, serve to explain the features of the claims.

[0012] Figure 1A This is a system block diagram illustrating an exemplary communication system suitable for implementing any of the various implementation schemes.

[0013] Figure 1B This is a system block diagram illustrating an example decomposed base station architecture suitable for implementing any of the various implementation schemes of a wireless communication system.

[0014] Figure 2 This is a component block diagram of an example computing and wireless modem system applicable to any implementation of various implementation schemes.

[0015] Figure 3 This is a component block diagram illustrating a software architecture, including a radio protocol stack for the user plane and control plane in wireless communication, applicable to any implementation of various implementation schemes.

[0016] Figure 4 This is a component block diagram illustrating example components applicable to source and destination wireless devices used with various implementation schemes.

[0017] Figure 5 This is a block diagram illustrating various aspects of mirror projection between a source wireless device and a destination wireless device.

[0018] Figure 6 This is a flowchart illustrating a method for supporting the mirroring of video content in a session established between a source wireless device and a destination wireless device, according to various implementation schemes.

[0019] Figure 7 This is a flowchart illustrating a method for supporting the mirroring of video content in a session established between a source wireless device and a destination wireless device, according to various implementation schemes.

[0020] Figure 8 This is a block diagram illustrating various aspects of mirror projection between a source wireless device and a destination wireless device according to various implementation schemes.

[0021] Figure 9 It is a component block diagram applicable to wireless devices implementing various implementation schemes.

[0022] Figure 10 This is a component block diagram illustrating example wireless devices applicable to various implementation schemes. Detailed Implementation

[0023] Various embodiments will be described in detail with reference to the accompanying drawings. Where possible, the same reference numerals will be used throughout the drawings to refer to the same or similar parts. References to specific examples and embodiments are for illustrative purposes and are not intended to limit the scope of the claims.

[0024] Various implementation schemes are provided for Systems and methods for end-to-end (E2E) stream transmission. Various implementations provide options for partial frame regions. Systems and methods for E2E stream transmission. In a WFD session, various implementations enable the encoding and transmission of only valid frame regions of content from the source wireless device to the destination wireless device. Various implementations also enable the decoding of only valid frame regions of content at the destination wireless device. Furthermore, various implementations enable WFD mirroring without pixel padding (e.g., without black pixel padding).

[0025] The terms “wireless device,” “computing device,” “user equipment,” and “UE” are used herein to refer to any or all of the following: endpoint or user equipment, including wireless devices, wireless routers, wireless devices, cellular phones, smartphones, portable computing devices, personal or mobile multimedia players, laptops, tablets, smartbooks, ultrabooks, handheld computers, wireless email receivers, internet-enabled multimedia cellular phones, medical devices and equipment, and devices that enable… Electronic devices, biometric sensors / devices, wearable devices (including smartwatches, smart clothing, smart glasses, smart wristbands, smart jewelry (e.g., smart rings, smart bracelets)), entertainment devices (e.g., wireless game controllers, music and video players, satellite radios, televisions, display units, etc.), wireless network-enabled Internet of Things (IoT) devices (including smart meters / sensors, industrial manufacturing equipment, large and small machines and appliances for home or business use), wireless communication elements in autonomous and semi-autonomous vehicles, UEs attached to or incorporated into various mobile platforms, GPS devices, and similar electronic devices including memory, wireless communication components, and programmable processors.

[0026] The term “radio resources” is used herein to refer to hardware such as modems, radios, processors, transceivers, transmitters, receivers, timers, voltage regulators, oscillators, amplifiers, filters, antennas, circuits, encoders, decoders, etc., and / or software that operates alone or in any combination for transmitting and / or receiving electromagnetic radiation to provide wireless communication services (such as cellular and mobile communication services).

[0027] The term "System-on-a-Chip" (SOC) is used herein to refer to a single integrated circuit (IC) chip containing multiple resources and / or processors integrated on a single substrate. A single SOC may contain circuitry for digital, analog, mixed-signal, and radio frequency functions. A single SOC may also include any number of general-purpose and / or specialized processors (digital signal processors, modem processors, video processors, etc.), memory blocks (e.g., ROM, RAM, flash memory, etc.), and resources (e.g., timers, regulators, oscillators, etc.). A SOC may also include software for controlling the integrated resources and processors, as well as software for controlling peripheral devices.

[0028] The term "System-in-Package" (SIP) may be used herein to refer to a single module or package containing multiple resources, computing units, cores and / or processors on two or more IC chips, a substrate, or a System-on-a-Chip (SoC). For example, a SIP may include a single substrate on which multiple IC chips or semiconductor dies are stacked in a vertical configuration. Similarly, a SIP may include one or more multi-chip modules (MCMs) on which multiple ICs or semiconductor dies are packaged into a single substrate. A SIP may also include multiple independent SoCs coupled together and packaged in close proximity via high-speed communication circuitry, such as on a single motherboard or in a single wireless device. The proximity of the SoCs facilitates high-speed communication and the sharing of memory and resources.

[0029] As used herein, the terms “network,” “system,” “wireless network,” “cellular network,” and “wireless communication network” can be used interchangeably to refer to part or all of a wireless network of carriers associated with wireless devices and / or subscriptions on those devices. The techniques described herein can be used in various wireless communication networks, such as Code Division Multiple Access (CDMA), Time Division Multiple Access (TDMA), FDMA, Orthogonal FDMA (OFDMA), Single Carrier FDMA (SC-FDMA), and others. Generally, any number of wireless networks can be deployed in a given geographical area. Each wireless network can support at least one radio access technology, which can operate on one or more frequencies or frequency ranges. For example, a CDMA network can implement Universal Terrestrial Radio Access (UTRA) (including the Wideband Code Division Multiple Access (WCDMA) standard), CDMA2000 (including IS-2000, IS-95, and / or IS-856 standards), etc. In another example, a TDMA network can implement GSM Enhanced Data Rate (EDGE), an evolution of GSM. Similarly, an OFDMA network can implement Evolved UTRA (E-UTRA) (including the LTE standard), IEEE 802.11, etc. IEEE802.16(WiMAX), IEEE 802.20, Etc. Reference may be made to wireless networks using the LTE standard; therefore, the terms "Evolved Universal Terrestrial Radio Access," "E-UTRAN," and "eNodeB" may be used interchangeably herein to refer to wireless networks. However, such citations are provided merely as examples and are not intended to exclude wireless networks using other communication standards.

[0030] LTE is a mobile network standard for 4G wireless communication for high-speed data, developed by 3GPP (3rd Generation Partnership Project) and specified in its Release 8 document series. In contrast to the circuit-switched (CS) model of cellular network standards, LTE has been designed to support packet-switched only (PS) services. Data services in LTE are provided over the Internet, while multimedia services are supported by the Internet Multimedia Subsystem (IMS) framework. The LTE standard is based on the evolution from Universal Mobile Telecommunications System (UMTS) radio access to the Evolved Universal Terrestrial Radio Access Network (E-UTRAN). E-UTRAN, together with the Evolved Packet Core (EPC) network (the core network compatible with LTE), constitutes the Evolved Packet System (EPS). While the access network in UMTS simulates circuit-switched connections for real-time services and packet-switched connections for data communication services, the Evolved Packet System (EPS) is purely based on the Internet Protocol (IP), and both real-time and data communication services are carried by the IP protocol.

[0031] 5G systems are advanced technologies developed from 4G LTE, and provide new radio access technologies (RATs) through the evolution of existing mobile communication network architectures. 5G systems can support, for example, extended LTE (eLTE) and non-3GPP access (e.g., WLAN).

[0032] One implementation option for employing advanced networks (such as 5G New Radio (5GNR) networks, future generations of system networks (e.g., sixth-generation (6G) or higher) networks) is a 5G SA network in which the 5G Radio Access Network (RAN) and 5G Core Network provide 5G services within a geographic region (such as a country). In this way, the 5G SA network can overlap with LTE network coverage within a geographic region (such as a country). The 5G SA network can exclusively include NR base stations, such as next-generation B nodes (gB nodes or gNBs).

[0033] Another implementation option for currently used advanced systems or networks (e.g., 5G systems or networks, 6G systems or networks, higher-generation systems or networks, etc.) is the 5G NSA network, in which a RAN (e.g., a RAN including both LTE base stations (such as LTE evolved B nodes (eNodeB or eNB)) and NR base stations (such as next-generation B nodes (gB nodes or gNB)) supporting both LTE (also known as 4G) and NR (also known as 5G) is connected to the LTE core network (e.g., an evolved packet core (EPC) network). In such a 5G NSA network, radio devices (sometimes referred to as user equipment (UE)) that support both LTE and NR communications can signal to the 5G NSA that the UE supports dual connectivity with the new radio (DCNR).

[0034] Standard supports source wireless devices via connection to destination wireless devices. (WFD) connections enable wireless sharing of multimedia content, including high-resolution images and high-definition (HD) video content, by mirroring and projecting multimedia content.

[0035] In various implementations, wireless devices in a wireless network (such as a WFD network) can be classified as source and / or destination depending on whether the device is transmitting or receiving content data. A source or source wireless device is a wireless device that transmits content (e.g., content frames, content packets, etc.) to one or more other wireless devices (i.e., destination devices). For example, a source wireless device may operate as a server providing frames or packets to other wireless devices in the wireless network. A destination or destination wireless device is a wireless device that receives content (e.g., content frames, content packets, etc.) from one or more other wireless devices. A wireless device may operate exclusively as a source, exclusively as a destination, or both as a destination and a source in a wireless network (such as a WFD network). As an example, a WFD network may include a UE acting as a source, which, according to... The standard wirelessly projects streaming Moving Picture Experts Group (MPEG) format images onto one or more nearby display units that act as one or more hosts.

[0036] This article discusses various examples of different wireless standards, wireless connections, wireless networks, and wireless media delivery platforms, especially... Connect to WFD, networks, and media delivery platforms. The discussion with WFD is provided only as an example to better illustrate various aspects of the different embodiments and is not intended to limit the claims. Or WFD, unless specifically stated otherwise. Other wireless standards, wireless connections, wireless networks, and wireless media delivery platforms (e.g., Apple) (etc.) can be used in various implementations or in conjunction with various implementations, and other wireless standards, wireless connections, wireless networks and wireless media delivery platforms can be replaced in the various examples discussed herein.

[0037] Wireless media delivery platforms (e.g., Apple) WiDi (etc.) can support screen mirroring (sometimes also called casting, screen casting, mirroring, or mirroring projection), where the content displayed on the source wireless device's display is provided to the destination wireless device's display. According to In a WFD session established between a source wireless device and a destination wireless device, the standard allows the source wireless device to mirror its video and / or audio output to the destination wireless device for output. The source wireless device encodes the content (e.g., video and / or audio data) output to its display and / or speakers to generate encoded content (e.g., encoded video and / or encoded audio data). The source wireless device then groups the encoded content (e.g., encoded video and / or encoded audio data) and transmits it to the destination wireless device for mirroring. The destination wireless device receives the encoded content (e.g., encoded video and / or encoded audio data), decodes the content (e.g., video and / or audio data), and outputs the content (e.g., video and / or audio data) to its display and / or speakers.

[0038] according to The standard for content mirroring from the source wireless device to the destination wireless device in a WFD session (sometimes also called screen mirroring, casting, screen projection, or mirroring) can be referred to as... An end-to-end (E2E) session, and the content transmitted from the source wireless device to the destination wireless device in a WFD session, can be referred to as... flow.

[0039] exist In an E2E session, the display resolution of the source wireless device and the display resolution of the destination wireless device can be different. For example, the source wireless device can be a smartphone with a display content resolution of 1080×2400, and the destination wireless device can be a television with a display resolution of 3840×2160.

[0040] In the current WFD mirroring, the source wireless device uses standard 720p / 1080p / 2160p video encoding-decoding resolution for WFD stream encoding-decoding and sending the video content to the destination wireless device. In the current WFD mirroring, the source wireless device maintains a constant frame aspect ratio and fills any area outside the source content with black pixels to create 720p / 1080p / 2160p source encoded frames. In the current WFD mirroring, the destination wireless device receives and decodes the standard 720p / 1080p / 2160p WFD stream. In the current WFD mirroring, the source wireless device encodes the WFD stream content at a frame resolution of 3840×2160p30, and the destination wireless device decodes the WFD stream content at the same frame resolution of 3840×2160p30.

[0041] The padding of black pixels in the current WFD mirror projection does not affect the stream bit rate because the encoding of black pixels does not add bytes to the compressed stream. However, the padded black pixels are processed in the same way as other pixels in both the source and destination wireless systems. Therefore, in the current WFD mirror projection where both the source and destination wireless devices are processing a frame resolution of 3840×2160p30, the power and resources of both the source and destination wireless devices may be wasted on processing redundant black-filled pixels.

[0042] For example, in current WFD mirroring, processors such as graphics processing units (GPUs) and data processing units (DPUs) need to generate black-filling pixels one after another, which incurs resource costs in terms of processor clock time and power consumption of processors and / or other hardware resources. As another example, in current WFD mirroring, the video encoding codec at the source wireless device needs to encode black-filling pixels and macroblocks one after another, which incurs resource costs in terms of encoding codec clock time and power consumption of video encoding codecs and / or other hardware resources. As yet another example, in current WFD mirroring, the video decoding codec at the destination wireless device needs to decode black-filling pixels and macroblocks one after another, which incurs resource costs in terms of decoding codec clock time and power consumption of video decoding codecs and / or other hardware resources. As yet another example, in current WFD mirroring, the destination wireless device's memory and display subsystem need to update / refresh black-filling pixels every frame, which incurs resource costs in terms of memory bandwidth and display subsystem clock time. The resource cost of such examples used to handle redundant black-filled pixels in the current WFD mirroring is a waste of power and performance for both the source and destination wireless devices.

[0043] Various implementations provide methods and systems for supporting the mirroring of video content within a session established between a source wireless device and a destination wireless device. Various implementations also provide methods for mirroring partial frame regions. Systems and methods for E2E streaming. In a WFD session, various implementations enable the encoding and transmission of only valid frame regions of content from the source wireless device to the destination wireless device. Various implementations enable the decoding of only valid frame regions of content at the destination wireless device. Various implementations enable WFD mirroring without pixel padding (e.g., without black pixel padding or other colored pixel padding).

[0044] In various implementations, in response to establishing a WFD session between the source and destination wireless devices for WFD mirroring, the source wireless device can generate and encode a first pure fill color frame (such as a pure black frame, pure blue frame, pure red frame, etc.) and transmit the encoded pure fill color frame to the destination wireless device. The resolution of the encoded pure fill color frame can match the full resolution of the destination wireless device and the WFD session. As an example, the resolution of the encoded pure fill color frame can be 3840×2160. The destination wireless device can receive the encoded pure fill color frame and decode it into a first full frame.

[0045] In various implementations, the source wireless device can determine the region of interest (ROI) for the projected video content to be mirrored in the WFD session. The ROI can be... The region of a full-resolution frame in an E2E session, where all pixels are associated with the content being displayed. In other words, the region of interest that is a valid frame area can be... An E2E session may involve a region of the full-resolution frame where pixel padding may not be required. For example, the region of interest for mirroring projected video content in a WFD session can be defined by the pixel width and pixel height of the source frame of the content to be mirrored, as well as the position coordinates of that source frame within the entire frame. The region of interest can be a unique region of the full frame that can be updated when the content is output from the source wireless device.

[0046] In various implementations, the source wireless device may transmit to the destination wireless device an indication of the region of interest (ROI) of the video content to be mirrored in a WFD session. To transmit the ROI indication, the source wireless device may transmit to the destination wireless device an indication of the pixel width and pixel height of the source frame of the content to be mirrored, as well as the position coordinates of that source frame within the entire frame. As an example, the ROI indication may be transmitted as part of a High-Efficiency Video Decoding (HEVC) refresh function and / or in one or more Supplemental Enhancement Information (SEI) messages transmitted from the source wireless device to the destination wireless device.

[0047] In various implementations, after transmitting the first plain-fill color frame, when generating and encoding a regular frame next, the source wireless device may generate and encode only the region of interest (ROI) of the video content. When the ROI remains unchanged, the source wireless device may generate and encode only the ROI of the video content. Because only the ROI is generated and encoded after the first plain-fill color frame is encoded and transmitted, frames after the first frame may be without plain-fill color frames, such as frames without black pixel fill. Encoding only the ROI allows the source wireless device to transmit only partial frames containing data corresponding to the ROI.

[0048] In various implementations, the receiving wireless device may receive encoded partial frames after receiving an encoded pure fill color frame as a first full frame. The receiving wireless device may decode the encoded partial frames to generate partial frames corresponding to the region of interest. The receiving wireless device may use the partial frames corresponding to the region of interest to update the full frame. Since only the coordinates corresponding to the region of interest can be updated, no resource cost is incurred for the full frame region outside the region of interest, because the full frame region outside the region of interest is not updated and retains the original pure fill color used in the first full frame.

[0049] As an example, a full-frame resolution of 3480×2160 can be established between the source and destination wireless devices. E2E Session. The source wireless device can generate and encode a pure black first frame at 3480×2160 resolution and transmit the encoded first frame to the destination wireless device. The destination wireless device can decode the first frame and output a pure black first frame. The source wireless device can determine that the region of interest for the video content is the 972×2160 source frame and can indicate the 972×2160 width / height and position coordinates of the source frame in the full frame to the destination wireless device, such as in an SEI message. After the first frame, the source wireless device can generate and encode only the region of interest for all frames, specifically only the 972×2160 frame region, and transmit these encoded partial frames to the destination wireless device. The destination wireless device can receive and decode the partial frames and update the previous full frame with the partial frames of the region of interest, thereby changing only the 972×2160 frame region at the time of each subsequent frame presentation. In this way, the regions of the entire frame outside the region of interest in the effective frame can remain pure black as the original first frame, and the source and destination wireless devices only need to process, generate, encode and / or decode the 972×2160 frames and corresponding macroblocks.

[0050] In various implementations, the source wireless device may determine that the effective frame region has changed. As an example, the effective frame region may change due to the source wireless device being rotated and / or application preferences changing due to an application running on the source wireless device. As a specific example, the effective frame region may change from 972×2160 to 1620×2160. In response to determining that the effective frame region has changed, the source wireless device may determine a new effective frame region of interest for the video content to be mirrored in the WFD session. In various implementations, the source wireless device may transmit an indication to the destination wireless device of the new effective frame region of interest for the video content to be mirrored in the WFD session.

[0051] In response to a change in the valid frame region, the source wireless device can generate and encode a new pure fill color frame (such as a pure black frame, pure blue frame, pure red frame, etc.) and transmit the encoded pure fill color frame to the destination wireless device. The resolution of the encoded pure fill color frame can match the full resolution of the destination wireless device and the WFD session. As an example, the resolution of the encoded pure fill color frame could be 3840×2160. The destination wireless device can receive the encoded pure fill color frame and decode it into a new full frame.

[0052] In various implementations, after transmitting a new plain-fill color frame, when generating and encoding a regular frame next, the source wireless device may generate and encode only the new active frame region of interest (NEFF) for the video content. When the NEFF region remains unchanged, the source wireless device may generate and encode only the NEFF region of interest for the video content. Because only the NEFF region of interest is generated and encoded after the new plain-fill color frame is encoded and transmitted, subsequent frames may be frames without a plain-fill color frame, such as frames without black pixel fill. Encoding only the NEFF region allows the source wireless device to transmit only a portion of the frame containing data corresponding to the NEFF region of interest.

[0053] In various implementations, the receiving wireless device may receive encoded partial frames after receiving a newly encoded pure fill color frame as a new full frame. The receiving wireless device may decode the encoded partial frames to generate partial frames corresponding to the region of interest in the new valid frame. The receiving wireless device may use the partial frames corresponding to the region of interest in the new valid frame to update the full frame. Since only the coordinates corresponding to the region of interest in the new valid frame can be updated, no resource cost is incurred for the full frame region outside the region of interest in the new valid frame, because the full frame region outside the region of interest in the new valid frame is not updated and retains the original pure fill color used in the new full frame.

[0054] Improvements to various implementation plans Power and performance of source and destination wireless devices in an E2E session. As an example, the clock time consumed by the GPU, DPU, codec, and decoder in a current WFD mirroring projection, compared to the clock time consumed by the GPU, DPU, codec, and decoder in various implementations for supporting... Clock times for mirroring in E2E sessions can be reduced by 50% or more. Various implementations can triple the frames per second (FPS) rate for WFD sessions, for example, increasing WFD session capability from 3840×2160p30 to 3840×2160p120. Various implementations can triple the resolution rate for WFD sessions, for example, increasing WFD session capability from 720×1280p30 to 3840×2160p3. Various implementations can reduce DPU, codec, and decoder clocks from 400 MHz to 200 MHz or less, because the number of pixels to be processed is reduced by 75% compared to those in current WFD mirroring.

[0055] Figure 1A This is a system block diagram illustrating an example communication system 100. Communication system 100 can be a 5G New Radio (NR) network, or any other suitable network (such as a Long Term Evolution (LTE) network). Although Figure 1A A 5G network is illustrated, but subsequent networks may include the same or similar elements. Therefore, references to 5G networks and 5G network elements in the following description are for illustrative purposes and are not intended to be limiting.

[0056] Communication system 100 may include a heterogeneous network architecture, which includes a core network 140 and various UEs (in... Figure 1A The communication system 100 may also include several base stations (exemplified as BS110a, BS110b, BS110c, and BS110d) and other network entities. A base station is an entity that communicates with the UE and may also be referred to as a Node B, an LTE evolved node B (eNodeB or eNB), an access point (AP), a radio headend, a transmit / receive point (TRP), a new radio base station (NR BS), a 5G Node B (NB), a next-generation Node B (gNodeB or gNB), and so on. Each base station can provide communication coverage for a specific geographic area. In 3GPP, the term "cell" can refer to the coverage area of ​​a base station, a base station subsystem serving that coverage area, or a combination thereof, depending on the context in which the term is used. The core network 140 can be any type of core network, such as an LTE core network (e.g., an evolved packet core (EPC) network), a 5G core network, etc.

[0057] Base stations 110a-110d can provide communication coverage for macro cells, pico cells, femto cells, another type of cell, or combinations thereof. Macro cells cover a relatively large geographic area (e.g., a radius of several kilometers) and allow unrestricted access by UEs with a service subscription. Pico cells cover a relatively small geographic area and allow unrestricted access by UEs with a service subscription. Femto cells cover a relatively small geographic area (e.g., a residential area) and allow restricted access by UEs associated with that femto cell (e.g., UEs in a Closed Subscriber Group (CSG)). A base station used for a macro cell may be referred to as a macro BS. A base station used for a pico cell may be referred to as a pico BS. A base station used for a femto cell may be referred to as a femto BS or a home BS. Figure 1A In the example illustrated herein, base station 110a may be a macro BS for macro cell 102a, base station 110b may be a pico BS for pico cell 102b, and base station 110c may be a femto BS for femto cell 102c. Base stations 110a-110d may support one or more (e.g., three) cells. The terms “eNB,” “base station,” “NR BS,” “gNB,” “TRP,” “AP,” “Node B,” “5G NB,” and “cell” may be used interchangeably herein.

[0058] In some examples, the cell may not be stationary, and the geographical area of ​​the cell may move depending on the location of the mobile base station. In some examples, base stations 110a-110d may interconnect with each other using any suitable transport network through various types of backhaul interfaces (such as direct physical connections, virtual networks, or combinations thereof), and interconnect to one or more other base stations or network nodes (not illustrated) in the communication system 100.

[0059] Base stations 110a-110d can communicate with the core network 140 via wired or wireless communication link 126. UEs 120a-120e can communicate with base stations 110a-110d via wireless communication link 122.

[0060] The wired communication link 126 can use a variety of wired networks (such as Ethernet, TV cable, telephone, fiber optic, and other forms of physical network connection), which can use one or more wired communication protocols, such as Ethernet, point-to-point protocol, high-level data link control (HDLC), advanced data communication control protocol (ADCCP), and transmit control protocol / Internet protocol (TCP / IP).

[0061] The communication system 100 may also include a relay station (such as relay BS110d). A relay station is an entity capable of receiving data transmissions from an upstream station (e.g., a base station or a UE) and transmitting those data transmissions to a downstream station (e.g., a UE or a base station). A relay station may also be a wireless device (e.g., a UE) capable of relaying transmissions for other UEs. Figure 1A In the example illustrated, relay station 110d can communicate with macro base station 110a and UE 120d to facilitate communication between base station 110a and UE 120d. A relay station may also be referred to as a relay base station, relay unit, repeater, etc.

[0062] The communication system 100 can be a heterogeneous network comprising different types of base stations (e.g., macro base stations, pico base stations, femto base stations, relay base stations, etc.). These different types of base stations may have different transmit power levels, different coverage areas, and different effects on interference in the communication system 100. For example, macro base stations may have high transmit power levels (e.g., 5 watts to 40 watts), while pico base stations, femto base stations, and relay base stations may have lower transmit power levels (e.g., 0.1 watts to 2 watts).

[0063] Network controller 130 can be coupled to a set of base stations and can provide coordination and control over these base stations. Network controller 130 can communicate with the base stations via backhaul. Base stations can also communicate with each other directly or indirectly, for example, via wireless or wired backhaul.

[0064] UEs 120a, 120b, and 120c can be distributed throughout the communication system 100, and each UE can be stationary or mobile. UEs can also be referred to as access terminals, terminals, mobile stations, subscriber units, stations, wireless devices, etc.

[0065] Macro base station 110a can communicate with communication network 140 on wired or wireless communication link 126. UEs 120a, 120b, and 120c can communicate with base stations 110a-110d on wireless communication link 122.

[0066] Wireless communication links 122 and 124 may include multiple carrier signals, frequencies, or frequency bands, each of which may include multiple logical channels. Wireless communication links 122 and 124 may utilize one or more radio access technologies (RATs). Examples of RATs that can be used in wireless communication links include: 3GPP LTE, 3G, 4G, 5G (such as NR), GSM, Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Microwave Access Global Interoperability (WiMAX), Time Division Multiple Access (TDMA), and other mobile phone communication technology cellular RATs. Other examples of RATs that can be used in one or more of the various wireless communication links within communication system 100 include mid-range protocols (such as... LTE-U, LTE-Direct, LAA, MuLTEfire) and relatively short-range RATs (such as ZigBee, Bluetooth and Bluetooth Low Energy (LE)).

[0067] Certain wireless networks (e.g., LTE) use Orthogonal Frequency Division Multiplexing (OFDM) on the downlink and Single-Carrier Frequency Division Multiplexing (SC-FDM) on the uplink. OFDM and SC-FDM divide the system bandwidth into multiple (K) orthogonal subcarriers, often referred to as tones, frequency slots, etc. Each subcarrier can be modulated using data. Generally, modulation symbols are transmitted using OFDM in the frequency domain and SC-FDM in the time domain. The spacing between adjacent subcarriers can be fixed, and the total number of subcarriers (K) can depend on the system bandwidth. For example, the subcarrier spacing could be 15 kHz, and the minimum resource allocation (called a "resource block") could be 12 subcarriers (or 180 kHz). Therefore, for system bandwidths of 1.25 MHz, 2.5 MHz, 5 MHz, 10 MHz, or 20 MHz, the nominal Fast File Transfer (FFT) size could be 128, 256, 512, 1024, or 2048, respectively. The system bandwidth can also be divided into subbands. For example, a subband can cover 1.08MHz (i.e., 6 resource blocks), and for system bandwidths of 1.25MHz, 2.5MHz, 5MHz, 10MHz, or 20MHz, there can be 1, 2, 4, 8, or 16 subbands, respectively.

[0068] While some implementations may use terminology and examples associated with LTE technology, some implementations are applicable to other wireless communication systems, such as New Radio (NR) or 5G networks. NR can utilize OFDM with a cyclic prefix (CP) on both the uplink (UL) and downlink (DL) and includes support for half-duplex operation using Time Division Duplex (TDD). A single component carrier bandwidth of 100 MHz can be supported. An NR resource block can span 12 subcarriers with a subcarrier bandwidth of 75 kHz over a duration of 0.1 milliseconds (ms).

[0069] Each radio frame may comprise 50 subframes, each 10 ms in length. Therefore, each subframe may have a length of 0.2 ms. Each subframe may indicate the link direction (i.e., DL or UL) for data transmission, and the link direction of each subframe may be dynamically switched. Each subframe may include DL / UL data and DL / UL control data.

[0070] Beamforming is supported and beam direction can be dynamically configured. Multiple-input multiple-output (MIMO) transmission with pre-decoded encoding is also supported. MIMO configuration in the DL can support up to eight transmit antennas, with up to eight streams and up to two streams per UE in multi-layer DL transmission. Multi-layer transmission with up to two streams per UE can also be supported.

[0071] Up to eight serving cells can be used to support the aggregation of multiple cells. Alternatively, NR can support different air interfaces in addition to the OFDM-based air interface.

[0072] Some UEs can be considered Machine-Type Communication (MTC) UEs, or evolved or enhanced Machine-Type Communication (eMTC) UEs. MTC and eMTC UEs include, for example, robots, drones, remote devices, sensors, meters, monitors, location tags, etc., which can communicate with a base station, another device (e.g., a remote device), or some other entity. Wireless computing platforms can provide connectivity to or to a network (e.g., a wide area network, such as the Internet) or a cellular network, for example, via wired or wireless communication links. Some UEs can be considered Internet of Things (IoT) devices, or can be implemented as NB-IoT (Narrowband Internet of Things) devices. UEs 120a-120e can be included within a housing that houses the components of UEs 120a-120e, such as processor components, memory components, similar components, or combinations thereof.

[0073] Generally, any number of communication systems and wireless networks can be deployed in a given geographical area. Each communication system and wireless network can support a specific Radio Access Technology (RAT) and can operate on one or more frequencies. A RAT may also be referred to as a radio technology, air interface, etc. A frequency may also be referred to as a carrier, frequency channel, etc. Each frequency can support a single RAT in a given geographical area to avoid interference between communication systems using different RATs. In some cases, 4G / LTE and / or 5G / NR RAT networks can be deployed. For example, a 5G Non-Standalone (NSA) network can use a 4G / LTE RAT on the 4G / LTE RAN side of a 5G NSA network, and simultaneously use a 5G / NR RAT on the 5G / NR RAN side of the 5G NSA network. The 4G / LTE RAN and 5G / NR RAN can be interconnected and connected to the 4G / LTE core network (e.g., an evolved packet core (EPC) network) in the 5G NSA network. Other example network configurations may include a 5G Standalone (SA) network, in which the 5G / NR RAN is connected to the 5G core network.

[0074] In some implementations, two or more UEs (e.g., exemplified as UE 120a and UE 120e) may communicate directly using one or more sidelink channels (e.g., without using base stations 110a-d as intermediaries). For example, UEs 120a-120e may communicate using peer-to-peer (P2P) communication, device-to-device (D2D) communication, WFD communication, vehicle-to-everything (V2X) protocols (which may include vehicle-to-vehicle (V2V) protocols, vehicle-to-infrastructure (V2I) protocols, or similar protocols), mesh networks, or similar networks, or combinations thereof. In this scenario, UEs 120a-120e may perform scheduling operations, resource selection operations, and other operations described elsewhere herein as performed by base stations 110a-110d.

[0075] In some specific implementations, one UE 120a-120e can provide content (such as multimedia content) to another UE 120a-120e. Content can be delivered from the originating UE (acting as the source radio device) to the receiving UE (acting as the destination radio device) of UE 120a-120e via connections 122 and 124. Methods such as WFD can be used. Various protocols, such as [list of protocols], are used to deliver content. As an example, a WFD session can be established between the source wireless device (or source endpoint) UE 120a-120e and the destination wireless device (or destination endpoint) UE 120a-120e to [deliver content]. E2E transmission stream mirrors video content from source wireless device UE 120a-120e to destination wireless device UE 120a-120e.

[0076] Figure 1B This is a system block diagram illustrating an exemplary decomposed base station 160 architecture suitable for implementing any of the various implementation schemes, which may be part of a communication system (e.g., communication system 100) such as a 5G (or later generation) network. Reference Figure 1A and Figure 1BThe decomposed base station 160 architecture may include one or more central units (CUs) 162, which may communicate directly with the core network 180 via a backhaul link, or indirectly with the core network 180 through one or more decomposed base station units, such as a near-real-time (Near-RT) RAN Intelligent Controller (RIC) 164 via an E2 link, or a non-real-time (Non-RT) RIC 168 associated with a Service Management and Orchestration (SMO) framework 166, or both. CUs 162 may communicate with one or more distributed units (DUs) 170 via corresponding midhaul links (such as F1 interfaces). DUs 170 may communicate with one or more radio units (RUs) 172 via corresponding fronthaul links. RUs 172 may communicate with corresponding UEs 120 via one or more radio frequency (RF) access links. In some implementations, a UE may be served simultaneously by multiple RUs 172.

[0077] Each of the units (i.e., CU 162, DU 170, RU 172), as well as the near-RT RIC 164, non-RT RIC 168, and SMO frame 166, may include one or more interfaces, or be coupled to one or more interfaces configured to receive or transmit signals, data, or information (collectively, signals) via wired or wireless transmission media. Each of the units, or an associated processor or controller that provides instructions to the communication interfaces of these units, may be configured to communicate with one or more other units via transmission media. For example, these units may include a wired interface configured to receive or transmit signals to one or more other units via a wired transmission media. Additionally, the unit may include a wireless interface that may include a receiver, transmitter, or transceiver (such as a radio frequency (RF) transceiver) configured to receive or transmit signals, or both, to one or more other units via a wireless transmission media.

[0078] In some aspects, the CU 162 can host one or more higher-level control functions. Such control functions may include Radio Resource Control (RRC), Packet Data Convergence Protocol (PDCP), Service Data Adaptation Protocol (SDAP), etc. Each control function can be implemented using an interface configured to signal to other control functions hosted by the CU 162. The CU 162 can be configured to handle user plane functionality (i.e., Central Unit-User Plane (CU-UP)), control plane functionality (i.e., Central Unit-Control Plane (CU-CP)), or a combination thereof. In some specific implementations, the CU 162 can be logically divided into one or more CU-UP units and one or more CU-CP units. When implemented in an O-RAN configuration, the CU-UP units can communicate bidirectionally with the CU-CP units via an interface such as an E1 interface. The CU 162 can be implemented to communicate with the DU 170 for network control and signaling as needed.

[0079] DU 170 may correspond to a logical unit comprising one or more base station functions for controlling the operation of one or more RU 172s. In some aspects, DU 170 may, at least in part, host one or more of the Radio Link Control (RLC) layer, the Media Access Control (MAC) layer, and one or more high physical (PHY) layers (such as modules for forward error correction (FEC) encoding and decoding, scrambling, modulation and demodulation, etc.) depending on functional splits (such as those defined by the 3rd Generation Partnership Project (3GPP). In some aspects, DU 170 may also host one or more low PHY layers. Each layer (or module) may be implemented using an interface configured to communicate signals with other layers (and modules) hosted by DU 170 or with control functions hosted by CU162.

[0080] Lower-layer functionality can be implemented by one or more RU 172s. In some deployments, an RU172 controlled by a DU 170 may correspond to a logical node that hosts RF processing functions or low-PHY layer functions (such as performing Fast Fourier Transform (FFT), Inverse FFT (iFFT), digital beamforming, Physical Random Access Channel (PRACH) extraction and filtering, or both, based at least in part on functional decomposition (such as lower-layer functional decomposition). In such architectures, the RU 172 may be implemented to handle over-the-air (OTA) communications with one or more UE 120s. In some specific implementations, the real-time and non-real-time aspects of control and user plane communications with the RU 172 may be controlled by the corresponding DU 170. In some scenarios, this configuration allows the DU 170 and CU 162 to be implemented in cloud-based radio access network (RAN) architectures such as vRAN architectures.

[0081] SMO framework 166 can be configured to support RAN deployment and provisioning of both non-virtualized and virtualized network elements. For non-virtualized network elements, SMO framework 166 can be configured to support the deployment of dedicated physical resources for RAN coverage requirements, which can be managed via operation and maintenance interfaces such as the O1 interface. For virtualized network elements, SMO framework 166 can be configured to interact with a cloud computing platform such as Open Cloud (O-Cloud) 176 to perform network element lifecycle management (such as instantiating virtualized network elements) via a cloud computing platform interface such as the O2 interface. Such virtualized network elements may include, but are not limited to, CU 162, DU 170, RU 172, and near-RT RIC 164. In some implementations, SMO framework 166 can communicate with the hardware aspects of the 4G RAN (such as Open eNB (O-eNB) 174) via the O1 interface. Additionally, in some implementations, SMO framework 166 can communicate directly with one or more RUs 172 via the O1 interface. SMO framework 166 may also include a non-RT RIC 168 configured to support the functionality of SMO framework 166.

[0082] The non-RT RIC 168 can be configured to include logical functions enabling non-real-time control and optimization of RAN elements and resources, including artificial intelligence / machine learning (AI / ML) workflows for model training and updates, or policy-based guidance for applications / features in the near-RT RIC 164. The non-RT RIC 168 can be coupled to or communicate with the near-RT RIC 164 (e.g., via an A1 interface). The near-RT RIC 164 can be configured to include logical functions enabling near real-time control and optimization of RAN elements and resources via an interface (e.g., via an E2 interface) through data collection and action, connecting one or more CU 162s, one or more DU 170s, or both, and O-eNBs to the near-RT RIC 164.

[0083] In some implementations, to generate AI / ML models to be deployed in the near-RT RIC 164, the non-RT RIC 168 may receive parameters or external enrichment information from an external server. This information can be utilized by the near-RT RIC 164 and can be received from non-network data sources or network functions at the SMO framework 166 or the non-RT RIC 168. In some examples, the non-RT RIC 168 or the near-RT RIC 164 may be configured to tune RAN behavior or performance. For example, the non-RT RIC 168 may monitor long-term trends and patterns in performance and employ AI / ML models to perform corrective actions via the SMO framework 166 (such as reconfiguration via O1) or by creating RAN management policies (such as A1 policies).

[0084] Figure 2 This is a component block diagram illustrating an example computing and wireless modem system 200 suitable for implementing any of the various implementation schemes. Various implementation schemes may be implemented on several single-processor and multi-processor computer systems, including system-on-a-chip (SOC) or system-in-package (SIP) systems.

[0085] refer to Figures 1A to 2 The illustrated example computing system 200 (which may be a SIP in some embodiments) includes two SOCs 202 and 204 coupled to a clock 206, a voltage regulator 208, and a transceiver 266 configured to transmit wireless communications to / receive communications from a UE (such as base station 110a) via an antenna (not shown). In some embodiments, the first SOC 202 may operate as a central processing unit (CPU) of the UE, which executes instructions by performing arithmetic, logic, control, and input / output (I / O) operations specified by instructions from a software application. In some embodiments, the second SOC 204 may operate as a dedicated processing unit. For example, the second SOC 204 may operate as a dedicated 5G processing unit responsible for managing high-capacity, high-speed (such as 5Gbps) or ultra-high frequency short-wavelength (such as 28GHz millimeter-wave spectrum) communications.

[0086] The first SOC 202 may include a digital signal processor (DSP) 210, a modem processor 212, a graphics processor (also known as a graphics processing unit (GPU)) 214, an application processor 216, one or more coprocessors 218 (such as vector coprocessors, data processing units (DPUs), etc.) connected to one or more of the processors, memory 220, custom circuitry 222, system components and resources 224, interconnect / bus modules 226, one or more codecs (e.g., video and / or audio codecs, video and / or audio decoder codecs, etc.), one or more temperature sensors 230, a thermal management unit 232, and a thermal power envelope (TPE) component 234. The second SOC 204 may include a 5G modem processor 252, a power management unit 254, an interconnect / bus module 264, multiple millimeter-wave transceivers 256, memory 258, and various additional processors 260, such as application processors, packet processors, etc.

[0087] Each processor 210, 212, 214, 216, 218, 252, 260 may include one or more cores, and each processor / core may perform operations independently of other processors / cores. For example, the first SOC 202 may include a processor running a first type of operating system (such as FreeBSD, LINUX, OS X, etc.) and a processor running a second type of operating system (such as MICROSOFT WINDOWS 10). Furthermore, any or all of processors 210, 212, 214, 216, 218, 252, 260 may be included as part of a processor cluster architecture (such as a synchronous processor cluster architecture, an asynchronous or heterogeneous processor cluster architecture, etc.).

[0088] The first SOC 202 and the second SOC 204 may include various system components, resources, and custom circuitry for managing sensor data, encoding, decoding, analog-to-digital conversion, wireless data transmission, and performing other specialized operations such as decoding data packets and processing encoded audio and video signals for presentation in web browsers or other display applications. For example, the system components and resources 224 of the first SOC 202 may include power amplifiers, voltage regulators, oscillators, phase-locked loops, peripheral bridges, data controllers, memory controllers, system controllers, access ports, timers, clocks, decoders, encoders, and other similar components to support processors and software clients running on the UE. System components and resources 224 or custom circuitry 222 may also include circuitry for interfacing with peripheral devices such as cameras, electronic displays, wireless communication devices, external memory chips, etc.

[0089] The first SOC 202 and the second SOC 204 can communicate via interconnect / bus module 250. Various processors 210, 212, 214, 216, and 218 can be interconnected via interconnect / bus module 226 to one or more memory elements 220, system components and resources 224, custom circuitry 222, codecs 233, and thermal management unit 232. Similarly, processor 252 can be interconnected via interconnect / bus module 264 to power management unit 254, millimeter-wave transceiver 256, memory 258, and various additional processors 260. Interconnect / bus modules 226, 250, and 264 may include arrays of reconfigurable logic gates or implement bus architectures (such as CoreConnect, AMBA, etc.). Communication can be provided by advanced interconnects, such as high-performance on-chip networks (NoC).

[0090] The first SOC 202 or the second SOC 204 may also include input / output modules (not illustrated) for communicating with external resources (such as clock 206 and voltage regulator 208). The external resources (such as clock 206 and voltage regulator 208) may be shared by two or more internal SOC processors / cores.

[0091] In addition to the example SIP 200 discussed above, some specific implementations can also be implemented in a wide variety of computing systems, which may include a single processor, multiple processors, multi-core processors, or any combination thereof.

[0092] Figure 3 This is a component block diagram illustrating a software architecture 300, including a radio protocol stack for the user plane and control plane in wireless communication, applicable to any of the various implementation schemes. Reference Figures 1A to 3UE 320 may implement software architecture 300 to facilitate communication between UE 320 (e.g., UE 120, 120a-120e, 200) and network device 350 of a communication system (e.g., 100). In various embodiments, layers in software architecture 300 may form logical connections with corresponding layers in the software of network device 350. Software architecture 300 may be distributed among one or more processors (e.g., processors 212, 214, 216, 218, 252, 260). Although described with respect to a single radio protocol stack, in a multi-subscriber identity module (SIM) UE, software architecture 300 may include multiple protocol stacks, each associated with a different SIM (e.g., in a dual-SIM wireless communication device, the two protocol stacks are associated with two SIMs respectively). Although described below with reference to the LTE communication layer, software architecture 300 may support any of the various standards and protocols used for wireless communication, and / or may include additional protocol stacks that support any of the various standards and protocols used for wireless communication.

[0093] Software architecture 300 may include a Non-Access Stratum (NAS) 302 and an Access Stratum (AS) 304. NAS 302 may include functions and protocols supporting packet filtering, security management, mobility control, session management, and services and signaling between the UE's SIM (such as SIM 204) and its core network 140. AS 304 may include functions and protocols supporting communication between the SIM (such as SIM 204) and entities of the supported access network (such as base stations). Specifically, AS 304 may include at least three layers (Layer 1, Layer 2, and Layer 3), each of which may contain various sublayers.

[0094] In the user plane and control plane, Layer 1 (L1) of AS 304 can be Physical Layer (PHY) 306, which supervises the functions of transmitting or receiving over the air interface via a radio transceiver (e.g., 266). Examples of such Physical Layer 306 functions may include Cyclic Redundancy Check (CRC) appendices, decoding blocks, scrambling and descrambling, modulation and demodulation, signal measurement, MIMO, etc. The Physical Layer may include various logical channels, including the Physical Downlink Control Channel (PDCCH) and the Physical Downlink Shared Channel (PDSCH).

[0095] In the user plane and control plane, Layer 2 (L2) of AS 304 can be responsible for the link between UE 320 and network device 350 on physical layer 306. In some implementations, Layer 2 may include Media Access Control (MAC) sublayer 308, Radio Link Control (RLC) sublayer 310 and Packet Data Convergence Protocol (PDCP) sublayer 312, and Serving Data Adaptation Protocol (SDAP) sublayer 317, each sublayer forming a logical connection terminated at network device 350.

[0096] In the control plane, Layer 3 (L3) of AS 304 may include a Radio Resource Control (RRC) sublayer 3. Although not shown, software architecture 300 may include additional Layer 3 sublayers, as well as various upper layers above Layer 3. In some implementations, RRC sublayer 313 may provide functions including broadcasting system information, paging, and establishing and releasing RRC signaling connections between UE 320 and network device 350.

[0097] In various implementations, SDAP sublayer 317 provides mapping between Quality of Service (QoS) streams and Data Radio Bearers (DRBs). In some specific implementations, PDCP sublayer 312 provides uplink functions including multiplexing between different radio bearers and logical channels, sequence numbering, handover data processing, integrity protection, encryption, and header compression. In the downlink, PDCP sublayer 312 provides functions including in-order delivery of data packets, duplicate data packet detection, integrity verification, decryption, and header decompression.

[0098] In the uplink, RLC sublayer 310 can provide segmentation and concatenation of upper-layer data packets, retransmission of lost data packets, and Automatic Repeat Request (ARQ). In the downlink, RLC sublayer 310 functions may include reordering of data packets to compensate for out-of-order reception, reassembly of upper-layer data packets, and ARQ.

[0099] In the uplink, MAC sublayer 308 provides functions including multiplexing between logical and transport channels, random access procedures, logical channel priorities, and hybrid ARQ (HARQ) operation. In the downlink, MAC layer functions may include intra-cell channel mapping, demultiplexing, discontinuous reception (DRX), and HARQ operation.

[0100] While the software architecture 300 provides the ability to transmit data over a physical medium, it may also include at least one host layer 314 to provide data delivery services to various applications in the UE 320. In some implementations, the application-specific functionality provided by the at least one host layer 314 provides an interface between the software architecture and the general-purpose processor 206.

[0101] In other implementations, software architecture 300 may include one or more higher logical layers (such as transport, session, presentation, application, etc.) that provide host layer functionality. For example, in some implementations, software architecture 300 may include a network layer (such as the Internet Protocol (IP) layer) where logical connections terminate at a Packet Data Network (PDN) gateway (PGW). In some implementations, software architecture 300 may include an application layer where logical connections terminate at another device (such as an end-user equipment, server, etc.). In some implementations, software architecture 300 may further include a hardware interface 316 in AS 304 between physical layer 306 and communication hardware (such as one or more radio frequency (RF) transceivers).

[0102] Figure 4 This is a block diagram illustrating example components of a source wireless device 448 and a destination wireless device 449 configured to stream directly to each other. (See reference) Figures 1A to 1B The source radio device 448 can be any radio device (e.g., UE 120, 120a-120e, 200, 320) that provides content (e.g., content frames, content packets, etc.) to the destination radio device 449. Similarly, the destination radio device can be any radio device (e.g., UE 120, 120a-120e, 200, 320) that receives content from the source radio device 448.

[0103] Source wireless device 448 may include media player component 450, transmission component 454, and display component 452. Destination wireless device 449 may include media player component 470, transmission component 462, and decoder component 466. Media player component 470 may transmit video frames to display component 472.

[0104] As used herein, the term "component" refers to a computer-related entity, such as, but not limited to, hardware, firmware, a combination of hardware and software, software, or software being executed, configured to perform a specific operation or function. For example, a component can be, but is not limited to, a process, processor, object, executable file, execution thread, program, and / or computer running on a processor. By way of example, both an application running on a computing device and the computing device itself can be referred to as a component. One or more components may reside within a process and / or execution thread, and components may reside on a processor or core and / or be distributed across two or more processors or cores. Furthermore, these components may execute from various non-transitory computer-readable media on which various instructions and / or data structures are stored. Components may communicate via local and / or remote procedures, function or procedure calls, electronic signals, data packets, memory read / write, and other known computer, processor, and / or process-related communication methods.

[0105] Media player components 450 and 470 may be combinations of one or more other components of the source wireless device 448 and the destination wireless device 449, respectively. For example, a media player component may be a combination of a network element component configured to retrieve data from a network, a decoder component configured to decode the retrieved compressed data, and a media presenter component configured to perform audiovisual (AV) synchronization operations. As an alternative example, a media player component may simply be a media presenter component that transmits media data (such as decoded video frames) to a display component, transmits decoded audio samples to a sound card, etc. In some embodiments, the media player component 450 of the source wireless device 448 may include components different from the media player component 470 of the destination wireless device 449.

[0106] The media player component 450 of the source wireless device 448 can transmit content to the transmission component 454 for transmission to the destination wireless device 449. The decoder component 466 can decode the content, and the resulting content can be processed via the AV synchronization operation of the media player component 450. The media player component 450 may include various internal components configured to support the playback of media streams, such as AV synchronization components, decoder components, codec components, etc. The media player component 450 can output content to the display component 452 for presentation to the user of the source wireless device 448. The media player component 450 can transmit notifications to the transmission component 454 of operations performed on the content (e.g., notification of presentation timestamp, notification of or decoding timestamp, notification of AV synchronization operation, notification of content presentation, notification of content discard, etc.). The display component 452 can control the screen, speakers, and / or other user interfaces to present content to the user.

[0107] The transmission component 454 of the source wireless device 448 can transmit packets, such as packets containing content, packets containing metadata, and packets containing messages, to the transmission component 462 of the destination wireless device 449. The transmission component 454 of the source wireless device 448 can receive packets, such as packets containing control information, session information, and reply messages, from the transmission component 462 of the destination wireless device 449. The transmission component 462 of the destination wireless device 449 can receive packets, such as packets containing content, packets containing metadata, and packets containing messages, from the transmission component 454 of the source wireless device 448. The transmission component 462 of the destination wireless device 449 can transmit packets, such as packets containing control information, session information, and reply messages, to the transmission component 454 of the source wireless device 448. As an example, transmission component 454 and transmission component 462 can each be a component configured to establish a Wi-Fi connection with each other and exchange communication through the established Wi-Fi connection (e.g., a WFD component). Components, Apple Components, WiDi components, etc.

[0108] The transmission component 454 of the source wireless device 448 may include a control component 456 configured to control the operation of the transmission component 454. As an example, the control component 456 may monitor the transmission of packets by the transmission component 454, control the transmission bandwidth of the transmission component 454, control the encoder component 458 to generate packets (e.g., transmission packets, presentation packets, etc.), instruct packets to be stored, retrieved, and / or removed from one or more caches and / or buffers 460, determine the capabilities of the destination wireless device 449 via capability messages, monitor notifications from other components of the source wireless device 448 (such as the media player component 450), and indicate the presentation status of packets handled by the media player component 450 to the destination wireless device 449, etc. The transmission component 454 may include an encoder component 458 configured to encode packets for transmission to the destination wireless device 449. For example, the encoder component 458 may generate transmission packets and presentation packets for transmission to the destination wireless device 449. In some operating modes, encoder component 458 can be used to encode frames captured from display component 452. Transmission component 454 may include one or more caches and / or buffers 460 configured to store packets and other data. As an example, one or more caches and / or buffers 460 may include transmission packet buffers (e.g., pre-transmission queues, etc.), which may be a first-in-first-out (FIFO) queue storing transmission packets for transmission to destination wireless device 449.

[0109] The transmission component 462 of the destination wireless device 449 may include a control component 464 configured to control the operation of the transmission component 462. As an example, the control component 464 may monitor the transmission component 462's reception of packets, instruct packets to be stored, retrieved, and / or removed from one or more caches and / or buffers 468, control the decoder component 466 to decode packets from the source wireless device 448, indicate capabilities to the source wireless device 448 via reply messages, determine the presentation status indicated by the source wireless device 448, discard decoded packets, and transmit decoded packets to the media player component 470 of the destination wireless device 449, etc.

[0110] The transmission component 462 may include a decoder component 466 configured to decode packets received from the source wireless device 448. For example, the decoder component 466 may decode both transmission packets and presentation packets received from the source wireless device 448. The transmission component 462 may include one or more caches and / or buffers 468 configured to store packets and other data. As an example, the one or more caches and / or buffers 468 may include a decoded packet buffer (e.g., a pre-transmitted data buffer, etc.) that may store data from decoded transmission packets received from the source wireless device 448.

[0111] The media player component 470 of the receiving wireless device 449 can receive content transmitted from the transmission component 462 from the source wireless device 448. The media player component 470 can decode the content, for example, via controlling the operation of the decoder component 466, and the resulting content can be processed through the AV synchronization operation of the media player component 470. The media player component 470 may include various internal components configured to support the playback of media streams, such as AV synchronization components, decoder components, codec components, etc. The media player component 470 (e.g., the presenter component of the media player component 470) can output content to the display component 472 for presentation to the user of the receiving wireless device 449. The display component 472 can control the screen, speakers, and / or other user interfaces to present content to the user.

[0112] Figure 5 This is a block diagram illustrating various aspects of mirror projection between source wireless device 501 and destination wireless device 502. Refer to Figures 1 to 502. Figure 5 The source radio device 501 (e.g., UE 120, 120a-120e, 200, 320, 448) and the destination radio device 502 (e.g., UE 120, 120a-120e, 200, 320, 449) may have already been based on The standards established WFD sessions with each other (e.g., (E2E session) to transmit a stream 550 for mirroring the display video content 503 of the source wireless device 501 onto the destination wireless device 502 for output on the display of the destination wireless device 502.

[0113] The display resolution of the source wireless device 501 and the display resolution of the destination wireless device 502 can be different. For example, the source wireless device 501 can be a smartphone with a display content resolution of 1080×2400, and the destination wireless device 502 can be a television with a display resolution of 3840×2160.

[0114] In the current WFD mirroring, the source wireless device 501 uses a standard 720P / 1080P / 2160P video encoding-decoding resolution for WFD stream encoding-decoding and sends video content 503 to the destination wireless device 502. In the current WFD mirroring, the source wireless device 501 maintains a constant frame aspect ratio and fills any area outside the source content 503 with black pixels 505 to create a 720P / 1080P / 2160P source encoded frame 504. In the current WFD mirroring, the destination wireless device 502 receives and decodes the video. Stream 550 contains regular 720p / 1080p / 2160p frames 504. In the current WFD mirroring, the source wireless device 501 projects the image at a frame resolution of 3840×2160p30. The content 503 of stream 550 is encoded, and the destination wireless device 502 encodes it at the same frame resolution of 3840×2160p30. Decode the contents of stream 550.

[0115] The padding of black pixels 505 in the current WFD mirror projection does not affect the bitrate because the encoding of black pixels 505 does not add bytes to the compressed stream. However, the padded black pixels 505 are processed in the same way as other pixels in both the source wireless device 501 and the destination wireless device 502. Therefore, when both the source wireless device 501 and the destination wireless device 502 are processing frame resolutions of 3840×2160p30, the power and resources of the source wireless device 501 and the destination wireless device 502 may be wasted on processing redundant padded black pixels 505.

[0116] Various implementations provide methods and systems for supporting the mirroring of video content in a session established between a source wireless device and a destination wireless device. Various implementations provide methods for partial frame regions. Systems and methods for E2E streaming. In a WFD session, various implementations enable the encoding and transmission of only valid frame regions of content from the source wireless device to the destination wireless device. Various implementations enable the decoding of only valid frame regions of content at the destination wireless device. Various implementations enable WFD mirroring without pixel padding (e.g., without black pixel padding or other colored pixel padding).

[0117] Figure 6 A process flowchart of an example method 600 for supporting mirroring of video content in a session established between a source wireless device and a destination wireless device, according to various implementation schemes, is shown. (Reference) Figures 1A to 6 Method 600 can be performed by a processor (such as 210, 212, 214, 216, 218, 252, 260) of a source radio device (such as UE 120, 120a-120e, 200, 320, 448, 501) in a session (such as a WFD session) established with a destination radio device (such as UE 120, 120a-120e, 200, 320, 448, 502). Implemented in E2E sessions, etc. (See reference) Figures 1A to 6The component used to perform each operation of method 600 may be one or more processors of a UE (such as UE 120, 120a-120e, 200, 320, 448, 501), for example, one or more processors 210, 212, 214, 216, 218, 252, 260. In various embodiments, the operation of method 600 may be performed by a source wireless device in response to a session established with a destination wireless device for mirroring video content.

[0118] In box 602, the processor may perform operations including generating and encoding a pure fill color frame. The generation and encoding of the pure fill color frame may be performed in response to establishing a session with the destination wireless device for mirroring video content. The pure fill color frame can be a full frame of any color, such as a pure black frame, a pure blue frame, a pure red frame, etc. The resolution of the encoded pure fill color frame may match the full resolution of the destination wireless device and the WFD session. As an example, the resolution of the encoded pure fill color frame may be 3840×2160. Components for performing the operations of box 602 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 448, 501), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0119] In block 604, the processor may perform operations including transmitting an encoded pure fill color frame to the destination wireless device. For example, the encoded pure fill color frame may be transmitted via... The E2E transport stream is used for transmission. The components used to perform the operation of box 604 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 448, 501), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0120] In box 606, the processor may perform operations including determining a valid frame region of interest. The valid frame region of interest may be... The region of a full-resolution frame in an E2E session, where all pixels are associated with the content being displayed. In other words, the region of interest that is a valid frame area can be... The region of the full-resolution frame in an E2E session, where pixel padding may not be required. For example, the region of interest for mirroring projected video content in a WFD session may be defined by the pixel width and pixel height of the source frame of the content to be mirrored, and the position coordinates of that source frame within the entire frame. The region of interest may be a unique region of the full frame that can be updated when the content is output by the source wireless device. Components for performing the operations of box 606 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 448, 501), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0121] In block 608, the processor may perform operations including transmitting an indication of a valid frame region of interest to the destination wireless device. To transmit the indication of the valid frame region of interest, the source wireless device may transmit to the destination wireless device an indication of the pixel width and pixel height of the source frame of the content to be mirrored, as well as the position coordinates of that source frame within the entire frame. As an example, the indication of the valid frame region of interest may be transmitted as part of a High-Efficiency Video Decoding (HEVC) refresh function and / or in one or more Supplemental Enhancement Information (SEI) messages transmitted from the source wireless device to the destination wireless device. Components for performing the operations of block 608 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 448, 501), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0122] In block 610, the processor may perform operations including generating and encoding video content as partial frames containing only the effective frame region of interest. In various embodiments, after transmitting the first plain-filled color frame, the source wireless device may generate and encode only the effective frame region of interest for the video content when generating and encoding a regular frame next. The source wireless device may generate and encode only the effective frame region of interest for the video content when the effective frame region of interest remains unchanged. Since only the effective frame region of interest is generated and encoded after the first plain-filled color frame is encoded and transmitted, frames after the first frame may be frames without plain-filled color frames, such as frames without black pixel fill. Encoding only the effective frame region allows the source wireless device to transmit partial frames containing only data corresponding to the effective frame region of interest. Components for performing the operations of block 610 may include one or more processors of a UE (such as UE 120, 120a-120e, 200, 320, 448, 501), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0123] In block 612, the processor can perform operations including transmitting partial frames to the destination wireless device. For example, the partial frames can be transmitted via... The E2E transport stream is used for transmission. The components used to perform the operation of block 612 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 448, 501), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0124] In determination box 614, the processor may determine whether a change has occurred in the valid frame region of interest. As an example, the valid frame region may change due to the source wireless device being rotated and / or application preferences changing due to an application running on the source wireless device. As a particular example, the valid frame region may change from 972×2160 to 1620×2160. Components used to perform the operation of determination box 614 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 448, 501), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0125] In response to determining that the region of interest in the valid frame has not changed (i.e., determining box 614 = "No"), the processor may continue to perform operations to generate, encode, and transmit partial frames in boxes 610 and 612. In response to determining that the region of interest in the valid frame has not changed (i.e., determining box 614 = "No"), the processor may generate a new, plain-fill-color frame in box 602. In boxes 604 through 612, the processor may determine a new region of interest in the valid frame, transmit an indication of the region of interest in the valid frame, and generate and encode partial frames.

[0126] Figure 7 A process flowchart of an example method 700 for supporting mirroring of video content in a session established between a source wireless device and a destination wireless device, according to various implementation schemes, is shown. (Reference) Figures 1A to 7 Method 700 can be performed by a processor (such as 210, 212, 214, 216, 218, 252, 260) of a destination wireless device (such as UE 120, 120a-120e, 200, 320, 449, 502) in a session (such as a WFD session) established with a source wireless device (such as UE 120, 120a-120e, 200, 320, 448, 501). Implemented in E2E sessions, etc. (See reference) Figures 1A to 7The component used to perform each operation of method 600 may be one or more processors of a UE (such as UE 120, 120a-120e, 200, 320, 449, 502), for example, one or more processors 210, 212, 214, 216, 218, 252, 260. In various embodiments, the operation of method 700 may be combined with method 600 ( Figure 6 The operation of method 700 can be performed by the destination wireless device in response to a session established with the source wireless device for mirroring video content. In various implementations, the operation of method 700 can be performed by the destination wireless device in response to a session established with the source wireless device for mirroring video content.

[0127] In block 702, the processor may perform operations including receiving an encoded pure fill color frame. Receiving the encoded pure fill color frame may be in response to establishing a session with the source wireless device for mirroring video content. For example, the encoded pure fill color frame may be received via… The E2E transport stream is received. The pure fill color frame can be a full frame of any color, such as a pure black frame, a pure blue frame, a pure red frame, etc. The resolution of the encoded pure fill color frame can match the full resolution of the destination wireless device and the WFD session. As an example, the resolution of the encoded pure fill color frame can be 3840×2160. Components used to perform the operations of box 702 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 449, 502), for example, one or more processors 210, 212, 214, 216, 218, 252, 260.

[0128] In block 704, the processor may perform operations including decoding an encoded pure fill color frame into a first full frame. Decoding the pure fill color full frame provides the destination wireless device with a single-color full-resolution frame, such as a full-resolution black frame, a full-resolution blue frame, etc. Components for performing the operations in block 704 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 449, 502), for example, one or more processors 210, 212, 214, 216, 218, 252, 260.

[0129] In box 706, the processor may perform operations including receiving an indication of a valid frame region of interest from the source wireless device. The valid frame region of interest may be... The region of a full-resolution frame in an E2E session, where all pixels are associated with the content being displayed. In other words, the region of interest that is a valid frame area can be... The region of full-resolution frame in an E2E session, where pixel padding may not be required. For example, the region of interest (ROI) for mirroring projected video content in a WFD session may be defined by the pixel width and pixel height of the source frame of the content to be mirrored, and the position coordinates of that source frame within the entire frame. The ROI may be a unique region of the entire frame that can be updated when the content is output from the source wireless device. An indication of the ROI may be a direct indication to the destination wireless device of the pixel width and pixel height of the source frame of the content to be mirrored, and the position coordinates of that source frame within the entire frame. As an example, the indication of the ROI may be received as part of an Efficient Video Decoding (HEVC) refresh function and / or in one or more Supplemental Enhancement Information (SEI) messages received from the source wireless device. The components used to perform the operation of box 706 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 449, 502), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0130] In block 708, the processor may perform operations including receiving encoded partial frames from the source wireless device. In various embodiments, the destination wireless device may receive encoded partial frames after receiving an encoded plain-filled color frame as a first full frame. For example, the encoded partial frame may be received via… The E2E transport stream is received. The components used to perform the operation of box 708 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 449, 502), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0131] In block 710, the processor may perform operations including decoding an encoded partial frame to generate a partial frame update for the region of interest. The destination wireless device may decode the encoded partial frame to generate a partial frame corresponding to the region of interest. The destination wireless device may use the partial frame corresponding to the region of interest to update the full frame. Since only the coordinates corresponding to the region of interest can be updated, no resource cost is incurred for the full frame region outside the region of interest, because the full frame region outside the region of interest is not updated and retains the original pure fill color used in the first full frame. Components for performing the operations of block 710 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 449, 502), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0132] In block 712, the processor may perform operations including displaying a new full frame. The new full frame may be a mixture of a plain fill color outside the valid frame area of ​​interest and updated content from the source radio device within the valid frame area of ​​interest. Components for performing the operations in block 712 may include one or more processors of the UE (such as UE 120, 120a-120e, 200, 320, 449, 502), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0133] In determination block 714, the processor may perform operations including determining whether to receive an encoded new pure fill color frame. In various embodiments, the source radio device may transmit an encoded new pure fill color frame when the region of interest changes. Components for performing the operations of block 714 may include one or more processors of a UE (such as UE 120, 120a-120e, 200, 320, 449, 502), such as one or more processors 210, 212, 214, 216, 218, 252, 260.

[0134] In response to determining that no new pure fill color frame has been received (i.e., determination box 716 = "No"), the processor may continue receiving encoded partial frames in box 708 and perform the operations of boxes 710, 712, and 714 to continue updating the active frame region of interest. In response to determining that a new pure fill color frame has been received (i.e., determination box 716 = "Yes"), the processor may decode the encoded pure fill color frame into a new full frame in box 704 and perform the operations of boxes 706, 708, 710, 712, and 714 to display a partial update of the new full frame for the new active frame region of interest.

[0135] Figure 8 This is a block diagram illustrating various aspects of mirror projection between a source wireless device 501 and a destination wireless device according to various implementation schemes. Refer to Figures 1 to 501. Figure 8 The source radio device 501 (e.g., UE 120, 120a-120e, 200, 320, 448) and the destination radio device 502 (e.g., UE 120, 120a-120e, 200, 320, 449) may have already been based on The standards established WFD sessions with each other (e.g., (E2E session) to transmit a stream 550 for mirroring the display video content 503 of the source wireless device 501 onto the destination wireless device 502 for output on the display of the destination wireless device 502. Figure 8 Instantiable method 600( Figure 6 ) and / or 700 ( Figure 7This is a specific implementation example of one or more operations. The first full frame 601 can be a pure fill color frame transmitted at the full resolution of the session, such as a pure black frame, a pure blue frame, etc. As an example, a full frame resolution of 3480×2160 can be established between the source wireless device and the destination wireless device. E2E session.

[0136] Source wireless device 501 can generate and encode a pure black first frame 601 at a resolution of 3480×2160 and transmit the encoded first frame 601 to destination wireless device 502. Destination wireless device 502 can decode the first frame 601 and output a pure black first frame. Source wireless device 501 can determine that the region of interest in the video content is the 972×2160 source frame, and can indicate the 972×2160 width / height and position coordinates of the source frame in the entire frame to destination wireless device 502, for example, in an SEI message. After the first frame, source wireless device 501 can generate and encode only the region of interest in all frames, specifically only the 972×2160 frame region, and transmit these encoded partial frames 603 to destination wireless device 502.

[0137] The receiving wireless device 502 can receive and decode a portion of frame 603 and update the previous full frame using a portion of the valid frame region of interest, thereby changing only the 972×2160 frame region during each subsequent frame presentation. In the newly presented updated frame 604, the region of the full frame 604 outside the valid frame region of interest 603 can remain pure black as in the original first frame 601, and the source wireless device 501 and the receiving wireless device 502 only need to process, generate, encode, and / or decode the 972×2160 frame and the corresponding macroblock.

[0138] Various implementation schemes can be implemented on various wireless devices (e.g., wireless devices 120, 120a-120e, 200, 320, 448, 449, 501, 502), examples of which are provided below. Figure 9 This example uses a smartphone 900. (See reference) Figures 1A to 9 The smartphone 900 may include a first SOC 202 (e.g., an SOC-CPU) coupled to a second SOC 204 (e.g., a 5G-capable SOC). The first SOC 202 and the second SOC 204 may be coupled to internal memories 906, 916, a display 912, and a speaker 914. Additionally, the smartphone 900 may include an antenna 904 for transmitting and receiving electromagnetic radiation that can be connected to a wireless data link and / or a cellular transceiver 266 coupled to one or more processors in the first SOC 202 and / or the second SOC 204. The smartphone 900 typically also includes menu selection buttons or a joystick switch 920 for receiving user input.

[0139] A typical smartphone 900 also includes a voice codec (CODEC) circuit 910 that digitizes sound received from the microphone into data packets suitable for wireless transmission and decodes the received voice data packets to generate an analog signal for the speaker to produce sound. Furthermore, one or more of the processor in the first SOC 202 and the second SOC 204, the wireless transceiver 266, and the CODEC 910 may include digital signal processor (DSP) circuitry (not shown separately).

[0140] The processors of the head-mounted device 1572, the wireless network computing device 1600, and the smartphone 900 can be any programmable microprocessor, microcomputer, or one or more multiprocessor chips capable of being configured via software instructions (applications) to perform various functions, including those described in the various embodiments below. In some wireless devices, multiple processors may be provided, such as one processor within SOC 204 dedicated to wireless communication functions and another within SOC 202 dedicated to running other applications. Typically, software applications may be stored in memories 906, 916, and then accessed and loaded into the processor. The processor may include sufficient internal memory to store application software instructions.

[0141] Various implementation schemes can also be implemented on various wireless devices (e.g., wireless devices 120, 120a-120e, 200, 320, 448, 449, 501, 502), such as Figure 10 The example shown is display unit 1000. (Reference) Figures 1A to 10 The display unit 1000 may include a first SOC 202 (e.g., an SOC-CPU) coupled to a second SOC 204 (e.g., a SOC with 5G capability). The first SOC 202 and the second SOC 204 may be coupled to a memory 1004. The display unit 1000 may include a speaker 1006, which may be connected to the SOCs 202 and 204 and configured to output sound. The display unit 1000 may also include one or more radio transceivers 266 (e.g., [missing information]) coupled to each other and / or coupled to the SOCs 202 and 204 for transmitting and receiving signals. (RF radio, etc.) and antenna. The transceiver 266 and antenna can be used to implement various wireless transmission protocol stacks and interfaces. The display unit 1000 may also include a touchpad 1008 and a screen 1012, both fully coupled to the SOCs 202 and 204.

[0142] As used in this application, the terms "component," "module," "system," etc., are intended to include computer-related entities such as, but not limited to, hardware, firmware, combinations of hardware and software, software, or software being executed, configured to perform specific operations or functions. For example, a component can be, but is not limited to, a process, processor, object, executable file, execution thread, program, and / or computer running on a processor. By way of illustration, both an application running on a wireless device and the wireless device itself can be referred to as a component. One or more components may reside within a process and / or execution thread, and components may reside on a processor or core and / or be distributed across two or more processors or cores. Furthermore, these components may execute from various non-transitory computer-readable media on which various instructions and / or data structures are stored. Components may communicate via local and / or remote processes, function or procedure calls, electronic signals, data packets, memory read / write, and other known network, computer, processor, and / or process-related communication methods.

[0143] Several different cellular and mobile communication services and standards are available and envisioned for the future, all of which are feasible and benefit from various implementation schemes. These services and standards include, for example, the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE) systems, 3rd Generation (3G), 4th Generation (4G), 5th Generation (5G), Global System for Mobile Communications (GSM), Universal Mobile Telecommunications System (UMTS), 3GSM, General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA) systems (e.g., cdmaOne, CDMA1020TM), Enhanced Data Rate Evolution of GSM (EDGE), Advanced Mobile Phone Systems (AMPS), Digital AMPS (IS-136 / TDMA), Evolved Data Optimization (EV-DO), Digital Enhanced Cordless Telecommunications (DECT), WiMAX, and Wireless Local Area Networks (WLAN). Protected Access I and II (WPA, WPA2), and Integrated Digital Enhanced Network (iDEN). Each of these technologies relates to the transmission and reception of, for example, voice, data, signaling, and / or content messages. It should be understood that any references to terms and / or technical details relating to individual telecommunications standards or technologies are for illustrative purposes only and are not intended to limit the scope of the claims to a particular communication system or technology, unless specifically stated in the language of the claims.

[0144] The various embodiments illustrated and described are provided merely as examples illustrating the various features of the claims. However, the features shown and described with respect to any given embodiment are not necessarily limited to the associated embodiment and may be used or combined with other embodiments shown and described. Furthermore, the claims are not intended to be limited to any one of the example embodiments. For example, one or more operations of method 600 and / or 700 may replace or be combined with one or more operations of method 600 and / or 700.

[0145] The following paragraphs describe specific implementation examples. While some of the following specific implementation examples are described based on example methods, other example implementations may include: example methods implemented by a computing device, as discussed in the following paragraphs, the computing device including a processor configured to perform operations of these example methods; example methods implemented by a computing device, as discussed in the following paragraphs, the computing device including components for performing the functionality of the example methods; and example methods implemented by a non-transitory processor-readable storage medium, as discussed in the following paragraphs, having processor-executable instructions stored thereon, the processor-executable instructions being configured to cause the processor of the computing device to perform the operations of the example methods.

[0146] Example 1. A method for supporting mirroring of video content in a session established between a source wireless device and a destination wireless device, the method comprising: in response to establishing a session with the destination wireless device for mirroring video content, generating and encoding a plain fill color frame by a processor of the source wireless device; transmitting the encoded plain fill color frame to the destination wireless device by the processor of the source wireless device; determining a region of interest (REF) by the processor of the source wireless device; transmitting an indication of the REF to the destination wireless device by the processor of the source wireless device; generating and encoding video content of only the REF as a partial frame by the processor of the source wireless device; and transmitting the partial frame to the destination wireless device by the processor of the source wireless device.

[0147] Example 2. According to the method of Example 1, the method further includes: determining, by the processor of the source wireless device, that a change has occurred in the region of interest (RI); in response to determining that the change has occurred in the RI, generating and encoding a new plain fill color frame by the processor of the source wireless device; transmitting the encoded new plain fill color frame to the destination wireless device by the processor of the source wireless device; determining, by the processor of the source wireless device, a new RI; transmitting, by the processor of the source wireless device, an indication of the new RI to the destination wireless device; generating and encoding, by the processor of the source wireless device, only the video content of the new RI as a new partial frame; and transmitting, by the processor of the source wireless device, the new partial frame to the destination wireless device.

[0148] Example 3. The method according to any one of Examples 1 or 2, wherein the session for mirroring video content is End-to-end (E2E) session.

[0149] Example 4. According to the method described in Example 3, the pure fill color frame is a pure black frame.

[0150] Example 5. The method according to any one of Examples 1 to 4, wherein the indication of the effective frame region of interest is an indication of the pixel height, pixel width and coordinates of the entire frame of the effective frame region of interest.

[0151] Example 6. The method according to Example 5, wherein the indication of the effective frame region of interest is a Supplemental Enhancement Information (SEI) message.

[0152] Example 7. A method for supporting mirroring of video content in a session established between a source wireless device and a destination wireless device, the method comprising: in response to establishing a session with the source wireless device for mirroring of video content, receiving, by a processor of the destination wireless device, an encoded pure fill color frame; decoding, by the processor of the destination wireless device, the encoded pure fill color frame into a first full frame; receiving, by the processor of the destination wireless device, an indication of a valid frame region of interest from the source wireless device; receiving, by the processor of the destination wireless device, an encoded partial frame from the source wireless device; decoding, by the processor of the destination wireless device, the encoded partial frame to generate a partial frame update for the valid frame region of interest; applying, by the processor of the destination wireless device, the partial frame update to the first full frame to generate a new full frame; and displaying, by the processor of the destination wireless device, the new full frame.

[0153] Example 8. The method according to Example 7, wherein the session for mirroring video content is End-to-end (E2E) session.

[0154] Example 9. According to the method described in Example 8, the pure fill color frame is a pure black frame.

[0155] Example 10. The method according to any one of Examples 7 to 9, wherein the indication of the effective frame region of interest is an indication of the pixel height, pixel width and coordinates of the entire frame of the effective frame region of interest.

[0156] Example 11. The method according to Example 10, wherein the indication of the effective frame region of interest is a Supplemental Enhancement Information (SEI) message.

[0157] The foregoing method descriptions and process flowcharts are provided as illustrative examples only and are not intended to require or imply that the operations of the various embodiments must be performed in the given order. As those skilled in the art will appreciate, the operations of the foregoing embodiments can be performed in any order. Words such as “afterward,” “then,” “next,” etc., are not intended to limit the order of operations; these words are used to guide the reader through the description of the method. Furthermore, any reference to singular claim elements (e.g., references using the articles “a,” “an,” or “described”) should not be construed as limiting that element to the singular.

[0158] The various exemplary logic blocks, modules, components, circuits, and algorithmic operations described in conjunction with the embodiments disclosed herein can be implemented as electronic hardware, computer software, or a combination of both. To clearly illustrate this interchangeability between hardware and software, various exemplary components, blocks, modules, circuits, and operations have been generally described above in terms of their functionality. Whether such functionality is implemented as hardware or software depends on the specific application and the design constraints imposed on the overall system. Those skilled in the art may implement the described functionality in different ways for each specific application, but such implementation decisions should not be construed as causing a departure from the scope of the claims.

[0159] Hardware for implementing the various exemplary logic units, logic blocks, modules, and circuits described in conjunction with the embodiments disclosed herein may be implemented or executed using a general-purpose processor, digital signal processor (DSP), application-specific integrated circuit (ASIC), field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic units, discrete hardware components, or any combination thereof. While the general-purpose processor may be a microprocessor, in alternative embodiments, the processor may be any conventional processor, controller, microcontroller, or state machine. The processor may also be implemented as a combination of receiver intelligent objects, such as a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors combined with a DSP core, or any other such configuration. Alternatively, some operations or methods may be performed by circuitry specific to a given function.

[0160] In one or more embodiments, the described functionality can be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functionality may be stored as one or more instructions or code on a non-transitory computer-readable storage medium or a non-transitory processor-readable storage medium. The operation of the methods or algorithms disclosed herein may be implemented in a processor-executable software module or processor-executable instructions, which may reside on a non-transitory computer-readable or processor-readable storage medium. A non-transitory computer-readable or processor-readable storage medium may be any storage medium that can be accessed by a computer or processor. By way of example and without limitation, such non-transitory computer-readable or processor-readable storage media may include RAM, ROM, EEPROM, FLASH memory, CD-ROM or other optical disc storage devices, magnetic disk storage devices or other magnetic storage intelligent objects, or any other medium that can be used to store desired program code in the form of instructions or data structures and is accessible by a computer. As used herein, disks and optical discs include compact optical discs (CDs), laser discs, optical discs, digital versatile optical discs (DVDs), floppy disks, and Blu-ray discs, wherein disks typically magnetically reproduce data, while optical discs optically reproduce data using lasers. The above combinations are also included within the scope of non-transitory computer-readable and processor-readable media. Additionally, the operation of a method or algorithm may reside as one or any combination or set of code and / or instructions on a non-transitory processor-readable and / or computer-readable storage medium, which may be incorporated into a computer program product.

[0161] The above description of the disclosed embodiments is provided to enable any person skilled in the art to implement or use the claims. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be applied to other embodiments without departing from the scope of the claims. Therefore, this disclosure is not intended to be limited to the embodiments shown herein, but should be granted the broadest scope consistent with the following claims and the principles and novel features disclosed herein.

Claims

1. A wireless device, comprising: Wireless transceiver; as well as a processor coupled to the wireless transceiver and configured to: generating and encoding a solid fill color frame in response to establishing a session with a sink wireless device for mirroring of video content; transmitting the encoded solid fill color frame to the sink wireless device; Determine the valid frame region of interest; transmitting an indication of the valid frame region of interest to the sink wireless device; Generate and encode the video content of only the valid frame area of ​​interest as a partial frame; as well as The partial frame is transmitted to the sink wireless device.

2. The wireless device of claim 1 , wherein the processor is further configured to: determining that a change has occurred in the active frame region of interest; generating and encoding, by a processor of the source wireless device, a new solid fill color frame in response to determining that the change has occurred in the active frame region of interest; transmitting the encoded new solid fill color frame to the sink wireless device; determining a new valid frame region of interest; transmitting an indication of the new valid frame region of interest to the sink wireless device; Generate and encode the video content of only the new valid frame area of ​​interest as a new partial frame; as well as The new partial frame is transmitted to the sink wireless device.

3. The wireless device of claim 1 , wherein the session for mirroring of video content is End-to-end (E2E) session. The wireless device of claim 3 , wherein the solid fill color frame is a solid black frame.

5. The wireless device of claim 1, wherein the indication of the active frame region of interest is an indication of pixel height, pixel width, and coordinates in a full frame of the active frame region of interest.

6. The wireless device of claim 5, wherein the indication of the active frame region of interest is a Supplemental Enhancement Information (SEI) message.

7. A method for supporting mirroring of video content in a session established between a source wireless device and a sink wireless device, the method comprising: generating and encoding, by a processor of the source wireless device, a solid fill color frame in response to establishing a session for mirroring of video content with the sink wireless device; transmitting, by the processor of the source wireless device to the sink wireless device, the encoded solid fill color frame; determining, by the processor of the source wireless device, a valid frame region of interest; transmitting, by the processor of the source wireless device to the sink wireless device, an indication of the valid frame region of interest; generating and encoding, by the processor of the source wireless device, video content of only the active frame region of interest as a partial frame; as well as The partial frame is transmitted by the processor of the source wireless device to the sink wireless device.

8. The method according to claim 7, further comprising: determining, by the processor of the source wireless device, that a change has occurred in the active frame region of interest; generating and encoding, by a processor of the source wireless device, a new solid fill color frame in response to determining that the change has occurred in the active frame region of interest; transmitting, by the processor of the source wireless device to the sink wireless device, the encoded new solid fill color frame; determining, by the processor of the source wireless device, a new valid frame region of interest; transmitting, by the processor of the source wireless device to the sink wireless device, an indication of the new valid frame region of interest; generating and encoding, by the processor of the source wireless device, video content of only the new valid frame region of interest as a new partial frame; as well as The new partial frame is transmitted by the processor of the source wireless device to the sink wireless device.

9. The method of claim 7, wherein the session for mirroring of video content is End-to-end (E2E) session.

10. The method of claim 9, wherein the solid fill color frame is a solid black frame.

11. The method of claim 7, wherein the indication of the active frame region of interest is an indication of pixel height, pixel width, and coordinates in a full frame of the active frame region of interest.

12. The method of claim 11, wherein the indication of the active frame region of interest is a Supplemental Enhancement Information (SEI) message.

13. A wireless device, comprising: Wireless transceiver; as well as a processor coupled to the wireless transceiver and configured to: receiving an encoded solid fill color frame in response to establishing a session with a source wireless device for mirroring of video content; Decoding the encoded pure fill color frame into a first full frame; receiving, from the source wireless device, an indication of a valid frame region of interest; receiving an encoded partial frame from the source wireless device; decoding the encoded partial frame to generate a partial frame update for the valid frame region of interest; applying the partial frame update to the first full frame to generate a new full frame; as well as The new full frame is displayed.

14. The wireless device of claim 13, wherein the session for mirroring of video content is End-to-end (E2E) session. The wireless device of claim 14 , wherein the solid fill color frame is a solid black frame.

16. The wireless device of claim 13, wherein the indication of the active frame region of interest is an indication of pixel height, pixel width, and coordinates in a full frame of the active frame region of interest.

17. The wireless device of claim 16, wherein the indication of the active frame region of interest is a Supplemental Enhancement Information (SEI) message.

18. A method for supporting mirroring of video content in a session established between a source wireless device and a sink wireless device, the method comprising: receiving, by a processor of the sink wireless device, an encoded solid fill color frame in response to establishing a session for mirroring of video content with the source wireless device; decoding, by the processor of the sink wireless device, the encoded solid fill color frame into a first full frame; receiving, by the processor of the sink wireless device, from the source wireless device, an indication of a valid frame region of interest; receiving, by the processor of the sink wireless device, an encoded partial frame from the source wireless device; decoding, by the processor of the sink wireless device, the encoded partial frame to generate a partial frame update for the active frame region of interest; applying, by the processor of the sink wireless device, the partial frame update to the first full frame to generate a new full frame; as well as The new full frame is displayed by the processor of the sink wireless device.

19. The method of claim 18, wherein the session for mirroring of video content is End-to-end (E2E) session.

20. The method of claim 19, wherein the solid fill color frame is a solid black frame.

21. The method of claim 18, wherein the indication of the active frame region of interest is an indication of pixel height, pixel width, and coordinates in a full frame of the active frame region of interest.

22. The method of claim 21, wherein the indication of the active frame region of interest is a Supplemental Enhancement Information (SEI) message.

Images