System for relocating data in video stream and method thereof

Abstract

A method and system (304) for relocating data associated with a video stream. The method includes identifying data from a plurality of frames of a video stream being played on a user device (302). The method includes identifying a plurality of regions (718, 720) and separating first data and second data from the identified data. The method includes determining a visibility index for the first data based on a plurality of parameters of the video stream, and relocating the first data in at least one of the plurality of regions (718, 720) based on the determined visibility index. The repositioned first data is visible on a user interface (308) associated with the target device (306).

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the field of video streaming, and more specifically, to a system and method of repositioning data, wherein the data includes at least one of lost data or data with reduced visibility associated with a video stream. BACKGROUND

[0002] Conventionally, a user would prefer to watch a video on an electronic device, such as a television. Further, with advancements in technology, a user has multiple options (e.g., a smart phone, a smart television, a laptop, etc.) to watch a video. Among the multiple options, a user prefers to watch a video on their smart phone as the smart phone is easy to carry and provides a comfortable viewing experience to the user. However, as shown in Figure 1 there are limitations in watching a video on a smart phone: when a user watches a video in a normal screen mode in the smart phone 102, in this case, the user encounters a problem of video resolution being unclear, which also hinders the readability of data (i.e., text information 104 provided in the video). Further, in another case, when a user watches a video in a full screen mode, in this case, the user experiences good video resolution, however, the data 104 provided in the video is lost.

[0003] Additionally, a user can face a problem of losing data in a video when watching the video in different scenarios. As shown in Figure 2 in one scenario, during multitasking 202 (i.e., when a user is streaming a video on a television and working on a smart phone), in this case, the user will not be able to see the data on the smart phone. In another scenario, when a user watches a video in the background (picture-in-picture mode) 204 on a smart phone while performing some other task on the smart phone, etc. In yet another scenario, when a user watches a video on a smart watch 206, and finally, when a user watches a video in low resolution 208. In all of the above scenarios, a user faces a problem of losing data while watching a video.

[0004] Thus, due to different aspect ratios, poor resolution, size of content, and a user’s divided attention, there is a trade-off between good video resolution and text information provided in the video.

[0005] In this regard, many technical solutions have been developed to overcome the above-mentioned problems. For example, in known technology, a filter is added for underexposed areas. Specifically, edges are determined and a smoothed version of the image is generated. Additionally, a grayscale version of the image is generated and the two images are compared and the underexposed areas are additionally exposed. However, the known technology focuses on providing illumination to underexposed areas. Thus, the technical problems discussed earlier remain in the prior art.

[0006] Thus, it is advantageously and desirably provided a system and method capable of maintaining data such that a user can read the data while eliminating or mitigating at least one of the above-mentioned technical problems. SUMMARY

[0007]

TECHNICAL SOLUTION

[0008] In an embodiment, the present disclosure discloses a system configured to reposition data associated with a video stream. The system includes a memory and at least one processor. The at least one processor is communicatively coupled with the memory. The at least one processor is configured to identify data from a plurality of frames of a video stream playing on a user equipment (UE). The data includes at least one of missing data or data having reduced visibility. The at least one processor is configured to identify a plurality of regions from the plurality of frames of the video stream. The at least one processor is configured to separate relevant data and irrelevant data from the identified data in at least one of the plurality of regions. The at least one processor is configured to determine a visibility index of the relevant data based on a plurality of parameters of the video stream. The at least one processor is configured to reposition the relevant data in at least one of the plurality of regions based on the determined visibility index. The repositioned relevant data is visible on a user interface associated with a target device.

[0009] In an embodiment, a method of repositioning data associated with a video stream is also disclosed herein. The method includes identifying data from a plurality of frames of a video stream playing on a user equipment (UE). The data includes at least one of missing data or data having reduced visibility. The method includes identifying a plurality of regions from the plurality of frames of the video stream. The method also includes separating relevant data and irrelevant data from the identified data in at least one of the plurality of regions. The method also includes determining a visibility index of the relevant data based on a plurality of parameters of the video stream. The method includes repositioning the relevant data in at least one of the plurality of regions based on the determined visibility index. The repositioned relevant data is visible on a user interface associated with a target device.

[0010] In an embodiment, the present disclosure provides a method for repositioning data associated with a video stream, the method comprising: identifying data from a plurality of frames of a video stream being displayed on a user equipment (UE), wherein the data comprises at least one of missing data or data having reduced visibility; identifying a plurality of regions from the plurality of frames of the video stream based on the identified content; separating relevant data and irrelevant data from the identified data provided in at least one region of the plurality of regions; determining a visibility index of the relevant data based on a plurality of parameters associated with the video stream; and repositioning the relevant data in at least one region of the plurality of regions based on the determined visibility index, wherein the repositioned relevant data is visible on a user interface associated with a target device.

[0011] Prior to identifying the data from the content associated with the plurality of frames, the method comprises: identifying at least one mode from a plurality of modes of the video stream, wherein the plurality of modes comprises one of a display mode and a hidden mode. The method can comprise: determining a visibility index of at least one frame of the plurality of frames based on a plurality of parameters, wherein the plurality of parameters comprises at least one of resolution, content size, brightness, distance, and orientation; and determining a state of data visibility in content of the video stream based on a state of a plurality of factors associated with the video stream and at least interest of a user, wherein the plurality of factors comprises at least one of a pause state of the video stream, a user interaction with the UE, a user interaction with a linked device of the UE, a user interaction with the target device, and missing content from the video stream, and wherein the state of data visibility comprises at least one of missing data or data having reduced visibility.

[0012] In some embodiments, to identify the data from the content associated with the plurality of frames, the method can comprise: transforming at least one frame of the plurality of frames into a plurality of grid blocks; and determining a visibility index of each block of the at least one frame based on a plurality of parameters to identify the data comprising at least one of missing data or data having reduced visibility, wherein the plurality of parameters comprises at least one of resolution, content size, brightness, distance, and orientation. Prior to identifying the plurality of regions from the plurality of frames of the video stream, the method for determining the plurality of regions can comprise: determining a first plurality of regions from the plurality of frames in an initial state of the video stream; determining a second plurality of regions from the plurality of frames of the video stream after a specified time interval; comparing the first plurality of regions and the second plurality of regions to determine the plurality of regions, wherein the plurality of regions comprises static regions and dynamic regions; and determining a visibility index of the determined plurality of regions.

[0013] After separating relevant data and irrelevant data from the identified data (the data is from at least one of the plurality of regions), the method can include determining at least one of the plurality of regions with the identified data, wherein the at least one of the plurality of regions is a static region; determining a visibility index of a plurality of blocks of the at least one of the plurality of regions; comparing the determined visibility index to a threshold visibility index; and based on the comparison, extracting the relevant data from the identified data in the at least one of the plurality of regions. To reposition the relevant data in the at least one of the plurality of regions, the method can include determining a size of the at least one of the plurality of regions, wherein the at least one of the plurality of regions is a static region; determining a contrast of the at least one of the plurality of regions to the relevant data when the size of the at least one of the plurality of regions is optimal for the relevant data; determining a visibility index of the at least one of the plurality of regions and the relevant data; and repositioning the relevant data in the at least one of the plurality of regions, wherein the repositioned relevant data is visible on a user interface associated with the target device. When the size of the at least one of the plurality of regions is not optimal for the relevant data, the method can include forming an overlay layer; identifying a shape of the overlay layer; dividing the relevant data into a plurality of portions to be placed within the identified shape of the overlay layer; and repositioning the relevant data in the at least one of the plurality of regions, wherein the repositioned relevant data is visible on a user interface associated with the target device. To form the overlay layer, the method can include extracting a color matrix of the at least one of the plurality of regions, wherein the at least one of the plurality of regions is a static region; selecting a color spectrum with contrasting colors; determining a color coefficient of each color provided in the color spectrum; determining a contrast coefficient of each color to the color matrix of the static region to select an optimal color; determining a visibility index to generate an optimal shape with the optimal color; applying the optimal color and the optimal shape to the overlay layer when the visibility index is optimal; and placing the overlay layer on a user interface of the target device, wherein the overlay layer is configured to place the plurality of portions of the relevant data on the user interface of the target device. Prior to placing the overlay layer on the user interface of the target device, the method can include selecting a background of the overlay layer, wherein the background is selected based on a combination of the static region, a background color of the static region corresponding to the relevant data, and a foreground color of the static region corresponding to the relevant data; and selecting a font of the plurality of portions of the relevant data. Responsive to receiving a user input indicating a switch from the video stream, the relevant data can be repositioned on the user interface of the target device.

[0014] In an embodiment, the present disclosure provides a system for repositioning data associated with a video stream, the system comprising: a memory; and at least one processor communicatively coupled with the memory, the at least one processor configured to: identify data from content associated with a plurality of frames of a video stream being displayed on a user equipment (UE), wherein the data comprises at least one of missing data or data having reduced visibility; identify a plurality of regions from the plurality of frames of the video stream based on the identified content; separate relevant data and irrelevant data from the identified data provided in at least one region of the plurality of regions; determine a visibility index of the relevant data based on a plurality of parameters associated with the video stream; and reposition the relevant data in the at least one region of the plurality of regions based on the determined visibility index, wherein the repositioned relevant data is visible on a user interface associated with a target device.

[0015] The target device can be one of the UE or different from the UE, in which the target device (306) is wirelessly connected with the UE. The target device can be at least one of a smart watch, a television, a desktop computer, a laptop computer, a smart phone. Prior to identifying the data from the content associated with the plurality of frames, the at least one processor can be configured to: identify at least one mode from a plurality of modes of the video stream, wherein the plurality of modes comprises one of a display mode and a hidden mode.

[0016] The at least one processor can be configured to: determine a visibility index of at least one frame of the plurality of frames based on a plurality of parameters, wherein the plurality of parameters comprises at least one of a resolution, a content size, a brightness, a distance, and an orientation; and determine a state of visibility of data in content of the video stream based on a state of a plurality of factors associated with the video stream and at least an interest of a user, wherein the plurality of factors comprises at least one of a paused state of the video stream, an interaction of the user with the UE, an interaction of the user with a linked device of the UE, an interaction of the user with the target device, and missing content from the video stream, and wherein the state of visibility of data comprises at least one of missing data or data having reduced visibility.

[0017] To identify data from content associated with a plurality of frames, the at least one processor can be configured to: transform at least one frame of the plurality of frames into a plurality of grid blocks; and determine a visibility index for each block of the at least one frame based on a plurality of parameters to identify data comprising at least one of missing data or data having reduced visibility, wherein the plurality of parameters comprises at least one of resolution, content size, brightness, distance, and orientation. Prior to identifying a plurality of regions from the plurality of frames of the video stream, to determine the plurality of regions, the at least one processor can be configured to: determine a first plurality of regions from the plurality of frames at an initial state of the video stream; determine a second plurality of regions from the plurality of frames after a specified time interval; compare the first plurality of regions and the second plurality of regions to determine the plurality of regions, wherein the plurality of regions comprises static regions and dynamic regions; and determine a visibility index for the determined plurality of regions. After separating relevant data and irrelevant data from the identified data (the data being from at least one region of the plurality of regions), the at least one processor can be configured to: determine at least one region of the plurality of regions having the identified data, wherein the at least one region of the plurality of regions is a static region; determine a visibility index for a plurality of blocks of the at least one region of the plurality of regions; compare the determined visibility index with a threshold visibility index; and based on the comparison, extract the relevant data from the identified data in the at least one region of the plurality of regions.

[0018] To relocate the relevant data in at least one of the plurality of regions, the at least one processor can be configured to: determine a size of at least one of the plurality of regions, wherein the at least one of the plurality of regions is a static region; determine a contrast of the at least one of the plurality of regions and the relevant data when the size of the at least one of the plurality of regions is optimal for the relevant data; determine a visibility index of the at least one of the plurality of regions; and relocate the relevant data in the at least one of the plurality of regions, wherein the relocated relevant data is visible on the user interface associated with the target device. When the size of the at least one of the plurality of regions is not optimal for the relevant data, the at least one processor can be configured to: form an overlay layer; identify a shape of the overlay layer; divide the relevant data into a plurality of portions to be placed within the identified shape of the overlay layer; and relocate the relevant data in the at least one of the plurality of regions, wherein the relocated relevant data is visible on the user interface associated with the target device. To form the overlay layer, the at least one processor can be configured to: extract a color matrix of the at least one of the plurality of regions, wherein the at least one of the plurality of regions is a static region; select a color spectrum with contrasting colors; determine a color coefficient of each color provided in the color spectrum; determine a contrast coefficient of each color to the color matrix of the static region to select an optimal color; determine a visibility index to generate an optimal shape with the optimal color; apply the optimal color and the optimal shape to the overlay layer when the visibility index is optimal; and place the overlay layer on the user interface of the target device, wherein the overlay layer is configured to place the plurality of portions of the relevant data on the user interface of the target device. Prior to placing the overlay layer on the user interface of the target device, the at least one processor can be configured to: select a background of the overlay layer, wherein the background is selected based on a combination of the static region, a background color of the static region corresponding to the relevant data, and a foreground color of the static region corresponding to the relevant data; and select a font of the plurality of portions of the relevant data. The relevant data can be relocated on the user interface of the target device in response to receiving a user input indicating a switch from the video stream.

[0019] In an embodiment, the present disclosure provides a method of repositioning data for display. The method can include computing a visibility index of a (complete) frame of a video stream being played in order to determine whether the frame includes non-visible data (e.g., missing data and / or data with reduced visibility). The method can include dividing the frame into a plurality of blocks. The method can include computing a visibility index of each block of the plurality of blocks. The method can include determining a plurality of regions in the frame, each region of the plurality of regions including a set of connected blocks (e.g., vertically or horizontally adjacent blocks with similar color / opacity values) among the plurality of blocks. The method can include identifying at least one static region (e.g., a region with little or no movement / change over a series of frames of the video stream) from among the plurality of regions. The method can include identifying at least one data-containing region containing relevant / extractable data (e.g., textual or numerical information) from among the plurality of regions. The method includes computing an average visibility index of the data-containing region using the computed visibility index of each block of the data-containing region. If the average visibility index is below a threshold, the method can include extracting the relevant / extractable data from the data-containing region. The method can include positioning the extracted relevant / extractable data in the static region for display. The visibility index will generally represent the readability / visibility of the content in the frame to a user. The readability / visibility of the content in the frame to a user can depend on, for example, zooming, PIP mode, screen resolution / size, user switching screens / devices, etc. The threshold can be based on the visibility index computed for the frame. The method can also include an initial detection of whether a user device is playing a video stream. A system configured to operate the method can also be provided.

[0020] In an embodiment, the present disclosure provides a computer-readable medium storing a computer program for operating any of the methods described herein.

[0021] For further elucidation of the advantages and features of the present disclosure, the application will be more thoroughly described and explained with reference to the drawings attached hereto, in which like or similar numerals refer to like or similar elements throughout the several views, of which: BRIEF DESCRIPTION OF DRAWINGS

[0022] These and other features, aspects, and advantages of the present disclosure will become easier to understand with further description and illustration presented below and will become more fully appreciated when considered in connection with the following figures, wherein like numbers refer to like items throughout the several figures, in which:

[0023] Figure 1 A scenario depicting at least one of missing data or data with reduced visibility in a device according to the prior art is shown;

[0024] Figure 2 various scenarios depicting at least one of missing data or data with reduced visibility according to the prior art are shown;

[0025] Figure 3 a block diagram of an environment of a system communicatively coupled with a user equipment (UE) and a target device according to an embodiment of the present disclosure is shown;

[0026] Figure 4 a block diagram of a system in communication with a user equipment and a target device according to an embodiment of the present disclosure is shown;

[0027] Figure 5a 、 Figure 5b 、 Figure 5c and Figure 5d various video streaming modes in a UE according to an embodiment of the present disclosure are shown;

[0028] Figure 6 forming a grid over at least one frame according to an embodiment of the present disclosure is shown;

[0029] Figure 7a and Figure 7b a process for determining a plurality of regions according to an embodiment of the present disclosure is shown;

[0030] Figure 8 and Figure 9 a process for repositioning relevant data on a user interface of a target device according to an embodiment of the present disclosure is shown;

[0031] Figure 10 a process for forming an overlay layer according to an embodiment of the present disclosure is shown;

[0032] Figure 11 a process for forming a background of an overlay layer according to an embodiment of the present disclosure is shown;

[0033] Figure 12 a method performed by a system according to an embodiment of the present disclosure is shown; and

[0034] Figure 13a 、 Figure 13b and Figure 13c various use cases of a system for repositioning at least one of missing data or data with reduced visibility according to an embodiment of the present disclosure are shown.

[0035] Furthermore, those skilled in the art will appreciate that the elements in the figures are illustrated for simplicity and clarity and that the actual implementation can be provided with additional or different elements, as desired. Further, the device can be shown in only a few exaggerated or schematic representations throughout the drawings for the purpose of simplification and clarity and, thus, the present disclosure can not be limited by the specific illustrations of the drawings. DETAILED DESCRIPTION

[0036] For the purposes of promoting an understanding of the principles of the present disclosure, reference will now be made to embodiments illustrated in the drawings, and specific language will be used to describe the same. It will nevertheless be understood that no limitation of the scope of the disclosure is intended by this specific language, and that others skilled in the art, upon reading the present disclosure, will readily appreciate many modifications to the illustrated systems as well as the principles disclosed herein. Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure belongs. The systems and examples provided herein are illustrative only and not intended to be limiting.

[0037] For example, the term "some" as used herein can be understood to mean "none," or "one," or "more than one," or "all." Accordingly, the term "none," "one," "more than one," "more than one, but not all," or "all" will fall within the definition of "some." It will be understood by those within the art that, in this application, terms such as "one," "a," or "an" are taken to mean "one or more" or "at least one," and terms such as "first," "second," "third," etc., merely

[0038] For example, any terms used herein, such as "comprising," "including," "containing," "characterized by," "comprised of," and the like, are to be construed as being inclusive (and not exclusive), unless otherwise indicated (e.g., by use of the restrictive language "consisting essentially of" or "consisting of"). Additionally, unless otherwise specified, such terms are not to be construed as excluding the possibility of additional features or elements.

[0039] A feature or element can be "a plurality" or "at least one" of something even if it is only used once, unless otherwise indicated. Additionally, the use of the term "plurality" or "at least one" does not exclude the possibility of there being a single feature or element, unless otherwise specified (e.g., by use of the restrictive language "a plurality of" or "at least one of").

[0040] Unless otherwise defined, all terms (including technical and / or scientific terms) used herein can be interpreted according to the meaning commonly used by a person of ordinary skill in the art.

[0041] Reference is made herein to some “embodiments”. It should be understood that embodiments are examples of possible implementations of any of the features and / or elements of the disclosure. Some embodiments have been described for purposes of illustrating a variety of manners in which the specific features and / or elements of the proposed disclosure satisfy the novelty, utility, and non-obviousness requirements of the patent statutes.

[0042] The use of the phrases and / or terms including but not limited to “first embodiment”, “another embodiment”, “alternative embodiment”, “one embodiment”, “embodiment”, “embodiments”, “some embodiments”, “other embodiments”, “additional embodiments”, “yet another embodiment”, “further embodiment” or variations thereof does not necessarily refer to the same embodiment. Unless otherwise specified, a number of specific features and / or elements described in connection with a number of embodiments can be found in one embodiment, or can be found in more than one embodiment, or can be found in all embodiments, or can not be found in any embodiment. Although a number of features and / or elements can be described herein in the context of only a single embodiment, or in the context of more than one embodiment, or in the context of all embodiments, the features and / or elements can be provided alone or in any appropriate combination, or not provided at all. Conversely, any feature and / or element described in the context of separate embodiments can alternatively be implemented together in the context of a single embodiment.

[0043] Any specific and all general details of the disclosure set forth herein are used in the above description of some embodiments only and therefore should not be construed to limit the proposed disclosure.

[0044] Embodiments of the disclosure will be described in detail below with reference to the attached drawings.

[0045] Figure 3 An environment 300 is shown of a system 304 communicatively coupled with a user device 302 and a target device 306, in accordance with embodiments of the disclosure. Figure 4 A block diagram of a system 304 connected with a user device 302 and a target device 306, in accordance with embodiments of the disclosure, is shown. Figure 5a 、 Figure 5b 、 Figure 5c And Figure 5d A number of video streaming modes in a UE 302, in accordance with embodiments of the disclosure, is shown. Figure 6 A grid is formed on at least one frame, in accordance with embodiments of the disclosure, is shown. Figure 7a And Figure 7bA process for determining a plurality of regions is shown in accordance with an embodiment of the disclosure. Figure 8 and Figure 9 A process for repositioning relevant data on a user interface 308 of a target device 306 is shown in accordance with an embodiment of the disclosure. Figure 10 A process for forming an overlay layer is shown in accordance with an embodiment of the disclosure. Figure 11 A process for forming a background of an overlay layer is shown in accordance with an embodiment of the disclosure.

[0046] In an embodiment, the user device 302 can include a plurality of applications, for example, a video player application, a video streaming application, a gallery, and the like. When a video is streamed in the user device (UE) 302, the system 304 can be configured to identify data associated with the video stream, wherein the data includes at least one of missing data or data with reduced visibility. Missing data can include data in the video stream that is outside of a screen area or viewport currently seen by the user. Data with reduced visibility can include data, such as text, that is too small for the user to comfortably recognize or read. Missing data or data with reduced visibility can be due to, for example, zooming, PIP mode, screen resolution / size, user switching screens / devices, and the like.

[0047] Further, the system 304 is configured to reposition the identified data associated with the video stream in a user interface 308 of a target device 306. In an embodiment, the target device 306 can be the same as the user device 302 without departing from the scope of the disclosure. In an embodiment, the target device 306 can be different from the user device 302 without departing from the scope of the disclosure, wherein the target device 306 can be wirelessly connected to the user device 302. In an embodiment, the target device 306 can be at least one of a smartphone, a television, a smartwatch, a desktop computer, a laptop computer, and the like without departing from the scope of the disclosure.

[0048] In an embodiment, the system 304 can include, but is not limited to, at least one processor (herein referred to as one or more processors, processor) 404, a memory 408, and a plurality of modules 412, among other examples detailed in subsequent paragraphs.

[0049] The system 304 can include an input / output (I / O) interface 432, a transceiver 430, a view system, a window manager, a user interface of the system 304, multitasking, a graphics layer with information about buffers, and drawing of a graphics buffer. In an embodiment, the view system can include creation of all surface views and overlay views without departing from the scope of the disclosure. In an embodiment, the window manager can provide dimensions of visible windows without departing from the scope of the disclosure.

[0050] Further, in some embodiments where the system 304 is implemented as a standalone entity at a server / cloud architecture, the system 304 can be in communication with a plurality of user devices to receive data from each of the plurality of user devices, and the details provided below with respect to the system 304 and the user device 302 also apply to the system 304 and the plurality of user devices.

[0051] In an example embodiment, the processor 404 can be operatively coupled to each of the I / O interface 432, the plurality of modules 412, the transceiver 430, and the memory 408. In one embodiment, the processor 404 can include a graphics processing unit (GPU) and / or an AI engine (AIE). In one embodiment, the processor 404 can include at least one data processor for executing processes in a virtual storage area network. The processor 404 can include specialized processing units such as an integrated system (bus) controller, memory management control unit, floating point unit, graphics processing unit, digital signal processing unit, etc. In one embodiment, the processor 404 can include a central processing unit (CPU), a graphics processing unit (GPU), or both. The processor 404 can be one or more general processors, digital signal processors, application specific integrated circuits, field programmable gate arrays, servers, networks, digital circuitry, analog circuitry, combinations thereof, or other now known or later developed devices for analyzing and processing data. The processor 404 can execute software programs, such as a hand-crafted (i.e., programmed) code, to perform desired operations.

[0052] The processor 404 can be configured to communicate with one or more input / output (I / O) devices via the I / O interface 432. In some embodiments, the processor 404 can communicate with the UE 302 and the target device 306 using the I / O interface 432. In some embodiments, the I / O interface 432 can be integrated within the user device 302. The I / O interface 432 can employ communication code division multiple access (CDMA), high-speed packet access (HSPA+), global system for mobile communications (GSM), long term evolution (LTE), WiMax, etc. In embodiments, the I / O interface 203 can be implemented using suitable devices such as, but not limited to, a display, a keyboard, a mouse, a touchscreen, a microphone, a speaker, etc. for outputting to and inputting from the system 304.

[0053] Through use of the I / O interface 432, the system 304 can communicate with the one or more I / O devices (specifically, the user device 302) from which the system 304 identifies missing data and the target device 306. For example, an input device can be an antenna, microphone, touchscreen, touchpad, storage device, transceiver, video device / source, etc. An output device can be a video display (e.g., cathode ray tube (CRT), liquid crystal display (LCD), light-emitting diode (LED), plasma, plasma display panel (PDP), organic light-emitting diode display (OLED), etc.), audio speaker, etc.

[0054] The processor 404 can be configured to communicate with a communication network via a network interface. In embodiments, the network interface can be the I / O interface 432. The network interface can connect to a communication network to enable the system 304 to connect with the UE 302 and / or the target device 306. The network interface can employ connection protocols including, but not limited to, direct connect, Ethernet (e.g., twisted pair 10 / 100 / 1000 Base T), transmission control protocol / internet protocol (TCP / IP), token ring, IEEE 802.11a / b / g / n / x, etc. The communication network can include, but is not limited to, direct interconnection, local area network (LAN), wide area network (WAN), wireless network (e.g., using Wireless Application Protocol), the Internet, etc. Through the use of the network interface and the communication network, the system 304 can communicate with other devices. The network interface can employ connection protocols including, but not limited to, direct connect, Ethernet (e.g., twisted pair 10 / 100 / 1000 Base T), transmission control protocol / internet protocol (TCP / IP), token ring, IEEE 802.11a / b / g / n / x, etc.

[0055] The transceiver 430 can be configured to receive signals from and / or transmit signals to the UE 302. In one embodiment, the database can be configured to store information required by the plurality of modules 412 and the processor 404 to perform one or more functions for relocating identified data on the UE 302 / target device 306.

[0056] In some embodiments, the memory 408 can be communicably coupled to the processor 404. The memory 408 can be configured to store data, as well as instructions executable by the processor 404. In one embodiment, the memory 408 can be disposed within the UE 302. In another embodiment, the memory 408 can be disposed within the system 304, remote from the UE 302. In yet another embodiment, the memory 408 can be in communication with the processor 404 within the system 304 via a bus. In still another embodiment, the memory 408 can be located at a location remote from the processor 404, and can be in communication with the processor 404 via a network. The memory 408 can include, but is not limited to, nonvolatile computer-readable storage media, such as various types of volatile and non-volatile storage media including, but not limited to, random access memory, read-only memory, programmable read-only memory, electrically programmable read-only memory, electrically erasable read-only memory, flash memory, magnetic tape or disk, optical media, and the like.

[0057] In one example, the memory 408 can include a cache or random access memory for the processor 404. In an alternative example, the memory 408 is separate from the processor 404, such as a cache memory, system memory, or other memory of the processor. The memory 408 can be an external storage device or database for storing data. The memory 408 can be operable to store instructions executable by the processor 404. The functions, acts or tasks illustrated in the figures or described in the specification can be performed by

[0058] In some embodiments, the plurality of modules 412 can be included within the memory 408. The memory 408 can also include a database for storing data. The plurality of modules 412 can include sets of instructions that are executable to cause the system 304, specifically the processor 404 of the system 304, to perform any one or more of the methods / processes disclosed herein. The plurality of modules 412 can be configured to perform the steps / operations of the present disclosure using the data stored in the database. For example, the plurality of modules 412 can be configured to perform the steps / operations disclosed in the method / process of Figures 5a to 10

[0059] In embodiments, each of the plurality of modules 412 can be a hardware unit that can be located external to the memory 408. Further, the memory 408 can include an operating system for performing one or more tasks of the system 304, which are performed by a general purpose operating system. ​

[0060] In one example, the modules 412 can include an identifying module 414, a determining module 416, a transforming module 418, a comparing module 420, a separating module 422, an extracting module 424, and a repositioning module 428. Each of the identifying module 414, the determining module 416, the transforming module 418, the comparing module 420, the separating module 422, the extracting module 424, and the repositioning module 428 can be in communication with each other. Further, each of the identifying module 414, the determining module 416, the transforming module 418, the comparing module 420, the separating module 422, the extracting module 424, and the repositioning module 428 can be in communication with the processor 404.

[0061] Further, the present disclosure contemplates a computer-readable medium that includes instructions or receives and executes instructions responsive to a propagated signal. Further, a propagated signal can be an electronically-magnetic signal, an optical signal, etc. Further, the instructions can be transmitted via a communication port or interface or a bus (not shown) over a network. The communication port or interface can be part of the processor 404 or a separate component. The communication port can be created in software or can be a physical connection in hardware.

[0062] The communication port can be configured to connect with a network, an external medium, a display, or any other component in the system, or a combination thereof. The connection with the network can be a physical connection such as a wired Ethernet connection, or can be established wirelessly. Similarly, the additional connections with other components of the system 304 can be physical or can be established wirelessly. The network can alternatively be connected directly to the bus. The architecture and standard operation of the memory 204, the processor 404, the transceiver 205, and the I / O interface 203 are not discussed in detail for brevity.

[0063] Further, in embodiments, the operation of the system 304 in repositioning the identified data on the user interface 308 of the target device 306 is explained in detail in the subsequent paragraphs in conjunction with Figures 3 to 11

[0064] The processor 404 in conjunction with the identifying module 414, the determining module 416, the transforming module 418, the comparing module 420, the separating module 422, the extracting module 424, and the repositioning module 428 can be configured to perform the specific operations explained in the subsequent paragraphs.

[0065] In embodiments, prior to identifying the data, the identifying module 414 can be configured to identify at least one mode from a plurality of modes of the video stream. In embodiments, the plurality of modes includes one of a display mode and a hidden mode, without departing from the scope of the present disclosure.

[0066] In embodiments, the processor 404 in conjunction with the identifying module 414, the determining module 416, the transforming module 418, the comparing module 420, the separating module 422, the extracting module 424, and the repositioning module 428 can be configured to perform the specific operations explained in the subsequent paragraphs. Figure 5a and Figure 5b ​In an embodiment, the video stream in the display mode can be identified through various techniques (e.g., ViewTreeObserver).

[0067] Specifically, when the video stream is detected at operation 502, the data imported from the source (e.g., disk / network) of the streaming video can be provided as input to the input buffer 512 of the input unit 508. Further, the data provided from the input buffer generates a continuous byte stream 514, and further, the continuous byte stream 514 is read / analyzed. In an embodiment, once the byte stream is analyzed, a plurality of raw images / frames can be decoded. In an embodiment, the plurality of raw images / frames can be provided as input to the image unit 510. Further, at operation 516, a plurality of frames are extracted from the plurality of raw images, and the plurality of frames are provided to the buffer information (media codec) 518. In an embodiment, the buffer information (media codec) 518 can include a media extractor, a media metadata retriever, and an adaptive manifest manager. The buffer information (media codec) 518 can generate DRM protected data.

[0068] In an embodiment, with reference to Figure 5a Without departing from the scope of the present disclosure, once the video stream is detected at operation 502, subsequently at operation 504, the display mode of the video stream can be identified at least on the foreground of the user interface of the user device 302, the background of the user interface of the user device 302, the interaction of the user with the linked device (multi-display) of the user device (UE) 302.

[0069] Further, once the display mode is identified and the plurality of frames are extracted from the plurality of raw images, at operation 506 and operation 522, surface information is extracted from the plurality of raw images / frames. In an embodiment, the plurality of raw images / frames can be rendered on a surface view / thrower, and further data / information is extracted from the surface view / thrower, for example, the size of the playing content, the location of the playing content, the color of the content, the capacity, the status of the video stream, the foreground, the background, the display ID of the UE 302, the DRM protected content. This configuration helps to identify the video stream in the display mode.

[0070] In the hidden mode, with reference to Figure 5c At operation 524, when the user exits from the video stream by pressing at least one of the home button and the back button in the UE 302, the video stream is moved to the background of the UE 302, as shown in operation 526. Specifically, with reference to Figure 5d When the user presses the home button or the back button on the UE 302, in this case, onPause of the video stream is identified. This configuration helps to determine whether the video stream is occurring in the hidden mode.

[0071] In an embodiment, once at least one of the plurality of modes of the video stream is determined, the determining module 416 can be configured to determine a visibility index (µ) of at least one frame of the plurality of frames of the video stream based on a plurality of parameters full The visibility index of the at least one frame is a numerical representation of the likelihood of the at least one frame being seen by the UE 302. In an embodiment, the plurality of parameters include at least one of resolution, content size, brightness, distance, and orientation, without departing from the scope of the present disclosure. In another embodiment, the plurality of parameters can also include a size of a user interface of the UE 302, a size of content of the video stream on the UE 302, a grid size, a type of the video stream on the UE 302, a type of content of the video stream on the UE 302, a distance of the user from the UE 302, an orientation of the user with respect to the UE 302, to identify whether the lost data is a continuous part of the video stream or any different overlay layer in the video stream, to identify a portion of a display unit of the UE 302 that can be blocked due to a system message (e.g., notification / prompt / other interference), to identify whether the video stream is turned off or switched to the background of the UE 302, to identify a number of connected displays / devices and whether any video stream is present on these devices. In an embodiment, the visibility index of the at least one frame can be determined by the following formula: Formula 1

[0072] In an embodiment, the best determined visibility index of the at least one frame can be less than 0.20, without departing from the scope of the present disclosure. In an embodiment, The standard deviation can represent the overall.

[0073] Further, after determining the visibility index, the determining module 416 can be configured to determine a data visibility of content of the video stream based on a plurality of factors associated with the video stream and / or at least one interest of the user. In an embodiment, the plurality of factors include at least one of a paused state of the video stream, an interaction of the user with the UE 302, an interaction of the user with a linked device of the UE 302, an interaction of the user with the target device 306, and a lost content from the video stream. In an embodiment, the interest / interaction of the user can be determined by various factors (e.g., changing a zoom scale of the video, adjusting a position of the video, and changing a focus from the video to other portions of the screen).

[0074] In an embodiment, the data visibility includes at least one of missing data or data with reduced visibility. Accordingly, at least one of missing data or data with reduced visibility can be determined and a relocation of the at least one of missing data or data with reduced visibility is determined based on a plurality of parameters. Table 1 provided below depicts an example of determining whether data should be relocated: [Table 1]

[0075] In an embodiment, after determining the state of data visibility, the processor 404 can be configured to identify data from content associated with a plurality of frames of a video stream. In an embodiment, the data includes at least one of missing data or data with reduced visibility without departing from the scope of the present disclosure. In an embodiment, the video stream can be displayed on the user device 302. Specifically, the transformation module 418 can be configured to transform at least one frame of the plurality of frames into a plurality of grid blocks. In an embodiment, at least one frame of the plurality of frames can be collected from the video stream and provided to a grid mapper. The grid mapper creates a custom grid 602 for the at least one frame and forms a plurality of blocks on the at least one frame as shown in Figure 6 In an embodiment, the plurality of blocks can include 256*256 grid blocks without departing from the scope of the present disclosure. In another embodiment, the plurality of blocks can have different sizes and / or different arrangements / quantities without departing from the scope of the present disclosure.

[0076] In an embodiment, the determination module 416 can be configured to determine a visibility index for each block of the at least one frame based on a plurality of parameters to identify data including at least one of missing data or data with reduced visibility. In an embodiment, the visibility index determined for each block can be able to assist in determining whether each block needs to be relocated when a threshold is calculated. In an embodiment, the visibility index for the at least one frame from the content can be stored in an array format with the visibility index for each block, where the visibility index for each block can be calculated by the previously discussed Equation 1. Similarly, the visibility index for each block of each frame can be determined in an alternative manner, which is not illustrated here for brevity. In an embodiment, the visibility index for each block can be stored in an array, which is represented by the following equation: [Equation 1]

[0077] In an embodiment, after determining the visibility index for each block, the processor 404 can be configured to determine a plurality of regions 718, 720 in the plurality of frames. Specifically, reference is made to Figure 7adetermining module 416 can be adapted to determine a first plurality of regions from the plurality of frames at an initial state of the video stream, as indicated at operation 702. At operation 704, a second plurality of regions can be determined from the plurality of frames of the video stream after a specified time 't'. Further, the comparison module 420 can compare the first plurality of regions and the second plurality of regions to identify a plurality of regions 718, 720, wherein the plurality of regions can be static regions 718 and dynamic regions 720. In an embodiment, the static regions 718 are regions that do not have visible movement over the plurality of frames. The static regions 718 can have two types, for example, a ROI having textual data (relevant data) and undesired data (irrelevant data). The relevant data is interpreted as meaningful and important data present in the static regions. Further, the irrelevant data is interpreted as data provided in the static regions that is not useful for the user. In an embodiment, the dynamic regions 720 are interpreted as regions that have movement (indicating important visual data on which regenerating data cannot be placed) over the plurality of frames, as indicated at operation 706. Figure 7b

[0078] In an embodiment, at operation 708, a first region table is generated from the determined plurality of regions 718, 720. Similarly, in an embodiment, at operation 710, the plurality of regions can be determined from the plurality of frames of the video stream multiple times (i.e., repeatedly after a specified X interval). Further, at operation 712, a second region table is formed based on the determined plurality of regions.

[0079] In an embodiment, at operation 714, the comparison module 420 can be configured to compare the first region table and the second region table to generate a final plurality of regions 718, 720 from the plurality of frames at operation 716. In an embodiment, the comparison of the plurality of regions 718, 720 corresponding to the plurality of frames can be performed constantly after a specified time interval to determine the plurality of regions 718, 720. In an embodiment, the plurality of frames are analyzed to identify whether any textual data / graphical representation is present in the plurality of frames, which can include relevant data. Once the data is identified, the region containing the data can be marked and stored in, for example, a hash map. Further, the marked data can be identified by periodically traversing the plurality of frames, and further, the static regions 718 or the dynamic regions 720 can be subsequently identified.

[0080] For better understanding, the determination of the static regions 718 can be explained through a non-limiting example. Referring to Figure 7b ​In one example, the color component of each block of at least one frame of the plurality of frames can be determined by the determining module 416 after a specified time interval. Further, the color component of each block of another frame of the plurality of frames can be determined by the determining module 416 in the initial state. Finally, the color component of the resultant frame can be determined by calculating the difference between the color component of at least one frame of the plurality of frames and the color component of another frame of the plurality of frames to determine the static region 718 from the plurality of regions 718, 720 in the plurality of frames. The color component of each block can be determined by the equation given below: [Equation 2]

[0081] In one example, the color component of each block can be calculated by considering the value of each block and considering the weighted average value of each block.

[0082] In an embodiment, after determining the plurality of regions 718, 720, the determining module 416 can be configured to determine the visibility index of the plurality of regions 718, 720 in accordance with the equation provided below: [Equation 3]

[0083] In an embodiment, without departing from the scope of the present disclosure, x represents the number of blocks in the region. Further, Table 2 and Figure 7b below show an example of the dependency of the visibility index of each block with the plurality of regions: [Table 2]

[0084] In one embodiment, with reference to Figure 7b After determining the plurality of regions 718, 720, the processor 404 (more specifically, the identifying module 414) is configured to identify the plurality of regions 718, 720 from the plurality of frames of the video stream based on the identified content.

[0085] In an embodiment, after identifying the plurality of regions 718, 720 from the plurality of frames, the processor 404 (more specifically, the separating module 422) separates the relevant data and the irrelevant data from the identified data (i.e., at least one of the lost data or the data with reduced visibility) included in at least one region of the plurality of regions 718, 720. In one example, without departing from the scope of the present disclosure, the separating module 422 can separate the relevant data and the irrelevant data from at least one of the lost data or the data with reduced visibility provided in either of the static region 718 and the dynamic region 720.

[0086] ​In an embodiment, after separating relevant data and irrelevant data in at least one of the missing data or data with reduced visibility provided from at least one of the plurality of regions 718, 720, the processor 404 (more specifically, the determining module 416) can be configured to determine at least one region of the plurality of regions 718, 720 having the identified data (i.e., at least one of the missing data or data with reduced visibility). In an embodiment, the at least one region of the plurality of regions 718, 720 can be the static region 718 (as shown in Figure 7b

[0087] In an embodiment, the determining module 416 can be configured to determine the visibility index (average) of the plurality of blocks of at least one region (i.e., the static region 718) of the plurality of regions 718, 720. In an embodiment, the comparing module 420 can be configured to compare the determined visibility index with a threshold visibility index. In an embodiment, if after the comparison, the visibility index is less than the threshold visibility index, the extracting module 424 extracts the relevant data from the identified data (i.e., at least one of the missing data or data with reduced visibility) in at least one region (i.e., the static region 718) of the plurality of regions. In an embodiment, the determining module 416 can be configured to determine the visibility index of the relevant data based on a plurality of parameters associated with the video stream.

[0088] In particular, once the static region 718 is determined, the text recognition engine / unit becomes active to identify the data in the static region 718 comprising at least one of the missing data or data with reduced visibility. In an embodiment, if the data comprising at least one of the missing data or data with reduced visibility is identified, a technique is implemented for checking whether the identified at least one of the missing data or data with reduced visibility contains a logo / important advertisement or dividing the static region 718 with text. In an embodiment, once the static region 718 is divided, the relevant data / text is extracted. In an embodiment, an input image file is provided in which color reduction and further ROI detection occurs. Further, after detecting the ROI, the filtering unit is configured to filter the data. In an embodiment, the filtering unit can be configured to perform geometric and morphological feature extraction, neural network based classification, and connected component analysis.

[0089] In an embodiment, the filtered data is sent to the text extraction, recognition, and storage unit. In an embodiment, the text extraction, recognition, and storage unit can include text region extraction and at least one optical character recognition. In an embodiment, the filtered data after being sent from the text extraction, recognition, and storage unit can provide the extracted relevant data.

[0090] ​In an embodiment, once the extracted relevant data is obtained, semantic analysis is activated to break down the relevant data to identify meaningful words, distinguish spacing, and color component changes.

[0091] Further, after semantic analysis, the extracted relevant data can be broken down into similar groups by padding from the start text to the end text in the static region 718. Thus, the relevant data is extracted and displayed.

[0092] In an embodiment, once the relevant data is extracted, the processor 404 (more specifically, the repositioning module 426) can be configured to reposition the relevant data in at least one region (i.e., the static region 718) of the plurality of regions 718, 720 based on the determined visibility index of the relevant data. The repositioned relevant data is visible on the user interface 308 associated with the target device 306.

[0093] In an embodiment, with reference to Figure 8 , to reposition the relevant data in at least one region of the plurality of regions, the at least one processor 404 (more specifically, the determination module 416) can be configured to analyze the extracted relevant data at operation 802 and also determine the screen region required for the relevant data at operation 804.

[0094] At operation 806, the determination module 416 can be configured to determine the size of at least one region (i.e., the static region 718) of the plurality of regions 718, 720. At operation 808, the determination module 416 can determine whether the size of the static region 718 is optimal. In an embodiment, at operation 814, the determination module 416 can be configured to determine the contrast of at least one region (i.e., the static region 718) of the plurality of regions and the relevant data when the size of at least one region of the plurality of regions 718, 720 is optimal for the relevant data. In an embodiment, the contrast can be 0.6 and 0.7 without departing from the scope of the present disclosure.

[0095] At operation 818, the determination module 416 can determine the visibility index of at least one region (i.e., the static region 718) of the plurality of regions 718, 720 and the relevant data. In an embodiment, the visibility index can be 0.87 and 0.89 without departing from the scope of the present disclosure.

[0096] At operation 820 and with reference to Figure 9part of the B part of the static region 718, when the size of the static region 718 is optimal (e.g., large enough to accommodate / contain the relevant data), the relevant data is repositioned in at least one of the plurality of regions (i.e., the static region 718). In an embodiment, after repositioning the relevant data, the visibility index of the relevant data is determined to determine whether the overlay layer is required or not, without departing from the scope of the present disclosure. Further, the repositioned relevant data is visible on the user interface 308 associated with the target device 306 and should not occlude any part of the user interface, for example, the play / pause option.

[0097] Further, in an embodiment, at operation 810, when the size of at least one of the plurality of regions 718, 720 (i.e., the static region 718) is not optimal for the relevant data, the processor 404 is configured to form an overlay layer, as shown in the B part of the static region 718. Figure 9

[0098] In an embodiment, to form the overlay layer, reference is made to Figure 10 At operation 1002, the processor 404 (more specifically, the extraction module 424) can be configured to extract the color matrix of at least one of the plurality of regions 718, 720 (i.e., the static region 718). In an embodiment, the processor 404 is configured to provide the extracted color matrix in a color blender at operation 1004. In an embodiment, in the color blender, the processor 404 can be configured to select a color spectrum with contrasting colors at operation 1006. Further, the determination module 416 can be configured to determine the color coefficient of each color provided in the color spectrum at operation 1008.

[0099] In an embodiment, the processor 404 can be configured to prepare a list of candidates of contrasting colors at operation 1010.

[0100] In an embodiment, the determination module 416 can be configured to determine the contrast coefficient of each color (candidate color) with the color matrix of the static region 718 at operations 1012 and 1014 to select the best color. In an embodiment, the determination module 416 can be configured to determine the visibility index to generate the best shape with the best color at operation 1016. In an embodiment, the visibility index can be calculated by the formula shown below: [Equation 4]

[0101] Further, the relationship between the values of the equation 4 is provided below: [Table 3]

[0102] ​In an embodiment, processor 404 may be configured to apply optimal color and optimal shape to the overlay layer at operation 1020, when the visibility index is optimal. In an embodiment, the optimal range of the visibility index may be... In an embodiment, when the optimal visibility index is not achieved at operation 1018, the determination module 416 re-determines the contrast coefficient of each color with the color matrix of the static region 718 to select the optimal color, etc., and this process continues until the optimal visibility index is achieved.

[0103] In an embodiment, after applying the optimal color and optimal shape to the overlay at operation 1022, the processor 404 can be configured to further select the background of the overlay. In an embodiment, as... Figure 11 As shown, a background is selected based on a combination of the color 1102 of the static region 718, the background color 1104 in the static region 718 corresponding to the relevant data, and the foreground color 1106 in the static region 718 corresponding to the relevant data; and the font 1108 of multiple portions of the relevant data is selected. In the embodiment, the background can be provided in two ways (e.g., using the same background as the background used in the shape of the extracted overlay). This method can be used / implemented when a similar background does not obstruct the visibility of the area where the overlay can be placed. Furthermore, different backgrounds can be implemented that are adjusted together with the background of the area where the overlay can be placed. In the embodiment, when the overlay is placed, if the original background obstructs visibility, the background of the overlay changes according to the background of the overlay area.

[0104] In an embodiment, once the background is implemented in the overlay and the optimal color is achieved in the overlay, the processor 404 can place the overlay on the user interface 308 of the target device 306.

[0105] In embodiments, the overlay can be placed on an area that maintains user viewing comfort. Furthermore, the processor 404 is configured to recognize the shape of the overlay. In embodiments, the shape / position of the overlay can be selected based on various parameters; for example, the color of the entire screen and areas that present the same color on the user interface 308 can be used to depict a static area with relevant data. Furthermore, the overlay can be placed on the user interface 308 so that all necessary content of the video stream is visible. Furthermore, the position of the overlay can vary on the static area 718 without departing from the scope of this disclosure. Furthermore, the overlay may not be placed on the user interface 308 where user interaction may occur, or it may overlap with a logo / important advertisement. In one example, user interaction may be a play / pause option, a forward / rewind option, and any other controls without departing from the scope of this disclosure. Finally, the position of the overlay can be represented by the following equation: [Equation 5]

[0106] In an embodiment, the shape of the overlay layer can be rectangular, oval, square or any other shape without deviating from the scope of the present disclosure, while maintaining the user's video streaming experience.

[0107] In an embodiment, the processor 404 can be configured to divide the relevant data into a plurality of portions to be placed within the identified shape of the overlay layer. In one example, the shape of the overlay layer can be determined in real time. In one example, the text blocks of the relevant data are divided into smaller blocks and, further, the static area 718 in which each block has to be placed is determined. If the blocks cannot be placed in the static area 718, each block is further segmented until a threshold is reached. Further, once the threshold is reached and the blocks still cannot fit into the static area 718, the relevant data can be divided into a plurality of portions, wherein the plurality of portions can be placed in the overlay layer, as shown in Figure 9

[0108] In an embodiment, the repositioning module 428 can be configured to reposition the relevant data in at least one of the plurality of areas, i.e., the static area 718. The repositioned relevant data is visible on the user interface 308 associated with the target device 306.

[0109] Figure 12 A method 1200 performed by the system 304, in accordance with an embodiment of the present disclosure, is shown.

[0110] The method 1200 can be performed by a programmed computing device, e.g., based on instructions retrieved from a non-transitory computer readable medium. The computer readable medium can include machine executable or computer executable instructions for performing all or part of the described methods. The computer readable medium can be, for example, digital memories, magnetic storage media such as magnetic disks and magnetic tapes, hard drives, or optically readable data storage media.

[0111] The method 1200 includes a series of operations, as shown in operations 1202 to 1210. The method 1200 can be performed by the system 304 in conjunction with the module 412, the details of which are explained in connection with Figure 12 Figures 3 to 10 The method 1200 begins at operation 1202.

[0112] ​​At operation 1202, the method 1200 includes identifying data from a plurality of frames of a video stream playing on the UE 302. The data includes at least one of missing data or data having reduced visibility. The method 1200 includes converting at least one frame of the plurality of frames into a plurality of grid blocks. Further, the method 1200 includes determining a visibility index for each block of the at least one frame based on a plurality of parameters to identify the data including at least one of missing data or data having reduced visibility.

[0113] Further, prior to identifying the data from the content associated with the plurality of frames, the method 1200 includes identifying at least one mode from a plurality of modes of the video stream. In an embodiment, the plurality of modes includes one of a display mode and a hidden mode. Further, the method 1200 includes determining a visibility index for at least one frame of the plurality of frames based on a plurality of parameters. The plurality of parameters includes at least one of resolution, content size, brightness, distance, and orientation. The method 1200 includes determining a state of visibility of the data in the content of the video stream based on a state of a plurality of factors associated with the video stream and at least interest of the user. In an embodiment, the plurality of factors includes at least one of a pause state of the video stream, an interaction of the user with the UE 302, an interaction of the user with a linked device of the UE 302, an interaction of the user with the target device 306, and missing content from the video stream. In an embodiment, the state of visibility of the data includes at least one of missing data or data having reduced visibility.

[0114] At operation 1204, the method 1200 includes identifying the plurality of regions 718, 720 from the plurality of frames of the video stream based on the identified content. In an embodiment, prior to identifying the plurality of regions from the plurality of frames of the video stream, for determining the plurality of regions 718, 720, the method 1200 includes determining a first plurality of regions from the plurality of frames in an initial state of the video stream (as shown in operation 702 of FIG. 7). Figure 7a The method 1200 includes determining a second plurality of regions from the plurality of frames of the video stream after a specified time interval (as shown in operation 704 of FIG. 7). The method 1200 includes comparing the first plurality of regions and the second plurality of regions to determine the plurality of regions. In an embodiment, the plurality of regions includes a static region 718 (as shown in operation 706 of FIG. 7) and a dynamic region 720 (as shown in operation 708 of FIG. 7). Further, the method 1200 includes determining a visibility index for the determined plurality of regions. Figure 7a Figure 7b Figure 7b

[0115] ​​​At operation 1206, the method 1200 includes separating relevant data and irrelevant data from the identified data provided in the at least one region of the plurality of regions 718, 720. After separating the relevant data and the irrelevant data from the identified data provided in the at least one region of the plurality of regions 718, 720, the method 1200 includes determining the at least one region of the plurality of regions 718, 720 having the identified data. In an embodiment, the at least one region of the plurality of regions 718, 720 is the static region 718. The method 1200 includes determining a visibility index of a plurality of blocks of the at least one region of the plurality of regions 718, 720. The method 1200 includes comparing the determined visibility index with a threshold visibility index. The method 1200 includes extracting the relevant data from at least one of missing data or data having reduced visibility in the at least one region of the plurality of regions 718, 720 based on the comparison (e.g., if the determined visibility index is less than the threshold visibility index).

[0116] At operation 1208, the method 1200 includes determining a visibility index of the relevant data based on a plurality of parameters associated with the video stream.

[0117] At operation 1210, the method 1200 includes repositioning the relevant data in the at least one region of the plurality of regions 718, 720 based on the determined visibility index. For example, if the determined visibility index of the result is acceptable (e.g., greater than a threshold, which can be related to the visibility index determined for the entire frame), the relevant data can be repositioned in the static region. In an embodiment, the repositioned relevant data is visible on the user interface 308 associated with the target device 306.

[0118] To reposition the relevant data in the at least one region of the plurality of regions 718, 720, the method 1200 includes determining a size of the at least one region of the plurality of regions 718, 720, wherein the at least one region of the plurality of regions 718, 720 is the static region 718. The method 1200 includes determining a contrast of the at least one region of the plurality of regions 718, 720 with the relevant data, wherein the size of the at least one region of the plurality of regions 718, 720 is optimal for the relevant data. The method 1200 includes determining a visibility index of the at least one region of the plurality of regions 718, 720 and the relevant data. The method 1200 includes repositioning the relevant data in the at least one region of the plurality of regions 718, 720, wherein the repositioned relevant data is visible on the user interface 308 associated with the target device 306.

[0119] In an embodiment, when the size of at least one of the plurality of regions 718, 720 is not optimal for the relevant data, the method 1200 includes forming an overlay layer. To form the overlay layer, the method 1200 includes extracting a color matrix of at least one of the plurality of regions, wherein the at least one of the plurality of regions 718, 720 is the static region 718. The method 1200 includes selecting a color spectrum with contrasting colors. The method 1200 includes determining a color coefficient of each color provided in the color spectrum. The method 1200 includes determining a contrast coefficient of each color with the color matrix of the static region to select an optimal color. The method 1200 includes determining a visibility index to generate an optimal shape with the optimal color. The method 1200 includes applying the optimal color and the optimal shape to the overlay layer when the visibility index is optimal.

[0120] Further, the method 1200 includes selecting a background of the overlay layer, wherein the background is selected based on a combination of the static region 718, a background color in the static region 718 corresponding to the relevant data, and a foreground color in the static region 718 corresponding to the relevant data. The method 1200 includes selecting a font of the plurality of portions of the relevant data.

[0121] After determining the background of the overlay layer, the method 1200 includes placing the overlay layer on the user interface 308 of the target device 306.

[0122] The method 1200 includes identifying a shape of the overlay layer. Further, the method 1200 includes dividing the relevant data into a plurality of portions to be placed within the identified shape of the overlay layer. Finally, the method 1200 includes repositioning the relevant data in at least one of the plurality of regions 718, 720, wherein the repositioned relevant data is visible on the user interface 308 associated with the target device 306. In an embodiment, the overlay layer is configured to place the plurality of portions of the relevant data on the user interface 308 of the target device 306.

[0123] In an embodiment, the relevant data is repositioned on the user interface 308 of the target device 306 in response to receiving a user input indicating a switch from the video stream, with reference to Figure 12 A to Figure 12 C elaborates on this point.

[0124] Figure 13a , Figure 13b and Figure 13c Figures illustrate a plurality of use cases of the system 304 for repositioning at least one of lost data or data with reduced visibility, in accordance with an embodiment of the present disclosure.

[0125] Reference is made to Figure 13aIn one example, a video stream is playing on the UE 302 (i.e., television) and the user is actively using the target device 306 (i.e., smart phone). The smart phone and the television are connected to each other through an application (e.g., intmDisplay). The application provides the total number of displays connected to the UE 302 and on which the user is currently working with the help of the touch and other user patterns detected on the display. Thus, in this use case, since the user is using the smart phone and a video stream is playing on the television, the relevant data can be visible on the smart phone.

[0126] Referring to Figure 13b When the user is streaming a video in PIP mode and exits from the video stream by pressing the home button, exit button, etc. (as shown in Figure 13b (1) and Figure 13b (2)), in this case, the processor 404 can obtain multiple frames / original images and create a virtual display with a new display ID (as shown in Figure 13b (3)). Further, a new surface is created for presenting the frames of the video stream. The property of the SurfaceView of the content / relevant data is extracted and it is displayed on the user interface 308 of the target device 306. Finally, the relevant data can be repositioned according to the context of the user interface 308 of the target device 306 and user interaction (as shown in Figure 13b (4) and Figure 13b (5)). In this case, the UE 302 and the target device 306 are the same without departing from the scope of the present disclosure.

[0127] Referring to Figure 13c , the UE 302 and the target device 306 can be connected to each other. In one example, the UE 302 can be a smart phone and the target device 306 can be a smart watch. Further, a video stream can be playing on the smart phone and the smart phone can become inactive due to multiple factors (e.g., power off, user exit from the video, etc.). In this case, the relevant data can be visible on the smart watch worn by the user.

[0128] As will be gathered, the disclosed system 304 and method 1200 can provide a comprehensive method for identifying and repositioning at least one of lost data or data with reduced visibility in a user interface 308 of a target device 306. The present configuration provides for repositioning relevant data from at least one of lost data or data with reduced visibility in a manner that does not impede user accessibility and viewability. Further, the system 304 and method 1200 provide for repositioning relevant data in multi-display scenarios, multi-device connection scenarios, and scenarios where a user exits from a video stream, thereby providing flexibility for a user to perform multitasking while accessing relevant data from a video stream.

[0129] While the present disclosure has been described using specific languages, it is not intended to limit the present disclosure to one or more particular phrases as set forth. As persons of ordinary skill in the art will readily understand, various modifications can be made to the method 1200 to implement the inventive concepts taught herein. The figures and the foregoing description provide illustration of examples of embodiments. Those skilled in the art will understand that one or more of the described elements can well be combined into a single function element. Alternatively, certain elements can be separated into multiple function elements. Elements from one embodiment can be added to another embodiment.

Claims

1. A method (1200) for relocating data associated with a video stream, the method (1200) comprising: Identify (1202) data from multiple frames of a video stream being played on the user equipment (UE) (302); Identify (1204) multiple regions (718, 720) from the multiple frames of the video stream. In at least one of the plurality of regions (718, 720), the first data and the second data are separated (1206) from the identified data; The visibility index of the first data is determined (1208) based on multiple parameters of the video stream; as well as Based on the determined visibility index, the first data is repositioned (1210) in at least one of the plurality of regions (718, 720), wherein the repositioned first data is visible on the user interface (308) associated with the target device (306).

2. The method (1200) according to claim 1, wherein, Before identifying the data (1202), the method (1200) includes: Identify at least one mode from a plurality of modes of the video stream, wherein the plurality of modes include a display mode and a hidden mode.

3. The method (1200) according to claim 2, wherein, The method (1200) includes: The data visibility of the video stream's content is determined based on multiple factors associated with the video stream. The plurality of factors include at least one of the following: the paused state of the video stream, the interaction between the user and the UE (302), the interaction between the user and the linked device of the UE (302), the interaction between the user and the target device (306), and the content lost from the video stream.

4. The method (1200) according to claim 1, wherein, The data identified (1202) includes: Convert at least one of the plurality of frames into a plurality of blocks; and The visibility index of each of the plurality of blocks is determined based on the plurality of parameters to identify the data, wherein the plurality of parameters includes at least two of resolution, content size, brightness, distance, and orientation.

5. The method (1200) according to claim 1, wherein, Before identifying the plurality of regions (1204), the method (1200) includes: In the initial state of the video stream, a first plurality of regions are determined from the plurality of frames; After a specified time interval, a second plurality of regions are determined from the plurality of frames; Compare the first plurality of regions and the second plurality of regions to determine whether the plurality of regions include static regions or dynamic regions; and Determine the visibility index of the identified static area.

6. The method (1200) according to claim 1, wherein, After separating (1206) the first data and the second data from the identified data in at least one of the plurality of regions (718, 720), the method (1200) includes: Determine at least one region among the plurality of regions (718, 720) that has the identified data, wherein the at least one region among the plurality of regions is a static region (718). Determine the visibility index of multiple blocks in at least one of the multiple regions (718, 720); The determined visibility index is compared with the threshold visibility index; and In response to a determined visibility index being less than the threshold visibility index, the first data is extracted from the identified data in at least one of the plurality of regions (718, 720).

7. The method (1200) according to claim 1, wherein, The first data includes repositioning (1210) in at least one of the plurality of regions (718, 720): Determine the size of at least one of the plurality of regions (718, 720), wherein the at least one of the plurality of regions (718, 720) is a static region (718). If the size of the static region (718) is optimal for the first data, determine the contrast between at least one of the plurality of regions and the first data; Determine the visibility index of the static region (718); and Based on the determined visibility index of the static area, the first data is repositioned in the static area (718), wherein the repositioned first data is visible on the user interface (308) associated with the target device (306).

8. The method (1200) according to claim 7, wherein, If the size of the static region (718) is not optimal for the first data, the method (1200) includes: Forming superimposed layers; Determine the shape of the superimposed layer; The first data is divided into multiple portions to be placed within the identified shape of the overlay layer; and The overlay is placed on the user interface (308) of the target device (306), wherein the overlay is configured to place the plurality of portions of the first data on the user interface (308) of the target device (306).

9. The method (1200) according to claim 8, wherein, Before placing the overlay on the user interface (308) of the target device (308), the method (1200) includes: The background of the overlay layer is selected based on a combination of the static region (718), the background color in the static region (718) corresponding to the first data, and the foreground color in the static region (718) corresponding to the first data; and Select the font of the plurality of parts of the first data.

10. The method (1200) according to claim 1, wherein, In response to receiving user input indicating a switch from the video stream, the first data is repositioned on the user interface (308) of the target device (306).

11. A system (304) configured to relocate data associated with a video stream, the system comprising: At least one processor (404), including processing circuitry; and Memory (408) stores instructions that, when executed individually or jointly by the at least one processor, cause the system to: Identify data from multiple frames of a video stream being played on the user equipment (UE) (302); Multiple regions (718, 720) are identified from the multiple frames of the video stream. In at least one of the plurality of regions (718, 720), the first data and the second data are separated from the identified data; The visibility index of the first data is determined based on multiple parameters of the video stream; as well as Based on the determined visibility index, the first data is repositioned in at least one of the plurality of regions (718, 720), wherein the repositioned first data is visible on the user interface (308) associated with the target device (306).

12. The system (304) according to claim 11, wherein, The target device (306) includes the UE (302) or different devices wirelessly connected to the UE (302).

13. The system (304) according to claim 12, wherein, The target device (306) is at least one of a smartwatch, a television, a desktop computer, a laptop computer, or a smartphone.

14. The system (304) according to claim 11, wherein, Before recognizing the data, the instructions, when executed individually or jointly by the at least one processor (404), also cause the system to: Identify at least one mode from a plurality of modes of the video stream, wherein the plurality of modes include a display mode and a hidden mode.

15. One or more non-transitory computer-readable storage media storing one or more computer programs including computer-executable instructions, which, when executed by at least one processor, cause a user device to perform operations, said operations including: Identify data from multiple frames of a video stream being played on the user equipment (UE) (302); Multiple regions (718, 720) are identified from the multiple frames of the video stream. In at least one of the plurality of regions (718, 720), the first data and the second data are separated from the identified data; The visibility index of the first data is determined based on multiple parameters of the video stream; as well as Based on the determined visibility index, the first data is repositioned in at least one of the plurality of regions (718, 720), wherein the repositioned first data is visible on the user interface (308) associated with the target device (306).

Images