Method for providing video cloud streaming service providing transparency information using renderer process and apparatus therefor

Abstract

Disclosed are a method for providing a video cloud streaming service providing transparency information using a renderer process, and an apparatus therefor. In the method for providing a video cloud streaming service according to the present invention, a video cloud streaming server may: generate transparency data on the basis of detection of a change in transparency in a renderer process in a browser; generate video element stream data on the basis of detection of a change in screen in a GPU process in the browser; synchronize the transparency data with the video element stream data; and stream the synchronized data to a user terminal.

Description

Method for providing a video cloud streaming service that provides transparency information using a renderer process and device therefor

[0001] The present invention relates to a method for providing a video cloud streaming service providing transparency information using a renderer process and an apparatus therefor, and more particularly, to a technology for providing a video cloud streaming service capable of providing transparency data using a renderer process in a browser running in a native Windows manner, and a technology for quickly discovering a cloud streaming server in which a transparency error has occurred, thereby improving service stability.

[0002] This invention claims the benefit of Korean Patent Application No. 10-2023-0093558, filed July 19, 2023, and Korean Patent Application No. 10-2023-0155734, filed November 10, 2023, the entire contents of which are incorporated herein by reference.

[0003] The rapid development of the Internet has resulted in a dramatic increase in personal communication speeds. This improvement has enabled users to access remote computers, download or upload large amounts of data, or use remote computer control programs to access remote computers as if logged in locally. Furthermore, with the development of diverse applications running on mobile devices like smartphones, virtualization technologies are increasingly being proposed to enable high-performance applications to run on relatively low-performance user devices.

[0004] Cloud streaming technology is a solution that runs applications or services requiring high-performance system resources on a cloud streaming server and delivers only the rendered results to the user's device. This allows high-performance video playback or 3D games even on low-performance devices.

[0005] Meanwhile, cloud streaming for user interfaces (UIs), such as menus or posters displayed on live channels, is primarily achieved through image encoding, as it facilitates transparency processing. However, image encoding has the disadvantage of incurring significant overhead when processing fast-changing screens, such as animations.

[0006] Therefore, the need for technology using video encoding based on video cloud streaming is being emphasized, but there is a problem that in order to obtain alpha data, which is transparency information, in the browser of video cloud streaming, it must be operated in the OSR (Off Screen Rendering) method.

[0007] The OSR method has lower performance than the Native Windows method, making it difficult to provide smooth service. In addition, the Native Windows method, which operates at higher performance than the OSR method, cannot provide transparency data in the GPU process that configures the final screen because the APIs for receiving GPU acceleration in each OS such as Windows / Linux do not support transparency.

[0008] Meanwhile, with the advent of transparent video cloud streaming, cloud streaming services can now simultaneously display live video and the video UI for video cloud streaming. However, if transparency information is incorrect, live video can become invisible to users, potentially leading to voice-of-customer (VOC) complaints.

[0009] As a method of detecting browser malfunctions due to resource problems within the cloud streaming server, a method was proposed to connect virtual terminals to multiple servers, receive ES frames for the same key input, decode them, and compare PSNRs to check the health of the cloud streaming servers.

[0010] However, this is a method that cannot verify transparency data, and similarly, it is possible to connect virtual terminals to the server and compare Alpha data, but there is a problem that it cannot be detected when traffic occurs and multiple servers all provide incorrect data.

[0011] The purpose of the present invention is to provide a video with transparency applied to a user by providing transparency data while using a Native Windows method that operates at high performance compared to the OSR method in video cloud streaming.

[0012] Another object of the present invention is to minimize performance degradation by reducing CPU usage for providing transparency data in a video cloud streaming service.

[0013] In addition, an object of the present invention is to improve service stability by quickly detecting a server in which a transparency error has occurred in a video cloud streaming service.

[0014] Additionally, it is an object of the present invention to reduce traffic when verifying transparency data of a cloud streaming server.

[0015] In addition, an object of the present invention is to solve a problem in which an error occurs in verification due to incorrect data being transmitted to multiple servers when verifying transparency data of a cloud streaming server.

[0016] A method for providing a video cloud streaming service according to the present invention for achieving the above-described purpose includes a step in which a video cloud streaming server generates transparency data based on detection of a transparency change in a renderer process within a browser; a step in which a video element stream data is generated based on detection of a screen change in a GPU processor within a browser; a step in which the transparency data and the video element stream data are synchronized; and a step in which the synchronized data is streamed to a user terminal.

[0017] At this time, the video cloud streaming server can generate the transparency data by performing capture and encoding based on the alpha value transmitted corresponding to the transparency change detection through the renderer process.

[0018] At this time, the video cloud streaming server can generate the video element stream data by performing capture and encoding based on the texture value transmitted corresponding to the screen change detection through the GPU processor.

[0019] At this time, the browser can be run in NATIVE WINDOWS mode.

[0020] At this time, the user terminal can display a screen with applied transparency change by combining the video element stream data and the transparency data.

[0021] At this time, the renderer process can perform the above transparency change detection based on the transparency change detection module.

[0022] At this time, synchronization can be performed based on the frame number (FRAME NUMBER).

[0023] In addition, the video cloud streaming server according to the present invention includes a processor that generates transparency data based on detection of a change in transparency in a renderer process within a browser, generates video element stream data based on detection of a change in screen in a GPU processor within a browser, synchronizes the transparency data with the video element stream data, and streams the synchronized data to a user terminal; and a memory that stores the transparency data and the video element stream data.

[0024] At this time, the processor can generate the transparency data by performing capture and encoding based on the alpha value transmitted corresponding to the transparency change detection through the renderer process.

[0025] At this time, the processor can generate the video element stream data by performing capture and encoding based on the texture value transmitted corresponding to the screen change detection through the GPU processor.

[0026] At this time, the browser can be run in NATIVE WINDOWS mode.

[0027] At this time, the user terminal can display a screen with applied transparency change by combining the video element stream data and the transparency data.

[0028] At this time, the renderer process can perform the above transparency change detection based on the transparency change detection module.

[0029] At this time, synchronization can be performed based on the frame number (FRAME NUMBER).

[0030] In addition, a transparency error detection method according to the present invention for achieving the above-described purpose includes: a step of implementing a transparency test mode in a browser of a cloud streaming server; a step of performing a transparency test through the transparency test mode when the cloud streaming server receives a transparency test signal from a transparency check server; and a step of detecting a transparency error by comparing a test value by the transparency test with a correct answer value of the transparency test mode stored in advance by the cloud streaming server.

[0031] At this time, the cloud streaming server may further include a step of transmitting a transparency error detection result to the transparency check server when the test value and the correct answer value are different, and the transparency check server may provide a notification to the server manager based on the transparency error detection result so that the cloud streaming server in which the transparency error occurred is excluded from the service.

[0032] At this time, the transparency test signal may include an alpha value for the transparency test.

[0033] At this time, the correct answer value of the transparency test mode can be stored according to the above alpha value.

[0034] At this time, the transparency check server can transmit a transparency test signal containing different alpha values ​​to multiple cloud streaming servers.

[0035] At this time, the transparency check server can transmit the transparency test signal to the plurality of cloud streaming servers at preset intervals.

[0036] In addition, the cloud streaming server according to the present invention includes a processor that implements a transparency test mode in a browser, performs a transparency test through the transparency test mode when receiving a transparency test signal from a transparency check server, and detects a transparency error by comparing a test value by the transparency test with a correct value of the transparency test mode stored in advance; and a memory that stores the correct value of the transparency test mode.

[0037] At this time, the processor transmits a transparency error detection result to the transparency check server when the test value and the correct answer value are different, and the transparency check server can provide a notification to the server manager so that the cloud streaming server in which the transparency error occurred is excluded from the service based on the transparency error detection result.

[0038] At this time, the transparency test signal may include an alpha value for the transparency test.

[0039] At this time, the correct answer value of the transparency test mode can be stored according to the above alpha value.

[0040] At this time, the transparency check server can transmit a transparency test signal containing different alpha values ​​to multiple cloud streaming servers.

[0041] At this time, the transparency check server can transmit the transparency test signal to the plurality of cloud streaming servers at preset intervals.

[0042] According to the present invention, it is possible to provide a video with transparency applied to a user by providing transparency data while using the Native Windows method, which operates at higher performance than the OSR method when streaming video to a cloud.

[0043] Additionally, the present invention can minimize performance degradation by reducing CPU usage for providing transparency data in a video cloud streaming service.

[0044] In addition, the present invention can improve service stability by quickly detecting a server in which a transparency error has occurred during a video cloud streaming service.

[0045] Additionally, the present invention can reduce traffic when verifying transparency data of a cloud streaming server.

[0046] In addition, the present invention can solve a problem in which an error occurs in verification due to incorrect data being transmitted to multiple servers when verifying transparency data of a cloud streaming server.

[0047] FIG. 1 is a block diagram illustrating a cloud streaming service system according to one embodiment of the present invention.

[0048] FIG. 2 is a flowchart illustrating a method for providing a video cloud streaming service that provides transparency information using a renderer process according to one embodiment of the present invention.

[0049] FIG. 3 is a diagram illustrating an example of a process for providing a video cloud streaming service according to the present invention.

[0050] FIG. 4 is a diagram showing an example of a component within a video cloud streaming server according to the present invention.

[0051] FIG. 5 is a flowchart illustrating in detail a process of generating transparency data in a method for providing a video cloud streaming service according to one embodiment of the present invention.

[0052] FIG. 6 is a detailed flowchart illustrating a process of generating video ES data among methods for providing a video cloud streaming service according to one embodiment of the present invention.

[0053] FIG. 7 is a block diagram illustrating a video cloud streaming server according to one embodiment of the present invention.

[0054] FIG. 8 is a block diagram illustrating an efficient transparency error detection system in a video cloud streaming environment according to an embodiment of the present invention.

[0055] FIG. 9 is a flowchart illustrating an efficient transparency error detection method in a video cloud streaming environment according to an embodiment of the present invention.

[0056] FIG. 10 is a diagram illustrating an example of transmitting a transparency test signal to a plurality of cloud streaming servers according to the present invention.

[0057] Figure 11 is a drawing showing an example of a transparency test process according to the present invention.

[0058] Figure 12 is a flowchart illustrating in detail a transparency error detection method according to one embodiment of the present invention.

[0059] FIG. 13 is a diagram illustrating a transparency error detection process according to one embodiment of the present invention.

[0060] FIG. 14 is a block diagram illustrating a cloud streaming server according to one embodiment of the present invention.

[0061] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. However, detailed descriptions of well-known functions or components that may obscure the gist of the present invention will be omitted in the following description and the accompanying drawings. It should also be noted that, where possible, identical components are indicated by the same reference numerals throughout the drawings.

[0062] The terms and words used in this specification and claims described below should not be interpreted as limited to their conventional or dictionary meanings, but should be interpreted with meanings and concepts that conform to the technical idea of ​​the present invention based on the principle that the inventor can appropriately define the concept of the term to best describe his or her invention. Therefore, the embodiments described in this specification and the configurations illustrated in the drawings are only the most preferred embodiments of the present invention and do not represent all of the technical idea of ​​the present invention. Therefore, it should be understood that there may be various equivalents and modified examples that can replace them at the time of this application. In addition, terms such as “first” and “second” are used to describe various components, and are used only for the purpose of distinguishing one component from another, and are not used to limit the components.

[0063] FIG. 1 is a block diagram illustrating a cloud streaming service system according to one embodiment of the present invention.

[0064] Referring to FIG. 1, a cloud streaming service system according to one embodiment of the present invention includes a video cloud streaming server (110), user terminals (120-1 to 120-N), and a network (130).

[0065] Cloud streaming service or cloud computing service is a service that allows users to store content in cloud storage on the Internet rather than in local storage, and access the cloud storage and use the content anytime, anywhere using an Internet-enabled device.

[0066] The video cloud streaming server (110) may include a video cloud streaming engine for processing a video cloud streaming service requested from a user terminal (120-1 to 120-N).

[0067] At this time, the video cloud streaming server (110) can create a predefined number of browsers at the time the video cloud streaming engine is executed, and the browser can download and execute HTML, CSS, JavaScript, etc. and render them on the screen.

[0068] At this time, HTML (Hyper Text Markup Language) provides the large framework of a web page, CSS (Cascading Style Sheets) manages design elements such as color and font, and JavaScript is a cross-platform, object-oriented scripting language that can be responsible for the operation of a web page.

[0069] Additionally, the video cloud streaming server (110) generates transparency data based on detection of transparency changes in the renderer process within the browser.

[0070] At this time, the browser can be run in NATIVE WINDOWS mode.

[0071] At this time, the transparency data can be generated by performing capture and encoding based on the alpha value transmitted corresponding to the detection of the transparency change through the renderer process.

[0072] At this time, the renderer process can perform the above transparency change detection based on the transparency change detection module.

[0073] Additionally, the video cloud streaming server (110) generates video element stream data based on screen change detection in the GPU processor within the browser.

[0074] At this time, the video element stream data can be generated by performing capture and encoding based on the texture value transmitted corresponding to the screen change detection through the GPU processor.

[0075] Additionally, the video cloud streaming server (110) synchronizes the transparency data with the video element stream data.

[0076] At this time, synchronization can be performed based on the frame number (FRAME NUMBER).

[0077] Additionally, the video cloud streaming server (110) streams synchronized data to the user terminal.

[0078] At this time, the user terminal can display a screen with transparency changes applied by combining video element stream data and transparency data.

[0079] User terminals (120-1 to 120-N) can request a cloud streaming service from a cloud streaming server (110), and receive an execution screen according to the request from the cloud streaming server (110) and provide it to the user. Accordingly, the user can feel as if a video or application corresponding to the requested service is running on the user terminal (120-1 to 120-N).

[0080] At this time, the user terminal (120-1 to 120-N) can synthesize RGB data and alpha data based on synchronization information and output the synthesized data to a user interface plain overlaid on a live channel.

[0081] At this time, the user terminal (120-1 to 120-N) can simultaneously perform video decoding for a live channel and video decoding for RGB data based on at least two video decoders.

[0082] At this time, the user terminals (120-1 to 120-N) may correspond to devices that are each connected to a communication network and can communicate with the cloud streaming server (110). For example, the user terminals (120-1 to 120-N) may correspond to various terminals such as all information and communication devices, multimedia terminals, wired terminals, fixed terminals, mobile communication terminals, and IP (Internet Protocol) terminals. In addition, the user terminals (120-1 to 120-N) may be mobile terminals having various mobile communication specifications such as cell phones, mobile phones, portable multimedia players (PMPs), mobile Internet devices (MIDs), smart phones, desktops, tablet computers (Tablet PCs), notebooks, netbooks, personal digital assistants (PDAs), smart TVs, and information and communication devices.

[0083] In addition, the user terminals (120-1 to 120-N) can input various information such as number and character information, set various functions, and transmit signals input in relation to the control of the functions of the user terminals (120-1 to 120-N) to the control unit through the input unit. In addition, the input unit of the user terminals (120-1 to 120-N) can be configured to include at least one of a keypad and a touchpad that generate an input signal according to the user's touch or operation. In this case, the input unit of the user terminals (120-1 to 120-N) is configured in the form of a single touch panel (or touch screen) together with the display unit of the user terminals (120-1 to 120-N) so that input and display functions can be performed simultaneously. In addition to input devices such as a keyboard, keypad, mouse, joystick, etc., the input unit of the user terminals (120-1 to 120-N) can use any form of input means that may be developed in the future.

[0084] In addition, the display unit of the user terminal (120-1 to 120-N) can display information about a series of operation states and operation results that occur during the performance of the functions of the user terminal (120-1 to 120-N). In addition, the display unit of the user terminal (120-1 to 120-N) can display menus of the user terminal (120-1 to 120-N) and user data input by the user. Here, the display unit of the user terminal (120-1 to 120-N) can be composed of a liquid crystal display (LCD), a thin film transistor LCD (TFT-LCD), a light emitting diode (LED), an organic light emitting diode (OLED), an active matrix OLED (AMOLED), a retina display, a flexible display, a three-dimensional (3 Dimension) display, etc. At this time, if the display unit of the user terminal (120-1 to 120-N) is configured in the form of a touch screen, the display unit of the user terminal (120-1 to 120-N) can perform some or all of the functions of the input unit of the user terminal (120-1 to 120-N).

[0085] In addition, the storage unit of the user terminal (120-1 to 120-N) is a device for storing data, and includes a main memory device and an auxiliary memory device, and can store application programs required for the functional operation of the user terminal (120-1 to 120-N). The storage unit of the user terminal (120-1 to 120-N) can largely include a program area and a data area. Here, when the user terminal (120-1 to 120-N) activates each function in response to a user request, it executes the corresponding application programs under the control of the control unit to provide each function.

[0086] In addition, the communication unit of the user terminal (120-1 to 120-N) may perform a function for transmitting and receiving data through a network with the cloud streaming server (110). Here, the communication unit of the user terminal (120-1 to 120-N) may include an RF transmitting means for up-converting and amplifying the frequency of a transmitted signal, an RF receiving means for low-noise amplifying a received signal and down-converting the frequency, etc. The communication unit of the user terminal (120-1 to 120-N) may include a wireless communication module. In addition, the wireless communication module is a configuration for transmitting and receiving data according to a wireless communication method, and when the user terminal (120-1 to 120-N) uses wireless communication, any one of a wireless network communication module, a wireless LAN communication module, and a wireless fan communication module may be used to transmit and receive data to and from the cloud streaming server (110). That is, the user terminal (120-1 to 120-N) connects to the network using a wireless communication module and can transmit and receive data with the cloud streaming server (110) through the network.

[0087] In addition, the control unit of the user terminal (120-1 to 120-N) may be a process device that drives an operating system (OS) and each component. For example, the control unit may control the entire process of connecting to the cloud streaming server (110). At this time, when the user terminal (120-1 to 120-N) connects to the cloud streaming server (110), the entire process of executing content such as a video or an application according to the user's request may be controlled, and a service use request may be transmitted to the cloud streaming server (110) simultaneously with the execution. At this time, information of the user terminal (120-1 to 120-N) required for user authentication may also be controlled to be transmitted together.

[0088] The network (130) provides a path for transmitting data between the cloud streaming server (110) and the user terminals (120-1 to 120-N), and is a concept encompassing both existing networks and networks that can be developed in the future. For example, the network (130) may be a wired or wireless local area network that provides communication between various information devices within a limited area, a mobile communication network that provides communication between mobiles and between mobiles and the outside world, a satellite communication network that provides communication between earth stations using satellites, or a wired or wireless communication network, or a combination of two or more of them. Meanwhile, the transmission method standard of the network is not limited to the existing transmission method standard, and may include all transmission method standards to be developed in the future.

[0089] This system allows us to provide users with transparent video by providing transparency data while using the Native Windows method, which operates at higher performance than the OSR method when streaming video to the cloud.

[0090] Additionally, performance degradation can be minimized by reducing CPU usage for providing transparency data in video cloud streaming services.

[0091] FIG. 2 is a flowchart illustrating a method for providing a video cloud streaming service that provides transparency information using a renderer process according to one embodiment of the present invention.

[0092] Referring to FIG. 2, a method for providing a video cloud streaming service that provides transparency information using a renderer process according to an embodiment of the present invention is such that a video cloud streaming server generates transparency data based on detection of a change in transparency in a renderer process within a browser (S210).

[0093] At this time, the renderer process may correspond to the rendering process for composing the screen to be displayed on the browser. For example, when an application execution screen is received from a web application server running a web application, the renderer processor can compute the received application execution screen and compose the screen so that it can be displayed on the browser of the video cloud streaming server.

[0094] At this time, the browser can be run in NATIVE WINDOWS mode.

[0095] Typically, obtaining alpha values ​​for transparency data from a video cloud streaming server requires the browser to operate in Off-Screen Rendering (OSR) mode. However, OSR mode may not be suitable for seamlessly delivering video cloud streaming services, as it has lower performance than native Windows mode.

[0096] However, when using the Native Windows method, which is higher performing than the OSR method, the APIs for receiving GPU acceleration in each operating system such as Windows / Linux do not support transparency, so there is no way to provide transparency data when composing the final screen.

[0097] To solve these problems, the present invention proposes a method for obtaining transparency data through a rendering process that calculates an image of a screen to be output to a browser while operating the browser in a native Windows manner when providing a video cloud streaming service.

[0098] That is, the GPU process that shows the execution screen of the application on the browser cannot express transparency in the native Windows way, but a method is proposed to provide transparency data to the user terminal that receives data through streaming so that the transparency of the screen can be expressed.

[0099] At this time, the video cloud streaming server can generate transparency data by performing capture and encoding based on the alpha value transmitted corresponding to the transparency change detection through the renderer process.

[0100] For example, during the rendering process of a web application, it is possible to detect whether a change in transparency occurs on the screen. If a change in transparency is detected on the application screen, the alpha value corresponding to the changed transparency can be captured and encoded to generate transparency data.

[0101] At this time, the renderer process can perform the above transparency change detection based on the transparency change detection module.

[0102] For example, a transparency change detection module can operate on a renderer process in the form of software and can detect when a transparency change occurs in an execution screen received from a web application.

[0103] At this time, the transparency change detection module is not limited to a specific module or algorithm, and can be used as something that can operate on the renderer process.

[0104] In addition, a method for providing a video cloud streaming service that provides transparency information using a renderer process according to an embodiment of the present invention is such that a video cloud streaming server generates video element stream data based on detection of screen changes in a GPU processor within a browser (S220).

[0105] At this time, the GPU process corresponds to a processing process for processing a screen shown to a user using a graphics processing unit (GPU), and can generate a video element stream such as H.264 / 265.

[0106] At this time, the video cloud streaming server can generate video element stream data by performing capture and encoding based on the texture value transmitted corresponding to the screen change detection through the GPU processor.

[0107] For example, a GPU process that processes screens displayed to the user can detect changes in the application execution screen received via a web application server. Therefore, when a screen change is detected, the texture values ​​corresponding to the changed screen can be captured and encoded to generate video element stream data.

[0108] In addition, a method for providing a video cloud streaming service that provides transparency information using a renderer process according to an embodiment of the present invention, a video cloud streaming server synchronizes transparency data and video element stream data (S230).

[0109] At this time, synchronization can be performed based on the frame number (FRAME NUMBER).

[0110] For example, synchronization can be performed by comparing the frame number corresponding to the alpha value passed from the renderer process with the frame number corresponding to the texture value passed from the GPU processor, and matching the alpha value and texture value corresponding to the same frame number.

[0111] In addition, a method for providing a video cloud streaming service that provides transparency information using a renderer process according to an embodiment of the present invention, a video cloud streaming server streams synchronized data to a user terminal (S240).

[0112] At this time, the user terminal can display a screen with transparency changes applied by combining video element stream data and transparency data.

[0113] For example, a user terminal can receive synchronized transparency data and video element stream data and then decode each piece of data. Thereafter, the decoded video element stream data and transparency data can be combined to display a video screen with applied transparency to the user.

[0114] User terminals (120-1 to 120-N) can request a cloud streaming service from a video cloud streaming server (110), and receive an execution screen according to the request from the video cloud streaming server (110) and provide it to the user. Accordingly, the user can feel as if a video or application corresponding to the requested service is running on the user terminal (120-1 to 120-N).

[0115] By providing a video cloud streaming service that provides transparency information using a renderer process like this, it is possible to provide users with videos with transparency applied by providing transparency data while using the Native Windows method that operates at high performance compared to the OSR method when streaming video cloud.

[0116] Additionally, performance degradation can be minimized by reducing CPU usage for providing transparency data in video cloud streaming services.

[0117] FIG. 3 is a diagram illustrating an example of a process for providing a video cloud streaming service according to the present invention.

[0118] Referring to FIG. 3, the video cloud streaming service according to the present invention can be provided through a process of capturing and encoding a screen processed through a browser (300) generated within a video cloud streaming server and streaming it to a user terminal.

[0119] At this time, the browser (300) can generate an application execution screen corresponding to the service requested from the user terminal by outputting the application execution screen received from the web application server to the browser (300) through the renderer process (310), GPU process (320), and main process (330).

[0120] At this time, the renderer process (310) may correspond to a rendering process for configuring a screen to be output to a browser. For example, when an application execution screen is received from a web application server executing a web application, the renderer processor may compute the received application execution screen to configure the screen so that it can be output to a browser of a video cloud streaming server.

[0121] At this time, the GPU process (320) corresponds to a processing process for processing a screen shown to a user using a graphics processing unit (GPU), and can generate a video element stream such as H.264 / 265.

[0122] At this time, the renderer process (310) can detect whether a change in transparency occurs on the application execution screen. If a change in transparency is detected on the application execution screen, an alpha value (311) corresponding to the changed transparency can be transmitted to the transparency capture and encoding module within the video cloud streaming server to generate transparency data.

[0123] At this time, the renderer process (310) can detect a change in transparency based on a transparency change detection module.

[0124] For example, the transparency change detection module can operate on the renderer process (310) in the form of software and can detect when a transparency change occurs in the execution screen received from the web application.

[0125] At this time, the GPU process (320) can detect a change in the screen of the application execution screen. If a change in the screen of the application execution screen is detected, a texture value (321) corresponding to the changed screen can be transmitted to the video capture and encoding module within the video cloud streaming server to generate video element stream data.

[0126] After this, the transparency data and video element stream data generated through the transparency capture and encoding module and the video capture and encoding module within the video cloud streaming server can be synchronized and streamed to the user terminal.

[0127] At this time, the main processor (330) can perform other tasks to create an application execution screen in the browser (300).

[0128] FIG. 4 is a diagram showing an example of a component within a video cloud streaming server according to the present invention.

[0129] Referring to FIG. 4, the video cloud streaming server according to the present invention may be composed of a transparency change detection unit (411), a transparency capture / compression unit (412), a screen change detection unit (421), a video capture / compression unit (422), a synchronization unit (430), a streaming unit (440), etc.

[0130] The transparency change detection unit (411) can detect transparency changes in the renderer process within the browser, and the transparency capture / compression unit (412) can generate transparency data based on the transparency change detection.

[0131] For example, the transparency change detection unit (411) can detect whether a transparency change occurs in the execution screen of a web application received from a web application server. If a transparency change is detected in the application execution screen, the transparency capture / compression unit (412) can capture and encode an alpha value corresponding to the changed transparency to generate transparency data.

[0132] The screen change detection unit (421) can perform screen change detection in the GPU processor, and the video capture / compression unit (422) can generate video element stream data based on the screen change detection.

[0133] For example, the screen change detection unit (421) can detect changes in the application execution screen received through the web application server. If a screen change is detected, the video capture / compression unit (422) can capture and encode the texture value corresponding to the changed screen to generate video element stream data.

[0134] The synchronization unit (430) can synchronize transparency data and video element stream data.

[0135] At this time, synchronization can be performed based on the frame number (FRAME NUMBER).

[0136] For example, synchronization can be performed by comparing the frame number corresponding to the alpha value passed from the renderer process with the frame number corresponding to the texture value passed from the GPU processor, and matching the alpha value and texture value corresponding to the same frame number.

[0137] The streaming unit (440) can stream synchronized data to a user terminal.

[0138] At this time, the user terminal can display a screen with transparency changes applied by combining video element stream data and transparency data.

[0139] For example, a user terminal can receive synchronized transparency data and video element stream data and then decode each piece of data. Thereafter, the decoded video element stream data and transparency data can be combined to display a video screen with applied transparency to the user.

[0140] By providing a video cloud streaming service that provides transparency information using a renderer process like this, it is possible to provide users with videos with transparency applied while using the Native Windows method that operates at higher performance than the OSR method when streaming video cloud, while also providing transparency data.

[0141] Additionally, performance degradation can be minimized by reducing CPU usage for providing transparency data in video cloud streaming services.

[0142] FIG. 5 is a flowchart illustrating in detail a process of generating transparency data in a method for providing a video cloud streaming service according to one embodiment of the present invention.

[0143] Referring to FIG. 5, in a method for providing a video cloud streaming service according to an embodiment of the present invention, a process for generating transparency data can first detect a change in transparency of an application execution screen received from a web application server based on a renderer process within a browser (S510).

[0144] After this, it is determined whether a change in transparency has been detected (S515), and if a change in transparency has been detected, transparency data can be generated by performing capture and encoding based on the alpha value transmitted corresponding to the detection of the change in transparency through the renderer process (S520).

[0145] For example, during the rendering process of a web application, it is possible to detect whether a change in transparency occurs on the screen. If a change in transparency is detected on the application screen, the alpha value corresponding to the changed transparency can be captured and encoded to generate transparency data.

[0146] Additionally, if no change in transparency is detected as a result of the judgment in step (S515), detection of a change in transparency can be repeatedly performed while the web application is running (S510, S515).

[0147] FIG. 6 is a detailed flowchart illustrating a process of generating video ES data among methods for providing a video cloud streaming service according to one embodiment of the present invention.

[0148] Referring to FIG. 6, in a method for providing a video cloud streaming service according to an embodiment of the present invention, a process for generating video ES data can first detect a screen change of an application execution screen received from a web application server based on a GPU process within a browser (S610).

[0149] After this, it is determined whether a screen change has been detected (S615), and if a screen change has been detected, capture and encoding can be performed based on the texture value transmitted corresponding to the screen change detection through the GPU process to generate video element stream data (S620).

[0150] For example, a GPU process that processes screens displayed to the user can detect changes in the application execution screen received via a web application server. Therefore, when a screen change is detected, the texture values ​​corresponding to the changed screen can be captured and encoded to generate video element stream data.

[0151] Additionally, if a screen change is not detected as a result of the judgment in step (S615), screen change detection can be repeatedly performed while the web application is running (S610, S615).

[0152] FIG. 7 is a block diagram illustrating a video cloud streaming server according to one embodiment of the present invention.

[0153] Referring to FIG. 7, a video cloud streaming server according to one embodiment of the present invention includes a communication unit (710), a processor (720), and a memory (730).

[0154] The communication unit (710) serves to transmit and receive information related to multiple user terminals via a communication network, such as the network illustrated in FIG. 1. In particular, the communication unit (710) according to one embodiment of the present invention may receive a request for a video cloud streaming service from a user terminal and provide a data packet corresponding to the video cloud streaming service requested by the user terminal to the user terminal.

[0155] The processor (720) generates transparency data based on detection of a change in transparency in the renderer process within the browser.

[0156] At this time, the renderer process may correspond to the rendering process for composing the screen to be displayed on the browser. For example, when an application execution screen is received from a web application server running a web application, the renderer processor can compute the received application execution screen and compose the screen so that it can be displayed on the browser of the video cloud streaming server.

[0157] At this time, the browser can be run in NATIVE WINDOWS mode.

[0158] Typically, obtaining alpha values ​​for transparency data from a video cloud streaming server requires the browser to operate in Off-Screen Rendering (OSR) mode. However, OSR mode may not be suitable for seamlessly delivering video cloud streaming services, as it has lower performance than native Windows mode.

[0159] However, when using the Native Windows method, which is higher performing than the OSR method, the APIs for receiving GPU acceleration in each operating system such as Windows / Linux do not support transparency, so there is no way to provide transparency data when composing the final screen.

[0160] To solve these problems, the present invention proposes a method for obtaining transparency data through a rendering process that calculates an image of a screen to be output to a browser while operating the browser in a native Windows manner when providing a video cloud streaming service.

[0161] That is, the GPU process that shows the execution screen of the application on the browser cannot express transparency in the native Windows way, but a method is proposed to provide transparency data to the user terminal that receives data through streaming so that the transparency of the screen can be expressed.

[0162] At this time, the video cloud streaming server can generate transparency data by performing capture and encoding based on the alpha value transmitted corresponding to the transparency change detection through the renderer process.

[0163] For example, during the rendering process of a web application, it is possible to detect whether a change in transparency occurs on the screen. If a change in transparency is detected on the application screen, the alpha value corresponding to the changed transparency can be captured and encoded to generate transparency data.

[0164] At this time, the renderer process can perform the above transparency change detection based on the transparency change detection module.

[0165] For example, a transparency change detection module can operate on a renderer process in the form of software and can detect when a transparency change occurs in an execution screen received from a web application.

[0166] At this time, the transparency change detection module is not limited to a specific module or algorithm, and can be used as something that can operate on the renderer process.

[0167] Additionally, the processor (720) generates video element stream data based on screen change detection in the GPU processor within the browser.

[0168] At this time, the GPU process corresponds to a processing process for processing a screen shown to a user using a graphics processing unit (GPU), and can generate a video element stream such as H.264 / 265.

[0169] At this time, the video cloud streaming server can generate video element stream data by performing capture and encoding based on the texture value transmitted corresponding to the screen change detection through the GPU processor.

[0170] For example, a GPU process that processes screens displayed to the user can detect changes in the application execution screen received via a web application server. Therefore, when a screen change is detected, the texture values ​​corresponding to the changed screen can be captured and encoded to generate video element stream data.

[0171] Additionally, the processor (720) synchronizes transparency data and video element stream data.

[0172] At this time, synchronization can be performed based on the frame number (FRAME NUMBER).

[0173] For example, synchronization can be performed by comparing the frame number corresponding to the alpha value passed from the renderer process with the frame number corresponding to the texture value passed from the GPU processor, and matching the alpha value and texture value corresponding to the same frame number.

[0174] Additionally, the processor (720) streams synchronized data to the user terminal (S240).

[0175] At this time, the user terminal can display a screen with transparency changes applied by combining video element stream data and transparency data.

[0176] For example, a user terminal can receive synchronized transparency data and video element stream data and then decode each piece of data. Thereafter, the decoded video element stream data and transparency data can be combined to display a video screen with applied transparency to the user.

[0177] The memory (730) stores transparency data and video element stream data.

[0178] In addition, the memory (730) stores various information generated in the process of providing a video cloud streaming service that provides transparency information using a renderer process according to an embodiment of the present invention as described above.

[0179] In some embodiments, the memory (730) may be configured independently from the video cloud streaming server to support a function for providing a video cloud streaming service that provides transparency information using a renderer process. In this case, the memory (730) may function as a separate large-capacity storage and may include a control function for performing the operation.

[0180] Additionally, the video cloud streaming server configured as described above may be implemented with one or more servers.

[0181] Meanwhile, the video cloud streaming server is equipped with memory and can store information within the device. In one embodiment, the memory is a computer-readable medium. In one embodiment, the memory may be a volatile memory unit, and in another embodiment, the memory may be a non-volatile memory unit. In one embodiment, the storage device is a computer-readable medium. In various other embodiments, the storage device may include, for example, a hard disk drive, an optical disk drive, or any other mass storage device.

[0182] By using a video cloud streaming server like this, we can provide transparency data while using the Native Windows method, which operates at higher performance than the OSR method when streaming video to the video cloud, and provide users with videos with transparency applied.

[0183] Additionally, performance degradation can be minimized by reducing CPU usage for providing transparency data in video cloud streaming services.

[0184] FIG. 8 is a block diagram illustrating an efficient transparency error detection system in a video cloud streaming environment according to an embodiment of the present invention.

[0185] Referring to FIG. 8, a transparency error detection system according to one embodiment of the present invention includes a transparency check server (810), a management server (820), a cloud streaming server (830-1 to 830-N), and a network.

[0186] Cloud streaming service or cloud computing service is a service that allows users to store content in cloud storage on the Internet rather than in local storage, and access the cloud storage and use the content anytime, anywhere using an Internet-enabled device.

[0187] The cloud streaming server (830-1 to 830-N) may include a video cloud streaming engine for processing a cloud streaming service requested from a user terminal.

[0188] At this time, the cloud streaming server (830-1 to 830-N) can create a predefined number of browsers when the cloud streaming engine is running, and the browser can download and execute HTML, CSS, and JavaScript, etc., and render them on the screen.

[0189] At this time, HTML (Hyper Text Markup Language) provides the large framework of a web page, CSS (Cascading Style Sheets) manages design elements such as color and font, and JavaScript is a cross-platform, object-oriented scripting language that can be responsible for the operation of a web page.

[0190] The present invention relates to a technology for detecting whether a transparency error occurs in a cloud streaming server (830-1 to 830-N) providing such a cloud streaming service. The transparency error detection process will be described in detail below.

[0191] Cloud Streaming Servers (830-1 to 830-N) implement a transparency test mode in the browser.

[0192] Additionally, when the cloud streaming server (830-1 to 830-N) receives a transparency test signal from the transparency check server, it performs a transparency test through the transparency test mode.

[0193] At this time, the transparency test signal may include an alpha value for transparency testing.

[0194] At this time, the transparency check server (810) can transmit a transparency test signal containing different alpha values ​​to multiple cloud streaming servers (830-1 to 830-N).

[0195] At this time, the transparency check server (810) can transmit a transparency test signal to multiple cloud streaming servers (830-1 to 830-N) at preset intervals.

[0196] Additionally, the cloud streaming server (830-1 to 830-N) detects transparency errors by comparing the test value by the transparency test with the correct value of the transparency test mode stored in advance.

[0197] Additionally, the cloud streaming server (830-1 to 830-N) transmits the transparency error detection result to the transparency check server (810) when the test value and the correct answer value are different.

[0198] At this time, the correct answer value of the transparency test mode can be stored according to the alpha value.

[0199] At this time, the transparency check server (810) can provide a notification to the server manager or management server (820) so that the cloud streaming server in which the transparency error occurred is excluded from the service based on the transparency error detection result.

[0200] At this time, transparency information generated for transparency testing in the present invention may only be used within the server and may not be transmitted to user terminals using the cloud streaming service.

[0201] Although not illustrated in FIG. 8, a user terminal that accesses a cloud streaming server (830-1 to 830-N) according to the present invention and uses a cloud streaming service can request the cloud streaming service from the cloud streaming server (830-1 to 830-N), and receive an execution screen according to the request from the cloud streaming server (830-1 to 830-N) and provide it to the user. Accordingly, the user can feel as if a video or application corresponding to the requested service is running on the user terminal.

[0202] Through such a system, service stability can be improved by quickly detecting servers that have transparency errors in video cloud streaming services.

[0203] Additionally, it can reduce traffic when verifying transparency data on cloud streaming servers.

[0204] Additionally, it can solve the problem of verification errors occurring when verifying transparency data of a cloud streaming server due to incorrect data being transmitted to multiple servers.

[0205] FIG. 9 is a flowchart illustrating an efficient transparency error detection method in a video cloud streaming environment according to an embodiment of the present invention.

[0206] Referring to FIG. 9, in an efficient transparency error detection method in a video cloud streaming environment according to one embodiment of the present invention, a cloud streaming server implements a transparency test mode in a browser (S910).

[0207] For example, a cloud streaming server according to one embodiment of the present invention can implement a transparency test mode through a transparency test module (1111) provided in a browser, as illustrated in FIG. 11.

[0208] In addition, in an efficient transparency error detection method in a video cloud streaming environment according to an embodiment of the present invention, when a cloud streaming server receives a transparency test signal from a transparency check server, the cloud streaming server performs a transparency test through a transparency test mode (S920).

[0209] At this time, the transparency test signal may include an alpha value for transparency testing.

[0210] At this time, the transparency check server can transmit a transparency test signal containing different alpha values ​​to multiple cloud streaming servers.

[0211] At this time, the transparency check server can transmit a transparency test signal to multiple cloud streaming servers at preset intervals.

[0212] Referring to FIG. 10, the transparency check server (1010) may transmit a transparency test signal to the cloud streaming servers (1020-1 to 1020-N) to calculate designated alpha values ​​at preset intervals. For example, a transparency test signal including an alpha value of 128 may be transmitted to browser #1 of the cloud streaming server (1020-1), and a transparency test signal including an alpha value of 200 may be transmitted to the browser of the cloud streaming server (1020-N).

[0213] In this way, the transparency check server (1010) can perform transparency error detection for all cloud streaming servers by transmitting transparency test signals containing different alpha values ​​at preset intervals to the cloud streaming servers (1020-1 to 1020-N) that constitute the system.

[0214] In addition, in an efficient transparency error detection method in a video cloud streaming environment according to an embodiment of the present invention, a cloud streaming server detects a transparency error by comparing a test value by a transparency test with an answer value of a transparency test mode stored in advance (S930).

[0215] At this time, the correct answer value of the transparency test mode can be stored according to the alpha value.

[0216] At this time, the transparency value generated for transparency testing in the browser is only used in the transparency error detection module (1120) as shown in Fig. 11, and is not transmitted to the user terminal connected to the cloud streaming server.

[0217] For example, the cloud streaming servers (1020-1 to 1020-N) illustrated in FIG. 10 can store in advance the correct values ​​for alpha values ​​128 and 200 transmitted as included in the transparency test signal from the transparency check server (1010).

[0218] Accordingly, when the cloud streaming server (1020-1) receives a transparency test signal from the transparency check server (1010), it generates a test value for the alpha value 128 included in the transparency test signal, and compares the test value with a previously stored correct value for the alpha value 128 to detect a transparency error. In this case, if the test value and the correct value are different, it can be determined that a transparency error has occurred in the cloud streaming server.

[0219] Specifically, referring to FIG. 11, the cloud streaming server may be equipped with a transparency test module (1111) within a browser (1110), and when a transparency test signal is received, the transparency test module (1111) may generate a test value.

[0220] After this, the correct answer value (1121) stored in the transparency error detection module (1120) within the cloud streaming server can be compared with the test value generated from the transparency test module (1111) to determine whether a transparency error has occurred.

[0221] In addition, although not shown in FIG. 9, an efficient transparency error detection method in a video cloud streaming environment according to an embodiment of the present invention can transmit a transparency error detection result to a transparency check server when a test value and an accurate value are different from each other.

[0222] At this time, the transparency check server can notify the server administrator to exclude the cloud streaming server where the transparency error occurred from the service based on the transparency error detection result.

[0223] For example, referring to FIG. 8, the transparency check server (810) can receive transparency error detection results from cloud streaming servers (830-1 to 830-N) and transmit information about the cloud streaming server in which the transparency error occurred to the management server (820), thereby maintaining overall service stability.

[0224] At this time, the management server (820) that has received information from the transparency check server (810) can stop the service of the cloud streaming server where a transparency error has occurred and control the correction of the error.

[0225] In such a video cloud streaming environment, an efficient transparency error detection method can be used to quickly detect servers where transparency errors occur during video cloud streaming services, thereby improving service stability.

[0226] Additionally, it can reduce traffic when verifying transparency data on cloud streaming servers.

[0227] Additionally, it can solve the problem of verification errors occurring when verifying transparency data of a cloud streaming server due to incorrect data being transmitted to multiple servers.

[0228] Figure 12 is a flowchart illustrating in detail a transparency error detection method according to one embodiment of the present invention.

[0229] Referring to FIG. 12, in a transparency error detection method according to one embodiment of the present invention, a cloud streaming server implements a transparency test mode in a browser (S1210).

[0230] For example, a cloud streaming server according to one embodiment of the present invention can implement a transparency test mode through a transparency test module (1111) provided in a browser, as illustrated in FIG. 11.

[0231] After this, it can be determined whether a transparency test signal is received from the transparency check server (S1215).

[0232] As a result of the judgment in step (S1215), if a transparency test signal has not been received, it is possible to wait until a transparency test signal is received from the transparency check server.

[0233] Additionally, as a result of the judgment in step (S1215), if a transparency test signal is received, a transparency test can be performed through the transparency test mode (S1220).

[0234] At this time, the transparency test signal may include an alpha value for transparency testing.

[0235] At this time, the transparency check server can transmit a transparency test signal containing different alpha values ​​to multiple cloud streaming servers.

[0236] At this time, the transparency check server can transmit a transparency test signal to multiple cloud streaming servers at preset intervals.

[0237] Referring to FIG. 10, the transparency check server (1010) may transmit a transparency test signal to the cloud streaming servers (1020-1 to 1020-N) to calculate designated alpha values ​​at preset intervals. For example, a transparency test signal including an alpha value of 128 may be transmitted to browser #1 of the cloud streaming server (1020-1), and a transparency test signal including an alpha value of 200 may be transmitted to the browser of the cloud streaming server (1020-N).

[0238] In this way, the transparency check server (1010) can perform transparency error detection for all cloud streaming servers by transmitting transparency test signals containing different alpha values ​​at preset intervals to the cloud streaming servers (1020-1 to 1020-N) that constitute the system.

[0239] After this, it is determined whether the test value by the transparency test and the correct answer value of the transparency test mode stored in advance are the same (S1225), and if the test value and the correct answer value are not the same, the transparency error detection result can be transmitted to the transparency check server (S1230).

[0240] Additionally, if the test value and the correct answer value are the same as the judgment result of step (S1225), it is determined that there is no transparency error and the process can be terminated.

[0241] At this time, the correct answer value of the transparency test mode can be stored according to the alpha value.

[0242] At this time, the transparency value generated for transparency testing in the browser is only used in the transparency error detection module (1120) as shown in Fig. 11, and is not transmitted to the user terminal connected to the cloud streaming server.

[0243] For example, the cloud streaming servers (1020-1 to 1020-N) illustrated in FIG. 10 can store in advance the correct values ​​for alpha values ​​128 and 200 transmitted as included in the transparency test signal from the transparency check server (1010).

[0244] Accordingly, when the cloud streaming server (1020-1) receives a transparency test signal from the transparency check server (1010), it generates a test value for the alpha value 128 included in the transparency test signal, and compares the test value with a previously stored correct value for the alpha value 128 to detect a transparency error. In this case, if the test value and the correct value are different, it can be determined that a transparency error has occurred in the cloud streaming server.

[0245] Specifically, referring to FIG. 11, the cloud streaming server may be equipped with a transparency test module (1111) within a browser (1110), and when a transparency test signal is received, the transparency test module (1111) may generate a test value.

[0246] After this, the correct answer value (1121) stored in the transparency error detection module (1120) within the cloud streaming server can be compared with the test value generated from the transparency test module (1111) to determine whether a transparency error has occurred.

[0247] At this time, the transparency check server can notify the server administrator to exclude the cloud streaming server where the transparency error occurred from the service based on the transparency error detection result.

[0248] For example, referring to FIG. 8, the transparency check server (810) can receive transparency error detection results from cloud streaming servers (830-1 to 830-N) and transmit information about the cloud streaming server in which the transparency error occurred to the management server (820), thereby maintaining overall service stability.

[0249] At this time, the management server (820) that has received information from the transparency check server (810) can stop the service of the cloud streaming server where a transparency error has occurred and control the correction of the error.

[0250] At this time, a cloud streaming server according to an embodiment of the present invention can provide a cloud streaming service to a user terminal.

[0251] Through this transparency error detection method, service stability can be improved by quickly detecting servers that have transparency errors in video cloud streaming services.

[0252] Additionally, it can reduce traffic when verifying transparency data on cloud streaming servers.

[0253] Additionally, it can solve the problem of verification errors occurring when verifying transparency data of a cloud streaming server due to incorrect data being transmitted to multiple servers.

[0254] FIG. 13 is a diagram illustrating a transparency error detection process according to one embodiment of the present invention.

[0255] Referring to FIG. 13, a transparency error detection process according to an embodiment of the present invention first implements a transparency test mode in a browser by a cloud streaming server (S1302).

[0256] For example, a cloud streaming server according to one embodiment of the present invention can implement a transparency test mode through a transparency test module (1111) provided in a browser, as illustrated in FIG. 11.

[0257] After this, when the transparency check server transmits a transparency test signal to the cloud streaming server (S1304), the cloud streaming server can perform a transparency test through the transparency test mode (S1306).

[0258] At this time, the transparency test signal may include an alpha value for transparency testing.

[0259] At this time, the transparency check server can transmit a transparency test signal containing different alpha values ​​to multiple cloud streaming servers.

[0260] At this time, the transparency check server can transmit a transparency test signal to multiple cloud streaming servers at preset intervals.

[0261] Referring to FIG. 10, the transparency check server (1010) may transmit a transparency test signal to the cloud streaming servers (1020-1 to 1020-N) to calculate designated alpha values ​​at preset intervals. For example, a transparency test signal including an alpha value of 128 may be transmitted to browser #1 of the cloud streaming server (1020-1), and a transparency test signal including an alpha value of 200 may be transmitted to the browser of the cloud streaming server (1020-N).

[0262] In this way, the transparency check server (1010) can perform transparency error detection for all cloud streaming servers by transmitting transparency test signals containing different alpha values ​​at preset intervals to the cloud streaming servers (1020-1 to 1020-N) that constitute the system.

[0263] After this, it is possible to determine whether a transparency error has occurred by comparing the test value by the transparency test with the correct value of the transparency test mode stored in advance (S1308).

[0264] If no transparency error has occurred as a result of the judgment in step (S1308), it is possible to wait until a transparency test signal is transmitted again from the transparency check server.

[0265] Additionally, if a transparency error occurs as a result of the judgment in step (S1308), the transparency error detection result can be transmitted to the transparency check server (S1310).

[0266] At this time, the correct answer value of the transparency test mode can be stored according to the alpha value.

[0267] At this time, the transparency value generated for transparency testing in the browser is only used in the transparency error detection module (1120) as shown in Fig. 11, and is not transmitted to the user terminal connected to the cloud streaming server.

[0268] For example, the cloud streaming servers (1020-1 to 1020-N) illustrated in FIG. 10 can store in advance the correct values ​​for alpha values ​​128 and 200 transmitted as included in the transparency test signal from the transparency check server (1010).

[0269] Accordingly, when the cloud streaming server (1020-1) receives a transparency test signal from the transparency check server (1010), it generates a test value for the alpha value 128 included in the transparency test signal, and compares the test value with a previously stored correct value for the alpha value 128 to detect a transparency error. In this case, if the test value and the correct value are different, it can be determined that a transparency error has occurred in the cloud streaming server.

[0270] Specifically, referring to FIG. 11, the cloud streaming server may be equipped with a transparency test module (1111) within a browser (1110), and when a transparency test signal is received, the transparency test module (1111) may generate a test value.

[0271] After this, the correct answer value (1121) stored in the transparency error detection module (1120) within the cloud streaming server can be compared with the test value generated from the transparency test module (1111) to determine whether a transparency error has occurred.

[0272] After this, the transparency check server can notify the management server to exclude the cloud streaming server in which the transparency error occurred from the service based on the transparency error detection result (S1312).

[0273] For example, referring to FIG. 8, the transparency check server (810) can receive transparency error detection results from cloud streaming servers (830-1 to 830-N) and transmit information about the cloud streaming server in which the transparency error occurred to the management server (820), thereby maintaining overall service stability.

[0274] After this, the management server can stop the service of the cloud streaming server where the transparency error occurred and control the correction of the error (S1314).

[0275] Through this transparency error detection process, service stability can be improved by quickly detecting servers that have transparency errors in video cloud streaming services.

[0276] Additionally, it can reduce traffic when verifying transparency data on cloud streaming servers.

[0277] Additionally, it can solve the problem of verification errors occurring when verifying transparency data of a cloud streaming server due to incorrect data being transmitted to multiple servers.

[0278] FIG. 14 is a block diagram illustrating a cloud streaming server according to one embodiment of the present invention.

[0279] Referring to FIG. 14, a cloud streaming server according to one embodiment of the present invention includes a communication unit (1410), a processor (1420), and a memory (1430).

[0280] The communication unit (1410) serves to transmit and receive information related to transparency error detection with the transparency check server and management server via a communication network such as the network illustrated in FIG. 8. In particular, the communication unit (1410) according to one embodiment of the present invention may receive a transparency test signal from the transparency check server and, if a transparency error is detected, transmit the transparency error detection result to the transparency check server.

[0281] Here, the network provides a path for transmitting data between the transparency check server, management server, and cloud streaming server, and is a concept encompassing both existing networks and networks that may be developed in the future. For example, the network may be a wired or wireless local area network that provides communication between various information devices within a limited area, a mobile communication network that provides communication between mobile devices and between mobile devices and the outside world, a satellite communication network that provides communication between earth stations using satellites, or a combination of two or more of the wired and wireless communication networks. Meanwhile, the network transmission method standard is not limited to existing transmission method standards and may include all transmission method standards that will be developed in the future.

[0282] The processor (1420) implements a transparency test mode in the browser.

[0283] For example, a cloud streaming server according to one embodiment of the present invention can implement a transparency test mode through a transparency test module (1111) provided in a browser, as illustrated in FIG. 11.

[0284] Additionally, the processor (1420) performs a transparency test through a transparency test mode when receiving a transparency test signal from a transparency check server.

[0285] At this time, the transparency test signal may include an alpha value for transparency testing.

[0286] At this time, the transparency check server can transmit a transparency test signal containing different alpha values ​​to multiple cloud streaming servers.

[0287] At this time, the transparency check server can transmit a transparency test signal to multiple cloud streaming servers at preset intervals.

[0288] Referring to FIG. 10, the transparency check server (1010) may transmit a transparency test signal to the cloud streaming servers (1020-1 to 1020-N) to calculate designated alpha values ​​at preset intervals. For example, a transparency test signal including an alpha value of 128 may be transmitted to browser #1 of the cloud streaming server (1020-1), and a transparency test signal including an alpha value of 200 may be transmitted to the browser of the cloud streaming server (1020-N).

[0289] In this way, the transparency check server (1010) can perform transparency error detection for all cloud streaming servers by transmitting transparency test signals containing different alpha values ​​at preset intervals to the cloud streaming servers (1020-1 to 1020-N) that constitute the system.

[0290] Additionally, the processor (1420) detects a transparency error by comparing the test value by the transparency test with the correct value of the transparency test mode stored in advance.

[0291] At this time, the correct answer value of the transparency test mode can be stored according to the alpha value.

[0292] At this time, the transparency value generated for transparency testing in the browser is only used in the transparency error detection module (1120) as shown in Fig. 11, and is not transmitted to the user terminal connected to the cloud streaming server.

[0293] For example, the cloud streaming servers (1020-1 to 1020-N) illustrated in FIG. 10 can store in advance the correct values ​​for alpha values ​​128 and 200 transmitted as included in the transparency test signal from the transparency check server (1010).

[0294] Accordingly, when the cloud streaming server (1020-1) receives a transparency test signal from the transparency check server (1010), it generates a test value for the alpha value 128 included in the transparency test signal, and compares the test value with a previously stored correct value for the alpha value 128 to detect a transparency error. In this case, if the test value and the correct value are different, it can be determined that a transparency error has occurred in the cloud streaming server.

[0295] Specifically, referring to FIG. 11, the cloud streaming server may be equipped with a transparency test module (1111) within a browser (1110), and when a transparency test signal is received, the transparency test module (1111) may generate a test value.

[0296] After this, the correct answer value (1121) stored in the transparency error detection module (1120) within the cloud streaming server can be compared with the test value generated from the transparency test module (1111) to determine whether a transparency error has occurred.

[0297] Additionally, the processor (1420) transmits the transparency error detection result to the transparency check server when the test value and the correct answer value are different.

[0298] At this time, the transparency check server can notify the server administrator to exclude the cloud streaming server where the transparency error occurred from the service based on the transparency error detection result.

[0299] For example, referring to FIG. 8, the transparency check server (810) can receive transparency error detection results from cloud streaming servers (830-1 to 830-N) and transmit information about the cloud streaming server in which the transparency error occurred to the management server (820), thereby maintaining overall service stability.

[0300] At this time, the management server (820) that has received information from the transparency check server (810) can stop the service of the cloud streaming server where a transparency error has occurred and control the correction of the error.

[0301] Memory (1430) stores the correct answer value of the transparency test mode.

[0302] Additionally, the memory (1430) stores various information generated during an efficient transparency error detection process in a video cloud streaming environment according to an embodiment of the present invention as described above.

[0303] In some embodiments, the memory (1430) may be configured independently from the cloud streaming server to support functions for efficient transparency error detection in a video cloud streaming environment. In this case, the memory (1430) may function as a separate large-capacity storage device and may include control functions for performing operations.

[0304] Additionally, the cloud streaming server configured as described above may be implemented with one or more servers.

[0305] Meanwhile, the cloud streaming server is equipped with memory and can store information within the device. In one embodiment, the memory is a computer-readable medium. In one embodiment, the memory may be a volatile memory unit, and in another embodiment, the memory may be a non-volatile memory unit. In one embodiment, the storage device is a computer-readable medium. In various other embodiments, the storage device may include, for example, a hard disk drive, an optical disk drive, or any other mass storage device.

[0306] In this way, through a cloud streaming server, service stability can be improved by quickly detecting servers that have transparency errors in video cloud streaming services.

[0307] Additionally, it can reduce traffic when verifying transparency data on cloud streaming servers.

[0308] Additionally, it can solve the problem of verification errors occurring when verifying transparency data of a cloud streaming server due to incorrect data being transmitted to multiple servers.

[0309] The functional operations and implementations of the subject matter described herein may be implemented as digital electronic circuits, or may be implemented as computer software, firmware, or hardware that includes the structures disclosed herein and their structural equivalents, or a combination of one or more of these. The implementations of the subject matter described herein may be implemented as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a tangible program storage medium for controlling the operation of a processing system or for execution by the same.

[0310] A computer-readable medium may be a machine-readable storage device, a machine-readable storage substrate, a memory device, a composition of matter for affecting a machine-readable radio signal, or a combination of one or more of these.

[0311] As used herein, the term "system" or "device" encompasses any mechanism, device, or machine for processing data, including, for example, a programmable processor, a computer, or a multiprocessor or computer. In addition to hardware, a processing system may include code that forms an execution environment for a computer program upon request, such as code constituting processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of these.

[0312] A computer program (also known as a program, software, software application, script, or code) may be written in any programming language, including compiled or interpreted languages, a priori or procedural languages, and may be deployed in any form, including as a standalone program, module, component, subroutine, or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program may be stored in a single file provided to a requested program, in multiple interacting files (e.g., a file storing one or more modules, subprograms, or portions of code), or as part of a file containing other programs or data (e.g., one or more scripts stored within a markup language document). A computer program may be deployed to be executed on a single computer or on multiple computers located at a single site or distributed across multiple sites and interconnected by a communications network.

[0313] Meanwhile, computer-readable media suitable for storing computer program instructions and data may include all forms of non-volatile memory, media and memory devices, including semiconductor memory devices such as EPROM, EEPROM and flash memory devices, magnetic disks such as internal hard disks or external disks, magneto-optical disks and CD-ROM and DVD-ROM disks. The processor and memory may be supplemented by, or incorporated into, special-purpose logic circuitry.

[0314] Implementations of the subject matter described herein may be implemented in a computing system that includes a back-end component, such as a data server, a middleware component, such as an application server, a front-end component, such as a user computer having a web browser or graphical user interface through which a user can interact with implementations of the subject matter described herein, or any combination of one or more of such back-end, middleware, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication, such as a communications network.

[0315] While this specification contains details of a number of specific implementations, these should not be construed as limitations on the scope of any invention or what may be claimed, but rather as descriptions of features that may be unique to particular embodiments of a particular invention. Likewise, certain features described herein in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented in multiple embodiments, either individually or in any suitable subcombination. Furthermore, although features may operate in a particular combination and may initially be described as being claimed as such, one or more features from a claimed combination may in some cases be excluded from that combination, and the claimed combination may be modified into a subcombination or variation of a subcombination.

[0316] Furthermore, while this specification depicts operations in the drawings in a particular order, this should not be construed as requiring that such operations be performed in the particular or sequential order depicted to achieve desired results, or that all depicted operations be performed. In certain instances, multitasking and parallel processing may be advantageous. Furthermore, the separation of the various system components of the embodiments described above should not be construed as requiring such separation in all embodiments, and it should be understood that the program components and systems described may generally be integrated together into a single software product or packaged into multiple software products.

[0317] As such, this specification is not intended to limit the invention to the specific terms presented. Accordingly, while the invention has been described in detail with reference to the examples described above, those skilled in the art will appreciate that modifications, variations, and variations may be made to these examples without departing from the scope of the invention. The scope of the invention is indicated by the claims below rather than the detailed description above, and all changes or modifications derived from the meaning and scope of the claims and their equivalents should be construed as being included within the scope of the invention.

[0318] As described above, the method for providing a video cloud streaming service that provides transparency information using a renderer process according to the present invention and the device therefor are not limited to the configurations and methods of the embodiments described above, but the embodiments may be configured by selectively combining all or part of each embodiment so that various modifications can be made.

[0319] According to the present invention, a video cloud streaming server can generate transparency data based on detection of a transparency change in a renderer process within a browser, generate video element stream data based on detection of a screen change in a GPU processor within a browser, synchronize the transparency data with the video element stream data, and stream the synchronized data to a user terminal. This allows the server to be operated so as to provide transparent video more effectively by providing transparency information while using a high-performance Native Windows method.

[0320] Furthermore, according to the present invention, when a cloud streaming server implements a transparency test mode in a browser and receives a transparency test signal from a transparency check server, the cloud streaming server performs a transparency test through the transparency test mode, and compares the test value obtained through the transparency test with the correct value of the transparency test mode stored in advance to detect transparency errors. Accordingly, a cloud streaming server in which a transparency error has occurred can be quickly detected, thereby improving service stability.

Claims

1. Video cloud streaming server, A step of generating transparency data based on detecting a change in transparency in the renderer process within the browser; A step of generating VIDEO ELEMENT STREAM data based on detection of screen changes in the GPU processor within the browser; A step of synchronizing the above transparency data and the above video element stream data; and Steps for streaming synchronized data to user terminals A method for providing a video cloud streaming service, characterized in that it includes:

2. In claim 1, The above video cloud streaming server A method for providing a video cloud streaming service, characterized in that the transparency data is generated by performing capture and encoding based on the alpha value transmitted corresponding to the detection of the change in transparency through the renderer process.

3. In claim 1, The above video cloud streaming server A method for providing a video cloud streaming service, characterized in that the video element stream data is generated by performing capture and encoding based on a texture value transmitted corresponding to the screen change detection through the GPU processor.

4. In claim 1, The above browser A method for providing a video cloud streaming service characterized by being operated in a native Windows manner.

5. In claim 1, The above user terminal A method for providing a video cloud streaming service, characterized in that the method combines the video element stream data and the transparency data to display a screen with applied transparency changes.

6. In claim 2, A method for providing a video cloud streaming service, wherein the above renderer process performs the above transparency change detection based on a transparency change detection module.

7. In claim 1, The above synchronization is A method for providing a video cloud streaming service, characterized in that it is performed based on a frame number (FRAME NUMBER).

8. A processor that generates transparency data based on detection of a change in transparency in a renderer process within a browser, generates video element stream data based on detection of a change in screen in a GPU processor within a browser, synchronizes the transparency data and the video element stream data, and streams the synchronized data to a user terminal; and Memory for storing the above transparency data and the above video element stream data A video cloud streaming server comprising:

9. Cloud streaming server, Steps to implement a transparency test mode in your browser; When receiving a transparency test signal from a transparency check server, a step of performing a transparency test through the transparency test mode; and A step of detecting a transparency error by comparing the test value by the above transparency test with the correct answer value of the above transparency test mode stored in advance. A method for detecting transparency errors, characterized by including:

10. A processor that implements a transparency test mode in a browser, performs a transparency test through the transparency test mode when receiving a transparency test signal from a transparency check server, and detects a transparency error by comparing a test value by the transparency test with a correct value of the transparency test mode stored in advance; and Memory for storing the correct answer value of the above transparency test mode A cloud streaming server comprising:

Images