System and method for canvas cleaning

Abstract

The present invention provides novel systems and methods for canvas cleanup. In various embodiments, a system and method includes: receiving a request to open a web page from a client device; replacing a first function with a second function via a proxy, the proxy being loaded into a browser; loading the web page from a network server into the browser; in response to attempting to execute the first function on the web page, executing the second function corresponding to the first function via the browser to generate a drawing for a predetermined time period; converting the drawing to an image via the proxy; and transmitting the image to the client device. Other aspects, embodiments, and features are also claimed and described.

Description

[0001] Copyright Notice

[0002] A portion of the disclosure of this patent document contains material that is subject to copyright protection. The copyright owner has no objection to the facsimile reproduction by anyone of the patent document or the patent disclosure, as it appears in the Patent and Trademark Office patent files or records, but otherwise reserves all copyright rights whatsoever. Technical Field

[0003] The present disclosure generally relates to methods, systems, and devices for remote browser isolation technology. Background Art

[0004] With the rapid development of the Internet, web browsing has become an indispensable part of daily life. However, with the convenience and accessibility of online resources, network threats (such as malware, ransomware and phishing attacks) are also increasing. These threats exploit vulnerabilities in web browsers to obtain unauthorized access to end-user systems, thereby endangering their sensitive information, financial assets and overall digital security. As the demand for web browsing continues to grow, research and development continue to advance web browsing technology to not only prevent unauthorized access to end-user systems, but also advance and enhance the user experience of web browsing. According to some methods, techniques such as canvas cleaning (sanitization) can be used. For example, the term "canvas cleaning" refers to a security mechanism that helps to remove unsafe (and potentially malicious) content from untrusted or potentially malicious web content before presenting it to users. Although existing canvas cleaning techniques have been proposed, for reasons explained below, the canvas cleaning techniques are not enough. Summary of the invention

[0005] In one aspect, a method for canvas cleanup includes: receiving a request to open a webpage from a client device; replacing a first function with a second function via an agent, the agent being loaded into a browser; loading the webpage from a network server into the browser; in response to attempting to execute the first function on the webpage, executing the second function corresponding to the first function via the browser to generate a drawing of a first time period; converting the drawing to an image via the agent; and transmitting the image to the client device.

[0006] In some embodiments, the proxy is loaded into the browser prior to loading the web page from the web server into the browser.

[0007] In some embodiments, the agent includes a proxy, and the replacing the predefined drawing function includes: redefining the first function as the second function.

[0008] According to some embodiments, the first function is configured to be executed using drawing parameters, and the second function is configured to be executed using the drawing parameters.

[0009] In some embodiments, the loading of the webpage includes: receiving the webpage from the network server; and generating a drawing object of a hypertext markup language canvas element on the browser based on the webpage. In some embodiments, the first function is configured to be executed based on the drawing object.

[0010] According to some embodiments, executing the second function includes: executing the second function on the browser based on the drawing object.

[0011] According to some embodiments, the method further comprises starting a timer for the first time period, wherein the timer is associated with the drawing object.

[0012] According to some embodiments, the drawing includes a first partial drawing. In some embodiments, the method further includes: executing, via the agent, the second function corresponding to the first function to generate a second partial drawing for a second time period; and aggregating, via the agent, the first partial drawing and the second partial drawing to generate an aggregated drawing. In some embodiments, the converting the drawing includes: converting the aggregated drawing to the image.

[0013] According to some embodiments, the method further comprises: receiving a user input for instructing to change the drawing object. In some embodiments, the changing the drawing object comprises the attempting to execute the first function.

[0014] In some embodiments, the attempting to execute the first function includes completing commands on the web page before calling the first function.

[0015] In some embodiments, the drawing includes an object drawn by the second function. In some embodiments, converting the drawing includes converting the object into the image.

[0016] In yet another aspect, a server includes: a memory storing a browser and an agent; a processor coupled to the memory; and a communication system coupled to the processor. In some embodiments, the processor coupled to the memory is configured to: receive a request to open a webpage from a client device via the communication system; replace a first function with a second function via an agent, the agent being loaded into the browser; load the webpage from a network server via the browser; in response to attempting to execute the first function on the webpage, execute the second function corresponding to the first function via the agent to generate a drawing of a first time period; convert the drawing to an image via the agent; and transmit the image to the client device via the communication system.

[0017] In some embodiments, the agent comprises a proxy, and to replace the first function, the processor is configured to: redefine the first function as the second function.

[0018] According to some embodiments, the first function comprises drawing parameters. In some embodiments, to execute the second function, the processor is configured to: execute the second function based on the drawing parameters.

[0019] According to some embodiments, in order to load the web page, the processor is configured to: receive the web page from the network server; and generate a drawing object of a hypertext markup language canvas element on the browser based on the web page, wherein the first function is configured to be executed based on the drawing object.

[0020] According to some embodiments, in order to execute the second function, the processor is configured to: execute the second function on the browser based on the drawing object.

[0021] According to some embodiments, the processor is further configured to: start a timer for the first time period, wherein the timer is associated with the drawing object.

[0022] According to some embodiments, the drawing includes a first partial drawing. In some embodiments, the processor is further configured to: execute, via the agent, the second function corresponding to the first function to generate a second partial drawing for a second time period; and aggregate, via the agent, the first partial drawing and the second partial drawing to generate an aggregated drawing. In some embodiments, to convert the drawing, the processor is configured to: convert the aggregated drawing into the image.

[0023] According to some embodiments, the processor is further configured to: receive user input for instructing to change the drawing object. In some embodiments, the changing the drawing object comprises the attempting to execute the first function.

[0024] In yet another aspect, a server includes: a memory storing a browser and an agent; a processor coupled to the memory; and a communication system coupled to the processor. In some embodiments, the processor coupled to the memory is configured to: receive a request to open a webpage from a client device via the communication system; load an agent to the browser; overwrite a first function with a second function via the agent; load the webpage from a network server via the browser; in response to attempting to execute the first function on the webpage, execute the second function corresponding to the first function via the agent to generate a drawing of a first time period; convert the drawing to an image via the agent; and transmit the image to the client device via the communication system.

[0025] Various modifications and additions may be made to the discussed embodiments without departing from the scope of the invention. For example, although the embodiments described above refer to particular features, the scope of the invention also includes embodiments having different combinations of features and embodiments that do not include all of the above features.

[0026] The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, and from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] A further understanding of the nature and advantages of particular embodiments may be achieved by reference to the remainder of the specification and drawings, wherein like reference numerals are used to refer to like components. In some instances, a sub-label is associated with a reference numeral to identify one of multiple similar components. When a reference numeral is referenced without specifying an existing sub-label, it is intended to refer to all such multiple similar components.

[0028] Figure 1 An example computing environment for canvas cleanup is described in accordance with various embodiments.

[0029] Figure 2 is a flow chart illustrating an example process for canvas cleanup according to various embodiments.

[0030] Figure 3 is a call flow diagram illustrating an example process for canvas cleanup according to various embodiments.

[0031] Figure 4is a schematic diagram illustrating example canvas cleanup according to various embodiments. DETAILED DESCRIPTION

[0032] The disclosed technology will now be discussed in detail with reference to the drawings briefly described above. In the following description, numerous specific details are set forth to illustrate the best mode for the applicant to practice the invention and to enable a person of ordinary skill in the art to make and use the invention. However, it will be apparent to a person of ordinary skill in the art that the invention may be practiced without many of these specific details. In other examples, well-known machines, structures, and method steps are not described in particular detail in order to avoid unnecessarily obscuring the invention. Unless otherwise indicated, similar components and method steps are referenced with similar reference numerals.

[0033] As described above, network threats that exploit vulnerabilities in web browsers are increasing. For example, HTML (Hypertext Markup Language) provides the use of canvas elements, which can load graphic data from multiple sources. In some cases, the data may include malicious parts, which may attempt to interfere with the correct loading of the browser and even endanger the user's data and security if the vulnerability in the browser is exploited. Recently, remote browser isolation technology attempts to solve the challenges brought by network-based threats. Remote browser isolation involves executing a web browser in a controlled and secure environment outside the user's local device or client device. The technology processes and renders web content on a remote server instead of directly rendering web pages on the user device, and only transmits safe visual information back to the user device. In this way, any potentially harmful code or malicious content remains isolated from the local environment, thereby substantially reducing the risk of network attacks and providing a safer web browsing experience. However, existing remote browser isolation technology can only be implemented by intercepting network traffic. Therefore, existing remote browser isolation technology is computationally expensive because all network traffic must be analyzed to determine which part of the web page needs to be processed in the remote browser. Additionally, existing remote browser isolation techniques must send more web browsing data from the predefined web server to the user device than necessary. Therefore, existing remote browser isolation techniques exhibit latency issues due to the overhead involved in monitoring network traffic between the predefined web server and the user device and transmitting visual content between the remote server and the user device. Additionally, there are challenges in maintaining a seamless user experience across various web applications, especially applications that rely heavily on real-time interaction and multimedia content.

[0034] In addition, existing remote browser isolation technologies sometimes modify the browser code of the local or client device to integrate custom code into the browser itself during compilation to intercept network traffic and transmit data to a remote server that includes the remote browser. However, this requires compiling each local browser again for integrating the custom code into the local browser. This ultimately results in unnecessary time using existing remote browser isolation technologies and exploiting another vulnerability attributed to the modified browser.

[0035] Some embodiments described herein provide solutions to these problems by intercepting function calls or code rather than intercepting network traffic. This can dynamically track canvas elements while exerting minimal or no impact on website performance. In addition, some embodiments described herein also provide a timer for converting an image sequence of a web page and sending it to a user or client device. This promotes real-time interaction without or with minimal delay, and the user or client device does not need to experience the overhead of receiving a large image file. In addition, the improved system and method for canvas cleaning also provide various other aspects described herein.

[0036] Figure 1 An example computing environment 100 for canvas cleanup according to various embodiments is described. In the example computing environment 100, a user may open a web page on a local browser in a client device 130. The client device 130 may then transmit a request to open the web page to the isolation server 101, and the isolation server 101 may transmit the request to the server 102 including the remote browser 110 through the communication network 120. Thus, the server 102 may receive (e.g., via the communication subsystem) the request to open the web page from the client device 130, and the proxy 112 may overwrite the first function (e.g., the canvas drawing function). In some examples, the isolation server 101 or the server 102 may load the proxy 112 to the remote browser 110. The server 102 may then load the web page from the web server 140 to the remote browser 110 through the communication network 120. The proxy 112 may then intercept the drawing function of the web page, execute the second function to generate a drawing, convert the drawing into an image, and transmit the image 134 to the client device 130 through the communication network 120.

[0037] Figure 1The embodiment shown in is an example of a computing system environment and is not intended to be limiting. The subsystems and components within the server 102, the isolation server 101, and the client device 130 may be implemented in hardware, firmware, software, or a combination thereof. In some examples, a user operating a client computing device 130 may utilize a client application to interact with the isolation server 101, and the isolation server 101 communicates with the server 102 to browse a web page from a network server 140. The client device 130 may be configured to transmit a request to open a web page to the server 102 through the communication network 120 and receive an image, which is converted from a drawing configured to be drawn based on a drawing function of the web page. Such a client application may be a web browser, an application based on a web browser, an independent software application (such as a mobile device application), or any other suitable client application configured to be communicatively connected to any network server 140 through the communication network 120. In some examples, the client device 130 may include a user device (e.g., a personal computer, a laptop computer, a tablet computer, a mobile device, or any other suitable user device). In other examples, client device 130 may include an intermediate device to forward, deliver and / or route data between server 101 and the terminal device. In further examples, client device 130 may include another server, a network server, or any other suitable device. In some examples, client device 130 may communicate only with quarantine server 101 to protect client device 130 from malicious or harmful content accessing client device 130.

[0038] In some examples, the client device 130, the isolation server 101, the server 102, the remote browser 110, the proxy 112, the web server 140, and / or any other disclosed device may be communicatively coupled via a communication network 120. The communication network 120 may be any type of network known in the art that supports data communication. As non-limiting examples, the communication network 120 may be a local area network (LAN; such as Ethernet, token ring, etc.), a wide area network (e.g., the Internet), an infrared or wireless network, a public switched telephone network (PSTN), a virtual network, a wired network, etc. The communication network 120 may use any available protocol, such as, for example, the Transmission Control Protocol / Internet Protocol (TCP / IP), Systems Network Architecture (SNA), Internet Packet Exchange (IPX), Secure Sockets Layer (SSL), Transport Layer Security (TLS), Hypertext Transfer Protocol (HTTP), Secure Hypertext Transfer Protocol (HTTPS), Institute of Electrical and Electronics Engineers (IEEE) 802.11 protocol suite or other wireless protocols, and the like.

[0039] In some examples, the isolation server 101 may include a processor coupled to a memory to perform various tasks on behalf of the client device 130. In some examples, the isolation server 101 may further include a communication subsystem to provide a communication interface from the server 101 and other computing devices via the communication network 120. Therefore, the processor can communicate with the client device 130 and the server 102 via the communication subsystem. In some examples, the isolation server 101 can verify the authorization of the client device 130 and act on behalf of the client device 130. In other examples, the isolation server 101 can store the agent 112 in the memory and load the agent 112 to the browser (e.g., remote browser) in the server 102. In some examples, the isolation server 101 may be the only entity that can interact with the agent 112 and the browser 110 of the server 102. In other examples, the isolation server 101 may interact with other computing devices or servers. In other examples, the isolation server 101 and the server 102 including the browser 110 and the agent 112 may be physically separated entities. In other examples, the isolation server 101 and the server 102 are logically separate processes that communicate with each other. In other examples, the isolation server 101 may be in the same or different cloud environment as the server 102.

[0040] In some examples, various different system components 104 may be implemented on a server 102. The server 102 may be configured to run one or more server software applications or services (e.g., web-based or cloud-based services) to support interaction with the browser 110, the isolation server 101, and / or the network server 140. For example, the system component 104 may include a processor 106, which may communicate (e.g., interface) with a number of peripheral subsystems in the system component 104 via a bus subsystem. These peripheral subsystems may include, for example, a memory 108, an input / output (I / O) subsystem, and a communication subsystem.

[0041] In some examples, processor 106 may be implemented as one or more integrated circuits (e.g., conventional microprocessors or microcontrollers). In an example, processor 106 may control the operation of server 102. Processor 106 may include a single-core and / or multi-core (e.g., quad-core, hexa-core, octa-core, deca-core, etc.) processor and a processor cache. Processor 106 may execute a variety of resident software processes embodied in program code and may maintain multiple concurrently executing programs or processes. In some examples, processor 106 may include a special-purpose processor (e.g., a digital signal processor (DSP), external, graphics-specific, and / or other processor).

[0042] In some examples, the bus subsystem provides a mechanism for the intended communication between the various components in the server 102. The bus subsystem may utilize a single bus or multiple buses. In some examples, the bus subsystem may include a memory bus, a memory controller, a peripheral bus, and / or a local bus using any of a variety of bus architectures (e.g., Industry Standard Architecture (ISA), Micro Channel Architecture (MCA), Enhanced ISA (EISA), Video Electronics Standards Association (VESA), and / or Peripheral Component Interconnect (PCI) bus, which may be implemented as a mezzanine bus manufactured in accordance with the IEEE P1386.1 standard).

[0043] In some examples, the system components 104 of the server 102 may further include a memory 108, which is a non-transitory storage medium. The memory 108 may include any suitable storage device or devices that can be used to store suitable data (e.g., the agent 112, the browser 110, etc.) and instructions that can be used by, for example, the processor 106 to perform the following in conjunction with Figure 2 At least a portion of the process 200 described herein. The memory 108 may include any suitable volatile memory, non-volatile memory, storage device, or any suitable combination thereof. For example, the memory 120 may include a random access memory (RAM), a read-only memory (ROM), an electrically erasable programmable read-only memory (EEPROM), one or more flash drives, one or more hard disks, one or more solid-state drives, one or more optical drives, etc.

[0044] In some examples, the system components 104 of the server 102 may further include a communication subsystem to provide a communication interface from the server 102 and external computing devices via a communication network 120 (including a local area network (LAN), a wide area network (WAN) (e.g., the Internet), and various wireless telecommunication networks). In some examples, the communication subsystem may include, for example, one or more network interface controllers (NICs) (e.g., Ethernet cards, asynchronous transfer mode NICs, token ring NICs, and the like), and one or more wireless communication interfaces (e.g., wireless network interface controllers (WNICs), wireless network adapters, and the like). Additionally and / or alternatively, the communication subsystem may include one or more modems (telephone, satellite, cable, ISDN), synchronous or asynchronous digital subscriber line (DSL) units, FireWire, or FireWire. interface, The communication subsystem may also include a radio frequency (RF) transceiver component for accessing wireless voice and / or data networks (e.g., using cellular telephone technology, advanced data network technology (e.g., 3G, 4G or EDGE (Enhanced Data Rates for Global Evolution), Wi-Fi (IEEE 802.11 family of standards or other mobile communication technologies), or any combination thereof), a global positioning system (GPS) receiver component, and / or other components.

[0045] In some examples, the communication subsystem may also receive input communications in the form of structured and / or unstructured data feeds, event streams, event updates, and the like on behalf of users who may access web pages or websites of the network server 140 via the server 102. In an example, the communication subsystem may be configured to receive input (e.g., website addresses, mouse click input, keyboard input, etc.) from users and / or other communication services in real time. In addition, the communication subsystem 232 may be configured to receive websites (e.g., hypertext markup language (HTML) files, JavaScript files, image files, video files, etc.) from the network server 140, or any other suitable data in the form of continuous data streams and / or non-continuous data. In some examples, the communication subsystem may output modified website data from the network server 140 to the client device 130. The various physical components of the communication subsystem may be connected via a computer network (e.g., the communication network 120), The bus or the like couples to removable components of server 102, and / or may be physically integrated onto a motherboard of server 102. In some examples, the communication subsystem may be implemented in whole or in part by software.

[0046] In some examples, the server 102 may further include a browser 110 (e.g., a remote browser). In some examples, the server 102 may assign threads or processing resources to the browser 110 so that the browser 110 can perform browsing tasks (e.g., loading web pages from a network server, rendering content, etc.). In some examples, the server 102 may assign a single thread to a tab of the browser 110 so that the tab performs browsing tasks until the tab is closed. The browser 110 may be a remote browser and / or an isolated browser, which is an application for accessing a website, wherein the browser may run in a sandbox or a restricted and isolated environment. In some examples, the server 102 may be the only entity that can interact with the browser and the agent. In some examples, the browser 110 downloads HTML files and other resources (e.g., JavaScript files) from the web server of the web page, and then processes the web page on the browser to transmit the rendered image to the local device 130 to display the content of the web page. Therefore, the local device 130 does not need to directly access the web page or website.

[0047] In some examples, the agent 112 may be a program, program code, agent (e.g., a JavaScript agent) stored in the memory 108 in the server 102 or the memory in the isolation server 101. For example, the agent 112 may be written in JavaScript or any other suitable programming language. The processor 106 of the server 102 or the processor of the isolation server 101 may load the agent 112 to the browser 110. In some examples, the processor 106 of the server 102 or the processor of the isolation server 101 may load the agent 112 from the memory to the browser 110 before loading the web page from the network server 140. In some examples, the agent 112 is configured to replace or overwrite the first function (e.g., canvas drawing function) and / or built-in function (e.g., JavaScript function) with a second function (e.g., a proxy drawing function defined by the agent), intercept the canvas element on the web page, and / or notify the server when the client device needs to redraw that canvas element. Moreover, the agent 112 can independently manage the timer and the conversion of the canvas. In some examples, the proxy 112 along with the browser 110 may be logically or physically separated from the isolation server 101. Figure 2 At least a portion of process 200 may be processed by browser 110 and / or agent 112 using processor 106 of server 102. In some examples, although isolation server 101 may load agent 112 into browser 110, isolation server 101 may not need to maintain communication until the agent sends a drawing to isolation server 101 to be transmitted to client device 130.

[0048] In some examples, the browser 110 may download website resources (e.g., HTML, JavaScript files) and / or access other resources (e.g., video streams or images) from the network server 140. The network server 140 may be computer software and underlying hardware that accepts requests via a suitable communication protocol (e.g., HTTP) to allow the browser 110 to download resources (e.g., HTML files, JavaScript files, images, CSS files, etc.) and / or interact with them. In some examples, the network server 140 responds to requests from the browser 110, thereby providing the browser with website resources and other resources through the Internet. However, the network server 140 is not necessarily a reliable source. The network server 140 may contain potentially harmful code or malicious content, which may endanger the user's data and security. Therefore, the browser may contain potentially harmful code or malicious content from the network server 140 in the isolated environment.

[0049] Due to the ever-changing nature of computers and networks, the description of the computing system 100 depicted in the figure is intended only as a specific example. Many other configurations with more or fewer components than the system depicted in the figure are possible. For example, custom hardware may also be used, and / or specific elements may be implemented in hardware, firmware, software, or a combination thereof. In addition, connections to other computing devices (e.g., network input / output devices) may be employed. Based on the disclosure and teachings provided herein, a person of ordinary skill in the art will understand other ways and / or methods to implement the various embodiments.

[0050] Figure 2 2 is a flow chart illustrating an example process 200 for processing network content (e.g., canvas cleanup) according to various embodiments. As described below, certain implementations may omit some or all of the illustrated features, and all embodiments may be implemented without some of the illustrated features. In some examples, a server 102 having a processor 106 and a memory 108 may be configured to perform the process 200. In other examples, a combination of servers 102 and / or isolated servers 101 (collectively, servers 102) may be configured to perform the process 200. In some examples, any suitable device or means for performing the functions or algorithms described below may perform the process 200.

[0051] At block 210, server 102 may receive a request from client device 130 to open a web page or website (eg, via isolation server 101). Figure 3 In step 302, the client device 130 may transmit a request to open a web page to the isolation server 101, and the isolation server 101 receives the request from the client device 130 and transmits the request to the server 102. For example, the request from the client device 130 to open the web page may include user input on the local browser of the client device 130 (e.g., website address input, mouse click or any input redirected to the web page, voice command to open the web page, or any other suitable user input). In some scenarios, the user may enter the website address on the local browser of the client device 130. Then, the local device 130 may transmit the website address to the server 102 (e.g., through the isolation server 101). In some examples, the web page may include a document, an HTML file, a JavaScript file, a CS file, an image, a video stream, or any suitable document on the World Wide Web, and may be identified by a unique uniform resource locator (URL).

[0052] In some examples, the local device 130, the isolation server 101 or the server 102 may determine whether the web page to be opened is safe. In some examples, the local device 130 may determine whether the web page is safe. For example, the local device 130 may store a list of safe or verified website addresses or domains in the memory of the local device 130. Then, when the user enters the website address on the local browser, the local device 130 may search for the website address in the list of safe or verified website addresses (e.g., using a lookup table, a hash algorithm, etc.). If the website address or domain is a safe or verified web page, the client device 130 may directly connect to the network server 140 of the safe web page and download the web page (e.g., HTML file, JavaScript file, CS file, image, video stream, etc.) or the website resource to be rendered on the local browser. In other examples, the client device 130 may transmit the user input to the server 102 (e.g., through the isolation server 101), and the server 102 or the isolation server 101 may determine whether the web page is safe or verified. If the website address or domain is secure or verified, server 102 may connect to web server 140 and provide web pages and / or web page resources from web server 140 to client device 130 (e.g., via isolation server 101) without using canvas cleanup process 200. In another example, server 102 may determine whether to perform process 200 by searching for any HTML canvas elements on the web page from web server 140. Server 102 may perform process 200 when an HTML canvas element is present on the web page.

[0053] At block 220, server 102 may replace the first function with the second function via proxy 112. In some examples, to replace the first function, server 102 or isolation server 101 may load proxy 112 from a memory of server 102 or isolation server 101 to browser 110. This is also Figure 3306 of . In some examples, the first function may include a predefined drawing function, an HTML canvas drawing function (e.g., a line, a path, a rectangle, a circle, an arc, a Bezier curve, etc.), a web standard function, a built-in drawing function (e.g., a built-in JavaScript drawing function), or any other suitable drawing function. Therefore, the user can use the first function without newly defining how to draw to the browser. However, by loading the agent 112, the server 102 may redefine the first function as a second function via the agent 112, and the second function may be a proxy drawing function. In some examples, the second function may be a function defined by the agent 112 to perform a task corresponding to the first function. For example, the second function may be a proxy drawing function to perform a predefined drawing function, an HTML canvas drawing function (e.g., a line, a path, a rectangle, a circle, an arc, a Bezier curve, etc.), a web standard function, a built-in drawing function (e.g., a built-in JavaScript drawing function), or any other suitable drawing function corresponding to the first function. Since the first function on the HTML canvas can be used for unauthorized access to the end user's system, the server may replace the first function with the second function via the agent 112. In another example, a function (e.g., a first function or a second function) may include a procedure or a set of statements that performs a task (e.g., calculates a value, generates a letter, draws a drawing, etc.) and outputs (e.g., a task result, a calculated value, a letter, a drawing, etc.). In another example, replacing the first function may include replacing or overwriting the first function with the second function. For example, the first function is defined in a parent class, and the second function is newly defined (e.g., in a child class) with the same function name, parameters, and return type. Then, when the first function, which is a predefined standard function, is called, the second function may be executed instead of the first function because the second function overrides the first function.

[0054] In some instances, the first function may include a predefined drawing function, an HTML canvas drawing function for drawing a drawing (e.g., a line, a path, a rectangle, a circle, an arc, a Bezier curve, etc.), a built-in drawing function (e.g., a built-in JavaScript drawing function), or any other suitable drawing function. In other instances, the agent 112 may include an agent (e.g., a JavaScript agent), code, a set of program statements for intercepting the first function and / or redefining the first function as a proxy drawing function, which is a second drawing function. Moreover, the agent may instruct a programming object, a wrapper, or a program function to intercept the first function and replace the first function with a second function. The agent may execute code in the same context as the function application before and after calling the function. In other instances, the agent 112 may have a prototype function and any other suitable function for a given object. In some instances, in conjunction with Figure 2The second function is described in block 240 of FIG. 1. In some examples, to load the proxy 112, the server 102 may insert the proxy 112 containing the programming code into a header to be executed before loading and executing the webpage. In other examples, to load the proxy 112, the server 102 may run, execute, or implement the proxy 112 to intercept and / or overwrite the first function to be used on the webpage. In some examples, before loading the proxy 112 into the browser 110, the server 102 may create a tab on the browser 110, such as Figure 3 . In such an example, to load the agent to the browser 110, the server 102 may load the agent 112 to a tab of the browser 110. Since the server 102 loads the agent 112 to the browser 110, the browser 110 does not need to be modified. The loading of the agent 112 may simplify the canvas cleanup process and improve versatility because any remote browser may be used for the canvas cleanup process without modification. Moreover, the loading of the agent 112 may improve the security of the browser because the agent 112 is only used when performing the canvas cleanup process and / or until processing resources are assigned to the browser and no standard browser needs to be modified.

[0055] Reference again Figure 2 At block 230, server 102 may load a web page from web server 140 via browser 110. Figure 3308 and 310 of . For example, the server 102 may provide a web page to the browser 110 at step 308, and the browser 110 loads the web page from the network server 140 at step 310. In some examples, the web page may include a document, an HTML file, a JavaScript file, a CSS file, etc. to render the content from the network server 140 on the browser. In another example, the web page may include an HTML canvas element. For example, in order to load a web page, the server 102 may receive the web page from the network server 140 and generate a drawing object of the HTML canvas element on the browser based on the web page. For example, the drawing object of the HTML canvas element may be defined from a rendering context class (e.g., HTMLCanvasElement.getContext(contextType)). In some examples, HTMLCanvasElement.getContext(contextType), which is a rendering context class, may return a drawing context or a drawing object on the canvas, which may be defined in an HTML file. In some examples, contextType can be '2d' to generate a drawing object representing a two-dimensional rendering context, 'webgl', 'webgl2', or 'webgpu' to generate a drawing object representing a three-dimensional rendering context, or any other suitable context type. In other examples, the drawing object of the HTML canvas element can include drawing functions (e.g., line, path, rectangle, circle, arc, Bezier curve, etc.). However, in some examples, the drawing functions can be used to gain unauthorized access to the end-user system.

[0056] Reference again Figure 2 At block 240, server 102 may execute, via agent 112, a second function corresponding to the first function to generate a plot of the first time period in response to attempting to execute the first function on the web page. In some examples, the first time period may include a predetermined time period (e.g., 0.1, 0.5, 1, 2, 5, 10 seconds, or any other suitable time period). This is also Figure 3 102. In some examples, the drawing may include a line, a path, a rectangle, a circle, a Bezier curve, or any suitable object that can be rendered on the browser 110. In some examples, the first function may be executed based on the same drawing object. In some examples, the first function may include drawing parameters. For example, the drawing parameters may be provided from the network server 140. In some scenarios, in order to execute the second function, the server 102 may execute the second function based on the drawing parameters. In some examples, attempting to execute the first function may include completing a command on the web page before calling the first function or preparing to execute the first function. In some examples, the calling of the first function may include executing the first function of the drawing object.

[0057] In some examples, the server 102 may receive user input indicating a change to a drawing object. In some examples, attempting to execute a first function may include changing a drawing object. For example, when a mouse pointer is over an HTML canvas, a web page includes drawing function code on the canvas. Then, in response to the position of the mouse pointer (i.e., the user input), the first function is configured to be executed. However, the server 102 may intercept the first function and execute the second function via a browser. In some examples, each indication triggered by the user input may include a unique identifier for a drawing function on the canvas. Therefore, the server 102 may respond differently to each indication. Indications of changes made to canvas elements may appear multiple times while the web page is running.

[0058] In some examples, the proxy 112 may include a utility function that is capable of hooking related functions within all related rendering contexts. For example, the utility function may work by replacing a function belonging to a specific prototype. By replacing a function, any object that utilizes that prototype may employ the proxy 112. A first function may be replaced with a dynamically created definition that calls a function of a proxy 112 (e.g., a proxy) with the same context as the first function (a reference to the first function to be used in the exception), and the original set of arguments used in the first function is passed to the proxy 112 to work the same as the first function. In other examples, the proxy 112 may replace a drawing object or context with a proxy object (e.g., HTMLCanvasElement.getContext(contextType)). In such examples, the browser 110 may use a predefined drawing method to generate a drawing based on the replaced drawing object.

[0059] In response to the attempt, server 102 may execute, via the agent, a second function corresponding to the first function to generate a plot for a time period. In some examples, the time period may include a predetermined time period (e.g., 0.1, 0.5, 1, 2, 5, 10 seconds, or any other suitable time period). This is also Figure 3. In some examples, the second function may be configured to draw a line, path, rectangle, circle, arc, Bezier curve, or any other suitable shape from a drawing object or rendering context class (e.g., HTMLCanvasElement.getContext(contextType)). In some examples, the second function may include beginPath(), arc(), moveTo(), lineTo(), stroke(), or any other suitable drawing function. In some examples, the second function may draw the same drawing that the corresponding first function should draw. In some examples, to execute the second function, the server 102 may execute the second function on the browser based on the drawing object. Therefore, the second function is executed according to the same drawing object on which the first function is based. For example, when the first function is configured to draw a circle based on a drawing object on the canvas, the second function may only draw a circle based on the same drawing object on the canvas. In some examples, the drawing may include objects drawn by the second function. Therefore, the web page can run normally without knowing that the proxy function is called instead of the first function. This is because the proxy 112 intercepts functions rather than intercepting network traffic as used in existing techniques.

[0060] In some examples, the server 102 may start a timer for the time period. In some examples, the timer may be associated with the drawing object. Figure 3 After starting the timer, the server 102 may operate on the canvas elements within the server environment. For example, the server 102 may call a proxy function (e.g., beginPath(), arc(), moveTo(), lineTo(), stroke(), etc. In another example, the time period of the timer may be static and may be configured differently for various web pages. Therefore, while the timer is active, all drawing calls associated with the same drawing object or context may continue to operate normally without resetting the timer.

[0061] In another example, the plot may be a first partial plot. The server may then, via the proxy, execute a second function corresponding to the first function to generate a second partial plot for a second time period. In some examples, the second time period may include a predetermined time period (e.g., 0.1, 0.5, 1, 2, 5, 10 seconds, or any other suitable time period). The server 102 may then, via the proxy, aggregate the first partial plot and the second partial plot to generate an aggregated plot.

[0062] Reference again Figure 2At block 250, the server 102 may convert the drawing, via the proxy, into an image. In some examples, the drawing may be a line, path, rectangle, circle, arc, Bezier curve, or any other suitable object on a canvas (e.g., drawn by an HTML canvas drawing function, a built-in drawing function, or any other suitable drawing function). In other examples, the image may be a binary representation of the canvas containing the drawing. For example, the image may be stored in a variety of formats (e.g., TIFF, JPEG, PNG, GIF, or any other suitable format). This is also Figure 3 . In the example of block 240 where the first and second partial drawings are aggregated into an aggregated drawing, the server 102 may convert the aggregated drawing into an image via the proxy. In the example where the drawing includes an object drawn by the second function, the server 102 may convert the object into an image via the proxy rather than the entire rectangular area containing the object. In some examples, the server 102 may convert the drawing into an image via the proxy 112 (e.g., using the "toDataURL" method of HTMLCanvasElement). By converting the actual canvas element rather than the area it renders on the screen, the effect of background images or videos below or above the canvas layer may be eliminated. In another example, by selecting an image format that supports transparency (e.g., PNG), the canvas may not obscure any elements that are intended to be behind it in the browser. Since the copy does not contain any other data except the information of the pixels themselves, it does not contain any malicious data that may be stored in such contexts due to vulnerabilities in the browser's implementation of rendering contexts. Furthermore, the image is the exact size of the canvas because it is a copy of the canvas pixels rather than generic drawing instructions, thereby providing better final image quality to the client device 130 .

[0063] In some examples, when a drawing may include multiple partial drawings based on a timer, server 102 may convert each partial drawing into an image via agent 112. In other examples, server 102 may aggregate multiple partial drawings via agent 112 to generate an aggregated drawing and convert the aggregated drawing into an image. In further examples, server 102 may convert each partial drawing into an image via agent 112. Then, server 102 may aggregate images corresponding to the partial drawings via agent 112 to generate an image.

[0064] Reference again Figure 2 At block 260, server 102 may transmit the image to client device 130 (e.g., via isolation server 101) (e.g., via proxy 112 and / or communication subsystem). Figure 3In some examples, the agent 112 may transmit the image in the sandbox to the isolation server 101, and the isolation server 101 transmits the image to the client device 130. The client device may then draw the resulting image on the drawing object, such as Figure 3 As shown in step 326 in .

[0065] This process can be repeated when the web page attempts to draw other content on the drawing object, such as Figure 3 As shown in step 328 in . Figure 4 2 is an example schematic diagram illustrating an example canvas cleanup. The proxy 112 loaded into the browser 110 may intercept the drawing function of the web page and replace the drawing function with the proxy function to generate a drawing. Then, the proxy 112 may generate a pixel-to-pixel copy of the image of the drawing and send the cleaned image to the client device 130. The client device 130 may create a bitmap version of the cleaned image and draw the image onto the canvas of the local browser. Therefore, the process 200 does not need to modify the browser 110 by loading the proxy 112 into the browser 110, and reduces the use of computing resources by intercepting drawing functions rather than network traffic.

[0066] Although specific features and aspects have been described with respect to some embodiments, those skilled in the art will recognize that numerous modifications are possible. For example, the methods and processes described herein may be implemented using hardware components, software components, and / or any combination thereof. In addition, although the various methods and processes described herein may be described with respect to specific structural and / or functional components for ease of description, the methods provided by the various embodiments are not limited to any specific structural and / or functional architecture, but may be implemented on any suitable hardware, firmware, and / or software configuration. Similarly, although specific functionality is attributed to a specific system component, unless the context otherwise dictates, this functionality need not be limited thereto and may be distributed among various other system components according to several embodiments.

[0067] Furthermore, although the procedures of the methods and processes described herein are described in a particular order for ease of description, various procedures may be reordered, added, and / or omitted according to various embodiments unless the context dictates otherwise. Furthermore, procedures described with respect to one method or process may be incorporated into other described methods or processes; similarly, system components described according to a particular structural architecture and / or with respect to one system may be organized in an alternative structural architecture and / or incorporated into other described systems. Thus, although various embodiments are described with or without particular features for ease of description and illustration of some aspects of those embodiments, various components and / or features described herein with respect to a particular embodiment may be replaced, added, and / or deleted from other described embodiments unless the context dictates otherwise. Thus, although several embodiments are described above, it will be understood that the present invention is intended to cover all modifications and equivalents within the scope of the appended claims.

Claims

1. A method for processing network content, comprising: Receiving a request from a client device to open a web page; replacing the first function with the second function via a proxy, the proxy being loaded into the browser; loading the web page from a web server into the browser; In response to attempting to execute the first function on the webpage, executing, via the browser, the second function corresponding to the first function to generate a plot of a first time period; converting the drawing into an image via the agent; and The image is transmitted to the client device.

2. The method of claim 1, wherein the proxy is loaded into the browser prior to loading the web page from the web server into the browser.

3. The method of claim 1, wherein the agent comprises an agent, and Wherein replacing the first function comprises: Redefine the first function as the second function. 4 . The method of claim 3 , wherein the first function is configured to be executed using drawing parameters, and wherein the second function is configured to be executed using the drawing parameters.

5. The method according to claim 1, wherein the loading the webpage comprises: receiving the web page from the web server; and A drawing object of a hypertext markup language canvas element is generated on the browser based on the webpage, wherein the first function is configured to be executed based on the drawing object.

6. The method of claim 5, wherein the performing of the second function comprises: The second function is executed on the browser based on the drawing object.

7. The method according to claim 5, further comprising: A timer is started for the first time period, wherein the timer is associated with the drawing object.

8. The method of claim 7, wherein the drawing comprises a first portion of the drawing, Wherein the method further comprises: executing, via the agent, the second function corresponding to the first function to generate a second portion of the plot for a second time period; and aggregating the first portion of the drawing and the second portion of the drawing via the proxy to produce an aggregated drawing, and Wherein said converting the drawing comprises converting the aggregated drawing into the image.

9. The method according to claim 5, further comprising: receiving a user input for instructing to change the drawing object, Wherein said changing said drawing object comprises said attempting to execute said first function.

10. The method of claim 1, wherein the attempting to execute the first function comprises: The commands on the web page are completed before calling the first function.

11. The method of claim 1 , wherein the plot includes objects plotted by the second function, and wherein converting the plot includes: Convert the object to the image.

12. A server, comprising: Storage, which stores browsers and proxies; a processor coupled to the memory; and a communication system coupled to the processor, wherein the processor coupled to the memory is configured to: receiving, via the communication system, from a client device a request to open a web page; replacing a first function with a second function via a proxy, the proxy being loaded into the browser; loading the web page from a web server via the browser; In response to attempting to execute the first function on the webpage, executing, via the agent, the second function corresponding to the first function to generate a plot of a first time period; converting the drawing into an image via the agent; and The image is transmitted to the client device via the communication system.

13. The server according to claim 12, wherein the agent comprises an agent, and In order to replace the first function, the processor is configured to: redefine the first function as the second function.

14. The server of claim 13, wherein the first function comprises drawing parameters, and Wherein, in order to perform the second function, the processor is configured to: The second function is executed based on the drawing parameters.

15. The server of claim 12, wherein to load the web page, the processor is configured to: receiving the web page from the web server; and A drawing object of a hypertext markup language canvas element is generated on the browser based on the webpage, wherein the first function is configured to be executed based on the drawing object.

16. The server of claim 15, wherein to perform the second function, the processor is configured to: The second function is executed on the browser based on the drawing object.

17. The server of claim 15, wherein the processor is further configured to: A timer is started for the first time period, wherein the timer is associated with the drawing object.

18. The server of claim 17, wherein the drawing comprises a first portion of a drawing, wherein the processor is further configured to: executing, via the agent, the second function corresponding to the first function to generate a second partial plot for a second time period; and aggregating the first portion of the drawing and the second portion of the drawing via the proxy to produce an aggregated drawing, and Wherein to convert the drawing, the processor is configured to: convert the aggregated drawing into the image.

19. The server of claim 15, wherein the processor is further configured to: receiving a user input for instructing to change the drawing object, Wherein said changing said drawing object comprises said attempting to execute said first function.

20. A server, comprising: Storage, which stores browsers and proxies; a processor coupled to the memory; and a communication system coupled to the processor, wherein the processor coupled to the memory is configured to: receiving, via the communication system, from a client device a request to open a web page; loading an agent into the browser; replacing the first function with the second function via the proxy; loading the web page from a web server via the browser; In response to attempting to execute the first function on the webpage, executing, via the agent, the second function corresponding to the first function to generate a plot of a first time period; converting the drawing into an image via the agent; and The image is transmitted to the client device via the communication system.

Images