System and procedure for canvas sanitization

Abstract

Novel systems and methods for canvas sanitization are provided. In various embodiments, a system and a method comprise: receiving a request to open a web page from a client device; replacing a first function with a second function via an agent, the agent being loaded into a browser; loading the web page from a web server into the browser; in response to an attempt to execute the first function on the web page, executing the second function, which corresponds to the first function, via the browser to generate a drawing for a predetermined period of time; converting the drawing into an image via the agent; and transmitting the image to the client device. Other aspects, embodiments, and features are also claimed and described.

Description

[0001] The present disclosure generally relates to methods, systems, and apparatus for remote browser isolation technology.

[0002] With the rapid growth of the internet, browsing the web has become an essential part of daily life. However, with the convenience and accessibility of online resources, the number of cyber threats such as malware, ransomware, and phishing attacks has also increased. These threats exploit vulnerabilities in web browsers to gain unauthorized access to end-users' systems, compromising their confidential information, financial assets, and overall digital security. As the demand for web browsing continues to rise, research and development continue to advance web browsing technologies, not only to prevent unauthorized access to end-user systems but also to enhance the user experience while browsing. Some approaches can utilize techniques such as canvas sanitization.For example, the term "canvas sanitization" refers to a security mechanism that helps remove or disinfect unsafe (and potentially malicious) content from untrusted or potentially malicious web content before it is presented to the user. While canvas sanitization techniques have been proposed, they are insufficient for the reasons explained below.

[0003] According to one aspect, there is provided a method for processing web content, comprising: Receiving a request to open a web page from a client device; Replacing a first function with a second function via an agent, wherein the agent is loaded into a browser; loading the web page from a web server into the browser; in response to an attempt to perform the first function on the web page, executing, via the browser, the second function corresponding to the first function to generate a drawing for a first period of time; Converting the drawing into an image about the agent; and Sending the image to the client device.

[0004] Advantageously, the agent is loaded into the browser before the web page is loaded from the web server into the browser.

[0005] Advantageously, the agent comprises a proxy, and wherein replacing the first function comprises: Redefining the first function into the second function.

[0006] Advantageously, the first function is configured to be executed with a drawing parameter, and wherein the second function is configured to be executed using the drawing parameter.

[0007] Advantageously, loading the website includes: Receiving the web page from the web server; and Creating a drawing object of a hypertext markup language canvas element on the browser based on the web page, wherein the first function is configured to be executed based on the drawing object.

[0008] Advantageously, the execution of the second function includes: Performing the second function in the browser based on the drawing object.

[0009] Advantageously, the method further comprises: Trigger a timer for the first time period, where the timer is linked to the drawing object.

[0010] Advantageously, the drawing comprises a first partial drawing, the method further comprising: Executing, via the agent, the second function corresponding to the first function to generate a second partial drawing for a second time period; and aggregating, by the agent, the first partial drawing and the second partial drawing to produce an aggregated drawing, and wherein transforming the drawing comprises: . Converting the summarized drawing into the image.

[0011] Advantageously, the method further comprises: Receiving user input to indicate a change to the drawing object, where the modification of the drawing object includes an attempt to perform the first function.

[0012] Attempting to execute the first function advantageously includes: executing a command on the web page before calling the first function.

[0013] Advantageously, the drawing comprises an object drawn by the second function, and where the transformation of the drawing includes: transformation of the object into the image.

[0014] In one aspect, a server comprises: a store that stores a browser and an agent; a processor coupled to the memory and a communication system coupled to the processor, wherein the processor coupled to the memory is configured to: to receive a request to open a web page from a client device via the communication system; to replace a first function with a second function via an agent, whereby the agent is loaded into the browser; to load the website from a web server via the browser; in response to an attempt to perform the first function on the website, performing, via the agent, the second function corresponding to the first function to generate a drawing for a first period of time; convert the drawing into an image via the agent; and to transmit the image to the client device via the communication system.

[0015] The advantage is that the agent includes a proxy, and wherein the processor is configured to replace the first function to: redefine the first function into the second function.

[0016] Advantageously, the first function comprises a character parameter and wherein the processor is configured to perform the second function to: to perform the second function based on the drawing parameter.

[0017] To load the web page, the processor is advantageously configured to: receive the web page from the web server; and to generate a drawing object of a hypertext markup language canvas element on the browser based on the web page, wherein the first function is configured to be executed based on the drawing object.

[0018] To perform the second function, the processor is advantageously configured to: to perform the second function of the browser based on the drawing object.

[0019] Advantageously, the processor is also configured to: to trigger a timer for the first time period, where the timer is connected to the drawing object.

[0020] It is advantageous that the drawing includes a first partial drawing, wherein the processor is further configured to: execute the second function corresponding to the first function via the agent to generate a second partial drawing for a second time period; and to aggregate the first partial drawing and the second partial drawing via the agent to create an aggregated drawing, and wherein the processor for converting the drawing is configured to: convert the aggregated drawing into the image.

[0021] Advantageously, the processor is also configured to: to receive user input to indicate a change to the drawing object, where the modification of the drawing object includes an attempt to perform the first function.

[0022] In one aspect, a server comprises: a store that stores a browser and an agent; a processor coupled to the memory and a communication system coupled to the processor, wherein the processor coupled to the memory is configured to: to receive a request to open a web page from a client device via the communication system; load an agent into the browser; to replace a first function with a second function via the agent; to load the website from a web server via the browser; in response to an attempt to perform the first function on the website, performing, via the agent, the second function corresponding to the first function to generate a drawing for a first period of time; to convert the drawing into an image via the agent; and to transmit the image to the client device via the communication system.

[0023] In one aspect, a method for canvas sanitization comprises: receiving a request to open a web page from a client device; replacing a first function with a second function via an agent, the agent loaded in a browser; loading the web page from a web server into the browser; in response to an attempt to perform the first function on the web page, executing the second function corresponding to the first function via the browser to generate a drawing for a first period of time; converting the drawing to an image via the agent; and transmitting the image to the client device.

[0024] In some embodiments, the agent is loaded into the browser before the web page is loaded from the web server into the browser.

[0025] In some embodiments, the agent comprises a proxy, and replacing the predefined drawing function comprises: redefining the first function into the second function.

[0026] According to some embodiments, the first function is configured to be executed using a drawing parameter, and the second function is configured to be executed using the drawing parameter.

[0027] In some embodiments, loading the web page comprises receiving the web page from the web server; and generating a drawing object of a hypertext markup language canvas element on the browser based on the web page. In some embodiments, the first function is configured to be executed based on the drawing object.

[0028] According to some embodiments, performing the second function comprises: performing the second function on the browser based on the drawing object.

[0029] According to some embodiments, the method further comprises: initiating a timer for the first time period, the timer being associated with the drawing object.

[0030] According to some embodiments, the drawing comprises a first partial drawing. In some embodiments, the method further comprises: performing, via the agent, the second function corresponding to the first function to generate a second partial drawing for a second period of time; and aggregating, via the agent, the first partial drawing and the second partial drawing to generate an aggregated drawing. In some embodiments, converting the drawing comprises converting the aggregated drawing into the image.

[0031] According to some embodiments, the method further comprises receiving user input to indicate a change to the drawing object. In some embodiments, changing the drawing object comprises attempting to perform the first function.

[0032] In some embodiments, attempting to perform the first function comprises: executing a command on the web page prior to calling the first function.

[0033] In some embodiments, the drawing includes an object drawn by the second function. In some embodiments, converting the drawing includes converting the object into the image.

[0034] In another aspect, a server comprises: a memory storing a browser and an agent; a processor coupled to the memory; and a communications system coupled to the processor. In some embodiments, the processor coupled to the memory is configured to: receive, via the communications system, a request to open a web page from a client device; replace, via an agent, a first function with a second function, the agent being loaded into the browser; load, via the browser, the web page from a web server; in response to an attempt to perform the first function on the web page, execute, via the agent, the second function corresponding to the first function to generate a drawing for a first period of time; convert, via the agent, the drawing into an image; and transmit, via the communications system, the image to the client device.

[0035] In some embodiments, the agent comprises a proxy, and to replace the first function, the processor is configured to: redefine the first function to the second function.

[0036] According to some embodiments, the first function includes a drawing parameter. In some embodiments, the processor is configured to perform the second function: perform the second function based on the drawing parameter.

[0037] According to some embodiments, the processor is configured to load the web page to: receive the web page from the web server; and generate a drawing object of a hypertext markup language canvas element on the browser based on the web page, wherein the first function is configured to be performed based on the drawing object.

[0038] According to some embodiments, the processor is configured to perform the second function to: perform the second function on the browser based on the drawing object.

[0039] According to some embodiments, the processor is further configured to: initiate a timer for the first time period, the timer associated with the drawing object.

[0040] According to some embodiments, the drawing includes a first partial drawing. In some embodiments, the processor is further configured to: via the agent, perform the second function corresponding to the first function to generate a second partial drawing for a second period of time; and via the agent, aggregate the first partial drawing and the second partial drawing to generate an aggregated drawing. In some embodiments, the processor is configured to transform the drawing to: convert the aggregated drawing into the image.

[0041] According to some embodiments, the processor is further configured to: receive user input to indicate a change to the drawing object. In some embodiments, changing the drawing object comprises attempting to perform the first function.

[0042] In another aspect, a server comprises: a memory storing a browser and an agent; a processor coupled to the memory; and a communications system coupled to the processor. In some embodiments, the processor coupled to the memory is configured to: receive, via the communications system, a request to open a page from a client device; load an agent into the browser; override, via the agent, a first function with a second function; load, via the browser, the web page from a web server; in response to an attempt to perform the first function on the web page, execute, via the agent, the second function corresponding to the first function to generate a drawing for a first period of time; convert, via the agent, the drawing into an image; and transmit, via the communications system, the image to the client device.

[0043] Various modifications and additions may be made to the described embodiments without departing from the scope of the invention. For example, while the embodiments described above refer to specific features, the scope of this invention also includes embodiments with different combinations of features and embodiments that do not include all of the features described above.

[0044] The details of one or more aspects of the disclosure are set forth in the accompanying drawings and the following description. Other features, objects, and advantages of the techniques described in this disclosure will be apparent from the description and drawings, as well as from the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] For a better understanding of the nature and advantages of specific embodiments, reference is made to the remaining parts of the specification and the drawings, in which similar components are designated by like reference numerals. In some cases, a reference numeral is associated with a sub-label to designate one of several similar components. When a reference numeral is referred to without specifying an existing sub-label, it refers to the entirety of such several similar components. Fig. 1 illustrates an example computing environment for canvas sanitization, in accordance with various embodiments. Fig. 2 is a flowchart illustrating an example method for canvas sanitization in accordance with various embodiments. Fig. 3 is a call flow diagram illustrating an example process for canvas sanitization according to various embodiments. Fig. 4 is a schematic diagram illustrating an example of canvas sanitization in accordance with various embodiments. DESCRIPTION IN DETAIL

[0046] The disclosed technology will now be discussed in detail with reference to the accompanying drawings briefly described above. In the following description, numerous specific details are set forth which illustrate the best mode contemplated by the applicant for carrying out the invention and which will enable one skilled in the art to make and use the invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without many of these specific details. In other instances, well-known machines, structures, and process steps have not been described in particular detail in order not to unnecessarily obscure the present invention. Unless otherwise indicated, like parts and process steps are designated by like reference numerals.

[0047] As described above, cyber threats exploiting vulnerabilities in web browsers are increasing. For example, HTML (Hypertext Markup Language) offers the use of canvas elements that can load graphic data from various sources. In some cases, the data may contain malicious components that may attempt to interfere with the browser's proper loading and even compromise user data and security if a browser vulnerability has been exploited. Recently, remote browser isolation technologies have attempted to address the challenges posed by web-based threats. Remote browser isolation involves running web browsers in a controlled and secure environment outside of the user's local or client device.Instead of rendering web pages directly on the user device, the technology processes and renders web content on remote servers, and only secure visual information is transmitted back to the user device. This way, any potentially harmful code or malicious content remains isolated from the local environment, significantly reducing the risk of cyberattacks and making web browsing more secure. However, existing remote browser isolation technologies can only be implemented by intercepting web traffic. Therefore, existing remote browser isolation technologies are computationally intensive, as they require analyzing all web traffic to determine which part of the web page needs to be processed in the remote browser. In addition, existing remote browser isolation technologies require sending more than the required data from the predefined web server to the user device.Therefore, existing remote browser isolation technologies suffer from latency issues due to the overhead associated with monitoring web traffic between the predefined web server and the user device and delivering visual content between the remote server and the user device. Furthermore, there are challenges in maintaining a seamless user experience across various web applications, especially those that rely heavily on real-time interactions and multimedia content.

[0048] Furthermore, existing remote browser isolation technologies sometimes modify the browser code of the local or client device to integrate custom code into the browser itself during compilation, intercepting web traffic and delivering data to a remote server containing the remote browser. However, this requires each local browser to be recompiled to integrate the custom code into the local browser. This ultimately results in unnecessary time spent using existing remote browser isolation technologies and the exploitation of another vulnerability due to the modified browser.

[0049] Some of the embodiments described herein provide solutions to these problems by intercepting function calls or code instead of intercepting web traffic. This enables dynamic tracking of canvas elements and has minimal or no impact on website performance. Furthermore, some of the embodiments described herein provide a timer for converting and delivering a web page image sequence to the user or client device. This facilitates real-time interactions with little to no latency, and the user or client device does not have to go through the hassle of receiving a large image file. In addition, the improved system and method for canvas sanitization also provide various other aspects described herein.

[0050] Fig. 1 illustrates an example computing environment 100 for canvas sanitization according to various embodiments. In the example computing environment 100, a user may open a web page on a local browser on a client device 130. Then, the client device 130 may submit a request to open the web page to an isolation server 101, and the isolation server 101 may submit the request to a server 102 including a remote browser 110 via a communications network 120. Thus, the server 102 (e.g., via the communications subsystem) may receive the request to open the web page from the client device 130, and the agent 112 may override a first function (e.g., a canvas drawing function). In some examples, the isolation server 101 or the server 102 may load an agent 112 onto the remote browser 110.Server 102 may then load the web page from a web server 140 to remote browser 110 over communications network 120. Agent 112 may then intercept a drawing function of the web page, execute a second function to generate a drawing, convert the drawing into an image, and transmit the image 134 to client device 130 over communications network 120.

[0051] The Fig. The embodiment shown in Figure 1 is an example of a computing system environment and is not intended to be limiting. The subsystems and components within server 102, isolation server 101, and client device 130 may be implemented in hardware, firmware, software, or combinations thereof. In some examples, a user operating client computing device 130 may use a client application to interact with isolation server 101, which communicates with server 102 to browse the web page from web server 140. Client device 130 may be configured to transmit a request to open the web page to server 102 and receive an image converted from a drawing configured to be drawn based on a drawing function of the web page over communication network 120.Such client applications may be a web browser, web browser-based applications, standalone software applications such as mobile device applications, or other suitable client applications configured to communicatively connect to a web server 140 via the communications network 120. In some examples, the client device 130 may be a user device (e.g., a personal computer, a laptop, a tablet, a mobile device, or other suitable user device). In other examples, the client device 130 may comprise an intermediary device for relaying, delivering, and / or forwarding data between the server 101 and a terminal device. In further examples, the client device 130 may comprise another server, a web server, or any other suitable device.In some examples, the client device 130 may only communicate with the isolation server 101 to protect the client device 130 from malicious or harmful content accessing the client device 130.

[0052] In some examples, client device 130, isolation server 101, server 102, remote browser 110, agent 112, web server 140, and / or any other disclosed devices may be communicatively coupled via communication network 120. Communication network 120 may be any type of network known in the data communications field. As non-limiting examples, communication network 120 may be a local area network (LAN; e.g., Ethernet, Token Ring, etc.), a wide area network (e.g., the Internet), an infrared or wireless network, a public switched telephone network (PSTN), a virtual network, a wired network, etc. Communication network 120 may use any available protocols, such asTransmission Control Protocol / Internet Protocol (TCP / IP), Systems Network Architecture (SNA), Internet Packet Exchange (IPX), Secure Sockets Layer (SSL), Transport Layer Security (TLS), Hypertext Transfer Protocol (HTTP), Secure Hypertext Transfer Protocol (HTTPS), Institute of Electrical and Electronics (IEEE) 802.11 Protocol Suite oder andere drahtlose Protokolle und dergleichen.

[0053] In some examples, the isolation server 101 may include a processor coupled to memory to perform various tasks on behalf of the client device 130. In some examples, the isolation server 101 may also include a communications subsystem to provide a communications interface between the server 101 and other computing devices via the communications network 120. Thus, the processor may communicate with the client device 130 and the server 102 via the communications subsystem. In some examples, the isolation server 101 may verify the authorization of the client device 130 and act on behalf of the client device 130. In further examples, the isolation server 101 may store the agent 112 in memory and load the agent 112 into a browser (e.g., a remote browser) on the server 102.In some examples, isolation server 101 may be the only entity capable of interacting with agent 112 and browser 110 of server 102. In other examples, isolation server 101 may interact with other computing devices or servers. In further examples, isolation server 101 and server 102, including browser 110 and agent 112, may be physically separate entities. In other examples, isolation server 101 and server 102 are logically separate operations that communicate with each other. In further examples, isolation server 101 may be located in the same or a different cloud environment than server 102, or may be separate from it.

[0054] In some examples, various system components 104 may be implemented on server 102. Server 102 may be configured to execute one or more server software applications or services, e.g., web-based or cloud-based services, to support interaction with browser 110, isolation server 101, and / or web server 140. System components 104 may include, for example, a processor 106 that can communicate (e.g., interface) with a number of peripheral subsystems within system components 104 via a bus subsystem. These peripheral subsystems may include, for example, memory 108, an input / output (I / O) subsystem, and a communications subsystem.

[0055] In some examples, processor 106 may be implemented as one or more integrated circuits (e.g., a conventional microprocessor or microcontroller). In one example, processor 106 may control the operation of server 102. Processor 106 may include single-core and / or multi-core processors (e.g., quad-core, hexa-core, octo-core, ten-core processors, etc.) and processor caches. Processor 106 may execute a variety of resident software operations embodied in program code and may manage multiple concurrently executing programs or operations. In some examples, processor 106 may include a specialized processor (e.g., a digital signal processor (DSP), an external, graphics application-specific, and / or other processor).

[0056] In some examples, the bus subsystem provides a mechanism for intended communication between the various components of the server 102. The bus subsystem may use a single bus or multiple buses. In some examples, the bus subsystem may include a memory bus, a memory controller, a peripheral bus, and / or a local bus using one of various bus architectures (e.g., Industry Standard Architecture (ISA), Micro Channel Architecture (MCA), Enhanced ISA (EISA), Video Electronics Standards Association (VESA), and / or Peripheral Component Interconnect (PCI) bus, which may be implemented as a mezzanine bus according to the IEEE P1386.1 standard).

[0057] In some examples, the system components 104 of the server 102 may further include a memory 108, which is a non-transitory storage medium. The memory 108 may include one or more suitable storage devices that may be used to store suitable data (e.g., of the agent 112, the browser 110, etc.) and instructions that may be used, for example, by the processor 106 to execute at least a portion of the processing described below in connection with Fig. 2. Memory 108 may include any suitable volatile memory, non-volatile memory, data storage, or any suitable combination thereof. For example, memory 120 may include random access memory (RAM), read-only memory (ROM), electronically erasable programmable read-only memory (EEPROM), one or more flash drives, one or more hard drives, one or more solid-state drives, one or more optical drives, etc.

[0058] In some examples, the system components 104 of the server 102 may further include the communications subsystem to provide a communications interface between the server 102 and external computing devices via the communications network 120, including local area networks (LANs), wide area networks (WANs) (e.g., the Internet), and various wireless telecommunications networks. In some examples, the communications subsystem may include, for example, one or more network interface controllers (NICs), such as Ethernet cards, Asynchronous Transfer Mode NICs, Token Ring NICs, and the like, and one or more wireless communications interfaces, such as wireless network interface controllers (WNICs), wireless network adapters, and the like.Additionally and / or alternatively, the communications subsystem may include one or more modems (telephone, satellite, cable, ISDN), synchronous or asynchronous digital subscriber line (DSL) units, FireWire® interfaces, USB® interfaces, and the like. The communications subsystem may also include radio frequency (RF) transceiver components for accessing wireless voice and / or data networks (e.g., using cellular phone technology, advanced data network technology such as 3G, 4G, or EDGE (Enhanced Data Rates for Global Evolution), Wi-Fi (IEEE 802.11 family standards or other mobile communications technologies, or any combination thereof), Global Positioning System (GPS) receiver components, and / or other components.

[0059] In some examples, the communications subsystem may also receive input communications in the form of structured and / or unstructured data feeds, event streams, event updates, and the like on behalf of a user accessing the web page or the website of the web server 140 via the server 102. In one example, the communications subsystem may be configured to receive input (e.g., a website address, a mouse click input, a keyboard input, etc.) in real time from users and / or other communication services. In addition, the communications subsystem 232 may be configured to receive the website (e.g., a Hypertext Markup Language (HTML) file, a JavaScript file, an image file, a video file, etc.) from the web server 140 or other suitable data in the form of continuous data streams and / or discontinuous data.In some examples, the communications subsystem may output modified web page data from the web server 140 to the client device 130. The various physical components of the communications subsystem may be detachable components coupled to the server 102 via a computer network (e.g., the communications network 120), a FireWire® bus, or the like, and / or they may be physically integrated onto a motherboard of the server 102. In some examples, the communications subsystem may be implemented in whole or in part by software.

[0060] In some examples, server 102 may also include browser 110 (e.g., a remote browser). In some examples, server 102 may allocate a thread or processing resource to browser 110 so that browser 110 can perform a browsing task (e.g., loading a web page from a web server, rendering content, etc.). In some examples, server 102 may allocate a single thread to a tab of browser 110 so that the tab performs a browsing task until the tab is closed. Browser 110 may be a remote browser and / or an isolated browser, i.e., an application for accessing websites where the browser may be running in a sandbox or restricted and isolated environment. In some examples, server 102 may be the only entity that can interact with the browser and the agent.In some examples, browser 110 downloads the HTML file and other resources (e.g., a JavaScript file) from the web page's web server and then processes the web page in the browser to deliver a rendered image to local device 130 to display the web page's content. Thus, local device 130 does not need to access the web page or website directly.

[0061] In some examples, the agent 112 may be a program, program code, or a proxy (e.g., a JavaScript proxy) stored in memory 108 in the server 102 or in memory in the isolation server 101. The agent 112 may, for example, be written in JavaScript or another suitable programming language. The processor 106 of the server 102 or the processor of the isolation server 101 may load the agent 112 into the browser 110. In some examples, the processor 106 of the server 102 or the processor of the isolation server 101 may load the agent 112 from memory into the browser 110 before the web page is loaded from the web server 140. In some examples, the agent 112 is configured to replace a first function (e.g., canvas drawing functions) and / or built-in functions (e.g., JavaScript functions) with a second function (e.g.,an agent-defined agent drawing function), intercepts a canvas element on the web page, and / or notifies the server when the client device needs to redraw that canvas element. Additionally, the agent 112 may independently manage a timer and the transformation of the canvas. In some examples, the agent 112, along with the browser 110, may be logically or physically separated from the isolation server 101. In some examples, at least a portion of the process 200 may be performed in conjunction with. Fig. 2 by the browser 110 and / or the agent 112 using the processor 106 of the server 102. In some examples, the isolation server 101 may load the agent 112 into the browser 110, but the isolation server 101 need not maintain communication until the agent sends the isolation server 101 the drawing to be delivered to the client device 130.

[0062] In some examples, browser 110 may download website resources (e.g., HTML, JavaScript files) from web server 140 and / or access other resources (e.g., a video stream or images). Web server 140 may be computer software and underlying hardware that accepts requests via an appropriate communications protocol (e.g., HTTP) to enable browser 110 to download and / or interact with resources (such as HTML files, JavaScript files, images, CSS files, etc.). In some examples, web server 140 responds to requests from browser 110 and makes website resources and other resources available to browsers over the Internet. However, web server 140 is not necessarily a reliable source. Web server 140 may contain potentially harmful code or malicious content that may compromise user data and security.Therefore, the browser can contain the potentially harmful code or malicious content of the web server 140 in an isolated environment.

[0063] Due to the ever-changing nature of computers and networks, the description of the computer system 100 shown in the figure is intended only as a specific example. Many other configurations with more or fewer components than the system shown in the figure are possible. For example, custom hardware could also be used and / or certain elements could be implemented in hardware, firmware, software, or a combination thereof. In addition, connection to other computing devices, such as network input / output devices, may be used. Based on the disclosure and teachings contained herein, one of ordinary skill in the art will appreciate other ways and / or methods of implementing the various embodiments.

[0064] Fig. 2 is a flowchart illustrating an example operation 200 for processing web content (e.g., canvas sanitization) in accordance with various embodiments. As described below, some or all of the illustrated features may be omitted in a particular implementation, and some of the illustrated features are not required for implementation of all embodiments. In some examples, server 102 may be configured with processor 106 and memory 108 to perform operation 200. In other examples, server 102 and / or isolation server 101 in combination (collectively, server 102) may be configured to perform operation 200. In some examples, any suitable device or means for performing the functions or algorithms described below may perform operation 200.

[0065] In block 210, the server 102 may receive a request to open a web page or website from a client device 130 (e.g., via the isolation server 101). Referring to step 302 of Fig. 3, the client device 130 may transmit the request to open a web page to the isolation server 101, and the isolation server 101 receives the request from the client device 130 and transmits the request to the server 102. For example, the request to open a web page from the client device 130 may include user input (e.g., a website address input, a mouse click or any input to be redirected to the web page, a voice command to open the web page, or other suitable user input) in a local browser of the client device 130. In some scenarios, the user may enter a website address in the local browser of the client device 130. Then, the local device 130 may transmit the website address to the server 102 (e.g., via the isolation server 101).In some examples, a web page may contain a document, an HTML file, a JavaScript file, a CSS file, an image, a video stream, or any suitable document on the World Wide Web and may be identified by a unique Uniform Resource Locator (URL).

[0066] In some examples, the local device 130, the isolation server 101, or the server 102 can determine whether the web page to be opened is secure. In some examples, the local device 130 can determine whether the web page is secure. For example, the local device 130 can store a list of secure or verified website addresses or domains in the memory of the local device 130. Then, when the user types a website address into the local browser, the local device 130 can look up the website address in the list of secure or verified website addresses (e.g., using a lookup table, a hash algorithm, etc.). If the website address or domain is a secure or verified web page, the client device 130 can connect directly to a web server 140 of the secure website and load the web page (e.g., an HTML file, a JavaScript file, a CSs file, an image, a video stream, etc.) or download a website resource to be rendered on the local browser. In other examples, the client device 130 may transmit the user input to the server 102 (e.g., via the isolation server 101), and the server 102 or the isolation server 101 may determine whether the web page is secure or verified. If the website address or domain is secure or verified, the server 102 may connect to the web server 140 and serve the web page and / or the web page resource from the web server 140 to the client device 130 (e.g., via the isolation server 101) without using the canvas sanitization operation 200. In further examples, the server 102 may determine whether to perform the operation 200 by locating each HTML canvas element on the web page from the web server 140. If an HTML canvas element is present on the web page, the server 102 may perform operation 200.

[0067] In block 220, the server 102 may replace a first function with a second function via an agent 112. In some examples, the server 102 or the isolation server 101 may load the agent 112 from the memory of the server 102 or the isolation server 101 into the browser 110 to replace the first function. This is also described in step 306 of Fig. 3. In some examples, the first function may comprise a predefined drawing function, an HTML canvas drawing function (e.g., line, path, rectangle, circle, arc, Bezier curve, etc.), a web standard function, a built-in drawing function (e.g., a built-in JavaScript drawing function), or another suitable drawing function. Thus, a user can use the first function without redefining how to draw in a browser. However, by loading the agent 112, the server 102, via the agent 112, can redefine the first function to the second function, which may be a drawing function of the agent. In some examples, the second function may be a function defined by the agent 112 to perform a task corresponding to the first function. For example, the second function may be an agent drawing function to use a predefined drawing function, an HTML canvas drawing function (e.g.,Line, path, rectangle, circle, arc, Bezier curve, etc.), a web standard function, a built-in drawing function (e.g., a built-in JavaScript drawing function), or another suitable drawing function corresponding to the first function. Because the first function on an HTML canvas can be exploited to gain unauthorized access to an end-user system, the server, via agent 112, may replace the first function with the second function. In further examples, a function (e.g., the first function or the second function) may comprise a procedure or set of instructions that performs a task (e.g., calculating a value, generating letters, drawing a drawing, etc.) with an output (e.g., the task result, the calculated value, the letters, the drawing, etc.). In further examples, replacing the first function may include replacing or overwriting the first function with the second function.For example, the first function is defined in a parent class, while the second function is redefined (e.g., in a child class) and has the same function name, parameters, and return type. Then, when the first function, which is a predefined standard function, is called, the second function can be executed and not the first, because the second function overrides the first.

[0068] In some examples, the first function may include a predefined drawing function, an HTML canvas drawing function for drawing a figure (e.g., line, path, rectangle, circle, arc, Bezier curve, etc.), a built-in drawing function (e.g., built-in JavaScript drawing function), or any other suitable drawing function. In further examples, the agent 112 may include a proxy (e.g., a JavaScript proxy), code, or a set of program instructions to intercept the first function and / or redefine the first function into an agent drawing function that is the second drawing function. Additionally, the proxy may specify a programming object, wrapper, or program function to intercept the first function and replace the first function with the second function. The proxy may execute code before and after the function call in the same context as the function application.In further examples, agent 112 may have a prototype function of a particular object and other suitable functions. In some examples, the second function is used in connection with block 240 of FIG. Fig. 2. In some examples, to load agent 112, server 102 may include agent 112 containing programming code in the header to be executed before the web page is loaded and executed. In further examples, to load agent 112, server 102 may start, execute, or cause agent 112 to intercept and / or override the first function used on the web page. In some examples, prior to loading agent 112 in browser 110, server 102 may create a tab in browser 110, as in step 304 of Fig. 3. In such examples, to load the agent into the browser 110, the server 102 may load the agent 112 into the tab of the browser 110. Because the server 102 loads the agent 112 into the browser 110, the browser 110 does not need to be modified. Loading the agent 112 can simplify a canvas sanitization operation and improve versatility, as any remote browser can be used for the canvas sanitization operation without modification. Furthermore, loading the agent 112 can improve the security of the browser, as the agent 112 is only used when the canvas sanitization operation is performed and / or until the processing resource is allocated to the browser, and the default browser does not need to be modified.

[0069] Referring again to Fig. 2, a block 230, the server 102 can load the web page from the web server 140 via the browser 110. This is also described in steps 308 and 310 of Fig. 3. For example, the server 102 may provide the web page to the browser 110 in step 308, while the browser 110 loads the web page from the web server 140 in step 310. In some examples, the web page may include a document, an HTML file, a JavaScript file, a CSS file, etc., to render the content of the web server 140 in the browser. In further examples, the web page may include an HTML canvas element. To load the web page, the server 102 may, for example, receive the web page from the web server 140 and create a drawing object of an HTML canvas element on the browser based on the web page. The drawing object of the HTML canvas element may, for example, be defined from a rendering context class (e.g., HTMLCanvasElement.getContext(contextType)). In some examples, HTMLCanvasElement.getContext(contextType) as the rendering context class returns a drawing context or a drawing object on the canvas, which can be defined in an HTML file. In some examples, the contextType can be "2d" to create a drawing object representing a two-dimensional rendering context, "webgl", "webgl2", or "webgpu" to create a drawing object representing a three-dimensional rendering context, or any other suitable context type. In further examples, the drawing object of the HTML canvas element can contain a drawing function (e.g., line, path, rectangle, circle, arc, Bezier curve, etc.). However, in some examples, the drawing function can be exploited to gain unauthorized access to an end-user system.

[0070] Referring again to Fig. 2, in block 240, the server 102, via the agent 112, may execute the second function corresponding to the first function to generate a drawing for a first period of time in response to an attempt to perform the first function on the web page. In some examples, the first period of time may comprise a predetermined period of time (e.g., 0.1, 0.5, 1, 2, 5, 10 seconds, or any other suitable period of time). This is also described in step 312 of Fig. 3. In some examples, the drawing may include a line, a path, a rectangle, a circle, a Bezier curve, or any other suitable object renderable on the browser 110. In some examples, the first function may be executed based on the same drawing object. In some examples, the first function may include a drawing parameter. The drawing parameter may be provided, for example, by the web server 140. In some scenarios, the server 102 may execute the second function based on the drawing parameter to perform the second function. In some examples, attempting to execute the first function may include completing a command on the web page prior to a call to the first function or being ready to execute the first function.In some examples, calling the first function may involve executing the first function of a drawing object.

[0071] In some examples, server 102 may receive user input to indicate a change to the drawing object. In some examples, attempting to execute the first function may involve changing the drawing object. For example, if the web page includes drawing function code on the HTML canvas when the mouse cursor is on the canvas. Then, in response to the position of the mouse cursor (i.e., the user input), the first function is configured to execute. However, server 102 may intercept the first function and, through the browser, execute the second function. In some examples, each indication triggered by user input may include a unique identifier for the drawing function on the canvas. This allows server 102 to respond differently to each indication. The indication of a change to the canvas element may occur multiple times while the web page is running.

[0072] In some examples, agent 112 may include a helper function that allows a relevant function to be hooked into all relevant rendering contexts. For example, the helper function may replace a function belonging to a specific prototype. Replacing a function allows all objects using that prototype to use agent 112. The first function may be replaced with a dynamically created definition that calls a function of agent 112 (e.g., a proxy) with the same context as the first function, passes a reference to the first function to be used in an exception, and passes the original set of arguments used in the first function to agent 112 so that it functions in the same way as the first function. In other examples, agent 112 may replace the drawing object or context (e.g., HTMLCanvasElement.getContext(contextType)) with an agent object.In such examples, the browser 110 may use the predefined drawing method to create a drawing based on the replaced drawing object.

[0073] In response to the attempt, the server 102, via the agent, may execute the second function corresponding to the first function to generate a drawing for a period of time. In some examples, the period of time may comprise a predetermined period of time (e.g., 0.1, 0.5, 1, 2, 5, 10 seconds, or any other suitable period of time). This is also described in steps 316 and 318 of Fig. 3. In some examples, the second function may be configured to draw a line, path, rectangle, circle, arc, Bezier curve, or other suitable shape from the drawing object or rendering context class (e.g., HTMLCanvasElement.getContext(contextType)). In some examples, the second function may include beginPath(), arc(), moveTo(), lineTo(), stroke(), or other suitable function for drawing. In some examples, the second function may draw the same drawing that the corresponding first function is intended to draw. In some examples, to execute the second function, the server 102 may execute the second function in the browser based on the drawing object. Thus, the second function is executed based on the same drawing object on which the first function is based.For example, if the first function is configured to draw a circle based on the drawing object on the canvas, the second function can simply draw a circle based on the same drawing object on the canvas. In some examples, the drawing may include an object drawn by the second function. This allows the web page to execute normally without noticing the call to the agent function instead of the first function. This is because the agent 112 intercepts a function and does not intercept web traffic, as is the case with existing technologies.

[0074] In some examples, the server 102 may initiate a timer for the time period. In some examples, the timer may be linked to the drawing object. As in Fig. 3, after triggering the timer on the canvas element, the server 102 may operate within the server environment. For example, the server 102 may call an agent function (e.g., beginPath(), arc(), moveTo(), lineTo(), stroke(), etc.). In further examples, the time period for the timer may be static and configured differently for different web pages. Thus, while the timer is active, all drawing calls associated with the same drawing object or context can continue normally without resetting the timer.

[0075] In further examples, the drawing may be a first partial drawing. Then, the server, via the agent, may perform the second function corresponding to the first function to generate a second partial drawing for a second time period. In some examples, the second time period may comprise a predetermined time period (e.g., 0.1, 0.5, 1, 2, 5, 10 seconds, or any other suitable time period). Then, the server 102, via the agent, may aggregate the first partial drawing and the second partial drawing to generate an aggregated drawing.

[0076] As in Fig. 2, the server 102 may convert the drawing into an image via the agent in block 250. In some examples, the drawing may be a line, a path, a rectangle, a circle, an arc, a Bezier curve, or other suitable object on the canvas (e.g., drawn by an HTML canvas drawing function, a built-in drawing function, or other suitable drawing function). In further examples, the image may be a binary representation of the canvas including the drawing. The image may, for example, be saved in various formats (e.g., TIFF, JPEG, PNG, GIF, or other suitable format). This is also performed in step 320 of Fig. 3. In the examples in block 240 with the aggregation of the first and second partial drawings to form the aggregated drawing, the server 102, via the agent, may convert the aggregated drawing into the image. In the examples where the drawing contains the object drawn by the second function, the server 102, via the agent, may convert the object, rather than the entire rectangular area containing the object, into the image. In some examples, the server 102, via the agent 112, may convert the drawing into an image, e.g., using the "toDataURL" method of HTMLCanvasElement. By converting the actual canvas element, rather than the area it renders on the screen, the effects of background images or videos below or above the canvas layer may be eliminated. In further examples, by choosing a specific image format (e.g.,PNG), which supports transparency, the canvas does not obscure elements that are intended to be behind it in the browser. Since the copy contains no further data besides information about the pixels themselves, it cannot contain any malicious data that might be stored in rendering contexts due to a vulnerability in the browser implementation of such contexts. Furthermore, the image is precisely sized for the canvas because it is a copy of the canvas's pixels rather than generic drawing instructions, providing the client device 130 with better final image quality.

[0077] In some examples, if the drawings may include multiple partial drawings based on the timer, the server 102, via the agent 112, may convert each partial drawing into an image. In other examples, the server 102, via the agent 112, may aggregate the multiple partial drawings to generate an aggregated drawing and convert the aggregated drawing into an image. In further examples, the server 102, via the agent 112, may convert each partial drawing into an image. Then, the server 102, via the agent 112, may aggregate the images corresponding to the partial drawings to generate an image.

[0078] Referring again to Fig. 2, the server 102 may transmit the image to the client device 130 (e.g., via the agent 112 and / or the communication subsystem) in block 260. This is also described in steps 322 and 324 of Fig. 3. In some examples, agent 112 may transmit the image in the sandbox to isolation server 101, which transmits the image to client device 130. The client device may then draw the resulting image onto the drawing object, as in step 326 in Fig. 3 shown.

[0079] This process can be repeated if the web page tries to draw something else on the drawing object, as in step 328 in Fig. 3 shown. Fig.Figure 4 is a schematic diagram showing an example of canvas sanitization. Agent 112 loaded into browser 110 can intercept the web page's drawing functions and replace them with agent functions to generate a drawing. Agent 112 can then create a pixel-by-pixel copy of the drawing image and send the sanitized image to client device 130. Client device 130 can create a bitmap version of the sanitized image and draw the image onto the local browser's canvas. In this way, process 200 does not need to modify browser 110 by loading agent 112 into browser 110, and it reduces the use of computing resources by intercepting drawing functions rather than web traffic.

[0080] While certain features and aspects have been described with respect to some embodiments, one skilled in the art will recognize that numerous modifications are possible. For example, the methods and operations described herein may be implemented using hardware components, software components, and / or any combination thereof. While various methods and operations described herein are described with respect to particular structural and / or functional components for convenience, the methods provided by various embodiments are not limited to any particular structural and / or functional architecture, but rather may be implemented on any suitable hardware, firmware, and / or software configuration.Although certain functions are attributed to particular system components, this functionality need not be limited to them and may be distributed among various other system components according to the various embodiments, unless the context dictates otherwise.

[0081] While the procedures of the methods and acts described herein are described in a particular order for convenience, unless the context dictates otherwise, various procedures may be rearranged, added, and / or omitted in accordance with various embodiments. Furthermore, the methods or procedures described with respect to one method or act may be incorporated into other described methods or acts; likewise, system components described according to a particular structural architecture and / or with respect to a system may be organized into alternative structural architectures and / or integrated into other described systems. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited non-patent literature

[0000] Transmission Control Protocol / Internet Protocol (TCP / IP), Systems Network Architecture (SNA), Internet Packet Exchange (IPX), Secure Sockets Layer (SSL), Transport Layer Security (TLS), Hypertext Transfer Protocol (HTTP), Secure Hypertext Transfer Protocol (HTTPS), Institute of Electrical and Electronics (IEEE) 802.11

[0052]

Claims

[1] A method for processing web content, comprising: Receiving a request to open a web page from a client device; Replacing a first function with a second function via an agent, wherein the agent is loaded into a browser; loading the web page from a web server into the browser; in response to an attempt to perform the first function on the web page, executing, via the browser, the second function corresponding to the first function to generate a drawing for a first period of time; Converting the drawing into an image about the agent; and Sending the image to the client device. [2] The method of claim 1, wherein the agent is loaded into the browser before the web page is loaded from the web server into the browser. [3] The method of claim 1, wherein the agent comprises a proxy, and wherein replacing the first function comprises: redefining the first function into the second function. [4] The method of claim 3, wherein the first function is configured to be executed using a drawing parameter, and wherein the second function is configured to be executed using the drawing parameter. [5] The method of claim 1, wherein loading the web page comprises: Receiving the web page from the web server; and Creating a drawing object of a hypertext markup language canvas element on the browser based on the web page, wherein the first function is configured to be executed based on the drawing object. [6] The method of claim 5, wherein performing the second function comprises: Execute the second function on the browser based on the drawing object. [7] The method of claim 5, further comprising: Trigger a timer for the first time period, where the timer is linked to the drawing object. [8] Server, including: a store that stores a browser and an agent; a processor coupled to the memory and a communication system coupled to the processor, wherein the processor coupled to the memory is configured to: to receive a request to open a web page from a client device via the communication system; to replace a first function with a second function via an agent, whereby the agent is loaded into the browser; to load the website from a web server via the browser; in response to an attempt to perform the first function on the website, performing, via the agent, the second function corresponding to the first function to generate a drawing for a first period of time; to convert the drawing into an image via the agent; and to transmit the image to the client device via the communication system. [9] The server of claim 8, wherein the agent comprises a proxy, and wherein, to replace the first function, the processor is configured to: redefine the first function to the second function. [10] Server, including: a store that stores a browser and an agent; a processor coupled to the memory and a communication system coupled to the processor, wherein the processor coupled to the memory is configured to: to receive a request to open a web page from a client device via the communication system; load an agent into the browser; to replace a first function with a second function via the agent; to load the website from a web server via the browser; in response to an attempt to perform the first function on the website, performing, via the agent, the second function corresponding to the first function to generate a drawing for a first period of time; convert the drawing into an image via the agent; and to transmit the image to the client device via the communication system.

Images