A web page same-screen method and system

Abstract

The present application provides a web page same-screen method and system, the system includes a server and a client, the method includes: the server receives a same-screen creation request, loads the target web page in the server browser instance to generate a target web page instance, and injects a first script, starts page monitoring, after the web page is loaded or the web page content changes are monitored, serializes the web page DOM to generate same-screen data, or draws the web page content on Canvas and obtains Canvas data to generate same-screen data, or generates a picture of the displayed content of the web page as same-screen data, and sends the same-screen data to the client. In addition, the web page operation event sent by the client is received and executed in the server browser instance. It can be seen that the present application browses the same-screen web page and collects operation events on the client, actually executes the web page on the server, and sends the execution result to the client for rendering, which can achieve fast same-screen for any web page or URL, and has the advantages of strong applicability, fast implementation and high stability.

Description

Technical Field

[0001] The present invention relates to the field of Internet communication technology, and in particular to a method and system for sharing web pages on the same screen. Background Art

[0002] Web pages on the same screen facilitate website visitors to directly interact and communicate through the website, greatly improving communication efficiency.

[0003] However, the existing web page sharing technology has many problems. For example, the realization of web page sharing requires customized development and writing a special sharing program, which has high implementation costs. Some sharing technologies are implemented by introducing a sharing library, which also requires modifying the web program and deploying the website. It is not possible to use any page immediately, so the implementation efficiency is low. It is limited to sharing a single page, or there is a phenomenon of screen sharing service interruption and reconnection after the web page jumps. If audio and video communication is included in the sharing process, there will also be audio and video interruption when the page jumps, and the effect is poor. New users joining the sharing process cannot be shared immediately. The user experience is poor because it requires all clients to refresh the page and restart the screen sharing from the beginning. During the screen sharing process, real-time recording is not possible and recording must wait until the screen sharing is complete, which takes a long time. When sharing the screen between multiple devices, the display of each terminal is inconsistent due to device differences, resulting in misaligned graffiti and affecting communication. For a single operation on a shared web page, it will be executed multiple times on all clients, that is, multiple requests will be sent for one operation, causing serious confusion in program operation. If a Canvas is present on the shared web page, a large amount of data will be generated during the screen sharing process, blocking communication.

[0004] Therefore, considering that the existing web page same-screen technology has the problems of high implementation cost and low efficiency, and there are problems with the same-screen effect and user experience, and it is easy to cause network problems and operation errors, it is urgent to propose a new web page same-screen implementation method to solve the above problems. Summary of the Invention

[0005] To address the above issues, the present invention provides a method and system for web page sharing on the same screen, aiming to efficiently and quickly establish sharing on the same screen based on any web page, while also resolving existing issues such as web page jumps leading to sharing on the same screen service interruption and audio interruption, poor real-time user joining experience, inability to record in real time, graffiti misalignment, repeated execution of multiple web page requests, and network congestion during Canvas synchronization. The technical solutions provided by the present invention are as follows:

[0006] In a first aspect, the present application provides a method for sharing web pages on the same screen, which is applied to a server and includes:

[0007] Creating a browser instance on the server, loading a target webpage in the browser instance, generating a target webpage instance, injecting a first script into the target webpage instance, and starting page monitoring;

[0008] After the target web page instance content is loaded or a change in the target web page instance content is detected, serializing the target web page instance DOM to generate serialized DOM data, and sending the serialized DOM data to the client as on-screen data, or rendering the target web page instance display content on a first Canvas, obtaining Canvas data based on the first Canvas, and sending the Canvas data as on-screen data to the client, or generating image data based on the target web page instance display content, and sending the image data to the client as on-screen data;

[0009] Also includes:

[0010] Receive the webpage operation event sent by the client, and execute the webpage operation contained therein;

[0011] Among them, the target web page is a URL or web page file used for the same screen; the first script at least includes starting page monitoring, DOM serialization processing and / or first Canvas drawing and obtaining Canvas data and / or generating image data, and executing web page operation functions based on received web page operation events; the Canvas data includes generating drawing data based on the drawing of the first Canvas or generating image data based on the display content of the first Canvas; the client realizes same-screen browsing and operation of the target web page instance content by accessing the same-screen web page.

[0012] Through the solution provided by this embodiment, for the target web page that the user needs to view on the same screen, the server provides a browsing execution environment for the target web page, and provides the execution result in the form of serialized DOM data, or Canvas data, or image data to the client that conducts the same-screen communication. The client collects web page operation events but does not execute them, but sends them to the browsing execution environment of the server for execution. In this way, it is ensured that there is only one execution end in the same-screen process, and all clients participating in the same-screen are mirror images of the execution end content, ensuring the accuracy of the same-screen content, and can be quickly implemented for any web page without modifying the page, and the implementation cost is low.

[0013] In one implementation of the present application, on the server side, a pre-made web page is configured as a same-screen web page, or the same-screen web page is calculated and generated based on the target web page instance information; a same-screen web page link is generated based on the URL of the same-screen web page and / or the correspondence between the same-screen web page and the target web page instance; and according to the target web page instance, the corresponding same-screen web page link is returned to the client.

[0014] In this implementation, whether a pre-made web page is used as the same-screen web page, or a dedicated same-screen web page is calculated and generated for the target web page instance, a second script must be included in the same-screen web page. The second script includes at least the functions of receiving and processing the same-screen data, and collecting and sending web page operation events.

[0015] In this implementation method, it also includes and establishes a correspondence between the target web page instance and the same-screen web page, and returns a same-screen web page link for accessing the same-screen web page. The same-screen web page link includes one or more of the same-screen web page URL and parameter information. The correspondence is that for each target web page instance, there is a same-screen web page corresponding to it, or there is a same-screen web page and parameter combination corresponding to it, that is, for each target web page instance, there is a same-screen web page link corresponding to it.

[0016] In this implementation, a pre-made web page is configured as a same-screen web page, which corresponds to multiple target web pages and provides services. The same-screen web page link contains parameter information, and the second script calculates and identifies the corresponding target web page instance based on the parameter information; the same-screen web page is calculated and generated based on the target web page instance information, and the same-screen web page corresponds to the target web page instance and provides services.

[0017] Through the above implementation method, a same-screen webpage and parameters are allocated or created for the target webpage that requires same-screen communication, and a correspondence between the same-screen webpage and the target webpage instance is established, and a same-screen webpage link is generated for the client to access and enter the same-screen communication.

[0018] In one implementation of the present application, the browser is a headless browser. By using a headless browser, a tab is instantiated to create a browser instance, and the target web page is run, thereby saving server-side computing resources and increasing the flexibility of program control.

[0019] In one implementation of the present application, after the server browser loads the target webpage and injects the first script, the process further includes:

[0020] Instantiate a corresponding same-screen service instance on the server side;

[0021] The same-screen service instance receives the same-screen data sent by the server-side browser instance and sends it to the client; the same-screen service instance receives the webpage operation event sent by the client and sends it to the server-side browser instance;

[0022] Among them, in the target web page instance, the first script is executed to establish a communication connection with the same-screen service instance; in the same-screen web page, the second script is executed to establish a communication connection with the same-screen service instance.

[0023] Through the above implementation method, the execution end and the browsing operation end of the same screen are connected through the same screen service instance, which facilitates the centralized management and control of a web page on the same screen.

[0024] In one implementation of the present application, generating same-screen data based on the target web page instance also includes the following methods: serializing the changed DOM nodes in the target web page instance to generate serialized DOM data, and generating the serialized DOM data as the same-screen data; or, based on the change of the first Canvas, generating differential image data for the differential part between two changes, and generating the differential image data as the same-screen data; or, generating drawing data based on the change of the first Canvas, and generating the drawing data as the same-screen data; or, generating differential image data based on the part of the changed display content of the target web page instance, and generating the differential image data as the same-screen data.

[0025] Through the above implementation method, by collecting incremental data or differentiated data, the amount of transmitted data can be reduced during the communication process and the efficiency of the same screen can be improved.

[0026] In one implementation of the present application, the server receives client browser and / or system parameter information sent by the client, performs calculations based on the client browser and / or system parameters according to predetermined calculation rules, obtains the target web page instance and browser instance settings, sets the target web page instance based on the target web page instance settings, and sets the browser instance based on the browser instance settings. The server also includes monitoring a client disconnection event, and when it is confirmed that the client is not the last client, performs calculations based on the browser and / or system parameter information of the remaining connected clients and the client browser parameters according to predetermined calculation rules, obtains the target web page instance and browser instance settings, sets the target web page instance based on the target web page instance settings, and sets the browser instance based on the browser instance settings.

[0027] Through this implementation, for example, each time a client joins or disconnects a shared screen, the optimal size of the target web page execution terminal is calculated based on the browser size of the participating clients. This resolves display issues caused by varying client browser sizes, ensuring that all clients receive the correct display. Therefore, this implementation effectively addresses device compatibility issues.

[0028] In one implementation of the present application, a target web page instance in the server-side browser instance is monitored or judged to have a web page jump; when it is confirmed that the target web page instance has a web page jump, the web page after the jump is used as a new target web page, the new target web page is loaded in the browser, and a new target web page instance is generated; and the first script is injected into the new target web page instance.

[0029] In combination with the above implementation method, it also includes setting the corresponding relationship between the same-screen service instance corresponding to the original target web page instance and the new target web page instance, or creating a new same-screen service instance corresponding to the new target web page instance, and recording the relationship between the new same-screen service instance and the same-screen service instance corresponding to the original target web page instance, and the relationship is equivalent to the relationship between the original target web page instance and the new target web page instance.

[0030] Through the above implementation method, web page jumps occur during the same-screen process, but the same-screen service remains continuous and the same-screen process is not interrupted, thereby obtaining a better user experience.

[0031] In one implementation of the present application, after a web page jump occurs in the target web page instance in the server-side browser instance, the same-screen service instance corresponding to the original target web page instance is set to continue to provide same-screen service for the new target web page instance, and the same-screen web page link corresponding to the same-screen service instance remains unchanged. Therefore, when a web page jump occurs in the target web page instance, the same-screen web page accessed by the client will not jump.

[0032] The beneficial effect achieved by the above implementation is that the audio and video communication service is included in the same-screen web page, and when a web page jump occurs in the target web page instance, the audio and video communication still maintains a continuous communication state.

[0033] In one implementation of the present application, the same-screen data and the corresponding time information are stored, and / or the web page operation events and the corresponding time information received from the client are stored to generate same-screen process recording data.

[0034] In the above implementation method, the same-screen process recording data is used for the same-screen interaction process playback. According to the time information, the same-screen data is received and processed in the same-screen web page, and / or, the web page operation event is received and the web page operation is simulated to execute the same-screen interaction process playback.

[0035] Through the above implementation method, real-time data recording and storage of communication process are achieved during the same screen process.

[0036] In one implementation of the present application, the target web page instance receives an instruction to switch to the graffiti working mode; image data is generated based on the current page content of the target web page instance, or the current page content of the target web page instance is drawn on a Canvas and Canvas data is obtained; the image data or the Canvas data is generated as same-screen data and sent to the client.

[0037] In the above implementation, the image data or the Canvas data is received by the client and used to generate a graffiti background, and the graffiti data is collected and drawn using the graffiti background size and position information as a relative reference.

[0038] Through the above implementation method, it is ensured that after entering the graffiti working mode during the same-screen communication, the screen displayed by all clients is completely consistent with the screen of the target web page instance on the server, providing a basis for executing synchronous graffiti and avoiding display differences and graffiti misalignment caused by devices.

[0039] In one implementation of the present application, the Canvas element in the target web page instance is monitored. When a change is detected, the Canvas data of the Canvas element is cached, communication processing is temporarily suspended, and monitoring is started according to the clock cycle. At the end of the next clock cycle, it is determined whether the Canvas element change has been received. When it is confirmed that the Canvas element change has not been received, the same-screen data is sent and the cached data is cleared. When it is confirmed that the Canvas element change has been received, the Canvas data of the current Canvas element is cached. The changes of the Canvas element include but are not limited to changes in the attributes of the Canvas element and the execution of Context events.

[0040] Through the above implementation, by caching Canvas data, the network congestion problem caused by too many Canvas events when loading a target web page containing a Canvas element and having a large number of Canvas drawing operations is avoided.

[0041] In a second aspect, the present application provides a method for displaying web pages on the same screen, which is applied to a client and includes:

[0042] Access a link to a web page on the same screen and load the web page on the same screen;

[0043] Receiving the same-screen data sent by the server, parsing the serialized DOM data contained therein, updating or replacing the DOM content of the same-screen client webpage based on the serialized DOM data, or parsing the Canvas data contained therein, creating a second Canvas in the same-screen client webpage, displaying an image or performing drawing on the second Canvas based on the Canvas data, or parsing the image data contained therein, and displaying the image based on the image data;

[0044] Also includes:

[0045] monitoring web page operation events occurring in the client, not processing them in the client, and sending the web page operation events to the server;

[0046] Among them, the same-screen webpage includes a second script, and the second script at least includes the functions of establishing a communication connection with the server, receiving and processing the same-screen data, and collecting and sending the webpage operation event.

[0047] Through the solution provided by this embodiment, the same-screen content is displayed on the client, and web page operation events are collected but not executed, and sent to the server for actual execution. This achieves the separation of execution and display of the same-screen process, multi-terminal same-screen browsing, and execution of a unified same-screen working mode.

[0048] In one implementation of the present application, when the client switches to the graffiti working mode, it receives the image data or Canvas data generated based on the web page content in the target web page instance and sent by the server, displays the image data on the client or performs drawing based on the Canvas data to generate graffiti background content; uses the graffiti background size and position information as a relative reference, collects and draws graffiti data, that is, collects and sends Canvas data based on the graffiti Canvas, receives graffiti Canvas data, and performs drawing on the graffiti Canvas to achieve graffiti communication.

[0049] Through the above implementation, the content screen of the client in Tuya working mode is completely consistent with the screen content in the server browser instance.

[0050] In one implementation of the present application, the client collects browser parameter information and / or system parameter information and sends it to the server. The browser parameter information and / or system parameter information are used by the server to calculate the setting parameters of the target web page instance and browser instance. The browser parameter information includes but is not limited to resolution, browser viewport size, etc.

[0051] Through the above implementation method, the client browser parameter information or system parameter information is sent so that the server can perform unified calculations, so that the server's operation and rendering results are compatible with the client browser or system. For example, the client sends the browser size to the server, and the server calculates and sets the optimal target web page instance size, so that all clients participating in the same-screen communication can obtain the correct display effect.

[0052] In a possible implementation of the present application, monitoring webpage operation events occurring in the client and sending them to the server further includes:

[0053] When it is determined that a webpage operation event of a specified type is monitored, the webpage operation event is executed locally and simultaneously sent to the server browser instance and other clients participating in the same screen;

[0054] Also includes:

[0055] The webpage operation event of the specified type is received, executed on the client, and a notification of an execution result of the webpage operation event of the specified type is received, and a correction operation is performed based on the notification of the execution result.

[0056] Through this implementation, specific web page events, such as touch and scroll, are synchronously sent to other clients for execution, reducing latency and improving on-screen efficiency. Furthermore, corrections are made based on the execution results of the target web page instance on the server to ensure the accuracy of the final on-screen results.

[0057] In one possible implementation of the present application, when it is determined that the event triggered by the web page operation is an operation of calling a local system or a local browser, the event is simultaneously transmitted to the local system or local browser for processing, and a local call completion event is triggered after the processing is completed. The web page operation event is sent to the server in the local call completion event, wherein the sent web page operation event includes local execution result information.

[0058] Through the above implementation method, when the event triggered by the web page operation is to call the local system or browser operation, it avoids confusion caused by all clients performing local operations. At the same time, it is beneficial to protect the client privacy and obtain consistent same-screen execution results.

[0059] On the third aspect, the present application also provides a web page same-screen system, including a server and a client.

[0060] The server includes a creation unit, a browsing operation execution unit, and a same-screen service unit, as follows:

[0061] The creation unit is configured to receive a same-screen creation request and create a browser instance and a same-screen service instance based on the same-screen creation request, and includes a same-screen request processing module, wherein:

[0062] The same-screen request processing module is configured to receive a same-screen creation request and, based on the target webpage URL information contained therein, notify the browsing operation execution unit to instantiate a tab in the server-side browser, load the target webpage to generate a target webpage instance, and notify the same-screen service unit to create a corresponding same-screen service instance;

[0063] The browsing operation execution unit is used to manage the server browser, create a browser instance on the server, and execute target web page browsing and operations in the browser instance, including a loading module, a communication module, a DOM data acquisition module / Canvas content processing module / image data acquisition module, and a web page operation event execution module, wherein:

[0064] The loading module is configured to load the target webpage to generate a target webpage instance, inject a first script into the target webpage instance, and start page monitoring;

[0065] The communication module is configured to establish communication with the corresponding same-screen service instance;

[0066] The DOM data acquisition module is configured to, after monitoring changes to the content of the target web page instance, serialize the DOM of the target web page instance, generate serialized DOM data, and send it to the same-screen service instance; the Canvas content processing module is configured to draw the web page content of the target web page instance on a first Canvas, generate Canvas data based on the first Canvas, and send it to the same-screen service instance; the image data acquisition module is configured to generate image data based on the display content of the target web page instance, and send the image data as same-screen data to the client;

[0067] The webpage operation event execution module is configured to execute the webpage operation event received from the client;

[0068] The same-screen service unit is used to create and manage a same-screen service instance. The same-screen service instance corresponds to the target webpage instance. The same-screen service instance provides data communication and control services for the client participating in the same-screen, including a same-screen data sending module and a webpage operation data receiving module, wherein:

[0069] The same-screen data sending module is configured to send the same-screen data to the client;

[0070] The webpage operation data receiving module is configured to receive the webpage operation event data sent by the client;

[0071] The client is used to access the same-screen web page link and execute the same-screen display of the corresponding target web page instance, including a communication module, a DOM data processing module / Canvas data processing module / image data display module, and a web page operation event collection module, wherein:

[0072] The communication module is configured to establish communication with the corresponding same-screen service instance;

[0073] The DOM data processing module is configured to receive the same-screen data sent by the same-screen service instance, parse the serialized DOM data contained therein, and update or replace the DOM content of the same-screen client webpage based on the serialized DOM data;

[0074] The Canvas processing module is configured to receive the same-screen data sent by the same-screen service instance, parse the Canvas data contained therein, create a second Canvas in the same-screen client webpage, and display an image or perform drawing on the second Canvas based on the Canvas data;

[0075] The image data display module is configured to receive the same-screen data sent by the same-screen service instance, parse the image data contained therein, and display the image based on the image data;

[0076] The web page operation event collection module is configured to monitor web page operation events occurring in the client, not process the events in the client, and send the web page operation events to the same-screen service instance.

[0077] In one implementation of the present application, the system further includes generating a correspondence between the browser instance, the same-screen service instance, and the client, wherein at the same time, the browser instance corresponds to one same-screen service instance, and the same-screen service instance provides same-screen services corresponding to the same-screen webpage of the target webpage instance to multiple clients. The system also includes calculating and returning a corresponding same-screen webpage link for the target webpage instance based on a preset same-screen webpage, or creating a webpage file corresponding to the target webpage instance and adding the second script to generate the same-screen webpage, and returning the same-screen webpage link.

[0078] In a possible implementation of the present application, the browsing operation execution unit or the same-screen service unit further includes a recording module, which collects and stores the same-screen data and corresponding time information, the web page operation events received from the client and the corresponding time information, and generates same-screen process recording data.

[0079] In one possible implementation of the present application, the client further includes an operating environment information collection module, the same-screen service instance further includes an operating environment parameter calculation module, and the browser operation execution unit further includes an operating environment parameter setting module, wherein:

[0080] The operating environment information collection module is configured to collect client browser and / or system parameter information and send it to the server;

[0081] The operating environment parameter calculation module is configured to perform calculations based on all the client browsers and / or system parameters participating in a same-screen session according to predetermined calculation rules to obtain setting parameters of the target web page instance and browser instance;

[0082] The operating environment parameter setting module is configured to set the target web page instance based on the setting parameters of the target web page instance, and to set the browser instance based on the setting parameters of the browser instance.

[0083] In a possible implementation of the present application, the browsing operation execution unit further includes a webpage content image acquisition module, and the client further includes a graffiti processing module, wherein:

[0084] The webpage content image acquisition module is configured to generate image data or canvas data based on the current page content of the target webpage instance after receiving the instruction to switch to the graffiti working mode;

[0085] The graffiti processing module is configured to receive the image data or canvas data generated by the webpage content image acquisition module, display the image data on the client or perform drawing based on the canvas data to generate graffiti background content.

[0086] In summary, the implementation principle of the present invention is: events are collected on the client side, executed by the server side and the results are returned to the client side, and the client side renders the final results to achieve the same screen. Therefore, the present invention realizes the complete separation of the execution environment and the browsing operation environment, and has at least the following beneficial effects: constructing the execution environment of the web page on the server side, which is convenient for the unified execution of web page operations and the control of the same screen process, and avoids confusion in program operation; displaying the same screen results and collecting operation events on the client side is convenient for protecting the privacy of the client; based on the flexibility of the server-side processing, it can realize fast same screen for any web page, without the need to modify the same screen web page, and it can be used immediately, which greatly reduces the implementation cost; based on the unified management of the same screen communication on the server side, it solves the problems of page jump interruption, audio and video communication interruption, graffiti dislocation and other problems in the prior art. It can be seen that the web page same screen method and system provided by this application has low implementation cost, high efficiency, and has improved accuracy, applicability and user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0087] Figure 1 A schematic diagram of a flow chart of a web page same-screen method provided in an embodiment of the present application;

[0088] Figure 2 A schematic diagram of a method for sharing web pages on the same screen provided by an embodiment of the present application;

[0089] Figure 3 A schematic diagram of the interactive process of a web page same-screen method provided in an embodiment of the present application;

[0090] Figure 4 A schematic diagram of the interactive process of a web page same-screen method provided in an embodiment of the present application;

[0091] Figure 5 A schematic diagram of a process for processing web page jumps in a web page same-screen method provided in an embodiment of the present application;

[0092] Figure 6 A schematic diagram of a process flow for recording data in a web page on-screen method provided in an embodiment of the present application;

[0093] Figure 7 A schematic diagram of the interactive process of a web page same-screen method provided in an embodiment of the present application;

[0094] Figure 8 A schematic diagram of the Canvas synchronization process of a web page on-screen method provided in an embodiment of the present application;

[0095] Figure 9 A schematic diagram of the interactive process of a web page same-screen method provided in an embodiment of the present application;

[0096] Figure 10 A schematic diagram of the interactive process of a web page same-screen method provided in an embodiment of the present application;

[0097] Figure 11 A schematic diagram of the architecture of a web page sharing system provided in an embodiment of the present application;

[0098] Figure 12 A schematic diagram of the architecture of a web page same-screen system provided in an embodiment of the present application. DETAILED DESCRIPTION

[0099] The implementation section of this application is only used to explain the application but is not intended to limit the application.

[0100] The following describes the embodiments of the present application in conjunction with the accompanying drawings. Obviously, the embodiments described are only part of the embodiments of the present application, rather than all the embodiments. Those skilled in the art will appreciate that with the development of technology and the emergence of new scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0101] Below, some terms in this application are explained to facilitate understanding by those skilled in the art.

[0102] 1) Target web page refers to the web page used for same-screen communication, that is, the user has the need to communicate on the same screen with a certain web page or URL, and uses the method provided in this application to achieve same-screen communication, then the web page or URL is defined as the target web page.

[0103] 2) Target web page instance refers to the target web page loaded and run in the server-side browser instance. For example, if N groups of users communicate on the same screen for a target web page, N browser instances need to be created on the server side, and the target web page needs to be loaded separately, thereby generating N target web page instances at the same time.

[0104] 3) On-screen webpages and clients: Users interact with a target webpage on the same screen by accessing a link to the same-screen webpage on a client. This interaction is achieved by accessing the same-screen webpage link from a client. The same-screen webpage has a different link from the target webpage, but when it runs, it loads the target webpage's display content in real time to achieve the same-screen effect. The client accessing the same-screen webpage is called the client.

[0105] 4) Same-screen service: performs same-screen business logic calculations. During the same-screen process, it provides same-screen business-related data communication and instruction transmission services for the same-screen web pages and target web page instances running on the client.

[0106] 5) "And / or" describes the association relationship between related objects and indicates that there can be three possible relationships. For example, "A and / or B" can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.

[0107] This application provides a web page same-screen method, the implementation principle is: operate in the client browser, execute in the server browser, realize the separation of web page display and execution, combine the client browser, server browser instance and same-screen service instance, logically form a complete browser, and provide multiple clients with same-screen communication. Figure 2The structural diagram of a web page same-screen method provided by an embodiment of the present application is shown, in which a same-screen creation request processing unit receives a same-screen creation request, and the same-screen creation request requires the creation of a same-screen communication based on a certain web page or URL, which at least contains the web page or URL information of the target web page. The same-screen creation request processing unit processes the same-screen creation request, including: creating a server-side browser instance 1 to run the target web page; at the same time, creating a corresponding same-screen service instance 1 to provide the same-screen service; returning the corresponding same-screen web page link (including the URL and / or parameters) for the client to access. Through the above steps, the client accesses the same-screen web page and performs same-screen communication on the target web page, wherein the browser instance 1 is the actual execution environment of the target web page, and the clients A, B, etc. are the content browsing and operation event collection environments of the target web page, thereby achieving the same-screen communication effect of actually executing the web page in the browser instance 1, and collecting operation events and displaying the same-screen results on the clients A and B.

[0108] The following, combined Figure 1 The flowchart shown in the figure illustrates a possible implementation step of this application on the server side. The example steps are as follows:

[0109] S101, creating a browser instance on the server, loading a target web page, and generating a target web page instance;

[0110] S102, injecting a first script into the target web page instance, instantiating a corresponding same-screen service instance on the server side, and starting page monitoring on the target web page instance;

[0111] S103, after the target web page instance content is loaded or a change in the target web page instance content is detected, serializing the target web page instance DOM to generate serialized DOM data and sending it to the client, or rendering the target web page instance display content on a Canvas and obtaining Canvas data and sending it to the client, or generating image data based on the target web page instance display content and sending it to the client;

[0112] S104: The same-screen service instance receives the web page operation event sent by the client and executes the web page operation contained therein.

[0113] As described above, S101 creates a browser instance on the server side, such as instantiating a Tab on the server side browser, and the browser instance provides an execution environment for the target web page. In S102, the first script includes but is not limited to starting page monitoring, DOM serialization processing and / or first Canvas drawing and obtaining Canvas data and / or generating image data, and executing web page operation functions based on received web page operation events. The corresponding instantiation of the same-screen service on the server side is equivalent to creating a same-screen service instance. S103 collects same-screen content data in the execution environment, including but not limited to: serialized DOM data, drawing web page content with the help of Canvas and obtaining Canvas data, collecting image data generated by displaying the content of the target web page instance, etc. The Canvas data includes drawing data generated based on the drawing of the first Canvas or generating image data based on the display of the first Canvas. Optionally, the display screen of the browser instance is obtained on the server side to generate image data and send it as same-screen data. The client accesses the same-screen web page link to achieve same-screen browsing and operation of the content of the target web page instance. After receiving the client's operation, step S104 is actually executed in the browser instance.

[0114] In the embodiment of the present application, the browser instance runs the target web page, the client browses the same-screen web page, and the same-screen service instance provides same-screen data and control services for both. Corresponding to a same-screen creation request, the browser instance runs the target web page to generate a target web page instance. For each target web page instance, there is a same-screen service instance and a same-screen web page link corresponding to it. The data sending and receiving are constrained by the corresponding relationship. It should be noted that the above implementation order of creating a browser instance and creating a same-screen service instance is not particularly limited. In the specific implementation, they can be executed simultaneously, or they can be executed according to a preset order. It should also be noted that in the embodiment of the present application, multiple same-screen exchanges can be created for a web page or URL, but for each same-screen exchange, a browser instance must be created for it, with an independent execution environment; and an independent same-screen service instance is created for each same-screen exchange; the same-screen web page link corresponding to each same-screen exchange is different, and is distinguished by different URLs or parameters.

[0115] The following, combined Figure 3The interactive flow diagram shown in the figure illustrates a possible implementation step of the web page same-screen method provided by the present application. As shown in the figure, the server includes: a creation unit for receiving and processing a same-screen communication creation request; a browsing operation execution unit for creating a browser instance to run the target web page, that is, the actual execution environment of the web page in the same-screen communication; a same-screen service unit for providing same-screen communication data and control services for the target web page instance and the same-screen web page, that is, the same-screen communication service unit; in addition, the example includes a client: client 1, for browsing and operating the same-screen web page, that is, the browsing and operation environment of the web page in the same-screen communication. The main steps are as follows:

[0116] In the same-screen request processing unit:

[0117] S301, receiving a same-screen creation request;

[0118] S302, processing the same-screen creation request, creating or configuring the same-screen webpage;

[0119] S303, sending the target web page URL, browser instance creation parameters, etc.;

[0120] S304, sending the same-screen service instance creation parameters, etc.;

[0121] S305, sending a web page link on the same screen;

[0122] In the browsing operation execution unit:

[0123] S306, creating a browser instance and instantiating a Tab;

[0124] S307, loading the target web page;

[0125] S308, injecting the first script;

[0126] S309, start monitoring;

[0127] S310, after loading is completed, the target webpage instance DOM is serialized to generate serialized DOM data, and on-screen data is generated based on the serialized DOM data;

[0128] In the same screen service unit:

[0129] S311, creating a same-screen service instance;

[0130] On client 1:

[0131] S312, accessing a web page on the same screen;

[0132] S313, running the second script;

[0133] In the browsing operation execution unit:

[0134] S314, sending same-screen data;

[0135] In the same screen service unit:

[0136] S315, sending same-screen data;

[0137] On client 1:

[0138] S316, parsing the serialized DOM data contained in the same-screen data, and updating the DOM based on the serialized DOM data;

[0139] S317, collecting web page operation events, not executing, and generating same-screen data;

[0140] 318, send same-screen data

[0141] In the same screen service unit:

[0142] S319, sending same-screen data;

[0143] In the browsing operation execution unit:

[0144] S320, receiving a webpage operation event and executing the webpage operation;

[0145] S321, monitoring a change in web page content, serializing the changed DOM nodes to generate serialized DOM data, and generating on-screen data based on the serialized DOM data;

[0146] S322, sending same-screen data;

[0147] In the same screen service unit:

[0148] S323, sending same-screen data;

[0149] S324, parsing the DOM string data contained in the same-screen data, and replacing the DOM based on the DOM string data.

[0150] In the above steps, S301 receives a same-screen creation request, and the request includes at least one of a web page file, a URL, a file that can be converted into a web page, and image information for creating same-screen communication. Optionally, in the same-screen creation request, the same-screen communication is created based on a file or image that can be converted into a web page, then in step S302, it should also include converting the file, image, etc. in the creation request into a web page file, generating a target web page, and storing and allocating the corresponding URL, etc. The purpose of S302 processing the same-screen creation request is to configure the execution environment and service environment of this same-screen communication, and to generate a same-screen web page link for the client to browse. Therefore, after the S302 processing is completed, S303 is executed to send data to the browsing operation execution unit, S304 is executed to send data to the same-screen service unit, and S305 is executed to send the same-screen web page link to the client. It should be noted that the sending of data in this application includes a method of first receiving a request and then returning data.

[0151] In a possible implementation method combined with the above embodiments, the execution environment, service environment, and same-screen link of the same-screen communication are configured in the same-screen service unit. The main difference is that after S302 processes the request to create a same-screen web page, S303 and S305 are not executed, and only S304 is executed to send the same-screen service instance creation parameters and other data to the same-screen service unit. In the same-screen service unit, S311 is executed to generate a same-screen service instance, and the browser instance configuration data generated for this same-screen communication is sent to the browsing operation execution unit, and the same-screen web page link is generated and returned to the client.

[0152] It should be noted that the above two methods of configuring the execution environment, service environment and generating the same-screen web page link are only used to illustrate the implementation principle of this application. There are also many transformations and combinations in the specific implementation, or some functions in some modules can be implemented or replaced in other modules. For example, in the above step S308, the first script is injected to send a connection request to the same-screen service unit for the first time, and the connection request information carries the same-screen service instance creation parameter information. The same-screen service unit creates a same-screen service instance based on the connection request information and starts the service.

[0153] After the execution environment, service environment, and browsing operation environment of a same-screen communication request related to the above embodiments are configured, in the browsing operation execution unit, steps S306-S310 are executed to create a browser instance and load the target web page and other operations. Among them, the first script injected by S308 includes but is not limited to establishing a communication connection with the same-screen service instance, starting page monitoring, DOM serialization processing, and restoring the execution of web page operations. After this step is executed, a communication connection is established between the target web page instance and the corresponding same-screen service instance. S309 starts page monitoring, and the monitoring is used to monitor changes in the web page content in the target web page instance, including DOM changes, etc. In addition, the execution order of S309 and S310 is not particularly limited. Based on the implementation principle of the present application, the execution environment is used to generate the execution result of the target web page, and the execution result includes two modes: DOM data and web page display image. Therefore, optionally, S310 can be replaced by: executing drawing on the target web page content on the first Canvas, obtaining Canvas data based on the first Canvas, and using the Canvas data as the same-screen data, wherein the Canvas data includes drawing data generated based on the drawing of the first Canvas or image data generated based on the display content of the first Canvas, such as generating Base64 data based on the current page display content, drawing instruction data for drawing the current page display content on the Canvas, etc. Optionally, S310 can also be replaced by: generating image data based on the display content of the target web page instance, and using the image data as the same-screen data, wherein the image data includes but is not limited to image files, Base64 data, etc.

[0154] Combined with the relevant steps of the above embodiments in the execution environment, the advantages of the embodiments of the present application are explained. Compared with the traditional web page same-screen technology, the web page itself needs to be modified and deployed, or a plug-in needs to be installed in the operating environment (client browser), but the above two methods will incur implementation costs or increase the difficulty of user use. In this application, the server-side control of the web page execution environment is adopted, so the flexibility and controllability are greatly enhanced. For example, creating a browser instance on the server side, injecting the first script into the page, refreshing the page, controlling the jump page, etc., all benefit from this implementation method.

[0155] In the above embodiment, the same-screen service instance in the same-screen service unit receives the same-screen data sent by the server-side browser instance and sends it to the client participating in the same-screen. Client 1 loads the same-screen webpage and runs the second script through S312 and S313. The second script includes but is not limited to establishing communication with the same-screen service instance, receiving the same-screen data, processing the same-screen data, collecting webpage operation events, and sending the webpage operation events.

[0156] After the client 1 receives the serialized DOM data, it parses the serialized DOM data and updates the same-screen web page DOM to obtain a display effect consistent with the target web page instance. Optionally, after the client 1 receives the Canvas data, based on the type of the Canvas data, it displays the picture data in the image element, or displays the picture in the Canvas element or performs corresponding drawing, and also obtains a display effect consistent with the target web page instance. Optionally, after the client 1 receives the picture data, it displays the picture based on the picture data. It should be noted that the above embodiment uses a client (i.e., client 1) as an example to illustrate the implementation principle of the present application. In a specific implementation scenario, there are one or more clients similar to client 1.

[0157] After a user performs a webpage operation on client 1, S317 collects the webpage operation event but does not execute it on the client. Specifically, the client monitors operation events, including but not limited to touch, click, input, change, and scroll, and then directly sends the detected webpage operation event to the same-screen service unit without performing any execution processing on the client. This processing method in a browsing operation environment embodies the principle of separating the execution and display (and operation) of webpages in this application.

[0158] The web page operation events collected in step S317 are eventually sent directly or indirectly to the target web page execution environment, and are actually executed in S320. Then S321 monitors and obtains the execution results, generates same-screen data and sends it to the web page same-screen client. The client executes S324 to obtain a same-screen effect consistent with the target web page instance displayed on the execution end. Preferably, S321 serializes the DOM nodes that have changed in the target web page instance, generates the serialized DOM data as incremental data, and sends the incremental data as the same-screen data. Optionally, S321 generates a differential image for the differential part between the two changes based on the changes in the first Canvas, and generates the differential image data as the same-screen data. Optionally, S321 generates differential image data based on the part of the target web page instance display content that has changed, and generates the differential image data as the same-screen data.

[0159] In a possible implementation, in the same-screen request processing unit or the same-screen service unit, a method for generating a same-screen webpage link includes:

[0160] Method 1: configure a pre-made web page as a same-screen web page, and the web page contains the second script. In this method, the same-screen web page can correspond to multiple target web pages, or multiple target web page instances provide same-screen client access and browsing services. Therefore, when processing the same-screen creation request, a correspondence between the target web page instance and the same-screen web page is established, and parameter information for connecting to the same-screen web page is generated and returned to generate a same-screen web page link. When the client accesses the same-screen web page link, the link must contain the parameter information returned by the same-screen creation request, such as UUID, and the second script calculates and identifies the corresponding target web page instance based on the parameter information;

[0161] Method 2: Create a new webpage on the server and add the second script, along with configuration information connecting to the corresponding on-screen service instance, to generate a dedicated on-screen webpage. This dedicated on-screen webpage corresponds to the target webpage or target webpage instance and provides services. In this method, a single on-screen webpage can provide on-screen client access and browsing services for a single on-screen exchange, without requiring parameter information when the client connects. Alternatively, a single on-screen webpage can provide on-screen client access and browsing services for a single target webpage, requiring the client to carry the on-screen exchange ID or similar information when connecting.

[0162] In one possible implementation, the same-screen creation request also includes room ID information, which is used by the server to create or allocate a corresponding same-screen service instance for a target web page and a room ID combination, as well as to create a browser instance. The same-screen service instance provides same-screen services for clients accessing the same target web page and the same room ID. The browser instance is created on the server corresponding to a target web page and a room ID. The browser instance loads the target web page and generates a target web page instance. After the target web page instance generates same-screen data, it is sent to the client corresponding to the room ID. The web page operation event sent by the client corresponding to the room ID is received and executed in the corresponding target web page instance.

[0163] In one possible implementation, a browser instance is created on the server side. The browser includes, but is not limited to, a browser engine, a browser created in a program, a headless browser, a browser without a graphical interface, etc. Preferably, the browser is a headless browser, which helps conserve server computing resources and increase program control flexibility.

[0164] In one possible implementation, please refer to Figure 4The flowchart described above refers to the client's browser size, resolution, and other information to calculate the rendering size of the target web page instance so that the rendering result of the target web page instance in the browser instance can be correctly displayed in each participating client. The detailed steps are as follows:

[0165] On client 1:

[0166] S401, collecting browser parameter information, such as viewport size;

[0167] S402, sending the same-screen client parameter data;

[0168] In the same screen service unit:

[0169] S403, performing calculations based on the client browser parameters according to predetermined calculation rules to obtain the target web page instance web page size setting parameters;

[0170] S404, sending the target web page instance web page size setting parameters;

[0171] In the browse operation execution unit:

[0172] S405, setting the target web page instance size and performing rendering;

[0173] S406, after loading is completed, the target web page instance DOM is serialized to generate serialized DOM data, and the same-screen data is generated based on the serialized DOM data;

[0174] S407, sending same-screen data;

[0175] In the same screen service unit:

[0176] S408, sending same-screen data;

[0177] On client 1:

[0178] S409: Parse the serialized DOM data contained in the same-screen data, and update the DOM based on it.

[0179] In a specific implementation scenario, there are many differences among the clients participating in the same screen, and it is difficult to ensure that the same screen results generated on the target web page can be normally compatible or displayed normally on all clients. For example, the same screen content display is incomplete due to the target web page instance size being too large and the client size being too small, resulting in a very poor user experience. In the above embodiment, taking the page size as an example, the implementation principle of the present application for calculating and setting the parameters of the target web page instance based on the parameters of the clients participating in the same screen is explained. In S403, the predetermined calculation rule is such as: calculating the minimum displayable device size, the purpose of which is to ensure that all the same screen clients can fully display the target web page instance content. In a possible implementation scenario, it also includes that the server receives a disconnection request sent by the client, and when it is confirmed that the client is not the last client, according to the browser parameter information of the remaining connected clients, based on the client browser parameters, performs calculations according to the predetermined calculation rules, obtains the target rendering size of the target web page instance, and sets the web page size or browser viewport size of the target web page instance based on the target rendering size.

[0180] In the above embodiments, the client browser parameter information includes but is not limited to resolution, browser viewport size, web page size, etc. The above embodiments take setting the web page size as an example to illustrate the implementation principle of this application to set the server browser instance and target web page instance parameters based on client parameters to improve the compatibility of client devices in same-screen communication. It should also be noted that the steps of the above embodiments only illustrate the principle of improving device compatibility of this application, and do not limit the execution order and functions of the steps therein. For example, in S406 and S409, the method of collecting and processing same-screen data can replace the serialized DOM data with Canvas data, image data, etc.

[0181] The beneficial effects achieved by the above embodiments are compatibility with the differences in same-screen client browsers and devices, optimization of the same-screen effect, and improvement of user experience.

[0182] In a possible implementation, after the target web page instance in the server browser jumps to the web page, please refer to Figure 5 The main steps are as follows:

[0183] S501, monitoring or determining that a target web page instance in the server browser instance has jumped to a web page;

[0184] S502, taking the redirected webpage as a new target webpage, loading the new target webpage in the browser, and generating a new target webpage instance;

[0185] S503, injecting the first script into the new target web page instance;

[0186] S504, after the new target web page instance is loaded, serialize the new target web page instance DOM to generate serialized DOM data and send it to the client, or draw the new target web page instance display content on Canvas and obtain Canvas data and send it to the client, or generate image data based on the new target web page instance display content and send it to the client.

[0187] In the above implementation, S501 monitors or judges whether a web page jump occurs in the target web page instance in the server-side browser instance. The monitoring includes but is not limited to monitoring through the server, monitoring in the target web page instance, monitoring in the same-screen web page, monitoring in the browser instance, etc. The judgment includes but is not limited to judging whether the corresponding target web page instance has changed in the same-screen service. Optionally, when a web page jump occurs in the target web page instance, a corresponding relationship is set between the current same-screen service instance and the new target web page instance and services are continued to be provided to the new target web page instance. Optionally, a new same-screen service instance is created based on the new target web page instance, and the new same-screen service instance provides same-screen services corresponding to the new target web page instance. Optionally, the same-screen service instance corresponding to the target web page instance before the jump is retained, and the association relationship between the same-screen service instances corresponding to the target web page instances before and after the jump is recorded. S504 and subsequent steps are the same as Figure 3 Example S310 and subsequent steps are basically the same and will not be repeated here.

[0188] In one possible implementation, after a web page jump occurs in the target web page instance in the server-side browser instance, the same-screen service instance corresponding to the original target web page instance is set to continue providing same-screen service for the new target web page instance, so the same-screen web page link corresponding to the same-screen service instance remains unchanged. Therefore, when a web page jump occurs in the target web page instance, the same-screen web page accessed by the client will not jump.

[0189] Through the above implementation, the audio and video communication service is included in the same-screen web page. When a web page jump occurs in the target web page instance, the audio and video communication still maintains a continuous communication state.

[0190] In a possible implementation, the same-screen process is recorded and the same-screen process recording data is generated, such as Figure 6 The main steps are as follows:

[0191] S601, collecting the same-screen data and corresponding time information, and / or collecting the webpage operation events received from the client and corresponding time information;

[0192] S602: Structured storage to generate same-screen process recording data.

[0193] In the above implementation, the same-screen process recording data is used to store and replay the communication process. According to the time information, the same-screen data is received and processed in the same-screen webpage, and the same-screen interaction process playback is performed. Optionally, the above S601 and S602 steps are executed in the browsing operation execution unit to generate the same-screen process recording data. Optionally, the above S601 and S602 steps are executed in the same-screen service unit to generate the same-screen process recording data. Optionally, the second script is used to create a recording module on the client, and the above S601 and S602 steps are executed to generate the same-screen process recording data. Optionally, a browser instance simulating a client is created on the server side, and it participates in the same-screen communication as a client. A script is injected into the client to create a recording module, and the above S601 and S602 steps are executed to generate the same-screen process recording data. Optionally, the data collected in S601 also includes one or more of audio data, video data, and graffiti data.

[0194] In one possible implementation, the browser running the target webpage or accessing the same-screen webpage is used to record the same-screen communication process in real time by recording a browser video, generating a video recording of the same-screen process. In this implementation, the browser running the target webpage or accessing the same-screen webpage includes but is not limited to a client browser accessing the same-screen webpage, a server-side browser instance running the target webpage, and a browser instance created on the server to simulate a client. The recorded browser video includes but is not limited to the browser's display screen, the audio played by the browser, and the audio data received by the browser.

[0195] In one possible implementation, Figure 7 The interactive flow chart shown in FIG. 1 shows an example of a method for processing graffiti instructions. The specific steps are as follows:

[0196] In the browse operation execution unit:

[0197] S701: Receive a command to switch to Tuya working mode;

[0198] S702, generating image data based on the current page content of the target web page instance, or performing drawing on the web page content Canvas of the target web page instance and obtaining Canvas data;

[0199] S703, sending same-screen data;

[0200] In the same screen service unit:

[0201] S704, sending same-screen data;

[0202] On client 1:

[0203] S705, displaying the image data or performing drawing based on the Canvas data to generate graffiti background content;

[0204] S706, collecting graffiti operations;

[0205] S707: Send tutu operation data.

[0206] In the above implementation method, when entering the graffiti working mode during the same-screen communication process, the display content of the target web page instance is first generated as a picture or drawing data and sent to the client participating in the same screen, thereby ensuring that after entering the graffiti working mode, the screen displayed by all clients is consistent with the display screen in the browser instance, and graffiti operations are performed based on the screen, and the graffiti content is positioned with reference to avoid graffiti misplacement problems caused by device differences.

[0207] In one possible implementation, Figure 8 As shown in the flowchart, this application also provides a method for processing a target web page containing a Canvas element, and the specific steps are as follows:

[0208] S801, monitor the Canvas property changes or its Context event execution;

[0209] S802, cache Canvas data;

[0210] S803, start clock monitoring;

[0211] S804, the clock cycle ends;

[0212] S805, determining whether the Canvas element change is received;

[0213] When receiving the Canvas element change:

[0214] S806, cache the current Canvas data;

[0215] When no Canvas element changes are received:

[0216] S807: Send the same-screen data and clear the cached data.

[0217] The following is an example of the above embodiment: a target web page loaded in a server browser instance includes a Canvas element, and the display content of the Canvas element is a chart. According to the implementation principle of the present application, the Canvas element generates a large number of web page operation events (drawing operation events of the display content in the Canvas during the generation process, such as drawing a chart) during the loading process of the target web page, and correspondingly generates multiple data communications. Figure 3S314 and S315 in the above embodiment may cause communication congestion. According to the method provided in the above embodiment, by caching Canvas data and executing communication after Canvas drawing is completed, the network congestion problem caused by too many Canvas events when loading a web page containing Canvas elements and with a large number of Canvas drawing operations is avoided.

[0218] In one possible implementation, Figure 9 The interactive process diagram shown in FIG. 1 is a flow chart showing the interaction process. The present application also provides a method for processing a web page event of a specified type during the web page on-screen process. The specific steps are as follows:

[0219] On client 1:

[0220] S901, monitoring a web page operation event of a specified type;

[0221] S902, executing the webpage operation event;

[0222] S903, sending the webpage operation event to other clients on the same screen;

[0223] S905, sending a webpage operation event to the same-screen service unit;

[0224] On client 2:

[0225] S904, receiving a web page operation event and executing the web page operation;

[0226] In the same screen service unit:

[0227] S906, sending a webpage operation event to the browsing operation execution unit;

[0228] In the browse operation execution unit:

[0229] S907, receiving a web page operation event, executing the web page operation, and collecting execution result data;

[0230] S908, sending execution result data to the same-screen service unit;

[0231] In the same screen service unit:

[0232] S909, S910, sending execution result data to client 1 and other clients;

[0233] On client 1:

[0234] S911, performing a correction operation based on the received execution result data;

[0235] On client 2:

[0236] S912: Perform a correction operation based on the received execution result data.

[0237] The above embodiments are aimed at some specific types of web page events such as touch, scroll, etc. Such web page events usually require a quick response. For example, if they are executed in the target web page instance and then the execution results are synchronized to the client, a delay will be generated, resulting in a poor user experience. Therefore, this embodiment is executed on the client while being synchronously sent to other clients for execution, thereby reducing the delay to improve the user experience. In addition, the execution results of the target web page instance on the server are referenced for correction to ensure the correctness of the final result on the same screen.

[0238] In one possible implementation, the client's web page operation event is processed if the local system or browser is called, such as Figure 10 The interactive process diagram shown in the figure shows that on client 1, step S1001 is executed. When it is determined that the event triggered by the web page operation is an operation that calls the local system or browser, the event is simultaneously transmitted to the local system or browser; S1005, the local system or browser performs processing, and after the processing is completed, a local call completion event is triggered. In the completion event, the web page operation data is sent to the server. In the target web page instance, step S1008 is executed. Based on the client execution result, the web page operation event is processed to simulate the execution effect. Optionally, after client 1 executes S1001, a notification is sent to the target web page instance, and S1004 is executed in the target web page instance to display a prompt or other processing executed on the client. Let's take an example to illustrate the above implementation steps. For example, if you operate the input element on the client to input text content, when the input focus is on the input element, it triggers the call of the local input method component. At this time, a notification is sent to the target web page instance, so that the target web page instance generates a related prompt "Client 1 is inputting" and displays it on the same screen to other clients on the same screen. When Client 1 completes the input of the input element, its change event is triggered. When the event is collected and sent to the server, the operation result data is also sent at the same time. The operation result data in this example is the string data entered this time. The above implementation method avoids confusion caused by all clients performing local operations when the event triggered by the web page operation is to call the local system or browser operation. It is also beneficial to protect client privacy and obtain consistent on-screen execution results.

[0239] In a possible implementation, the first script and the second script in the embodiment of the present application also include the function of establishing a real-time voice conversation or video conversation between the target web page instance and the same-screen web page accessed by the client participating in the same-screen communication, so as to realize audio and video communication during the same-screen communication process.

[0240] In one possible implementation, the first script in the embodiment of the present application further includes creating a graffiti Canvas in the target web page instance, collecting graffiti data, and sending the graffiti data as on-screen data; receiving the on-screen data and parsing the graffiti data contained therein, and performing drawing on the graffiti Canvas. The second script in the embodiment of the present application further includes creating a graffiti Canvas in the on-screen web page browsed by the client, collecting graffiti data, and sending the graffiti data as on-screen data; receiving the on-screen data and parsing the graffiti data contained therein, and performing drawing on the graffiti Canvas.

[0241] In a possible implementation, the same-screen process recording data is generated in the same-screen service instance or the server-side browser instance, and also includes recording one or more of the audio data, video data, and graffiti data in the same-screen communication process to fully record the communication process.

[0242] Optionally, data communication is achieved between the server browser instance, the client, and the same-screen service instance using a WebSocket communication method. Optionally, communication is achieved between the server browser instance and the client participating in the same-screen communication using a WebRTC (Web Real-Time Communication) API.

[0243] In one possible implementation, a message digest is generated based on the recorded data of the same-screen process, and the message digest is saved by the creator, participants, etc. of the same-screen communication, and is used to verify the recorded data of the same-screen communication process and to store evidence of the same-screen communication process. Alternatively, a message digest is generated based on the data after each update of the recorded data of the same-screen process, and the message digest is saved by the creator, participants, etc. of the same-screen communication, and is used to verify the recorded data of each communication state and to store evidence of the same-screen communication state. In the above, the algorithm for generating the message digest is such as MD5, etc.

[0244] In a possible implementation, the server browser instance is replaced by a client browser that participates in the same-screen communication.

[0245] In one possible implementation, an inline frame containing the target webpage content is created within the target webpage instance. Correspondingly, an inline frame containing the same-screen webpage content is created within the same-screen webpage. For example, the webpage content to be shared is run within an IFrame, while the audio, video, and same-screen operations are run within a MainFrame. This embodiment provides the method for maintaining continuity and uninterrupted same-screen services and audio and video when the target webpage redirects.

[0246] Based on the web page same screen method proposed in the embodiment of the present application, the embodiment of the present application also provides a web page same screen system, please refer to Figure 11 , including server S and client C, where:

[0247] The server S includes a creation unit S01, a browsing operation execution unit S02, and a same-screen service unit S03, as follows:

[0248] The creation unit S01 is configured to receive a same-screen creation request and create a browser instance and a same-screen service instance based on the same-screen creation request, and includes a same-screen request processing module S011, wherein:

[0249] The same-screen request processing module S011 is configured to receive a same-screen creation request and, based on the target webpage URL information contained therein, notify the browsing operation execution unit S02 to instantiate a tab in the server-side browser, load the target webpage to generate a target webpage instance, and notify the same-screen service unit S03 to create a corresponding same-screen service instance;

[0250] The browsing operation execution unit S02 is used to manage the server browser, create a browser instance S021 on the server, and execute target web page browsing and operations in the browser instance S021, including a loading module S0212, a communication module S0211, a DOM data acquisition module S0213-1 / Canvas content processing module S0213-2 / image data acquisition module S0213-3, and a web page operation event execution module S0214, wherein:

[0251] The loading module S0212 is configured to load the target webpage to generate a target webpage instance, inject a first script into the target webpage instance, and start page monitoring;

[0252] The communication module S0211 is configured to establish communication with the corresponding same-screen service instance S031;

[0253] The DOM data acquisition module S0213-1 is configured to, after monitoring changes to the content of the target web page instance, serialize the DOM of the target web page instance, generate serialized DOM data, and send it to the on-screen service instance S031; the Canvas content processing module S0213-2 is configured to draw the web page content of the target web page instance on a first Canvas, generate Canvas data based on the first Canvas, and send it to the on-screen service instance S031; the image data acquisition module S0213-3 is configured to generate image data based on the display content of the target web page instance, and send the image data as on-screen data to the on-screen service instance S031;

[0254] The webpage operation event execution module S0214 is configured to execute the webpage operation event received from the client;

[0255] The same-screen service unit S03 is used to create and manage a same-screen service instance S031. The same-screen service instance corresponds to the target webpage instance. The same-screen service instance provides data communication and control services for the client participating in the same-screen session. It includes a same-screen data sending module S0311 and a webpage operation data receiving module S0312, wherein:

[0256] The same-screen data sending module S0311 is configured to send the same-screen data to the client C;

[0257] The webpage operation data receiving module S0312 is configured to receive the webpage operation event data sent by the client C.

[0258] The client C is used to access the same-screen web page link and execute the same-screen display of the corresponding target web page instance, including a communication module C01, a DOM data processing module C02-1 / Canvas data processing module C02-2 / image data display module C02-3, and a web page operation event collection module C03, wherein:

[0259] The communication module C01 is configured to establish communication with the corresponding same-screen service instance S031;

[0260] The DOM data processing module C02-1 is configured to receive the same-screen data sent by the same-screen service instance S031, parse the serialized DOM data contained therein, and update or replace the DOM content of the same-screen client webpage based on the serialized DOM data;

[0261] The Canvas processing module C02-2 is configured to receive the same-screen data sent by the same-screen service instance S031, parse the Canvas data contained therein, create a second Canvas in the same-screen client webpage, and display an image or perform drawing on the second Canvas based on the Canvas data;

[0262] The picture data display module C02-3 is configured to receive the same-screen data sent by the same-screen service instance S031, parse the picture data contained therein, and display the picture based on the picture data;

[0263] The web page operation event collection module C03 is configured to monitor web page operation events occurring in the client, not process the events in the client, and send the web page operation events to the same-screen service instance S031.

[0264] In a feasible implementation scenario, the system further includes generating a correspondence between the browser instance S021, the same-screen service instance S031, and the client C, wherein at the same time, the browser instance corresponds to one same-screen service instance, and the same-screen service instance provides a same-screen service of a same-screen webpage corresponding to the target webpage instance to multiple clients;

[0265] It also includes calculating and returning a corresponding same-screen web page link for the target web page instance based on a preset same-screen web page, or creating a web page file corresponding to the target web page instance and adding the second script to generate the same-screen web page, and returning the same-screen web page link.

[0266] In a possible implementation scenario, the browsing operation execution unit S02 or the same-screen service unit S03 or the client C further includes a recording module, which collects and stores the same-screen data and corresponding time information, the web page operation events received from the client and the corresponding time information, and generates same-screen process recording data.

[0267] In a possible implementation scenario, the client C further includes an operating environment information collection module C04, the same-screen service instance S031 further includes an operating environment parameter calculation module S0313, and the browser operation execution unit S02 further includes an operating environment parameter setting module S0215, wherein:

[0268] The operating environment information collection module C04 is configured to collect client browser and / or system parameter information and send it to the server;

[0269] The operating environment parameter calculation module S0313 is configured to perform calculations based on all the client browsers and / or system parameters participating in a same-screen session according to predetermined calculation rules to obtain the setting parameters of the target web page instance and browser instance S021;

[0270] The operating environment parameter setting module S0215 is configured to set the target web page instance based on the setting parameters of the target web page instance, and to set the browser instance based on the setting parameters of the browser instance S021.

[0271] In a possible implementation scenario, the browsing operation execution unit S02 further includes a webpage content image acquisition module S0216, and the client further includes a graffiti processing module C05, wherein:

[0272] The web page content image acquisition module S0216 is configured to generate image data or canvas data based on the current page content of the target web page instance after receiving the instruction to switch to the graffiti working mode;

[0273] The graffiti processing module C05 is configured to receive the image data or canvas data generated by the webpage content image acquisition module, display the image data on the client C or perform drawing based on the canvas data to generate graffiti background content.

[0274] It should also be noted that the server in this application refers to a logical server. In a specific implementation scenario, there may be one or more actual servers that logically implement the server functions mentioned in this application. Similarly, the various modules in this application are limited to logical functional divisions. In a specific implementation scenario, the functions of some modules may be combined or split to generate new modules, or existing modules may be replaced with modules with similar functions.

[0275] According to the web page screen sharing method and system provided by the above application, events are collected on the client, executed by the server, and then the full or differential data is returned to the client, and the client renders the results. The ultimate beneficial effect is that any web page can be used and quickly shared on the screen. The client can enter and participate in the shared screen at any time, and solves the problems existing in the prior art such as page jump interruption, audio and video communication interruption, graffiti misalignment, and inability to record in real time. Therefore, the method provided by this application has low implementation cost and high efficiency, and improves accuracy, applicability, and user experience.

[0276] The above description is merely a specific implementation of the embodiments of the present application, but the scope of protection of the embodiments of the present application is not limited thereto. Any changes or substitutions that can be easily conceived by a person skilled in the art within the technical scope disclosed in the embodiments of the present application should be included in the scope of protection of the embodiments of the present application. Therefore, the scope of protection of the embodiments of the present application should be based on the scope of protection of the claims.

Claims

1. A web page same-screen method, the method is applied to a server, characterized in that: include: Creating a browser instance on the server, loading a target webpage in the browser instance, generating a target webpage instance, injecting a first script into the target webpage instance, and starting page monitoring; After the target web page instance content is loaded or a change in the target web page instance content is detected, serializing the target web page instance DOM to generate serialized DOM data, and sending the serialized DOM data to the client as on-screen data, or rendering the target web page instance display content on a first Canvas, obtaining Canvas data based on the first Canvas, and sending the Canvas data as on-screen data to the client, or generating image data based on the target web page instance display content, and sending the image data as on-screen data to the client; Also includes: Receive the webpage operation event sent by the client, and execute the webpage operation contained therein; Among them, the target web page is a URL or web page file used for the same screen; the first script at least includes starting page monitoring, DOM serialization processing and / or first Canvas drawing and obtaining Canvas data and / or generating image data, and executing web page operation functions based on received web page operation events; the Canvas data includes generating drawing data based on the drawing of the first Canvas or generating image data based on the display content of the first Canvas; the client realizes same-screen browsing and operation of the target web page instance content by accessing the same-screen web page.

2. The web page same-screen method according to claim 1, characterized in that: Also includes: Configuring a pre-made web page as a same-screen web page, or generating the same-screen web page by calculation based on the target web page instance information; generating a same-screen webpage link based on the URL of the same-screen webpage and / or the corresponding relationship between the same-screen webpage and the target webpage instance; Returning the corresponding same-screen web page link to the client according to the target web page instance; The same-screen webpage is pre-set or generated, and each of the same-screen webpages must include a second script, which includes at least the functions of receiving and processing the same-screen data and collecting and sending webpage operation events. Each target webpage instance has a corresponding same-screen webpage link, and the same-screen webpage link includes the same-screen webpage URL and / or parameter information. A pre-made webpage is configured as a same-screen webpage, and the same-screen webpage corresponds to multiple target webpages and provides services. The same-screen webpage link includes parameter information, and the second script calculates and identifies the corresponding target webpage instance based on the parameter information. The same-screen webpage is generated based on the target webpage instance information, and the same-screen webpage corresponds to the target webpage instance and provides services.

3. The web page same-screen method according to claim 1, characterized in that: Also includes: The browser is a headless browser.

4. The web page same-screen method according to claim 2, characterized in that: After the server browser loads the target webpage and injects the first script, the method further includes: Instantiate a corresponding same-screen service instance on the server side; The same-screen service instance receives the same-screen data sent by the server-side browser instance and sends it to the client; the same-screen service instance receives the webpage operation event sent by the client and sends it to the server-side browser instance; Among them, in the target web page instance, the first script is executed to establish a communication connection with the same-screen service instance; in the same-screen web page, the second script is executed to establish a communication connection with the same-screen service instance.

5. The web page same-screen method according to claim 1, characterized in that: Generating same-screen data based on the target webpage instance further includes: The DOM nodes that have changed in the target web page instance are serialized to generate serialized DOM data, and the serialized DOM data is generated as the same-screen data; or, based on the change of the first Canvas, differential image data is generated for the differential part between two changes, and the differential image data is generated as the same-screen data; or, based on the change of the first Canvas, drawing data is generated, and the drawing data is generated as the same-screen data; or, based on the part of the target web page instance display content that has changed, differential image data is generated, and the differential image data is generated as the same-screen data.

6. The web page same-screen method according to claim 1, characterized in that: The server also includes receiving client browser and / or system parameter information sent by the client, performing calculations according to predetermined calculation rules based on the client browser and / or system parameters to obtain setting parameters of the target webpage instance and the browser instance, setting the target webpage instance based on the setting parameters of the target webpage instance, and setting the browser instance based on the setting parameters of the browser instance; It also includes that the server monitors the client disconnection event, and when it is confirmed that the client is not the last client, according to the browser and / or system parameter information of the remaining connected clients, based on the client browser parameters and according to predetermined calculation rules, calculates the setting parameters of the target web page instance and the browser instance, sets the target web page instance based on the setting parameters of the target web page instance, and sets the browser instance based on the setting parameters of the browser instance.

7. The method according to claim 4, characterized in that Also includes: Monitor or determine whether a target web page instance in the server browser instance jumps to a web page; After confirming that the target web page instance has jumped, the jumped web page is used as a new target web page, and the new target web page is loaded in the browser to generate a new target web page instance; The first script is injected into the new target web page instance.

8. The web page same-screen method according to claim 1, characterized in that: include: Storing the same-screen data and corresponding time information, and / or the webpage operation events and corresponding time information received from the client, to generate same-screen process recording data; Among them, the same-screen process recording data is used for the same-screen interaction process playback. According to the time information, the same-screen data is received and processed in the same-screen web page, and / or, the web page operation event is received and the web page operation is simulated to perform the same-screen interaction process playback.

9. The web page same-screen method according to claim 1, characterized in that: include: Receiving an instruction to switch to Tuya working mode in the target webpage instance; Generate image data based on the current page content of the target web page instance, or draw the current page content of the target web page instance on a Canvas and obtain Canvas data; The image data or the Canvas data is generated as same-screen data and sent to the client.

10. The web page same-screen method according to claim 1, characterized in that: Also includes: Monitor the Canvas element in the target web page instance, and when a change is detected, cache the Canvas data of the Canvas element, temporarily suspend communication processing, and start monitoring according to the clock cycle. At the end of the next clock cycle, determine whether the Canvas element change is received. When it is confirmed that the Canvas element change is not received, send the same-screen data and clear the cached data. When it is confirmed that the Canvas element change is received, cache the Canvas data of the current Canvas element.

11. A web page same screen method, the method is applied to a client, characterized in that: include: Access the same-screen webpage link and load the same-screen webpage; Receive the same-screen data sent by the server, parse the serialized DOM data contained therein, and update or replace the DOM content of the same-screen client webpage based on the serialized DOM data, or parse the Canvas data contained therein, create a second Canvas in the same-screen client webpage, and display an image or perform drawing on the second Canvas based on the Canvas data, or parse the image data contained therein and display the image based on the image data; Also includes: monitoring web page operation events occurring in the client, not processing them in the client, and sending the web page operation events to the server; Among them, the same-screen web page link corresponds to the target web page instance; the target web page instance is generated by loading the target web page by the browser instance created in the server; the same-screen web page contains a second script, which at least includes establishing a communication connection with the server, receiving and processing the same-screen data, and collecting and sending the web page operation event functions.

12. The web page same-screen method according to claim 11, characterized in that: When the client switches to Tuya working mode, the following steps are also included: Receiving image data or canvas data generated based on the webpage content in the target webpage instance and sent by the server, displaying the image data on the client or performing drawing based on the canvas data to generate graffiti background content; The graffiti data is collected and drawn using the graffiti background size and position information as a relative reference.

13. The web page same-screen method according to claim 11, characterized in that: Also includes: The client collects browser parameter information and / or system parameter information and sends it to the server. The browser parameter information and / or system parameter information is used by the server to calculate the setting parameters of the target web page instance and the browser instance.

14. The web page same-screen method according to claim 11, characterized in that: Monitoring webpage operation events occurring in the client and sending them to the server also includes: When it is determined that a webpage operation event of a specified type is monitored, the webpage operation event is executed locally and simultaneously sent to the server browser instance and other clients participating in the same screen; Also includes: The webpage operation event of the specified type is received, executed on the client, and a notification of an execution result of the webpage operation event of the specified type is received, and a correction operation is performed based on the notification of the execution result.

15. The web page same-screen method according to claim 11, characterized in that: include: When it is determined that the event triggered by the web page operation is an operation of calling a local system or a local browser, the event is simultaneously transmitted to the local system or the local browser for processing. After the processing is completed, a local call completion event is triggered, and the web page operation event is sent to the server in the local call completion event, wherein the sent web page operation event includes local execution result information.

16. A web page same screen system, characterized in that: The system includes a server and a client. The server includes a creation unit, a same-screen service unit, and a browsing operation execution unit. The creation unit is configured to receive a same-screen creation request and create a browser instance and a same-screen service instance based on the request, including: a same-screen request processing module configured to receive a same-screen creation request and, based on the target webpage URL information contained therein, notify the browsing operation execution unit to instantiate a tab in the server-side browser, load the target webpage to generate a target webpage instance, and notify the same-screen service unit to create a corresponding same-screen service instance; The same-screen service unit is configured to create and manage a same-screen service instance, the same-screen service instance corresponding to the target webpage instance, and the same-screen service instance providing data communication and control services for clients participating in the same-screen service, including: A same-screen data sending module, configured to send same-screen data to the client; A webpage operation data receiving module configured to receive webpage operation event data sent by the client; The browsing operation execution unit is used to manage the server browser, create a browser instance on the server, and execute target web page browsing and operations in the browser instance, including: A loading module configured to load the target webpage to generate a target webpage instance, inject a first script, and start page monitoring; A communication module configured to establish communication with the corresponding same-screen service instance; a DOM data acquisition module configured to, after monitoring a change in the content of the target web page instance, serialize the DOM of the target web page instance and generate serialized DOM data as the same-screen data; A Canvas content processing module configured to draw the webpage content of the target webpage instance on a first Canvas, and generate Canvas data as the same-screen data based on the first Canvas; A picture data acquisition module configured to generate picture data based on the display content of the target web page instance, and use the picture data as the same-screen data; a webpage operation event execution module, configured to execute a webpage operation event received from the client; The client is used to access the same-screen webpage link and execute the same-screen display of the corresponding target webpage instance, including: A communication module configured to establish communication with the corresponding same-screen service instance and receive the same-screen data; a DOM data processing module configured to parse the serialized DOM data contained in the same-screen data, and update or replace the DOM content of the client webpage based on the serialized DOM data; A canvas processing module configured to parse canvas data contained in the same-screen data, create a second canvas in the client webpage, and display an image or perform drawing on the second canvas based on the canvas data; a picture data display module configured to parse the picture data contained in the same-screen data and display pictures based thereon; The web page operation event collection module is configured to monitor web page operation events occurring in the client, not process them in the client, and send the collected web page operation event data to the same-screen service instance based on the collected data.

17. The system according to claim 16, wherein: Also includes: Generate a correspondence between the browser instance, the on-screen service instance, and the client, wherein at the same time, the browser instance corresponds to one on-screen service instance, and the on-screen service instance provides on-screen services of the on-screen webpage corresponding to the target webpage instance to multiple clients; It also includes calculating and returning a corresponding same-screen web page link for the target web page instance based on a preset same-screen web page, or creating a web page file corresponding to the target web page instance and adding a second script to generate the same-screen web page, and returning the same-screen web page link.

18. The system according to claim 16, wherein: The browsing operation execution unit or the same-screen service unit or the client also includes a recording module, which collects and stores the same-screen data and corresponding time information, the web page operation events received from the client and the corresponding time information, and generates same-screen process recording data.

19. The system according to claim 16, wherein: The client further includes an operating environment information collection module, the same-screen service instance further includes an operating environment parameter calculation module, and the browser operation execution unit further includes an operating environment parameter setting module, wherein: The operating environment information collection module is configured to collect client browser and / or system parameter information and send it to the server; The operating environment parameter calculation module is configured to perform calculations based on all the client browsers and / or system parameters participating in a same-screen session according to predetermined calculation rules to obtain setting parameters of the target web page instance and browser instance; The operating environment parameter setting module is configured to set the target web page instance based on the setting parameters of the target web page instance, and to set the browser instance based on the setting parameters of the browser instance.

20. The system according to claim 16, wherein: The browsing operation execution unit further includes a webpage content image acquisition module, and the client further includes a graffiti processing module, wherein: The webpage content image acquisition module is configured to generate image data or canvas data based on the current page content of the target webpage instance after receiving the instruction to switch to the graffiti working mode; The graffiti processing module is configured to receive the image data or canvas data generated by the webpage content image acquisition module, display the image data on the client or perform drawing based on the canvas data to generate graffiti background content.

Images