A web page explanation recording and playing method

Abstract

The application provides a webpage commentary recording and playing method. The recording method comprises the following steps: loading a target webpage, loading a first program comprising a commentary recording function; receiving an instruction to start recording, loading a recording control component, starting webpage content change monitoring and audio and video data collection; after the webpage is loaded or the content change is monitored, collecting webpage data and a time of occurrence to generate first commentary data; collecting audio and video to generate second commentary data; and storing the commentary data. The playing method comprises the following steps: accessing a playing website, loading a second program comprising a commentary playing function; loading a playing component, downloading commentary data, receiving an instruction to start playing, reading the commentary data, and starting a timer; when the time is confirmed to be consistent with the time included in the commentary data or when an event bound with the commentary data is executed, executing the commentary data playing. It can be seen that the method of the application is specifically used for recording and playing commentary for a webpage, does not need to rely on other software, and has smaller data and higher image quality than data generated by ordinary screen recording.

Description

Technical Field

[0001] The embodiments of the present invention relate to the fields of Internet technology and communication, and in particular to a method for recording and playing webpage commentary. Background Technology

[0002] Web pages are essential tools for daily information exchange and business transactions, providing a good user experience and efficient information services in a diversified, innovative, multi-channel, barrier-free, and user-friendly manner. For example, people obtain information, read notices, and understand procedures through government websites; they convey information by sharing web pages; they share Word documents for friends to read via web pages; and they conduct business through online banking. In these scenarios, government personnel need to further explain the key points of notices and procedures to those seeking services; sharers need to explain the content they share; sharing Word documents requires explanation of their content; and online banking services require guidance on operational procedures. However, web pages only contain content specified by the designers. Therefore, in existing implementations, users typically achieve these needs by recording their screen, creating PowerPoint presentations, or creating help documents. Screen recordings generate large files, which are not conducive to network transmission; creating PowerPoint presentations or help documents is time-consuming and laborious, hindering rapid publication.

[0003] Therefore, the above-mentioned problems regarding the explanation of web pages urgently need to be solved using new technologies. Summary of the Invention

[0004] To address the aforementioned problems, this invention provides a method for recording and playing webpage commentary, which aims to quickly generate commentary data for webpages and perform webpage commentary playback based on the commentary data, thereby improving the communication efficiency and user experience of webpages.

[0005] The technical solution provided by this invention is as follows:

[0006] In a first aspect, the present invention provides a method for recording webpage commentary, the method being used to record commentary for a target webpage, the method comprising:

[0007] Load the target webpage, and load a first program for performing the narration recording function, the first program including narration recording control and narration data acquisition function;

[0008] The system receives a start recording command to initiate narration recording and loads the narration recording control component, initiates monitoring of changes in the target webpage content, and initiates audio and / or video data acquisition. The narration recording control component includes one or more of the following: end recording, pause recording, drawing tool, and narration tool.

[0009] After the target webpage content is initially loaded, webpage data is collected to generate first narration data, or webpage data and corresponding time information are collected to generate the first narration data; after a change in the target webpage content is detected, the changed webpage data and the time of occurrence are collected to generate the first narration data; the audio and / or video data is collected to generate second narration data, or the audio and / or video data and corresponding time information are collected to generate second narration data; wherein, the first narration data and the second narration data are two types of narration data;

[0010] Send the explanation data to the server;

[0011] The narration data is stored on the server side. The narration data is used to provide playback data for the narration playback webpage. The narration playback webpage includes a second program for performing the narration playback function. The second program loads the narration data and narration playback controls to play the narration.

[0012] The webpage data includes one or more of the following: webpage HTML data, webpage CSS data, serialized webpage data, Canvas data generated based on the webpage HTML display content, and webpage content image data.

[0013] The above implementation method involves recording webpage data and the process of its changes during the explanation of the webpage to obtain the first explanation data, and recording audio and video to obtain the second explanation data. The two types of data are stored and applied to playback, ultimately achieving the same playback effect as screen recording.

[0014] In one possible implementation, after detecting a change in the target webpage content, the webpage data showing differences before and after the change is collected as the changed webpage data. Alternatively, the nature of the differences is simultaneously collected and added to the first explanatory data. The nature of the differences includes addition, replacement, and deletion. This implementation only records the differentiated portion of the webpage data, significantly reducing the data storage size of the first explanatory data.

[0015] In one possible implementation, the webpage data is serialized to generate serialized webpage data. The first narration data is then generated based on the serialized webpage data. Furthermore, the changed webpage data is serialized, and the changed node information is recorded simultaneously. In this implementation, performing serialization processing facilitates the storage and transmission of webpage data and is beneficial for calculating webpage data during narration playback.

[0016] In one possible implementation, the first narration data is generated based on the webpage data or the serialized webpage data. According to preset keyframe setting rules, after confirming that the data conforms to the preset keyframe setting rules, all webpage data of the target webpage is recorded as the first narration data, and a keyframe identifier is set for the first narration data. The first narration data with the keyframe identifier includes all webpage data at the corresponding time.

[0017] In one possible implementation, after the target webpage receives the command to start recording the narration and starts recording, it starts cursor event listening. After listening to the cursor event, it collects the cursor position information and the time information to generate third narration data, which is another type of narration data.

[0018] In another possible implementation, after the client receives the instruction to start recording commentary, it generates a first recording commentary request and sends it to the server; it receives a webpage for recording commentary or a URL corresponding to the webpage returned by the server, loads the webpage for recording commentary or accesses the URL for recording commentary to load the target webpage content, and the first program starts recording commentary. In this implementation, the webpage for recording commentary is a webpage generated based on the target webpage content and containing the first program. After receiving the recording commentary request from the target webpage, the server obtains the target webpage URL based on the request, further obtains the webpage content of the target webpage, adds the first program to the webpage content of the target webpage, generates the webpage for recording commentary, and returns the webpage for recording commentary, the webpage content, or the corresponding URL.

[0019] In another possible implementation, the webpage narration is recorded on the server side. The main steps include:

[0020] After the client receives the instruction to start recording the narration, it generates a second recording narration request and sends it to the server.

[0021] On the server side, the system receives and processes the second recording and narration request, obtains the target webpage URL based on the second recording and narration request, loads the target webpage in the server browser, loads the first program, starts narration recording and loads the narration recording control component, starts monitoring changes in the target webpage content, collects webpage data to generate the first narration data, loads a screen-to-screen communication program, establishes screen-to-screen communication with the client, collects webpage data to generate synchronous webpage data and sends it to the client.

[0022] On the client side, a screen-sharing communication mechanism is established between the client and the webpage loaded by the server browser; the synchronized webpage data is received and loaded to synchronize the content displayed on the client and the server browser; user operation monitoring is initiated, and user operation data is collected and sent to the server after a user operation event is detected; audio and / or video data collection is initiated, and the audio and / or video data is collected and sent to the server.

[0023] On the server side, the system receives the user operation data, executes the user operation events contained therein, and after detecting a change in the content of the target webpage, collects the changed webpage data and the time of occurrence to generate the first narration data; it also receives the audio and / or video data and generates the second narration data based on it; and stores the narration data.

[0024] The first program includes one or more forms, such as browser plugins and JavaScript code.

[0025] Among the above possible implementation methods, establishing a browsing environment for the target webpage on the server side and establishing screen-to-screen communication between the server browser and the client, receiving user operations in the client browser and executing them on the server side, and recording narration data based on the server side, this implementation method is beneficial for executing recording control on the server side and avoids transmitting a large amount of recording data to the server side, thus reducing the amount of data transmitted over the network.

[0026] Through the above implementation methods, it is possible to record and generate narration data for web pages. The narration data is used to provide playback data for the narration playback web page of the target web page, and to re-narrate the narration process of the target web page.

[0027] Secondly, the present invention also provides a method for playing webpage commentary, for playing webpage commentary, the method comprising:

[0028] Access the commentary playback URL and load a second program for performing the commentary playback function. The second program includes commentary playback control and commentary data loading functions.

[0029] Load the narration playback component and download the narration data. The narration data includes one or more of the following: first narration data, second narration data, and third narration data. The narration playback component includes one or more of the following: playback time display, play, pause, jump, fast forward, rewind, speed adjustment, and stop. The first narration data includes webpage data, the second narration data includes audio and / or video data, and the third narration data includes cursor position data. Each type of narration data may or may not contain time information. When the time information is not included, playback is executed by binding to a specified event by default.

[0030] Upon receiving a playback command, the system initiates narration playback, reads the narration data, starts a timer, and executes narration data playback when one of the following conditions is met: Condition 1, when the timer time is confirmed to match the time information contained in the narration data according to a predetermined time calculation rule; Condition 2, when the event executed by the second program is determined to be a specified event bound in the narration data; wherein, the narration data playback is determined based on the type of the narration data, including: calculating and displaying webpage content based on the first narration data, playing audio and / or video based on the second narration data, and setting the display cursor based on the third narration data;

[0031] The webpage data includes one or more of the following: webpage HTML data, webpage CSS data, serialized webpage data, Canvas data generated based on the webpage HTML display content, and webpage content image data.

[0032] Through the above implementation method, the second program loads the narration data and executes playback control. According to the time or event recorded in each narration data, the corresponding narration data is played, and audio and video data are played at the same time to reproduce the narration process of the target webpage.

[0033] In one possible implementation, when the first narration data is played and it is confirmed to be changed serialized web page data, based on the changed serialized web page data stored in the first narration data, a corresponding first data node is searched in the currently displayed objectified web page data based on the changed node information contained therein. The first data node is then processed according to the nature of the difference contained in the first narration data, including: if the nature of the difference is confirmed to be an addition, adding the content contained in the changed node information to the first data node; if the nature of the difference is confirmed to be a deletion, deleting the first data node; and if the nature of the difference is confirmed to be a replacement, replacing the first data node with the content contained in the changed node information.

[0034] In one possible implementation, upon receiving a fast-forward, rewind, or jump-to-a-specific-position playback instruction, the system calculates the first time point to which the player should jump based on the instruction, and plays audio and / or video data based on this first time point. Based on the first time point, it searches backward for the nearest first narration data containing a keyframe identifier or all the webpage data. The system obtains the corresponding time information of the found first narration data as a second time point. Based on the time information contained in the first narration data, it obtains a first narration data queue containing the second time point, the first time point, and the first narration data in between. Elements in the first narration data queue are extracted in chronological order. Based on the webpage data contained in these elements, and using all the webpage data contained in the first element of the first narration data queue as a basis, webpage data calculations are performed sequentially to obtain the webpage data corresponding to the first time point and replace the currently displayed webpage content. Finally, the timer is modified based on the first time point to continue playback.

[0035] Each of the above implementation methods calculates webpage data corresponding to a certain playback time or event based on the first narration data, displays the webpage to obtain the visual effect, plays audio and video based on the second narration data, and plays operation actions based on the third narration data, combining the playback of multiple types of narration data to generate a playback effect similar to a recorded screen.

[0036] Thirdly, the present invention also provides a computer storage medium storing a software program, which, when read and executed by one or more processors, can implement the method provided in any of the above-described embodiments.

[0037] Fourthly, the present invention also provides a computer program product that, when read and executed by one or more processors, can implement the method provided in any of the above-described embodiments.

[0038] As can be seen, the recording and display method of this invention uses webpage data recording, which has the advantages of small data storage, small network transmission volume, and fast transmission speed compared to ordinary screen recording. Furthermore, the playback and narration method of this invention obtains the playback screen by displaying webpage data based on the webpage, which has the advantage of lossless image quality compared to ordinary screen recording. Attached Figure Description

[0039] Figure 1 This is a schematic diagram of a webpage narration recording process provided in an embodiment of the present invention;

[0040] Figure 2 This is a schematic diagram illustrating a process for recording differentiated webpage data, provided by an embodiment of the present invention.

[0041] Figure 3 This is a schematic diagram of a process for recording serialized web page data, provided by an embodiment of the present invention.

[0042] Figure 4 This is a flowchart illustrating the process of setting keyframes for narration data, provided as an embodiment of the present invention.

[0043] Figure 5 This is a schematic diagram of another webpage narration recording process provided in an embodiment of the present invention;

[0044] Figure 6 This is a schematic diagram of a webpage narration recording process provided by an embodiment of the present invention, which generates a webpage or URL for recording narration on the server side.

[0045] Figure 7 This is a schematic diagram illustrating a process for recording webpage narration on a server side, as provided in an embodiment of the present invention.

[0046] Figure 8 This is a schematic diagram of a webpage commentary playback process provided in an embodiment of the present invention;

[0047] Figure 9 This is a schematic diagram of a process for playing serialized web page data, provided by an embodiment of the present invention.

[0048] Figure 10 This is a schematic diagram of a process for fast-forwarding, rewinding, or jumping to a designated position to play narration data, provided by an embodiment of the present invention.

[0049] Figure 11 This is a schematic diagram of a webpage narration recording interface provided in an embodiment of the present invention. Detailed Implementation

[0050] The embodiments section of this invention is for illustrative purposes only and is not intended to limit the scope of the invention.

[0051] The embodiments of the present invention are described below. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. Those skilled in the art will understand that, with the development of technology and the emergence of new scenarios, the technical solutions provided by the embodiments of the present invention are also applicable to similar technical problems.

[0052] Web pages, as important tools for information dissemination and business processing, often require explanations and the generation of explanatory data. Current technologies primarily rely on screen recording for web page explanations, which generates large amounts of data and is inefficient for network transmission. To address these issues, this invention provides a method for recording web page explanations based on the web page itself. This method uses programs embedded within the web page to complete the recording, generating explanation data by collecting web page data, audio / video data, and operational data. This results in smaller data volumes and facilitates network transmission. This invention also provides a method for playing web page explanations based on web page programs. Playback is performed by restoring and displaying web page data, playing audio, and recreating operational effects within the web page, with completely lossless image quality.

[0053] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings:

[0054] Example 1

[0055] Figure 1 This is a flowchart of an embodiment of the webpage explanation recording method of the present invention, as shown below. Figure 1 As shown, the method in this embodiment may include the following steps:

[0056] S101, Load the target webpage and load the first program used to perform the narration recording function;

[0057] For example, in this invention, the webpage for which the narration is recorded is the target webpage. The first program includes narration recording control and narration data acquisition functions. Preferably, the first program is included in the target webpage in the form of a JS library or JS code segment, independent of the original target webpage content, which facilitates rapid implementation. Preferably, after the first program is loaded, a function button to start narration recording is displayed on the target webpage, such as... Figure 11 The "Record Narration" button is shown in Figure A.

[0058] S102, Start narration recording, load narration recording control component; start target webpage content change monitoring; start audio and / or video data acquisition; acquire current target webpage data to generate first narration data;

[0059] In this step, a user operation event is received to initiate narration recording, and the narration recording control component is loaded, such as... Figure 11As shown in Figure B, the loaded narration recording control components include, but are not limited to, ending recording, pausing recording, drawing tools, narration tools, and recording time display components. The drawing tools include drawing control tools and a drawing canvas, with features such as starting drawing, exiting drawing, brush settings, color settings, and shape settings. Optionally, upon receiving a command to start drawing, the drawing control components such as brush settings, color settings, and shape settings are displayed, and a transparent drawing canvas is created on top of the target webpage content to receive drawing operations. Narration tools are auxiliary tools used for explanation, such as color changing, flashing, and border tools.

[0060] Start a listener for changes to the target webpage content, using a function such as MutationObserver to monitor these changes.

[0061] This invention records and displays content based on webpage data. Therefore, after the webpage loads, it collects the current target webpage data to generate first explanatory data. This first explanatory data is a type of explanatory data and is stored as a single explanatory data record. It then initiates target webpage content change monitoring to record subsequent display content. Optionally, when collecting the current target webpage data to generate the first explanatory data, the first program determines whether to collect the webpage data based on preset recording settings. For example, in a graffiti control tool, the graffiti canvas is defined as being within the webpage data collection range, while other tools and components outside the graffiti canvas are not. The explanatory tool is not within the webpage data collection range, but its effects are within the webpage data collection range.

[0062] Initiate audio and / or video data acquisition, using a camera to capture the narrator's audio and video data to record audio and video narration content.

[0063] S103A, detects changes in the content of the target webpage, collects the changed webpage data and the time information of the occurrence, and generates the first interpretation data;

[0064] In this step, the changed webpage data is collected to generate the first narration data. It's important to note that in this invention, monitoring changes in webpage content and collecting webpage data do not necessarily target the same objects. For example, the MutationObserver function monitors changes in webpage content, such as HTML and CSS. The webpage data collected to generate the first narration data includes, but is not limited to, webpage HTML data, webpage CSS data, serialized webpage data, Canvas data generated based on the webpage HTML display content, and webpage content image data. For instance, Canvas data generated based on the webpage HTML display content involves drawing on a Canvas based on the webpage's display content. One or more of the data, images, or drawing instructions contained in this Canvas are collected to generate the first narration data. In the above steps, the occurrence time information corresponds to the time when the webpage changes. For example, after starting narration recording in S102, a timer is set to start timing, and the timer data is collected in this step as the occurrence time information. Alternatively, after starting narration recording in S102, the start time is recorded, and the occurrence time is collected in this step as the occurrence time information.

[0065] S103B, acquires audio and / or video data to generate second narration data;

[0066] In this step, the second narration data is another type of narration data, which includes audio and / or video data. A second narration data record is generated without interruption of the narration recording process; in this case, time information does not need to be recorded. Multiple second narration data records are generated if the narration recording process allows pausing. Optionally, a new second narration data record is created after starting or continuing narration recording, acquiring data in the form of an audio or video stream.

[0067] S104, stores the explanation data;

[0068] In this step, storing the narration data includes sending it to the server and storing it on the server. Preferably, storing the narration data on the server also includes generating a URL for playing the narration. The URL corresponds to a narration playback webpage for playing the narration data. The narration playback webpage includes a second program for performing the narration playback function. When a user accesses the narration playback webpage, the second program is loaded, and the second program loads the narration data and narration playback controls to play the narration. Optionally, a message digest generation algorithm is used to generate a message digest for the currently stored narration data and display the message digest on the webpage. Further, each time new narration data is stored, a message digest is generated again based on the currently stored narration data or its corresponding message digest and the new narration data, and the newly generated message digest is displayed on the webpage. This implementation helps prevent tampering with the narration data, and the final generated message digest helps verify the consistency of the narration data.

[0069] In this embodiment, the narration is recorded directly from the webpage itself, without the need for other software or tools. Based on the characteristic of webpages displaying content based on webpage data, this invention generates display content by recording webpage data, and simultaneously generates audio and video narration content by recording the narrator's audio and video data. Playback is controlled by a program based on these two aspects of data, achieving the same narration effect as ordinary screen recording. The beneficial effects are: small recorded data volume, easy to save and transmit over the network; lossless image quality when played back through the webpage program; no need for other software or tools to perform recording; ordinary screen recording is easily interfered with by other programs, such as receiving system notifications during recording, while the recording in this invention is completed by the webpage itself, thus unaffected by other running programs; content is collected and displayed based on webpage data, not limited by webpage type, both dynamic and static webpages can be recorded; not limited by webpage updates, recording is based on the current content of the webpage; the recorded content can be controlled, such as filtering sensitive information; it stores the internal webpage data changes during webpage browsing operations, rather than simply displaying the output.

[0070] Example 2

[0071] Figure 2 This is a schematic diagram of a process for recording differentiated webpage data according to the present invention. This embodiment is based on embodiment one, S103A, as follows: Figure 2 As shown, the method in this embodiment may include:

[0072] S201, a change in the content of the target webpage has been detected;

[0073] In this step, we listen for changes in webpage content, such as changes to the webpage's HTML, DOM, and CSS. For example, we use MutationObserver to monitor changes made to the webpage's DOM tree.

[0074] S202, Calculate the webpage data that differs before and after the webpage content changes, and the nature of the differences;

[0075] In this step, based on the nature of the webpage data to be collected, the webpage data showing differences before and after the content changes is calculated and collected as differentiated webpage data. For example, if the webpage data is HTML data, the HTML data showing differences before and after changes is collected. Alternatively, if the webpage data is Canvas data generated based on the HTML content of the webpage, this method involves drawing on a corresponding Canvas based on changes in webpage content, and collecting the Canvas data that changes each time or recording the drawing instructions executed each time, as differentiated webpage data. Additionally, this step simultaneously collects the properties of the differences and adds them to the first explanatory data. The properties of the differences include, but are not limited to, additions, replacements, and deletions.

[0076] S203, Obtain the webpage data of the difference portion and the nature of the difference, and generate the first explanatory data;

[0077] This step ultimately generates the first explanatory data based on the differences in the webpage data.

[0078] This embodiment generates first explanatory data based on partial execution data collection of webpage data differences. The beneficial effects are that it further reduces the amount of data collected and stored, making large-scale explanatory data storage possible and facilitating rapid network transmission.

[0079] Example 3

[0080] Figure 3 This is a schematic diagram illustrating a process for recording serialized webpage data according to the present invention, such as... Figure 3 As shown, the method in this embodiment may include:

[0081] S301, Load the target webpage;

[0082] S302, serialize the webpage data of the target webpage to generate serialized webpage data, and generate first explanatory data based on the serialized webpage data;

[0083] In this step, the webpage data is serialized according to the nature of the collected webpage data. For example, after the target webpage content is loaded, the target webpage DOM is serialized to generate serialized DOM data, and the serialized DOM data is used to generate the first explanatory data. The target webpage DOM contains object methods and properties, which facilitates the comparison calculation in step S303, and serialization facilitates the persistent storage of the explanatory data.

[0084] S303, after detecting a change in the content of the target webpage, the first explanatory data is generated based on the serialized data of the changed webpage, and the information of the changed nodes is recorded at the same time;

[0085] In this step, depending on the nature of the collected webpage data, such as serialized webpage data, after detecting a change in the target webpage content, the target webpage DOM is serialized to generate serialized DOM data. Optionally, based on the detected change nodes in the target webpage DOM, the information of the changed nodes is recorded, and the changed parts are serialized, that is, the differing parts are serialized, stored, and used to generate the first explanatory data.

[0086] This embodiment provides a method for serializing and storing narration data, which has the following advantages: it facilitates the permanent preservation of narration data; it facilitates the accurate construction of web page data when playing narration; and it facilitates network transmission.

[0087] Example 4

[0088] Figure 4 This is a schematic diagram of a process for setting keyframes for interpretive data according to the present invention, as shown below. Figure 4 As shown, the method in this embodiment may include:

[0089] S401, collect webpage data of the target webpage;

[0090] S402, perform a judgment according to the preset keyframe setting rules, and after confirming that the rules are met, record all the web page data of the target web page as the first explanation data, and set keyframe identifiers for the first explanation data;

[0091] In the above steps, the keyframe is used to identify that the corresponding data record contains all the webpage data at the corresponding time. When the keyframe is loaded, it replaces all the webpage data in the narration playback environment. Then, based on the keyframe data, subsequent non-keyframe data is read in to calculate its corresponding webpage data. The keyframe setting rules are used to determine whether to set a keyframe, such as when the webpage is first loaded, after a page navigation jump, when the time interval between the previous keyframe reaches a specified duration, when the changes to the webpage reach a specified proportion, or when a specified type of component in the webpage changes.

[0092] The keyframe setting method provided in this embodiment, combined with the differentiated webpage data recording method mentioned in Embodiment 2 above, records all webpage data in the keyframes and differentiated webpage data in non-keyframes. This combination facilitates the calculation of webpage data at any point during narration playback. For specific calculation methods, refer to relevant embodiments of narration playback. The beneficial effects of this implementation are that the keyframes reduce the computational load of acquiring and calculating webpage data at any point during playback, improve running speed, facilitate the implementation of functions such as fast forward, rewind, and jumping to a specified position in narration playback, and provide data support for these functions.

[0093] Example 5

[0094] Figure 5 This is a flowchart of another embodiment of the webpage narration recording method of the present invention, as shown in the figure below. Figure 5 As shown, the method in this embodiment may include the following steps:

[0095] S501, Load the target webpage, load the first program used to perform the narration recording function;

[0096] S502, Start narration recording, load narration recording control component; Start target webpage content change monitoring; Start audio and / or video data acquisition; Start cursor event monitoring; Acquire current webpage data to generate first narration data;

[0097] The difference between this step and Example 1 is that it adds listening to cursor events, such as cursor movement events.

[0098] S503A detects changes in the content of the target webpage, collects the changed webpage data and the time of occurrence, and generates the first interpretation data;

[0099] S503B, acquires audio and / or video data to generate second narration data;

[0100] The S503C listens to cursor events, collects cursor position and time information, and generates third-party interpretation data.

[0101] This step collects cursor position and corresponding time information based on cursor events, and the resulting third interpretive data is another type of interpretive data.

[0102] S504, stores commentary data;

[0103] This embodiment adds the collection of user cursor information during the narration recording process and generates a new type of narration data, which is used to play the user cursor actions during narration playback, further enhancing the narration experience of the target webpage. Except for the monitoring of the cursor and data collection, the other steps are the same as in Embodiment 1, and will not be described again here.

[0104] Example 6

[0105] Figure 6 As another embodiment of the present invention, such as Figure 6 As shown, the method in this embodiment may include the following steps:

[0106] In client 10:

[0107] S601, Generate the first recording narration request;

[0108] In this step, the first recording and narration request is used to record narration for a specified target webpage. It should be noted that the specified target webpage can be any webpage, but it does not contain the first program of this application for performing the narration recording function; that is, narration cannot be recorded directly from that webpage. For example, if webpage A does not contain the first program, a first recording and narration request is generated on webpage A to record narration for webpage A. Similarly, if webpage B does not contain the first program, a first recording and narration request is generated on webpage A to record narration for webpage B.

[0109] S602, Send the first recording and commentary request;

[0110] In server 20:

[0111] S603, process the first recording and narration request, obtain the target webpage URL, further obtain the webpage content of the target webpage, add the first program to the webpage content of the target webpage, and generate a webpage or URL for recording and narration;

[0112] In this step, after receiving the first recording request, the server 20 obtains the URL information of the target webpage based on the request, accesses and obtains the webpage content based on the URL, and then adds the first program for performing the narration recording function to the obtained webpage content, so that the generated new webpage or its corresponding URL has the narration recording function. For example, if a first narration recording request is received from webpage A to record narration for webpage A, the implementation principle of this step is that webpage A itself does not contain the narration recording function. The server generates a new webpage with the narration recording function based on its webpage content and the first program. The user performs narration recording through the new webpage or the corresponding URL. Optionally, the server 20 stores the new webpage and generates a corresponding URL to return to the client 10 for starting recording. Optionally, a preset URL is returned to the server. The preset URL contains the first program. The user accesses the preset URL, and the server 20 dynamically loads the target webpage data content.

[0113] S604, returns the webpage or corresponding URL used for recording the narration;

[0114] In client 10:

[0115] S605, receive a webpage or URL for recording narration, load the webpage or access the URL to load the target webpage, and the first program starts recording narration;

[0116] This step obtains a webpage or URL containing the recording and narration function from the server 20 to start recording. The execution steps after starting the narration recording are basically the same as those in Embodiment 1 and Embodiment 3, and will not be repeated here.

[0117] This embodiment provides a method for generating a webpage or URL for recording commentary for a specified webpage. The specified webpage itself does not have the function of recording commentary. After processing by the server, a webpage or URL that can be used for recording commentary is obtained. Therefore, the beneficial effects are: it is applicable to recording commentary for any webpage, and for the target webpage that needs to be recorded commentary, no additional code is required to record commentary. The implementation cost is low and the application range is wide.

[0118] Example 7

[0119] Figure 7 As another embodiment of the present invention, such as Figure 7 As shown, the method in this embodiment may include the following steps:

[0120] In client 10:

[0121] S701, Generate a second recording narration request;

[0122] In this step, the second recording and narration request is used to record narration for a specified target webpage, which can be any webpage that does not contain the first program. For example, if webpage A does not contain the first program, a second recording and narration request is generated for recording narration for webpage A. Similarly, if webpage B does not contain the first program, a second recording and narration request is generated for recording narration for webpage B from webpage A.

[0123] S702, send a second recording narration request;

[0124] In server 20:

[0125] S703 processes the second recording and narration request, obtains the target webpage URL, loads the target webpage in the headless browser, loads the first program, loads the screen-sharing communication program, and establishes screen-sharing communication with the client.

[0126] In this step, a browsing and execution environment for the target webpage is created on the server side. The headless browser is a server-provided target webpage browsing and execution environment. This method performs narration recording in the server-side environment. The target webpage and the first program are loaded in the server-side browser to generate the server-side narration recording environment. Additionally, this method establishes screen-to-screen communication between the server-side browser and the client to perform narration recording and operation control. Optionally, after processing the second recording request on the server side, a URL for the client to perform narration recording and operation control is generated and returned to the client. After receiving the URL, the client loads the corresponding webpage content and establishes screen-to-screen communication with the server-side browser.

[0127] S704: Collect webpage data, generate first narration data storage, and generate synchronous webpage data for transmission;

[0128] This step includes two functions: firstly, recording the initial narration data on the server side; and secondly, generating snapshot data of the target webpage to be sent to the client, so that the client and server browsers can view the target webpage on the same screen.

[0129] S705, send synchronized webpage data;

[0130] In client 10:

[0131] S706 receives and loads synchronized web page data, establishes display synchronization with the server-side headless browser, and starts listening for user operations.

[0132] This step establishes simultaneous screen communication between the server and browser, receives target webpage snapshot data sent by the server for synchronous display, and starts user operation monitoring to collect user interaction actions on the target webpage.

[0133] S707, for acquiring audio and video data;

[0134] S708, sends audio and video data;

[0135] The above steps involve collecting and sending audio and video data for narration from the client for the server to receive and store.

[0136] In server 20:

[0137] S709 receives audio and video data and generates second narration data;

[0138] In client 10:

[0139] S710 collects user operation data;

[0140] S711, sends user operation data;

[0141] The above steps, based on the establishment of screen mirroring between the client and server-side headless browsers, involve collecting user actions from client 10 and sending them to server 20 for actual execution. Optionally, client 10 can start cursor event listening to collect cursor position information and send it to the server.

[0142] In server 20:

[0143] S712 receives user operation data, executes user operations, generates first explanation data, and collects synchronized webpage data.

[0144] This step is based on the client's operations on the target webpage, is actually executed on the server, and the narration data is recorded on the server. Simultaneously, synchronization webpage data for synchronization is generated for the client. Optionally, third narration data is generated and stored by receiving cursor position information and corresponding time information sent by client 10.

[0145] S713, send synchronized webpage data;

[0146] In client 10:

[0147] S714 receives synchronized webpage data and performs webpage display synchronization processing;

[0148] This step ensures that the content displayed on the client side is consistent with the content displayed in the headless browser on the server side.

[0149] The above embodiments provide a method for establishing a server-side browser and client-side screen mirroring, collecting user operations on the client side, and actually running the target webpage and executing user operations on the server side, recording webpage narration based on the target webpage running in the server-side browser. In this embodiment, the webpage content displayed on the client side is consistent with the server-side headless browser, which can be understood as a snapshot of the target webpage's operation. In the above embodiments, the headless browser can browse any webpage based on the second recording narration request; therefore, this implementation supports recording narration for any webpage. The collection and storage of narration data on the server side in this embodiment is basically the same as in the previous embodiments, and will not be repeated here. The beneficial effects of this embodiment are that it facilitates the execution of target webpage operation control and recording control on the server side, and avoids transmitting a large amount of recording data to the server side, reducing the amount of network data transmitted; in the client-side recording narration data method, when the webpage is closed after receiving the recording end instruction, the generated narration data is not completely uploaded to the server side, which may lead to the loss of narration data. This embodiment effectively avoids this problem.

[0150] Example 8

[0151] Figure 8 This is a flowchart of embodiment eight of the webpage explanation playback method of the present invention, as shown below. Figure 1As shown, the method in this embodiment may include the following steps:

[0152] S801, access the commentary playback URL and load the second program used to perform the commentary playback function;

[0153] In this step, the second program is a program for executing narration playback, which includes narration playback control and narration data loading functions. The playback control functions include, but are not limited to, displaying playback time, playing, pausing, skipping, fast forwarding, rewinding, speed adjustment, and stopping. The narration data loading function includes loading first narration data, second narration data, and third narration data. Optionally, the narration data loading function also includes a data preloading function, that is, preloading a specified time range or quantity of narration data.

[0154] S802, loads the narration playback component and downloads the narration data;

[0155] This step involves displaying components for controlling the playback of the narration after it has started. Optionally, before starting playback, a play narration button and narration summary information (such as duration and author) are displayed. After starting playback, control buttons such as playback time display, playback timeline, fast forward, rewind, speed adjustment, and stop are displayed.

[0156] S803, after receiving the playback command, starts the narration playback, reads the narration data, and starts the timer;

[0157] In this step, the timer is used to calculate the playback time of the narration data.

[0158] S804, according to the predetermined time calculation rules, when it is confirmed that the timer time matches the time information contained in the narration data, or when it is determined that the event executed by the second program is the specified event bound in the narration data, the narration data playback is executed, including: calculating and displaying webpage content based on the first narration data, playing audio and / or video based on the second narration data, and setting the display cursor based on the third narration data;

[0159] This step provides two ways to play the narration data. The first is based on time control, where the narration data is played when the timer's time matches the time contained in the narration data. For example, if a piece of the first narration data contains a time information of 20 seconds, meaning the narration record was generated 20 seconds after the recording started, playback will begin when the timer reaches 20 seconds. The second method is based on determining playback based on events in the playback page and events bound to the narration data. For example, if a start playback event is bound to the first narration data that records the initial webpage data of the target webpage, then the narration data record will be played after playback is started.

[0160] In the above embodiments, the second program plays back the narration data to reproduce the browsing, operation, and explanation process during the recording. This invention employs a composite playback method, which uses the first narration data to reproduce the changes in webpage display content at various time points, uses the second narration data to play audio and video data corresponding to the recording scene, and uses the third narration data to reproduce the cursor position and movement at various time points. Therefore, the beneficial effect of this embodiment is that by using several narration data composite playback methods, the same playback effect as the screen-recorded video can be reproduced using the webpage operating environment, with completely lossless image quality. Users can view the target webpage narration without the need for other software; the narration content can be played directly within the webpage.

[0161] Example 9

[0162] Figure 9 This is a schematic diagram illustrating a process for playing serialized webpage data according to the present invention, such as... Figure 9 As shown, the method of this embodiment may include:

[0163] S901, Read the changed serialized web page data stored in the first explanation data;

[0164] S902, based on the changed node information contained therein, find the corresponding first data node in the currently displayed objectified web page data;

[0165] S903, determine the nature of the differences contained in the first interpretation data;

[0166] When the nature of the difference is determined to be increasing:

[0167] S904, add the content of the changed node information to the first data node;

[0168] When the property of the difference is deletion:

[0169] S905, Delete the first data node;

[0170] When the nature of the difference is determined to be substitution:

[0171] S906, replace the first data node with the content contained in the changed node information;

[0172] In the above steps, the second program plays the first narration data based on the serialized webpage data. For example, if the webpage data is DOM data, a DOM object model is constructed on the client side. S901 receives the changed serialized webpage data, which, after deserialization, includes node information, node change properties, and node change data. S902 finds the corresponding data node, i.e., the first data node, in the DOM object model based on the node information in the received serialized webpage data. S903 determines the node change properties and performs corresponding processing based on these properties. Optionally, when the first narration data contains keyframe identifiers, the DOM object model is reconstructed based on the webpage data it contains.

[0173] In this embodiment, the client plays the first narration data, performs playback based on the data object model (e.g., DOM object model) built on the client, receives the first narration data to calculate and update the data object model, and generates webpage display data based on the data object model, thus realizing the playback processing of serialized webpage data.

[0174] Example 10

[0175] Figure 10 This is a schematic diagram illustrating a process for playing narration data by fast-forwarding, rewinding, or jumping to a designated position according to the present invention, such as... Figure 10 As shown, the method of this embodiment may include:

[0176] S1001, received a fast forward, rewind, or jump to a specified position playback command;

[0177] S1002, calculate the first time point to which the jump will take place;

[0178] This step calculates a first time point based on user-triggered playback commands such as fast forward, rewind, or jump to a specified position. The first time point is the time point the user intends to jump to. Optionally, upon receiving the fast forward playback command, the first time point is obtained by adding a preset time length backward from the current time; upon receiving the rewind playback command, the first time point is obtained by adding a preset time length forward from the current time; upon receiving the jump to a specified position playback command, the first time point is obtained based on the playback progress control.

[0179] S1003, plays audio and / or video data based on the first time point;

[0180] S1004: Based on the first time point, search backwards for the most recent first narration data that has a keyframe identifier or contains all web page data, and obtain its corresponding time information as the second time point according to the search result;

[0181] This step obtains the second time point through the first time point. Its purpose is to obtain the webpage data corresponding to the first time point specified by the user to jump to its playback. However, the present invention records the narration data for display by listening to changes in webpage content, that is, it records every change in webpage content. The second time point is used to obtain the time point with the most complete webpage data that is closest to the first time point.

[0182] S1005, Obtain the first explanatory data queue containing the second time point, the first time point, and the data between the two;

[0183] S1006, extract the elements in the first explanation data queue in chronological order, and based on the web page data contained therein, perform web page data calculations sequentially using all the web page data contained in the first element in the first explanation data queue as a basis, to obtain the web page data corresponding to the first time point and replace the currently displayed web page content.

[0184] The above steps are based on the time point closest to the specified playback position that has a keyframe identifier or contains all webpage data. Webpage data calculations are performed sequentially based on the first narration data between the second and first time points to ultimately obtain and display the webpage data corresponding to the specified playback position. The webpage data calculation refers to performing calculations based on the webpage content data at a given moment and the changed webpage data recorded in the first narration data, according to the nature of the changes, to obtain new webpage content data.

[0185] S1007, and based on the first time point, modify the timer to continue playback;

[0186] The timer is set accordingly based on the time information corresponding to the playback position specified by the user.

[0187] Optionally, when the fast-forward instruction is received in step S1001 above or the specified playback position is a position backward from the current position, after calculating and obtaining the first time point in S1002, the first narration data between the current time point and the first time point is obtained as the first narration data queue, and then S1006 is executed to obtain the web page data corresponding to the first time point and replace the currently displayed web page content.

[0188] The method provided in this embodiment is based on the data recording method of the present invention and realizes playback functions similar to those of traditional videos, such as fast forward, rewind, or jumping to a specified position, which facilitates flexible playback control for users.

[0189] The above are some basic embodiments of this application, used to illustrate the implementation principle of the invention. It is evident that the present invention provides a new method for recording and playing narration on web pages. Its technical feature lies in replacing traditional video recording methods with the collection of web page data, and achieving the playback effect of traditional screen recording by separately recording several types of narration data and then synthesizing them for playback. Its beneficial effects include: compared to traditional screen recording, the amount of recorded data is very small, and the transmission speed is fast; the playback effect is obtained by restoring the data using a web page program, with completely lossless image quality; and it achieves flexible playback control like traditional video.

[0190] It should be noted that in the above embodiments of the method of this application, some steps can be replaced by similar steps, and some steps can be combined, decomposed, or implemented in an adjusted order. Therefore, the above embodiments do not constitute a limitation on the webpage explanation recording and playback method proposed in this application.

[0191] This application also provides a computer program product that, when read and executed by one or more processors, can implement the method provided in any of the above embodiments.

[0192] This application also provides a computer-readable storage medium storing a software program that is read by one or more processors and executes the steps of the webpage narration recording and playback method of any of the above embodiments.

[0193] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the methods described above. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0194] The above are merely specific embodiments of this application, but the protection scope of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in this application should be included within the protection scope of this application. Therefore, the protection scope of this application should be determined by the scope of the claims.

Claims

1. A method for recording webpage commentary, characterized in that, The method is used to record narration for a target webpage, and the collected data is limited to the target webpage's original data, including: The target webpage is loaded, and a first program for performing the narration recording function is loaded. The first program includes narration recording control and narration data acquisition function, wherein the narration data acquisition function directly acquires the native data of the target webpage. The system receives a start recording command to start narration recording and loads the narration recording control component. It also starts monitoring changes in the target webpage content and starts audio and / or video data acquisition. The narration recording control component includes one or more of the following: end recording, pause recording, drawing tool, and narration tool. The canvas data of the drawing tool and the effect data of the narration tool are integrated into the native data acquisition range of the target webpage. After the target webpage content is initially loaded, webpage data is collected to generate first narration data, or webpage data and corresponding time information are collected to generate the first narration data; after a change in the target webpage content is detected, the changed webpage data and the time of occurrence are collected to generate the first narration data; the audio and / or video data is collected to generate second narration data, or the audio and / or video data and corresponding time information are collected to generate second narration data; wherein, the first narration data and the second narration data are two types of narration data, and the first narration data is the data source of the webpage content; Send the explanation data to the server; The narration data is stored on the server side. The narration data is used to provide playback data for the narration playback webpage of the target webpage. The narration playback webpage includes a second program for performing the narration playback function. The core of the second program is to parse the original data of the webpage and restore the display. The second program loads the narration data and the narration playback control to play the narration. The webpage data includes one or more of the following: webpage HTML data, webpage CSS data, serialized webpage data, Canvas data generated based on the webpage HTML display content, and webpage content image data.

2. The method according to claim 1, characterized in that, The process of collecting changed webpage data and the time of occurrence to generate the first explanatory data also includes: After detecting a change in the content of the target webpage, the webpage data that differs before and after the change is collected as the changed webpage data. Alternatively, the nature of the difference is collected and added to the first explanatory data. The nature of the difference includes one of addition, replacement, or deletion.

3. The method according to claim 1, characterized in that, The process of collecting the webpage data and the changed webpage data to generate the first explanatory data further includes: The webpage data or the changed webpage data is serialized to generate serialized webpage data. The first explanatory data is generated based on the serialized webpage data. The changed webpage data is serialized, and the changed node information is recorded.

4. The method according to any one of claims 1 or 3, characterized in that, Generating the first narration data based on the webpage data or the serialized webpage data further includes: According to the preset keyframe setting rules, after confirming that the preset keyframe setting rules are met, all web page data of the target web page is recorded as the first narration data, and keyframe identifiers are set for the first narration data.

5. The method according to claim 1, characterized in that, After the target webpage receives the command to start recording the narration and begins recording, the process further includes: Start cursor event listening. After the cursor event is listened to, collect cursor position information and time information to generate third narration data. The third narration data is a type of narration data. The cursor position information is precisely bound to the time information of the first narration data for the visualization and restoration of the cursor during playback.

6. The method according to claim 1, characterized in that, After receiving the command to start recording the narration, it also includes: The client generates the first recording and narration request and sends it to the server. The first program receives a webpage or a URL corresponding to the webpage returned by the server for recording commentary, loads the webpage for recording commentary or accesses the URL for recording commentary to load the content of the target webpage, and starts recording commentary. The webpage used for recording commentary is generated based on the content of the target webpage and contains the first program. After receiving the recording commentary request sent by the target webpage, the server obtains the URL of the target webpage based on the request, further obtains the webpage content of the target webpage, adds the first program to the webpage content of the target webpage, generates the webpage used for recording commentary, and returns the webpage used for recording commentary, the webpage content, or the corresponding URL. The first program only injects data collection functions into the target webpage.

7. The method according to claim 1, characterized in that, After receiving the command to start the explanation, it also includes: The client generates a second recording narration request and sends it to the server. On the server side, the second recording and narration request is received and processed; the target webpage URL is obtained based on the second recording and narration request; the target webpage is loaded in the server browser; the first program is loaded; narration recording is started and the narration recording control component is loaded; the monitoring of changes in the target webpage content is started; webpage data is collected to generate the first narration data; a screen-to-screen communication program is loaded; screen-to-screen communication is established with the client; webpage data is collected to generate synchronous webpage data and sent to the client; the synchronous webpage data is a snapshot of the webpage content. On the client side, a screen-sharing communication mechanism is established between the client and the webpage loaded by the server browser; the synchronized webpage data is received and loaded to synchronize the content displayed on the client and the server browser; user operation monitoring is initiated, and user operation data is collected and sent to the server after a user operation event is detected; audio and / or video data collection is initiated, and the audio and / or video data is collected and sent to the server. On the server side, the system receives the user operation data, executes the user operation events contained therein, and after detecting a change in the content of the target webpage, collects the changed webpage data and the time of occurrence to generate the first narration data; it also receives the audio and / or video data and generates the second narration data based on it; and stores the narration data. The first program includes one or more forms, such as browser plugins and JavaScript code.

8. A method for playing webpage commentary, characterized in that, The method is used to play webpage narration recorded by any one of the methods described in claims 1-7, and to calculate and reconstruct the webpage content based on the collected native webpage data, including: Access the commentary playback URL and load a second program for executing the commentary playback function. The second program includes commentary playback control and commentary data loading function. The commentary data loading function loads native webpage data and differentiated change data. The commentary playback control function is a data-based playback timing control. Load the narration playback component and download the narration data. The narration data includes one or more of the following: first narration data, second narration data, and third narration data. The narration playback component includes one or more of the following: playback time display, play, pause, jump, fast forward, rewind, speed adjustment, and stop. The first narration data includes webpage data, the second narration data includes audio and / or video data, and the third narration data includes cursor position data. Each type of narration data may or may not contain time information. When the time information is not included, playback is executed by binding to a specified event by default. Upon receiving a playback command, the system initiates narration playback, reads the narration data, starts a timer, and executes narration data playback when one of the following conditions is met: Condition 1, when the timer time is confirmed to match the time information contained in the narration data according to a predetermined time calculation rule; Condition 2, when the event executed by the second program is determined to be a specified event bound to the narration data; wherein, the narration data playback is determined based on the type of the narration data, including: calculating and displaying webpage content based on the first narration data, playing audio and / or video based on the second narration data, and setting a display cursor based on the third narration data; during the narration data playback process, the execution of webpage events not used for playback control is blocked; The webpage data includes one or more of the following: webpage HTML data, webpage CSS data, serialized webpage data, Canvas data generated based on the webpage HTML display content, and webpage content image data.

9. The method according to claim 8, characterized in that, When playing the first narration data and confirming that it is changed serialized web page data, the method further includes: Based on the changed serialized webpage data stored in the first explanatory data, a corresponding first data node is searched in the currently displayed objectified webpage data based on the changed node information contained therein. The first data node is then processed according to the nature of the difference contained in the first explanatory data, including: when the nature of the difference is confirmed to be an addition, adding the content contained in the changed node information to the first data node; when the nature of the difference is confirmed to be a deletion, deleting the first data node; and when the nature of the difference is confirmed to be a replacement, replacing the first data node with the content contained in the changed node information.

10. The method according to claim 8, characterized in that, After receiving a fast forward, rewind, or jump to a specified position playback command, it also includes: Calculate the first time point to jump to based on the fast forward, rewind, or jump to a specified position playback command, and play audio and / or video data based on the first time point; based on the first time point, search backward for the nearest first narration data with a keyframe identifier or containing all the webpage data, obtain its corresponding time information as the second time point based on the found first narration data, obtain the first narration data queue containing the second time point, the first time point, and the first narration data between the two based on the time information contained in the first narration data, extract the elements in the first narration data queue in chronological order, and based on the webpage data contained therein, perform webpage data calculations sequentially based on all the webpage data contained in the first element in the first narration data queue to obtain the webpage data corresponding to the first time point and replace the currently displayed webpage content, and modify the timer based on the first time point to continue playback.

Images