Web operation video generation method and device, equipment, medium and program product

Abstract

The invention discloses a Web operation video generation method and device, equipment, a medium and a program product, and relates to the technical field of video generation, and the method comprises the steps: generating an operation task queue based on a predefined configuration file after a host Web application is loaded; and executing the operation task instructions in the operation task queue in sequence, after any operation task instruction causing page view change is executed, starting a visual feedback buffer window with a preset duration, after the visual feedback buffer window is closed, suspending execution of subsequent operation task instructions, and controlling the media recording controller to enter a dynamic waiting state. During the dynamic waiting state, non-linear timeline control is performed on capture and writing of media stream data based on the monitored multi-dimensional state of the page view. And after all the operation task instructions are executed, packaging and exporting the media stream data recorded by the media recording controller, thereby reducing the post-processing cost of the web operation video.

Description

Technical Field

[0001] This application relates to the field of video generation technology, and in particular to web-based video generation methods, apparatus, devices, media, and program products. Background Technology

[0002] With the rapid development of internet technology and the widespread application of Rich Internet Applications (RIAs) and Single Page Applications (SPAs), the interactive complexity and business process length of web applications continue to increase. In scenarios such as product demonstrations, operational training, automated testing and certification, and compliance audits, recording the web front-end operation process in video format and creating replayable and documentable demonstration videos has become an important requirement. Compared to static screenshots or text descriptions, operation videos can more intuitively reflect user behavior and system response processes, possessing high practical and dissemination value.

[0003] In related technologies, the generation of web operation videos mainly relies on two types of solutions: one is screen recording based on the system or browser plugins, which generates video by continuously capturing screen images; the other is server-side automated recording based on headless browsers, which drives page execution through scripts and takes screenshots or records the screen. However, both of these solutions have significant shortcomings: screen recording cannot perceive the loading status and business logic inside the web application, and the recording process is linearly bound to physical time. When network requests or page rendering take a long time, a large number of invalid waiting frames will be generated, resulting in lengthy videos and high post-editing costs. On the other hand, the server-side headless browser solution not only consumes a lot of computing resources, but is also prone to rendering deviations due to inconsistencies between the operating environment and the actual user environment. In addition, existing browser-side recording solutions generally rely on memory caching of media data, which is easily limited by browser memory in long-term or high-resolution recording scenarios, posing risks such as memory overflow and page crashes, making it difficult to meet the application requirements of enterprise-level long-process, stable recording. Summary of the Invention

[0004] The main objective of this application is to provide a method, apparatus, device, storage medium, and program product for generating web-based operational videos, aiming to solve the problem of high cost in recording and generating web-based front-end operational videos in related technologies.

[0005] To achieve the above objectives, this application provides a web-based video generation method, the method comprising: After the host web application is loaded, an operation task queue is generated based on the predefined configuration file read by the injected automated execution engine; The scheduler executes operation task instructions in the operation task queue in sequence. After executing any operation task instruction that causes a change in the page view, a visual feedback buffer window of a preset duration is started. After the visual feedback buffer window is closed, the execution of subsequent operation task instructions is suspended, and the media recording controller is controlled to enter a dynamic waiting state. During the dynamic waiting state, the capture and writing of media stream data are controlled non-linearly based on the multi-dimensional state of the monitored page view. After all operation task instructions are executed, the media stream data recorded by the media recording controller will be encapsulated and exported to obtain the web operation video.

[0006] In one embodiment, the step of generating an operation task queue based on a predefined configuration file read by the injected automated execution engine after the host web application has been loaded includes: After the host web application has finished loading, inject the operation task instruction automation execution engine and media recording controller into the page context; The predefined configuration file is read through the automated execution engine; The parser traverses the predefined configuration files and performs validity checks on each object in the predefined configuration files based on preset validation rules. If the predefined configuration file is valid, the scheduler will convert each object in the predefined configuration file into a task in memory, encapsulate each task into a corresponding operation task instruction, and generate an operation task queue.

[0007] In one embodiment, before the step of performing non-linear time-axis control on the capture and writing of media stream data based on the monitored multi-dimensional state of the page view, the method further includes the step of: By monitoring the frequency of DOM node changes through the Document Object Model observer, or by monitoring the pixel difference rate through requesting animation frames, the page view is determined to be in a stable state when the frequency of node changes or the pixel difference rate is lower than the corresponding preset threshold. Maintain an active request counter. Once the active request counter reaches zero and maintains a preset buffer duration, determine that the page view is in a stable network state. Monitor the audio context state or speech synthesis playback progress, and determine that the page view is in an audio-ready state when the audio stream finishes playing or enters a silent period.

[0008] In one embodiment, the step of non-linear time-axis control of media stream data capture and writing based on the monitored multi-dimensional state of the page view includes: Continue capturing media stream data while an active audio context is detected. When it is detected that the page view is in an audio-ready state and the page view is in an unstable state or the network is unstable, the capture of media stream data is paused or redundant frames are discarded so that the physical time elapsed is not included in the video timeline. When it is detected that the page view is in an audio-ready state and the page view is in a stable state and the network is stable, the capture of media stream data is resumed, and the scheduler is notified to release the suspended state and trigger the next operation instruction.

[0009] In one embodiment, the step of performing non-linear time-axis control on the capture and writing of media stream data based on the monitored multi-dimensional state of the page view further includes: By setting the time slice parameters of the media recording controller, the media recording controller can output media stream data slices according to the set output rules; By using asynchronous callbacks, media stream data slices are written in real time to the storage object of the host web application's local index database. This allows the streaming media data slices in the storage object to be read in batches according to time sequence when exporting media stream data, using database cursors. During streaming transmission, the streaming media data slices are dynamically encapsulated into the target video file.

[0010] In one embodiment, the step of performing non-linear time-axis control on the capture and writing of media stream data based on the monitored multi-dimensional state of the page view further includes: By maintaining the mapping relationship between the logical clock and the physical clock, if the waiting time exceeds the preset waiting time during the dynamic waiting state, the writing of media stream data is suspended through the media recording controller. The recovery write operation is performed through the media recording controller only when the page view is in an audio-ready state and the page view and network are in a stable state.

[0011] Secondly, to achieve the above objectives, this application further provides a web-operated video generation apparatus, the apparatus comprising: The operation queue generation module generates an operation task queue based on a predefined configuration file read by the injected automated execution engine after the host web application has finished loading. The scheduling module, through the scheduler, executes the operation task instructions in the operation task queue in sequence. After executing any operation task instruction that causes a change in the page view, it starts a visual feedback buffer window of a preset duration. After the visual feedback buffer window is closed, it suspends the execution of subsequent operation task instructions and controls the media recording controller to enter a dynamic waiting state. The recording control module, during the dynamic waiting state, performs non-linear time axis control on the capture and writing of media stream data based on the multi-dimensional state of the monitored page view; The video export module encapsulates and exports the media stream data recorded by the media recording controller after all operation task instructions have been executed, thus obtaining the web operation video.

[0012] Thirdly, to achieve the above objectives, this application further provides a Web-operated video generation device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the above-described Web-operated video generation method.

[0013] Fourthly, to achieve the above objectives, this application further provides a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above-described Web operation video generation method.

[0014] Fifthly, to achieve the above objectives, this application further provides a computer program product, which includes a computer program that, when executed by a processor, implements the steps of the above-described Web operation video generation method.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: 1. Automatically eliminates network loading and meaningless waiting, compressing 5 minutes of physical world operations into a 2-minute high-density demonstration video.

[0016] 2. By utilizing a local index database as an "unlimited buffer," it breaks through the browser's memory limitations and supports high-definition recording for several hours.

[0017] 3. Ensure that the operation steps, voice narration and visual changes are strictly aligned, thus solving the problem of audio-visual asynchrony.

[0018] 4. The visual buffering mechanism effectively captures and retains instantaneous feedback such as button clicks, enhancing the viewing experience of the video. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0020] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of the first process of the Web operation video generation method in the embodiments of this application.

[0022] Figure 2 This is a schematic diagram of the second process of the Web operation video generation method in the embodiments of this application.

[0023] Figure 3 A schematic diagram of the module connections for a web-based video generation device.

[0024] Figure 4 A schematic diagram of the structure of a device for generating video for web operation.

[0025] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0026] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0027] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0028] Rich Internet applications and single-page applications have become widespread, leading to a growing demand for operation demonstrations, automated testing documentation, and user tutorial videos for web applications. Currently, generating such videos primarily relies on the following two technical approaches: Traditional screen recording uses system-level screen recording software (such as OBS) or browser extensions. Its core logic is "linear recording," meaning that for every second of physical time that passes, the video duration increases by one second.

[0029] Disadvantages: It cannot perceive the internal state of the web application. When encountering slow network loading or long data processing times, the recorded video will contain a large amount of redundant waiting footage, resulting in lengthy videos that lack focus. Post-production editing requires manual removal of these segments, which is costly.

[0030] Server-side recording based on headless browser: Start a headless browser on the server to execute the script and take screenshots / record the screen (such as Puppeteer).

[0031] Disadvantages: Extremely high resource consumption; and the rendering difference between the server environment and the user's actual environment may lead to video distortion. Furthermore, regardless of the solution, during long-duration high-definition recording, the media stream data generated on the browser side typically resides in heap memory. Due to browser memory limitations on single pages (usually 1-2GB), long-term recording can easily lead to memory overflow and page crashes, failing to meet the needs of enterprise-level long-process presentations. Therefore, there is an urgent need for a client-side video generation solution that can "automatically cut out invalid waiting time" and is "not limited by memory."

[0032] Based on this, the main solution of this application embodiment is as follows: by injecting an automated execution engine and a media recording controller into the host web application, an operation task queue is generated and scheduled according to a predefined configuration file; after the page view changes, a dynamic waiting state is entered; by comprehensively considering multiple dimensions of page view status such as DOM changes, network requests, and audio status, non-linear timeline control is performed on the capture and writing of the media stream; redundant frames are paused or discarded during page instability or waiting, so that physical waiting time is not included in the video timeline; at the same time, a streaming storage method of media stream slicing and local index database is adopted to achieve stable recording with long process and low memory consumption; finally, after the task is completed, the operation video is dynamically packaged and exported in logical time order, thereby effectively reducing the duration redundancy, resource consumption, and overall generation cost of web front-end operation video generation.

[0033] Specifically, this application provides a web-based video generation method, referring to... Figure 1 and Figure 2 In this embodiment, the Web-based video generation method includes steps S10 to S40: Step S10: After the host web application is loaded, an operation task queue is generated based on the predefined configuration file read by the injected automated execution engine.

[0034] In one feasible implementation, step S10 includes steps A10 to A40: Step A10: After the host web application has finished loading, inject the operation task instruction automation execution engine and media recording controller into the page context; Step A20: Read the predefined configuration file through the automated execution engine; Step A30: The parser traverses the predefined configuration file and performs legality verification on each object in the predefined configuration file based on preset verification rules; Step A40: If the predefined configuration file is valid, the scheduler converts each object in the predefined configuration file into a task in memory, encapsulates each task into a corresponding operation task instruction, and generates an operation task queue.

[0035] Specifically, after the host web application loads, a JavaScript automation execution engine and a media recording controller are injected first. The automation execution engine reads a predefined JSON format configuration file, and then the parser iterates through the JSON array in the configuration file, validating the legality of each instruction in the array according to the validation rules. After successful validation, the scheduler converts the JSON object into a task queue in memory, with each task containing a specific execution function and termination condition.

[0036] Step S20: The scheduler executes the operation task instructions in the operation task queue in sequence. After executing any operation task instruction that causes a change in the page view, a visual feedback buffer window of a preset duration is started. After the visual feedback buffer window is closed, the execution of subsequent operation task instructions is suspended, and the media recording controller is controlled to enter a dynamic waiting state.

[0037] Specifically, after the scheduler executes the operation task instruction, it does not immediately interrupt recording. Instead, it executes visual temporary logic, maintaining the recording state for a period of time through a preset duration visual feedback buffer window, for example, maintaining the recording state for 100ms. This method ensures that the visual effects triggered by the operation instruction are fully recorded in the video, such as recording the "click state" or "ripple effect" of the operation button. After the visual feedback buffer window closes, the execution of subsequent operation task instructions in the task queue is suspended, and the recording control phase begins, controlling the media recording controller to enter a dynamic waiting state.

[0038] Step S30: During the dynamic waiting state, based on the monitored multi-dimensional state of the page view, non-linear time axis control is performed on the capture and writing of media stream data.

[0039] Step S40: After all operation task instructions are executed, the media stream data recorded by the media recording controller is encapsulated and exported to obtain the Web operation video.

[0040] In one possible implementation, steps B10 to B30 are included before step S30: Step B10: Monitor the frequency of DOM node changes through the Document Object Model observer, or monitor the pixel difference rate by requesting animation frames. When the frequency of node changes or the pixel difference rate is lower than the corresponding preset threshold, determine that the page view is in a stable state.

[0041] Step B20: Maintain the active request counter. Once the active request counter reaches zero and maintains a preset buffer duration, determine that the page view is in a stable network state.

[0042] Step B30: Monitor the audio context state or speech synthesis playback progress. When the audio stream playback ends or enters a silent period, determine that the page view is in the audio ready state.

[0043] Step S30 includes steps C10 to C30: Step C10: When an active audio context is detected, continue capturing media stream data; Step C20: When it is detected that the page view is in an audio ready state and the page view is in an unstable state or the network is unstable, pause the capture of media stream data or discard redundant frames so that the physical time elapsed is not included in the video timeline. Step C30: When it is detected that the page view is in an audio-ready state and the page view is in a stable state and the network is stable, resume the capture of media stream data, notify the scheduler to release the suspended state, and trigger the next operation instruction.

[0044] During the execution of the above steps, steps D10 to D20 are also included: Step D10: By setting the time slice parameters of the media recording controller, the media recording controller outputs media stream data slices according to the set output rules.

[0045] Step D20 uses asynchronous callbacks to write media stream data slices into the storage object of the host web application's local index database in real time. This allows the streaming media data slices in the storage object to be read in batches according to time sequence when exporting media stream data, using a database cursor. During streaming transmission, the streaming media data slices are dynamically encapsulated into the target video file.

[0046] And steps E10 to E20: In step E10, by maintaining the mapping relationship between the logical clock and the physical clock, if the waiting time exceeds the preset waiting time during the dynamic waiting state, the writing of media stream data is suspended through the media recording controller.

[0047] Step E20: Until the page view is in an audio-ready state and the page view is in a stable state and the network is stable, perform a recovery write operation through the media recording controller.

[0048] For example, by activating a multidimensional state detector, the browser's underlying interface can be used to listen for the following metrics on the page: Using the MutationObserver interface of the Document Object Model (DOM) observer, it captures changes in page elements in real time, much like a surveillance camera. If the page structure changes more than 10 times within 500ms (configurable), it is determined that the page is currently rendering.

[0049] The system intercepts network requests (XMLHttpRequest or fetch) through a proxy mechanism, effectively taking over the browser's sending channel. It maintains a counter and only determines the network is idle when all sent data requests have returned (the counter reaches zero) and the system remains silent.

[0050] If the current step involves speech synthesis, listen for the onend (playback ended) event of the speech synthesis object to ensure that the narration has been completely played.

[0051] At the same time, maintain two timelines: a physical timeline (the actual elapsed time) and a video timeline (the time when the final video is generated).

[0052] In this example scenario, after the user clicks "Login," the API response is slow, and the page loads in a 5-second loop. Using traditional recording methods, the video would also show a 5-second loop. However, in this example, the monitoring method described above detects that a network request has been initiated, indicating an unstable network connection. At this point, the media recording controller calls `MediaRecorder.pause` to pause the capture of media stream data, entering a sleep detection mode and ceasing to write video frames, thus allowing physical time to continue while freezing the video time. After 5 seconds, the data is returned, the DOM is updated, and the page view returns to a stable network state. The media recording controller then calls `MediaRecorder.resume()` to resume the capture of media stream data. Using this method, after the video is exported, the "Login Successful" screen immediately follows the "click" action in the video, with the lengthy 5 seconds in between automatically "edited out" (or only a very short transition animation is retained).

[0053] Furthermore, this example also sets configurable time slices when initializing the recording controller, for example, causing the mediaRecorder to generate a Blob slice every 1 second. Subsequently, unlike traditional solutions that store the Blob in an array (consuming RAM), this example performs the following operations: Open the IndexedDB database, initiate a read-write transaction, directly store the Blob binary data into the object store, record the index ID, release the memory reference of the Blob object, and force garbage collection. This ensures that there is almost no video data in memory when recording ends. The export process is as follows: Create a Blob combiner; use IndexedDB's openCursor to traverse all slices; read the slices sequentially and dynamically construct the final video file; trigger the browser to download.

[0054] This mechanism allows this example to record high-definition video files exceeding 2GB even on a low-spec computer with only 4GB of RAM, because at any given moment, only the data for the current second exists in memory.

[0055] In addition, to prevent automatic recording from getting stuck in an infinite loop (e.g., network requests suspending and not returning), this example uses a dual circuit breaker mechanism: If network requests do not return to zero within the preset maximum waiting time, the page view is forcibly determined to be in a stable network state, and the capture and writing of media stream data continues to prevent the recording process from deadlocking.

[0056] Use a Web Worker or Service Worker independent of the main thread to monitor the main thread's heartbeat in real time. If a script execution error or page crash event is captured, stop recording immediately and use the atomicity of database transactions to ensure that the media has been written to IndexedDB.

[0057] For example, if the predefined configuration file DSL is configured with `waitStrategy: "network_idle"`, but the interface times out and does not return, the scheduler will trigger a WARN log, forcing it to be considered "stable" and resuming recording. This ensures that the video is not interrupted; although a loading screen may be recorded, evidence of the event is preserved. This example starts a separate Web Worker thread as a "daemon" during initialization: Heartbeat monitoring: The main thread sends a heartbeat to the Worker every 1 second; Crash handling: If the Worker does not receive a heartbeat for more than 3 seconds (indicating a main thread crash / freeze), the Worker thread cannot directly manipulate the DOM, but can directly take over the IndexedDB connection; Data recovery: The Worker immediately marks the current recording session as "abnormally terminated." When the user refreshes the page or reopens the browser, the system detects this marker and automatically retrieves the remaining segment data from IndexedDB for video synthesis. This ensures that even in the extreme case of a browser crash, the video evidence from the last second before the crash is permanently preserved.

[0058] Furthermore, this example demonstrates how the system utilizes the Web Audio API to mix audio streams for video generation scenarios that require voice narration (such as TTS), and employs the "maximum duration principle" to ensure precise synchronization of audio and video.

[0059] This example utilizes the Web Audio API to create an audio context as the core environment for audio processing: The original audio stream of the host web application page (such as background audio in a video) and the audio stream generated by TTS are connected to the same MediaStreamDestination node; the output audio stream of this MediaStreamDestination node is passed as a parameter to the constructor of the media recording controller. This ensures that the final recorded video file contains a complete, mixed audio track.

[0060] In addition, the scheduler uses the maximum duration principle to determine the minimum recording duration T_record for the current operation step to avoid truncating the voice narration: Computational visual duration (T_visual): The video duration required for page visuals (DOM changes) and network requests to reach stability; Calculate audio duration (T_audio): The estimated total playback duration of the associated TTS speech stream, which can be pre-calculated by the underlying SpeechSynthesis Engine when parsing the text, or precisely determined by monitoring playback progress events; Actual recording duration (T_record): T_record = Max(T_visual, T_audio). Only when T_record is reached, that is, when all indicators meet the stability conditions, will the system notify the scheduler to release the suspension and trigger the next operation instruction.

[0061] In special scenarios where the page view is stable but the narration has not finished playing, the scheduler will suspend the execution of subsequent operation instructions, but will not execute MediaRecorder.pause(). Since the page DOM and network are stable, the image captured by the media recording controller will no longer change, i.e., it will continue to record static frames. Recording will continue until the TTS audio stream notifies that playback has ended through the onend event (T_audio is achieved). This ensures that the audio narration in the exported video will not be truncated, and the picture and audio will always maintain semantic alignment, completely solving the problem of audio-visual desynchronization.

[0062] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the web operation video generation method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0063] This application also provides a web-based video generation device; please refer to [reference needed]. Figure 3 The web-based video generation device includes: The operation queue generation module 10 is used to generate an operation task queue based on a predefined configuration file read by the injected automated execution engine after the host web application has been loaded.

[0064] The scheduling module 20 is used to execute operation task instructions in the operation task queue in sequence through the scheduler. After executing any operation task instruction that causes a change in the page view, a visual feedback buffer window of preset duration is started. After the visual feedback buffer window is closed, the execution of subsequent operation task instructions is suspended, and the media recording controller is controlled to enter a dynamic waiting state.

[0065] The recording control module 30 is used to perform non-linear time axis control on the capture and writing of media stream data based on the multi-dimensional state of the monitored page view during the dynamic waiting state.

[0066] The video export module 40 is used to encapsulate and export the media stream data recorded by the media recording controller after all operation task instructions have been executed, so as to obtain the web operation video.

[0067] The Web operation video generation apparatus provided in this application, employing the Web operation video generation method in the above embodiments, can solve the technical problem of high cost in recording and generating Web front-end operation videos in related technologies. Compared with related technologies, the beneficial effects of the Web operation video generation apparatus provided in this application are the same as those of the Web operation video generation method provided in the above embodiments, and other technical features in the Web operation video generation apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0068] This application provides a web-operated video generation device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the web-operated video generation method in the above embodiments.

[0069] The following is for reference. Figure 4 This document illustrates a structural diagram of a web-operated video generation device suitable for implementing embodiments of this application. The web-operated video generation device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 4 The web-based video generation device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this application.

[0070] like Figure 4As shown, the web-operated video generation device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory 1002 (ROM) or a program loaded from a storage device 1003 into a random access memory 1004 (RAM). The random access memory 1004 also stores various programs and data required for the operation of the web-operated video generation device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 (I / O interface) is also connected to the bus 1005. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the web-operated video generation device to communicate wirelessly or wiredly with other devices to exchange data. Although a web-operated video generation device with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0071] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0072] The Web-based video generation device provided in this application, employing the Web-based video generation method described in the above embodiments, can solve the technical problem of high cost in recording and generating Web front-end operation videos in related technologies. Compared with related technologies, the beneficial effects of the Web-based video generation device provided in this application are the same as those of the Web-based video generation method provided in the above embodiments, and other technical features of this Web-based video generation device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0073] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0074] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0075] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the Web-based video generation method in the above embodiments.

[0076] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0077] The aforementioned computer-readable storage medium may be included in the Web-operated video generation device; or it may exist independently and not assembled into the Web-operated video generation device.

[0078] The aforementioned computer-readable storage medium carries one or more programs. When these programs are executed by the Web-operated video generation device, the Web-operated video generation device: after the host Web application is loaded, generates an operation task queue based on a predefined configuration file read by the injected automated execution engine; executes the operation task instructions in the operation task queue sequentially through a scheduler; after executing any operation task instruction that causes a change in the page view, it initiates a visual feedback buffer window of a preset duration; after the visual feedback buffer window is closed, it suspends the execution of subsequent operation task instructions and controls the media recording controller to enter a dynamic waiting state; during the dynamic waiting state, it performs non-linear time axis control on the capture and writing of media stream data based on the monitored multi-dimensional state of the page view; after all operation task instructions are executed, it encapsulates and exports the media stream data recorded by the media recording controller to obtain the Web-operated video.

[0079] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0080] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0081] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0082] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described Web operation video generation method, thereby solving the technical problem of high cost of recording and generating Web front-end operation videos in related technologies. Compared with related technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the Web operation video generation method provided in the above embodiments, and will not be repeated here.

[0083] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the Web-based video generation method described above.

[0084] The computer program product provided in this application can solve the technical problem of high cost in generating web front-end operation videos in related technologies. Compared with related technologies, the beneficial effects of the computer program product provided in this application are the same as those of the web operation video generation method provided in the above embodiments, and will not be repeated here.

[0085] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A method for generating web-based video, characterized in that, The method includes: After the host web application is loaded, an operation task queue is generated based on the predefined configuration file read by the injected automated execution engine; The scheduler executes operation task instructions in the operation task queue in sequence. After executing any operation task instruction that causes a change in the page view, a visual feedback buffer window of a preset duration is started. After the visual feedback buffer window is closed, the execution of subsequent operation task instructions is suspended, and the media recording controller is controlled to enter a dynamic waiting state. During the dynamic waiting state, the capture and writing of media stream data are controlled non-linearly based on the monitored multi-dimensional state of the page view. After all the operation task instructions are executed, the media stream data recorded by the media recording controller is encapsulated and exported to obtain the Web operation video.

2. The Web-based video generation method as described in claim 1, characterized in that, The step of generating an operation task queue based on a predefined configuration file read by the injected automated execution engine after the host web application has been loaded includes: After the host web application has finished loading, inject the operation task instruction automation execution engine and media recording controller into the page context; The predefined configuration file is read through the automated execution engine; The parser traverses the predefined configuration file and performs legality checks on each object in the predefined configuration file based on preset verification rules. If the predefined configuration file is valid, the scheduler will convert each object in the predefined configuration file into a task in memory, encapsulate each task into a corresponding operation task instruction, and generate an operation task queue.

3. The Web-based video generation method as described in claim 1, characterized in that, Before the step of performing non-linear time-axis control on the capture and writing of media stream data based on the monitored multi-dimensional state of the page view, the method further includes the following step: The frequency of DOM node changes is monitored by the Document Object Model observer, or the pixel difference rate is monitored by requesting animation frames. When the frequency of node changes or the pixel difference rate is lower than the corresponding preset threshold, the page view is determined to be in a stable state. Maintain an active request counter. When the active request counter reaches zero and maintains a preset buffer duration, determine that the page view is in a stable network state. Monitor the audio context state or speech synthesis playback progress, and determine that the page view is in an audio-ready state when the audio stream playback ends or enters a silent period.

4. The Web-based video generation method as described in claim 3, characterized in that, The step of non-linear time-axis control of media stream data capture and writing based on the monitored multi-dimensional state of the page view includes: When an active audio context is detected, the capture of the media stream data is maintained; When it is detected that the page view is in an audio ready state and the page view is in an unstable state or a network unstable state, the capture of the media stream data is paused or redundant frames are discarded, so that the physical time elapsed is not included in the video timeline. When it is detected that the page view is in an audio-ready state and the page view is in a stable state and the network is stable, the capture of the media stream data is resumed, and the scheduler is notified to release the suspended state and trigger the next operation instruction.

5. The Web-based video generation method as described in claim 4, characterized in that, The step of performing non-linear time-axis control on the capture and writing of media stream data based on the monitored multi-dimensional state of the page view further includes: By setting the time slice parameters of the media recording controller, the media recording controller outputs media stream data slices according to the set output rules; Using asynchronous callbacks, the media stream data slices are written to the storage object of the local index database of the host web application in real time, so that when exporting media stream data, the streaming media data slices in the storage object are read in batches according to time sequence using database cursors, and the streaming media data slices are dynamically encapsulated into target video files during streaming transmission.

6. The Web-based video generation method as described in claim 5, characterized in that, The step of performing non-linear time-axis control on the capture and writing of media stream data based on the monitored multi-dimensional state of the page view further includes: By maintaining the mapping relationship between the logical clock and the physical clock, if the waiting time exceeds the preset waiting time during the dynamic waiting state, the writing of media stream data is suspended by the media recording controller. The recovery write operation is performed through the media recording controller only when the page view is in an audio-ready state and the page view is in a stable state and the network is stable.

7. A web-based video generation device, characterized in that, The device includes: The operation queue generation module is used to generate an operation task queue based on a predefined configuration file read by the injected automated execution engine after the host web application has finished loading. The scheduling module is used to execute operation task instructions in the operation task queue in sequence through the scheduler. After executing any operation task instruction that causes a change in the page view, a visual feedback buffer window of preset duration is started. After the visual feedback buffer window is closed, the execution of subsequent operation task instructions is suspended, and the media recording controller is controlled to enter a dynamic waiting state. The recording control module is used to perform non-linear time axis control on the capture and writing of media stream data based on the monitored multi-dimensional state of the page view during the dynamic waiting state. The video export module is used to encapsulate and export the media stream data recorded by the media recording controller after all the operation task instructions have been executed, so as to obtain the Web operation video.

8. A web-operated video generation device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the Web-based video generation method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the Web-based video generation method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the Web-based video generation method as described in any one of claims 1 to 6.

Images