Cross-domain video recording method, device, equipment, storage medium and program product

Abstract

The application discloses a cross-domain video recording method and device, equipment, storage medium and program product, relates to the technical field of video generation, and the method comprises the steps that when a recording task is started, a host control agent in a host domain is used to inject a controlled agent script and a session verification token into a target subdomain to build a secure transmission tunnel; a subdomain state change package transmitted by the controlled agent is acquired through the secure transmission tunnel, and a global stability index is determined in combination with the stability state of the host domain; the recording state of a media recording controller is controlled based on the global stability index, and a host domain media stream and / or a subdomain media stream recorded by the media recording controller is acquired; the host domain media stream and the subdomain media stream are subjected to coordinate rectification processing and clock alignment processing, and a cross-domain Web operation recording video is obtained. The application realizes the accurate recording of the cross-domain Web operation video which eliminates the perceptual vacuum and positioning distortion.

Description

Technical Field

[0001] This application relates to the field of video generation technology, and in particular to cross-domain video recording methods, apparatus, devices, storage media, and program products. Background Technology

[0002] In a microservice architecture, due to the browser's same-origin policy, the web frontend's ability to control video recording is limited to the host browser page. When cross-domain nested subpages are involved, the control end cannot penetrate the sandbox to obtain the rendering state within the subdomain, resulting in a perception vacuum and positioning distortion issues at cross-domain boundaries for non-linear recording logic. Due to the browser's same-origin policy, the main page script cannot access the DOM tree of cross-domain sub-documents or listen to their network requests. This prevents the main control unit from sensing the loading progress within the sub-domain, causing recording control to fail at cross-domain boundaries because it cannot obtain view stabilization signals.

[0003] When the recording stream involves high-resolution screens, viewport scaling, or complex scrolling nesting, there is a non-linear offset between the recorded logical operation coordinates and the captured physical pixel image. Because the dynamic offset within the subdomain cannot be obtained in real time, the interactive guidance annotations during playback cannot be accurately aligned. Summary of the Invention

[0004] The main objective of this application is to provide a cross-domain video recording method, apparatus, device, storage medium, and program product, aiming to solve the problems of perception vacuum and positioning distortion in cross-domain video recording of Web front-end operations in related technologies.

[0005] To achieve the above objectives, this application provides a cross-domain video recording method, the method comprising: When the recording task starts, the controlled agent script and session verification token are injected into the target subdomain through the master agent in the host domain to build a secure transmission tunnel; The subdomain state change packets transmitted by the controlled agent are obtained through the secure transmission tunnel, and the global stability index is determined in combination with the stability status of the host domain. Based on the global stability index, the recording state of the media recording controller is controlled to obtain the host domain media stream and / or subdomain media stream recorded by the media recording controller. The host domain media stream and the subdomain media stream are subjected to coordinate correction and clock alignment processing to obtain cross-domain web operation recording video.

[0006] In one embodiment, the step of injecting the controlled agent script and session verification token into the target subdomain through the master agent within the host domain to construct a secure transmission tunnel when the recording task is started includes: When the recording task starts, a random string is generated by the main control agent as the session verification token; Controlled proxy scripts can be sent to the target subdomain by dynamically creating script tags or injecting execution environments. The control agent receives a handshake request from the controlled agent, verifies the source, and then sends the session verification token back to the controlled agent so that all event data packets sent by the controlled agent carry the session verification token. The session verification token is bound to the lifecycle of the current recording task, and the control agent only processes data packets carrying the session verification token.

[0007] In one embodiment, the step of performing coordinate correction and clock alignment processing on the host domain media stream and the subdomain media stream includes: The controlled agent acquires the scroll offset and scaling factor of each nested carrier in the target subdomain, as well as the current physical pixel ratio of the target subdomain, through the secure transmission tunnel. A recursive algorithm is used to accumulate the scroll offset and scaling factor of each nested carrier and then explicitly introduce the physical pixel ratio to perform coordinate correction processing on the subdomain media stream, unifying it to the pixel coordinates of the host and the media stream.

[0008] In one embodiment, the step of performing coordinate correction and clock alignment processing on the host domain media stream and the subdomain media stream further includes: When a recording task starts, monitor the start / stop status of the media recording controller. When the media recording controller switches states, determine the current video logical time, physical recording time, and video frame index, and update the three-dimensional mapping table. When the master agent receives the event data packet, it determines the transmission delay of the controlled agent based on the round-trip time of the secure transmission tunnel measured by the heartbeat packet. Based on the transmission delay of the controlled agent and the video logic time of click operation events in the subdomain media stream, the synchronization logic time of the host domain media stream is determined. The three-dimensional mapping table is searched to determine the video frame index of the host domain media stream at the synchronization logical time. Combined with the start / stop status of the media recording controller, the video frame index of the target subdomain when the click operation event occurs is mapped to the video frame index of the master control terminal at the synchronization logical time, thus completing the clock alignment process.

[0009] In one embodiment, the method further includes: The heartbeat packet loss time of the controlled agent is accumulated. When the heartbeat packet loss time is greater than a preset threshold, the synchronization wait lock between the controlled agent and the controlled agent is disconnected.

[0010] Secondly, to achieve the above objectives, this application also provides a cross-domain video recording method applied to a target subdomain, the method comprising: The controlled proxy monitors the frequency of DOM node changes in the target subdomain. If the number of DOM node changes exceeds the preset number of changes within a specified time and the number of incomplete requests in the target subdomain drops to the preset number of requests, a subdomain state change packet is sent to the host domain through a secure transmission tunnel so that the host domain can control the recording status of the media recording controller.

[0011] Thirdly, to achieve the above objectives, this application further provides a cross-domain video recording apparatus, the apparatus comprising: The channel construction module is used to inject the controlled agent script and session verification token into the target subdomain through the master agent in the host domain when the recording task starts, thereby building a secure transmission tunnel. The state data transmission module is used to obtain the subdomain state change packet transmitted by the controlled agent through the secure transmission tunnel, and determine the global stability index by combining it with the stability state of the host domain. The recording control module is used to control the recording status of the media recording controller based on the global stability index, and to obtain the host domain media stream and subdomain media stream recorded by the media recording controller. The video acquisition module is used to perform coordinate correction and clock alignment processing on the host domain media stream and the subdomain media stream to obtain cross-domain web operation recording video.

[0012] Fourthly, to achieve the above objectives, this application further provides a cross-domain video recording device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the cross-domain video recording method described above.

[0013] Fifthly, to achieve the above objectives, this application further provides a storage medium, which is a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it implements the steps of the above-described cross-domain video recording method.

[0014] Sixthly, to achieve the above objectives, this application further provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the cross-domain video recording method described above.

[0015] One or more technical solutions proposed in this application have at least the following technical effects: 1. Through a distributed proxy architecture, the "perception vacuum" caused by the browser's same-origin policy is effectively eliminated, and the ability to control the non-linear timeline is seamlessly extended from a single host domain to a multi-level cross-domain nested environment.

[0016] 2. By explicitly introducing the device pixel ratio (DPR) and the recursive multiplication logic of multi-layered nested scaling factors into the coordinate mapping model, the problem of "positioning distortion" after high resolution screens and offset is solved.

[0017] 3. The adaptive sampling and reporting logic based on the difference triggering mechanism significantly reduces the computational load of the main thread when monitoring the DOM and rendering status in real time, ensuring high smoothness of the automated execution process and stable frame rate of the recorded video stream.

[0018] 4. By utilizing a transmission delay compensation algorithm based on round-trip time (RTT), a precise three-dimensional mapping relationship between logical and physical clocks was constructed, ensuring the synchronization accuracy of video and audio after invalid segments were removed, and avoiding timing offsets caused by cross-domain command transmission.

[0019] 5. The introduction of session verification tokens enables logical isolation between different recording tasks. Combined with a heartbeat-based circuit breaker recovery mechanism, it effectively prevents deadlock in the recording process caused by subdomain crashes or illegal jumps. Attached Figure Description

[0020] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, those skilled in the art can obtain other drawings based on these drawings without creative effort.

[0022] Figure 1 This is a flowchart illustrating the cross-domain video recording method in an embodiment of this application.

[0023] Figure 2 This is a schematic diagram of the model framework in the embodiments of this application.

[0024] Figure 3 This is a schematic diagram of the coordinate recursive correction logic in the embodiments of this application.

[0025] Figure 4 This is a schematic diagram of the delay compensation mechanism in the embodiments of this application.

[0026] Figure 5 This is a schematic diagram of the module connections for a cross-domain video recording device.

[0027] Figure 6 This is a schematic diagram of a cross-domain video recording device.

[0028] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0029] It should be understood that the specific embodiments described herein are merely illustrative of the technical solutions of this application and are not intended to limit this application.

[0030] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0031] The main solution of this application embodiment is as follows: by building a master proxy in the host domain, a controlled proxy script and session verification token are injected into the target subdomain to establish a secure transmission tunnel, bypassing the perception restriction of the same-origin policy; the tunnel is used to obtain the view changes and request status of the subdomain in real time to determine the global stability index, thereby accurately controlling the recording timing; at the same time, a recursive algorithm is used to accumulate multi-layer nested offsets and scaling factors for coordinate correction, and the subdomain operation is mapped to the host domain video frame based on the heartbeat packet round-trip delay to complete clock alignment, and finally a cross-domain web operation recording video is generated that eliminates the perception vacuum and positioning distortion.

[0032] Specifically, this application provides a cross-domain video recording method, referring to... Figure 1 The cross-domain video recording method includes steps S10~S40: Step S10: When the recording task starts, the controlled agent script and session verification token are injected into the target subdomain through the master agent in the host domain to build a secure transmission tunnel.

[0033] It should be noted that the cross-domain secure collaborative communication mechanism constructed in this embodiment is a distributed functional extension of the Web automation execution engine and media recording controller. The master agent is deployed in the automation execution engine of the host Web application (host domain), while the controlled agent is injected into the cross-domain subdomain as an execution plugin. The two achieve state consensus through an encrypted instruction tunnel, thereby extending the non-linear timeline control scope described in the main case from a single host domain to a multi-layered cross-domain nested environment.

[0034] In one feasible implementation, step S10 includes steps A10 to A30: Step A10: When the recording task starts, a random string is generated by the main control agent as a session verification token.

[0035] Step A20: The controlled proxy script is sent to the target subdomain by dynamically creating a script tag or by injecting the execution environment.

[0036] Step A30: Obtain the handshake request from the controlled agent, verify the source, and send a session verification token back to the controlled agent so that all event data packets sent by the controlled agent carry the session verification token. The session verification token is bound to the lifecycle of the current recording task, and the master agent only processes data packets carrying the session verification token.

[0037] Specifically, this implementation establishes an encrypted command tunnel that is bound to the recording lifecycle. When the master agent injects into the controlled agent, it issues a session verification token generated based on the currently recorded session. It is understandable that secure injection into the controlled agent can be achieved when the controlled subdomain's Content-Security-Policy (CSP) allows specific script injection or there is trusted communication authorization between the master and slave domains, ensuring that the controlled agent within the security sandbox can be mounted correctly.

[0038] Understandably, the master agent only processes data packets carrying the session verification token. Through this mechanism, which is strongly bound to the lifecycle of a single recording task, a unique recording collaboration effect is achieved: on the one hand, in a cross-domain asynchronous environment where a recording task is unexpectedly interrupted and quickly restarted, the master agent can directly intercept and discard the delayed state packets (i.e., 'zombie packets') left over from the previous round of tasks, eliminating the timing pollution of the current recording timeline by historical asynchronous callbacks; on the other hand, this also provides strict physical and logical isolation for multiple independent recording tasks running concurrently under the same host domain, preventing instruction crosstalk between different recording streams.

[0039] In one example, refer to Figure 2 The system runs within the host web application environment. Once the automation execution engine reads the predefined configuration file and generates an operation task queue, the master control agent starts. Between the host page and nested cross-domain subpages, a secure tunnel is initialized using the media recording controller's time-slice parameters and local database storage capabilities, employing a 32-bit random token to ensure that the timing of cross-domain asynchronous data writes is strictly aligned with the host domain's logical clock.

[0040] When a recording task starts, the master agent generates a 32-bit random string as a session verification token using `crypto.getRandomValues()`. The master client sends the controlled agent's script and token to the subdomain via dynamically created script tags or execution environment injection. After loading, the controlled agent sends a `HANDSHAKE_REQUEST` to `window.parent`. The master client verifies the origin and sends back the token. The controlled agent caches this token in memory, and all subsequent event packets carry this token in the header or payload for identity verification.

[0041] Step S20: Obtain the subdomain state change packet transmitted by the controlled agent through the secure transmission tunnel, and determine the global stability index by combining it with the stability status of the host domain.

[0042] Step S30: Based on the global stability index, control the recording status of the media recording controller and obtain the host domain media stream and / or subdomain media stream recorded by the media recording controller.

[0043] Specifically, to address the performance overhead risk caused by high-frequency eavesdropping, this embodiment proposes a low-load state consensus logic: the controlled agent does not transmit the original signal in real time, but instead reports the stability factor vector (number of DOM nodes W) through a difference-triggered mechanism. dom Unprocessed requests W net State change packets are only sent through the tunnel when the frequency of DOM changes or the number of network requests exceeds a preset jitter threshold σ.

[0044] The master control unit uses a weighted formula to calculate the global stability index. S global : S global = α f ( State host )+ β g ( State sub ) Where, α, β For dynamic weights, State host This represents the stable state of the host domain. State sub This represents the stability state of the target subdomain.

[0045] In one feasible implementation, the method applied to the target subdomain includes step S21: Step S21: Monitor the DOM node change frequency of the target subdomain through the controlled proxy. If the number of DOM node changes exceeds the preset number of changes within a specified time and the number of incomplete requests of the target subdomain drops to the preset number of requests, then send the subdomain status change packet to the main control proxy through the secure transmission tunnel.

[0046] For example, to optimize performance and avoid stuttering caused by continuous monitoring, this implementation demonstrates how a controlled agent can intelligently report its status: In this example, the controlled proxy uses MutationObserver to listen to the subdomain's main container. A jitter threshold is set. That is, more than 10 nodes change within 100ms, which is considered unstable.

[0047] If the current And the previous cycle If not, then no reporting will be triggered.

[0048] If the current The value dropped sharply to 2 (rendering was nearing completion), and When the number of incomplete requests drops from 5 to 0, the system determines that a substantial shift has occurred in the status and immediately sends a status change packet to the tunnel.

[0049] After receiving the above signal, the main control unit calculates the result using a formula, taking into account the stability of the host environment. .when When the value is below the set threshold, it indicates that the network status is unstable or that a non-target event has occurred. At this time, the media recording controller is put into a dynamic waiting state to ensure that the final recorded video does not contain loading rotation animations or white screen segments.

[0050] Step S40: Perform coordinate correction and clock alignment processing on the host domain media stream and the subdomain media stream to obtain the cross-domain web operation recording video.

[0051] In one feasible implementation, step S40 includes steps B10 to B60: Step B10: Obtain the scroll offset and scaling factor of each nested carrier in the target subdomain collected by the controlled agent through a secure transmission tunnel, as well as the current physical pixel ratio of the target subdomain.

[0052] Step B20 employs a recursive algorithm to explicitly introduce physical pixel ratios after accumulating the scroll offsets and scaling factors of each nested carrier layer, in order to perform coordinate correction processing on the subdomain media stream and unify it to the pixel coordinates of the host and the media stream.

[0053] Step B30: When the recording task starts, monitor the start / stop status of the media recording controller. When the media recording controller switches states, determine the current video logical time, physical recording time, and video frame index, and update the three-dimensional mapping table.

[0054] Step S40: When the master agent receives the event data packet, it determines the transmission delay of the controlled agent based on the round-trip time of the secure transmission tunnel measured by the heartbeat packet.

[0055] Step S50: Determine the synchronization logic time of the host domain media stream based on the transmission delay of the controlled agent and the video logic time of the click operation event in the subdomain media stream.

[0056] Step S60: Search the three-dimensional mapping table to determine the video frame index of the host domain media stream at the synchronization logical time. Combined with the start / stop status of the media recording controller, map the video frame index of the target subdomain when the click operation event occurs to the video frame index of the master control terminal at the synchronization logical time to complete the clock alignment process.

[0057] Specifically, refer to Figure 3 To address the "positioning distortion" problem, this embodiment establishes a recursive dynamic coordinate compensation model: The controlled agent transmits the scroll offset within the subdomain via tunneling. x, y and scaling factor And the current physical pixel ratio (DPR).

[0058] The master control unit uses a recursive algorithm to accumulate the offsets at each level and explicitly introduces DPR for physical pixel conversion, calculating the composite coordinates in the unified physical video stream in real time. , The calculation method is as follows: in, These are the logical starting coordinates of the subdomain container within the host environment. This formula ensures that, in a Retina screen environment, the annotation position is perfectly aligned with the physical pixels in the video stream.

[0059] Reference Figure 4 To distinguish it from the commonly known simple `performance.now()`, the core of this invention lies in constructing a clock mapping model with delay compensation: Specifically, the three-dimensional mapping table in this application is not statically preset, but dynamically maintained by the main control agent based on the state changes of the media recording controller during the recording task execution. It is used to record segmented breakpoints during the non-linear recording process. To handle the culling time during recording, the table also records the recording's "RESUME" and "PAUSE" states: Record the initial offset when the recording task starts: = { State:'RESUME', , , }

[0060] Whenever a global stability metric triggers a state transition in the media recording controller, the master control agent collects the system's absolute logical time at the current instant. And read the total duration of the currently accumulated physical video. and total number of video frames Insert a new row into the mapping table: When a pause is triggered: Insert a record row. = {State: 'PAUSE', , , After this, physical time stops accumulating.

[0061] When a resume is triggered: Insert a new record row. = {State: 'RESUME', , , }

[0062] The table is represented in memory as a list... The mapping sequence is arranged in ascending order, with each node representing the "starting anchor point" of a valid recorded segment.

[0063] When the controlled agent captures an interaction event (such as a click) in the target subdomain, the alignment steps are as follows: The transmission delay of the command tunnel is estimated in real time using the heartbeat round-trip time (RTT). .

[0064] The controlled agent carries the subdomain's local timestamp when sending event packets. When the master receives the subdomain's logical time... At that time, calculate the synchronization logical time of the event from the perspective of the host domain: Search for the matching expression in the three-dimensional mapping table. Maximum index row This line represents the valid video recording segment to which the click event belongs.

[0065] according to The State identifier performs segmented mapping computation: like If the state is RESUME (i.e., the event occurred during the normal recording period), then the video frame index corresponding to the subdomain click event is determined as follows: F target like The status is PAUSE, indicating that the corresponding frame was not written to the physical video stream when the event occurred. The system directly maps and snaps the event to the last frame before the pause, i.e.: Visual feedback (such as click animation, ripple effect) of subdomain click events can be directly drawn or annotated on the physical video stream. On the frame.

[0066] For example, after video recording is complete, the system needs to align the logical events occurring in the subdomain with the physical video frames. If the RTT of the current command tunnel is measured to be 40ms via heartbeat packets, then the estimated transmission delay is... The controlled agent records the video logic timing of click events. .

[0067] The master control unit calculates the synchronization time: Find the physical recording time. We found that 5020ms corresponds to the 151st frame of the video (assuming 30fps).

[0068] Thus, even if some loading time is removed during the recording process, the system can still pass { , , A mapping table ensures that the click animation appears precisely at frame 151 of the video during playback.

[0069] Understandably, in cross-domain scenarios, due to browser same-origin policies and network jitter, when events from the subdomain are propagated back, the recording of the host domain may be in a "stability waiting" pause. Traditional linear time compensation (such as NTP) will cause alignment failures because it ignores recording breakpoints. This solution ensures that even in the "time gaps" of non-continuous recording, asynchronous events can be accurately traced back and locked to physical video frames by recording state change anchor points. arrive The direct mapping eliminates multiple timing distortions caused by cross-domain instruction delays, host domain main thread blocking, and media encoder buffer delays, achieving frame-level alignment of "audio, video, and interaction" in cross-domain web operation recording under Retina high-resolution screen environment.

[0070] Furthermore, the method also includes step C10: Step C10: Accumulate the heartbeat packet loss time of the controlled agent. When the heartbeat packet loss time is greater than a preset threshold, disconnect the synchronization wait lock with the controlled agent.

[0071] Specifically, this implementation provides a health monitoring method based on heartbeat interaction; once the heartbeat loss exceeds a threshold... This immediately triggers a circuit breaker and resumes recording to prevent deadlock.

[0072] For example, assume that the master agent and the controlled agent maintain heartbeat communication every 100ms. During recording, if a cross-domain subpage causes severe blocking of the main thread due to performing extremely complex business calculations, results in an uncaught memory overflow, or causes an unauthorized page redirection, the controlled agent will be unable to continue sending heartbeat packets.

[0073] The main control unit accumulates the heartbeat loss time in real time. When the heartbeat packet loss exceeds the preset circuit breaker recovery threshold Tmax (for example, Tmax=500ms, which means 5 consecutive heartbeat cycles are lost), the system determines that the encrypted command tunnel has been disconnected or the subdomain has lost contact.

[0074] Upon triggering the circuit breaker, the master control agent immediately and forcibly disconnects the synchronization wait lock with the controlled subdomain. The media recording controller on the master end no longer waits indefinitely for the subdomain to return stability metrics or logical clocks, but forcibly releases the current "suspended" state and resumes recording based on the master end's current system physical clock (Tphys). The system will mark this abnormal time point in the recording log and continue executing subsequent operation queues, thus completely avoiding a permanent deadlock in the entire automated testing and video recording task due to the crash of a single cross-domain subpage.

[0075] It should be noted that the above examples are only for understanding this application and do not constitute a limitation on the cross-domain video recording method of this application. Any simple modifications based on this technical concept are within the protection scope of this application.

[0076] This application also provides a cross-domain video recording device; please refer to... Figure 5 The cross-domain video recording device includes: The channel construction module 10 is used to inject the controlled agent script and session verification token into the target subdomain through the master agent in the host domain when the recording task starts, thereby building a secure transmission tunnel.

[0077] The state data transmission module 20 is used to obtain the subdomain state change packets transmitted by the controlled agent through a secure transmission tunnel, and determine the global stability index by combining the stability status of the host domain.

[0078] The recording control module 30 is used to control the recording status of the media recording controller based on global stability indicators, and to obtain the host domain media stream and subdomain media stream recorded by the media recording controller.

[0079] The video acquisition module 40 is used to perform coordinate correction and clock alignment processing on the host domain media stream and the subdomain media stream to obtain cross-domain web operation recorded video.

[0080] The cross-domain video recording apparatus provided in this application, employing the cross-domain video recording method in the above embodiments, can solve the technical problems of perception vacuum and positioning distortion in cross-domain recording of Web front-end operation videos in related technologies. Compared with related technologies, the beneficial effects of the cross-domain video recording apparatus provided in this application are the same as those of the cross-domain video recording method provided in the above embodiments, and other technical features in the cross-domain video recording apparatus are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0081] This application provides a cross-domain video recording device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the cross-domain video recording method in the above embodiments.

[0082] The following is for reference. Figure 6 This document illustrates a structural schematic diagram of a cross-domain video recording device suitable for implementing embodiments of this application. The cross-domain video recording device in these embodiments may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Description), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 6 The cross-domain video recording device shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0083] like Figure 6 As shown, the cross-domain video recording device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.), which can perform various appropriate actions and processes according to a program stored in the read-only memory 1002 (ROM) or a program loaded from the storage device 1003 into the random access memory 1004 (RAM). The random access memory 1004 also stores various programs and data required for the operation of the cross-domain video recording device. The processing unit 1001, the read-only memory 1002, and the random access memory 1004 are interconnected via a bus 1005. An input / output interface 1006 (I / O interface) is also connected to the bus 1005. Typically, the following systems can be connected to the input / output interface 1006: input devices 1007 including, for example, touchscreens, touchpads, keyboards, mice, image sensors, microphones, accelerometers, gyroscopes, etc.; output devices 1008 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 1003 including, for example, magnetic tapes, hard disks, etc.; and communication devices 1009. Communication device 1009 allows the cross-domain video recording device to communicate wirelessly or wiredly with other devices to exchange data. Although a cross-domain video recording device with various systems is shown in the figure, it should be understood that it is not required to implement or possess all the systems shown. More or fewer systems may be implemented alternatively.

[0084] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0085] The cross-domain video recording device provided in this application, employing the cross-domain video recording method in the above embodiments, can solve the technical problems of perception vacuum and positioning distortion in cross-domain recording of Web front-end operation videos in related technologies. Compared with related technologies, the beneficial effects of the cross-domain video recording device provided in this application are the same as those of the cross-domain video recording method provided in the above embodiments, and other technical features in this cross-domain video recording device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0086] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0087] The above are merely specific embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0088] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the cross-domain video recording method in the above embodiments.

[0089] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0090] The aforementioned computer-readable storage medium may be included in the cross-domain video recording device; or it may exist independently and not assembled into the cross-domain video recording device.

[0091] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0092] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0093] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0094] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described cross-domain video recording method. This solves the technical problems of perception vacuum and positioning distortion in cross-domain video recording via web front-end operation in related technologies. Compared with related technologies, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the cross-domain video recording method provided in the above embodiments, and will not be repeated here.

[0095] This application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps of the cross-domain video recording method described above.

[0096] The computer program product provided in this application can solve the technical problems of perception vacuum and positioning distortion in cross-domain video recording of Web front-end operation in related technologies. Compared with related technologies, the beneficial effects of the computer program product provided in this application are the same as those of the cross-domain video recording method provided in the above embodiments, and will not be repeated here.

[0097] The above are only some embodiments of this application and do not limit the patent scope of this application. All equivalent structural transformations made under the technical concept of this application and using the contents of the specification and drawings of this application, or direct / indirect applications in other related technical fields, are included in the patent protection scope of this application.

Claims

1. A cross-domain video recording method, characterized in that, When applied to a host domain, the method includes: When the recording task starts, the controlled agent script and session verification token are injected into the target subdomain through the master agent in the host domain to build a secure transmission tunnel; The subdomain state change packets transmitted by the controlled agent are obtained through the secure transmission tunnel, and the global stability index is determined in combination with the stability status of the host domain. Based on the global stability index, the recording state of the media recording controller is controlled to obtain the host domain media stream and / or subdomain media stream recorded by the media recording controller. The host domain media stream and the subdomain media stream are subjected to coordinate correction and clock alignment processing to obtain cross-domain web operation recording video.

2. The cross-domain video recording method as described in claim 1, characterized in that, The step of injecting the controlled agent script and session verification token into the target subdomain through the master agent within the host domain to construct a secure transmission tunnel when the recording task is started includes: When the recording task starts, a random string is generated by the main control agent as the session verification token; Controlled proxy scripts can be sent to the target subdomain by dynamically creating script tags or injecting execution environments. The master agent receives a handshake request from the controlled agent, verifies the source, and then sends the session verification token back to the controlled agent so that all event data packets sent by the controlled agent carry the session verification token. The session verification token is bound to the lifecycle of the current recording task, and the master agent only processes data packets carrying the session verification token.

3. The cross-domain video recording method as described in claim 1, characterized in that, The steps of performing coordinate correction and clock alignment processing on the host domain media stream and the subdomain media stream include: The controlled agent acquires the scroll offset and scaling factor of each nested carrier in the target subdomain, as well as the current physical pixel ratio of the target subdomain, through the secure transmission tunnel. A recursive algorithm is used to accumulate the scroll offset and scaling factor of each nested carrier and then explicitly introduce the physical pixel ratio to perform coordinate correction processing on the subdomain media stream, unifying it to the pixel coordinates of the host and the media stream.

4. The cross-domain video recording method as described in claim 3, characterized in that, The step of performing coordinate correction and clock alignment processing on the host domain media stream and the subdomain media stream further includes: When a recording task is started, the start / stop status of the media recording controller is monitored. When the status of the media recording controller changes, the current video logical time, physical recording time, and video frame index are determined, and the three-dimensional mapping table is updated. When the master agent receives the event data packet, it determines the transmission delay of the controlled agent based on the round-trip time of the secure transmission tunnel measured by the heartbeat packet. Based on the transmission delay of the controlled agent and the video logic time of the click operation event in the subdomain media stream, the synchronization logic time of the host domain media stream is determined. The three-dimensional mapping table is searched to determine the video frame index of the host domain media stream at the synchronization logical time. Combined with the start / stop status of the media recording controller, the video frame index of the target subdomain when the click operation event occurs is mapped to the video frame index of the master control terminal at the synchronization logical time, thus completing the clock alignment process.

5. The cross-domain video recording method as described in claim 4, characterized in that, The method further includes: The heartbeat packet loss time of the controlled agent is accumulated. When the heartbeat packet loss time is greater than a preset threshold, the synchronization wait lock between the controlled agent and the controlled agent is disconnected.

6. A cross-domain video recording method, characterized in that, Applied to a target subdomain, the method includes: The controlled proxy monitors the frequency of DOM node changes in the target subdomain. If the number of DOM node changes exceeds the preset number of changes within a specified time, and the number of incomplete requests in the target subdomain drops to the preset number of requests, then a subdomain state change packet is sent to the host domain through a secure transmission tunnel, so that the host domain can control the recording status of the media recording controller.

7. A cross-domain video recording device, characterized in that, The device includes: The channel construction module is used to inject the controlled agent script and session verification token into the target subdomain through the master agent in the host domain when the recording task starts, thereby building a secure transmission tunnel. The state data transmission module is used to obtain the subdomain state change packet transmitted by the controlled agent through the secure transmission tunnel, and determine the global stability index by combining it with the stability state of the host domain. The recording control module is used to control the recording status of the media recording controller based on the global stability index, and to obtain the host domain media stream and subdomain media stream recorded by the media recording controller. The video acquisition module is used to perform coordinate correction and clock alignment processing on the host domain media stream and the subdomain media stream to obtain cross-domain web operation recording video.

8. A cross-domain video recording device, characterized in that, The device includes: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the cross-domain video recording method as described in any one of claims 1 to 6.

9. A storage medium, characterized in that, The storage medium is a computer-readable storage medium, and a computer program is stored on the storage medium. When the computer program is executed by a processor, it implements the steps of the cross-domain video recording method as described in any one of claims 1 to 6.

10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the cross-domain video recording method as described in any one of claims 1 to 6.

Images