Video generation method and apparatus

Abstract

This application provides a video generation method and apparatus. The method includes: acquiring historical operation data generated by operating an application in a client, and acquiring historical state data generated by the application responding to operations; running the application according to the chronological order of the operations, based on the historical operation data and historical state data, to obtain a display page; and generating a video displaying the operation of the application in the client based on the display page. This method can recreate the process of operating the application in the client while saving significant storage space. Using big data technology to execute the video generation method of this application can greatly improve the efficiency of video generation.

Description

Technical Field

[0001] This application relates to the fields of computer and communication technology, and in particular to a video generation method and apparatus. Background Technology

[0002] In existing technologies, screen recording or screenshots can be used to record the process of a client operating an application, thereby providing proof of the application's operation process.

[0003] However, users of the application may forget to record or take screenshots. Even if the application has built-in screen recording or screenshot functions, the images and videos generated by recording or screenshotting will take up a lot of storage space, which is very inconvenient. Summary of the Invention

[0004] This application aims to provide a video generation method and apparatus that can save storage space to a certain extent while reproducing the process of operating a client application.

[0005] According to one aspect of the embodiments of this application, a video generation method is provided, comprising: acquiring historical operation data generated by operating an application in a client, and acquiring historical state data generated by the application in response to the operation; running the application based on the historical operation data and the historical state data in chronological order of the operation to obtain a display page; and generating a video displayed on the client based on the display page.

[0006] According to one aspect of the embodiments of this application, a video generation apparatus is provided, comprising: an acquisition module configured to acquire historical operation data generated by a client operating an application, and acquire historical state data generated by the application in response to the operation; a running module configured to run the application based on the historical operation data and the historical state data in chronological order of the operation to obtain a display page; and a generation module configured to generate a video of the application being operated on the client based on the display page.

[0007] In one embodiment of this application, based on the foregoing scheme, the historical operation data includes the click position of the application in the client, and the historical state data includes the historical click page state data generated by the application in response to the click operation at the click position; the running module is configured to: pre-embed nodes in the application and set the style of the nodes to floating icons; control the floating icons to be displayed at the click position; and modify the page state data of the page corresponding to the click position to the historical click page state data.

[0008] In one embodiment of this application, based on the foregoing scheme, the historical operation data includes the sliding position generated by sliding the application in the client, and the running module is configured to: slide the page corresponding to the sliding position according to the sliding position.

[0009] In one embodiment of this application, based on the foregoing scheme, the historical operation data includes interface requests generated by calling the application interface in the client; the historical state data includes return page state data obtained by the application in response to the interface request; and the running module is configured to: control the application to send the interface request; intercept the interface request sent by the application; search for historical return page state data corresponding to the interface request in the historical state data based on the intercepted interface request; and generate a return page based on the historical return page state data corresponding to the interface request.

[0010] In one embodiment of this application, based on the foregoing scheme, the running module is configured to: obtain the interface address targeted by the interface request; and search for the return data corresponding to the interface address in the historical operation data, and use it as the return data.

[0011] In one embodiment of this application, based on the foregoing scheme, the historical operation data includes the subcomponent identifier corresponding to the subcomponent in the application modified in the client, and the historical state data includes the historical subcomponent state data generated by the application responding to the modification of the subcomponent in the client. The running module is configured to: obtain the subcomponent identifier in the historical operation data; modify the subcomponent state data corresponding to the subcomponent identifier to the historical subcomponent state data; and modify the page where the subcomponent is located based on the historical subcomponent state data.

[0012] In one embodiment of this application, based on the foregoing scheme, the historical operation data includes a page identifier corresponding to the page state of the application modified in the client, and the historical state data includes historical page state data generated by the application responding to the modification of the page state in the client. The running module is configured to: obtain the page identifier in the historical operation data; and modify the page state data corresponding to the page identifier to the historical page state data.

[0013] In one embodiment of this application, based on the foregoing scheme, the acquisition module is configured to: before operating the application in the client, add monitoring code to the member function of the application's code to monitor state change events; and collect the changed state data corresponding to the state change events as the historical state data.

[0014] In one embodiment of this application, based on the foregoing scheme, the generation module is configured to: take a screenshot of the display page to obtain a display screenshot; obtain the display duration of the display screenshot based on the operation time corresponding to the operation; and combine the display screenshots corresponding to the display page according to the display duration in the order in which the display page is generated to obtain the video.

[0015] According to one aspect of the embodiments of this application, a computer-readable program medium is provided, which stores computer program instructions that, when executed by a computer, cause the computer to perform any of the methods described above.

[0016] According to one aspect of the embodiments of this application, an electronic device is provided, including: a processor; and a memory storing computer-readable instructions, which, when executed by the processor, implement the method as described in any of the preceding claims.

[0017] According to one aspect of the embodiments of this application, a computer program product or computer program is provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various alternative embodiments described above.

[0018] The technical solutions provided by the embodiments of this application may include the following beneficial effects:

[0019] In some embodiments of this application, the technical solutions provided involve acquiring historical operation data generated by operating the application in the client and acquiring historical state data generated by the application's response to operations. The application is then run based on the historical operation data and historical state data in chronological order of the operations to obtain a display page. Based on the display page, a video showing the operation of the application in the client is generated. Compared with storing images or videos in the prior art, storing historical operation data and historical state data requires less space, which can save a lot of storage space while restoring the process of operating the application in the client.

[0020] It should be understood that the above general description and the following detailed description are merely exemplary and do not limit this application. Attached Figure Description

[0021] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the specification, serve to explain the principles of this application.

[0022] Figure 1A schematic diagram of an exemplary system architecture to which the technical solutions of the embodiments of this application can be applied is shown;

[0023] Figure 2 A flowchart illustrating a video generation method according to an embodiment of this application is shown schematically;

[0024] Figure 3 This illustration schematically shows a process diagram of a video generation method according to an embodiment of the present application;

[0025] Figure 4 A block diagram of a video generation apparatus according to one embodiment of this application is shown schematically;

[0026] Figure 5 This is a hardware diagram of an electronic device according to an exemplary embodiment. Detailed Implementation

[0027] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this application more comprehensive and complete, and to fully convey the concept of the exemplary embodiments to those skilled in the art.

[0028] Furthermore, the described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. Numerous specific details are provided in the following description to give a thorough understanding of embodiments of this application. However, those skilled in the art will recognize that the technical solutions of this application can be practiced without one or more of the specific details, or other methods, components, apparatuses, steps, etc., can be employed. In other instances, well-known methods, apparatuses, implementations, or operations are not shown or described in detail to avoid obscuring various aspects of this application.

[0029] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0030] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily need to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0031] Cloud technology refers to a hosting technology that unifies a series of resources such as hardware, software, and networks within a wide area network or local area network to achieve data computing, storage, processing, and sharing.

[0032] Cloud technology is a collective term for network technologies, information technologies, integration technologies, management platform technologies, and application technologies applied to the cloud computing business model. It can form resource pools, providing flexible and convenient on-demand access. Cloud computing technology will become a crucial support. Backend services of technical network systems require substantial computing and storage resources, such as video websites, image websites, and many portal websites. With the rapid development and application of the internet industry, every item may have its own identification mark in the future, requiring transmission to backend systems for logical processing. Data at different levels will be processed separately, and various industry data will all require robust system support, which can only be achieved through cloud computing.

[0033] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that attempts to understand the essence of intelligence and produce a new kind of intelligent machine that can react in a way similar to human intelligence. AI studies the design principles and implementation methods of various intelligent machines, enabling them to possess the functions of perception, reasoning, and decision-making.

[0034] Artificial intelligence (AI) is a comprehensive discipline encompassing a wide range of fields, including both hardware and software technologies. Fundamental AI technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies primarily include computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0035] With the research and advancement of artificial intelligence (AI) technology, AI is being studied and applied in various fields, such as smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, autonomous driving, drones, robots, smart healthcare, and smart customer service. It is believed that with the development of technology, AI will be applied in more fields and play an increasingly important role.

[0036] The solutions provided in this application relate to big data processing technology in the fields of cloud technology and artificial intelligence. Big data refers to data sets that cannot be captured, managed, and processed within a certain time frame using conventional software tools. It represents massive, rapidly growing, and diverse information assets that require new processing models to achieve stronger decision-making, insight discovery, and process optimization capabilities. With the advent of the cloud era and the artificial intelligence era, big data has attracted increasing attention. Big data requires specialized technologies to effectively process large amounts of data within a tolerable time frame. Technologies suitable for big data include massively parallel processing databases, data mining, distributed file systems, distributed databases, cloud computing platforms, the Internet, and scalable storage systems. Using big data technology to execute the video generation method of this application can greatly improve the efficiency of video generation, as illustrated in the following embodiments:

[0037] Figure 1 A schematic diagram of an exemplary system architecture 100 to which the technical solutions of the embodiments of this application can be applied is shown.

[0038] like Figure 1 As shown, system architecture 100 may include client 101, network 102, data collection server 103, and backtracking server 104. Network 102 serves as the medium for providing communication links between client 101, data collection server 103, and backtracking server 104. Network 102 may include various connection types, such as wired communication links, wireless communication links, etc., which are not limited herein.

[0039] It should be understood that Figure 1 The number of client 101, network 102, data collection server 103, and backtracking server 104 shown is merely illustrative. Depending on implementation needs, there can be any number of client 101, network 102, data collection server 103, and backtracking server 104. For example, server 103 can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. Client 101 can be a smartphone, tablet, laptop, desktop computer, smart speaker, smartwatch, etc., but is not limited to these.

[0040] In one embodiment of this application, the backtracking server 104 obtains historical operation data generated by operating the application in the client by the data collection server 103, and obtains historical state data generated by the application response operation. According to the time sequence of the operation, the application is run based on the historical operation data and historical state data to obtain a display page. Based on the display page, a video showing the operation of the application in the client is generated. Compared with storing pictures or videos in the prior art, storing historical operation data and historical state data requires less space, which can save a lot of storage space while restoring the process of operating the application in the client.

[0041] It should be noted that the video generation method provided in this application embodiment is generally executed by the backtracking server 104, and correspondingly, the video generation device is generally located in the backtracking server 104. However, in other embodiments of this application, the data collection server 103 and the client 101 may also have similar functions to the backtracking server 104, thereby executing the video generation method provided in this application embodiment.

[0042] The implementation details of the technical solutions in the embodiments of this application are described in detail below:

[0043] Figure 2 The flowchart illustrating one embodiment of the video generation method according to this application is shown. The execution entity of the video generation method may be a server, for example, a... Figure 1 The backtracking server 104 shown is illustrated.

[0044] Reference Figure 2 As shown, the video generation method includes at least steps S210 to S230, which are described in detail below:

[0045] In step S210, historical operation data generated by operating the application in the client is obtained, and historical state data generated by the application in response to the operation is obtained.

[0046] In one embodiment of this application, the application can run on a client system, or the application can run on other applications. For example, the application can be a mini program, which can be used without downloading and installing. Users can open the application by scanning a QR code or searching for it.

[0047] In one embodiment of this application, historical operation data can be: operation data generated by operations performed on the application in the client within a time period prior to acquiring historical operation data, wherein the operation can be performed by a robot through the client or by a user through the client. For example, when the application receives a click or touch operation on the application in the client, the operation data can include click data generated by clicking or touching the application in the client, and the click data can include click location, click force, click duration, etc., and the click location can include the clicked page and click coordinates, etc.; when the application receives a swipe operation on the application in the client, the operation data can include swipe data generated by swiping the application in the client, and the swipe data can include swipe location, swipe duration, etc.; when the application receives an interface call operation on the application in the client, the operation data can include interface call data generated by the interface call on the application in the client, and the interface call data... The data can include interface requests generated by calling the application's API in the client and the corresponding returned page state data; when the application receives an operation to modify a subcomponent in the client, the operation data can include the subcomponent modification data generated by modifying the subcomponent in the client, and the subcomponent modification data can include the identifier of the modified subcomponent, etc.; when the application receives an operation to modify a page in the client, it can be considered that an operation to modify the page state is performed in the client, and the operation data can include the page state modification data generated by modifying the page state in the client, and the page state modification data can include the identifier of the modified page received by the application, etc.

[0048] In one embodiment of this application, the returned page status data corresponding to the interface request can be the page status data corresponding to the page generated from the data returned by the interface request. In one embodiment of this application, the subcomponent identifier can be one or more of the following: subcomponent name, subcomponent position, subcomponent number, or other information corresponding to the subcomponent. The page identifier can be one or more of the following: page name, page position, page number, or other information corresponding to the page.

[0049] In one embodiment of this application, the modification of the page state can be based on user actions. In another embodiment of this application, the modification of the page state can be based on the application's automatic page carousel, in which case the entity executing the operation to modify the application's page state is the application itself. In yet another embodiment of this application, the modification of the page state can be based on the application automatically calling an interface, in which case the entity executing the operation to modify the application's page state is also the application itself. For example, when the application jumps from the initialization page to the application's user interface, the application automatically calls an interface within the application to generate the user interface, and the page state data of the initialization page is modified to match the page state data of the user interface.

[0050] In one embodiment of this application, historical state data may include: page state data and sub-component state data generated by the application response operation during a certain period of time before the historical state data is acquired. The page state data is used to render the page, and the sub-component state data is used to render the sub-component. For example, when an application receives a click or touch operation from the client, the page state data can include click page state data generated by the application in response to the click location, and the page after the click can be rendered based on the click page state data; when an application receives a swipe operation from the client, the page state data may remain unchanged; when an application receives an API call operation from the client, the page state data can include the return page state data obtained by the application in response to the API call, and the page state data of the page corresponding to the API call can be modified to the return page state data, and the return page after the API call can be rendered based on the return page state data; when an application receives an operation to modify a subcomponent from the client, the subcomponent state data can include subcomponent state data generated by the application in response to the modification of the subcomponent from the client, and the modified subcomponent can be rendered based on the subcomponent state data generated by the modification of the subcomponent from the client; when an application receives an operation to modify the page from the client, it can be considered that the operation to modify the page state is performed on the application from the client, and the page state data can include the modified page state data generated by the application in response to the modification of the page state from the client, and the modified page can be rendered using the modified page state data.

[0051] In one embodiment of this application, before operating the application, page scrolling events can be monitored by monitoring the application's native event onPageScroll, and the position information of the scrolling position can be collected. The position information may include the page where the position is located, the position coordinates, etc.

[0052] In one embodiment of this application, monitoring code can be added to a member function of the application code before the application is operated to monitor page state change events; the changed page state data corresponding to the page state change events can be collected as historical state data, wherein the member function can be the setData function. The method of this embodiment can be used to monitor sub-component change events and collect changed sub-component state data.

[0053] Since changes in the state of an application page can be controlled by modifying the page state data, and modifying the page state data can only be done by calling member functions, the setData function is overridden here to monitor state changes and collect the page state data after the page state changes. The setData function is used to send data from the logic layer to the view layer.

[0054] The following is the code logic implementation for overriding setData and monitoring:

[0055] function RECDataListen(RECtx, extData = {}) { / / The setData function is overridden. RECtx needs to be passed the current page identifier. If you need to override the setData function of the child component, you need to pass the child component identifier to extData.

[0056]

[0057] In one embodiment of this application, the historical operation data and historical status data to be acquired can be determined based on the process of operating the application in the client as needed, and the historical operation data and historical status data generated during the operation can be acquired.

[0058] In one embodiment of this application, historical operation data and historical status data can be stored on a server, in the cloud, on a blockchain, on a client, or in another storage system, and historical operation data and historical status data can be obtained from the server, in the cloud, on a blockchain, on a client, or in another storage system.

[0059] In one embodiment of this application, user historical operation data and historical state data can be recorded by JavaScript script, and the corresponding operations can be recorded. JavaScript (hereinafter referred to as "JS") is a lightweight, interpreted or just-in-time compiled high-level programming language with function priority.

[0060] In one embodiment of this application, the server may include a data collection server and a backtracking server. The data collection server uploads historical operation data and historical status data and then sends them to the backtracking server. The backtracking server executes the video generation method of this application.

[0061] In one embodiment of this application, historical operation data and historical status data can be organized into JSON data and stored in the application's runtime memory. When the business process on the application side ends, the collected data is reported to the data collection server.

[0062] Continue to refer to Figure 2 In step S220, the application is run according to the time sequence of the operations, based on historical operation data and historical status data, to obtain the display page.

[0063] In one embodiment of this application, when the operation to be displayed is a click or touch operation on the application in the client, historical operation data may include the click position of the application in the client, and historical state data may include historical click page state data generated by the application in response to the click operation at the click position. The page corresponding to the click position can be used as the page before the click or touch operation. Nodes can be pre-embedded in the page corresponding to the click position of the application, and the style of the nodes can be set as floating icons. The floating icons are controlled to be displayed at the click position to obtain the page during the click or touch operation. The page state data of the page corresponding to the click position is modified to historical click page state data. The page after the click or touch operation is generated based on the historical click page state data. The page before the click or touch operation, the page during the click or touch operation, and the page after the click or touch operation are all used as the display page.

[0064] In this embodiment, the style of the floating icon can be set as needed, and can be a circle or an arrow, etc.

[0065] This embodiment simulates the process of clicking the application in the client by embedding points in the application, and simulates the result of clicking the application in the client by modifying the page state data, thus obtaining a display page that shows the application clicked in the client.

[0066] In one embodiment of this application, the node may be a DOM node.

[0067] In one embodiment of this application, when the operation to be displayed is a swipe operation on the application in the client, the historical operation data may include the swipe position generated by swiping the application in the client. The page corresponding to the swipe position can be swiped according to the swipe position, and the pages before, during and after the swipe are all used as the display pages. The swipe position may include the page position before the swipe and the page position after the swipe ends. The swipe position may also include the page position at a set time interval during the swipe process.

[0068] In one embodiment of this application, when the operation to be displayed is an operation that calls the application interface in the client, the historical operation data may include the interface request generated by calling the application interface in the client and the corresponding return data of the interface request. The application can be controlled to send the interface request, and then the interface request sent by the application can be intercepted. Based on the intercepted interface request, the historical return page status data corresponding to the interface request is searched in the historical status data. The return page is generated based on the historical return page status data corresponding to the interface request. Both the page that sent the interface request and the return page are used as the display page, thereby avoiding secondary interface calls when running the application based on the historical operation data and historical status data.

[0069] In one embodiment of this application, the interface address targeted by the interface request can be obtained, and the return data corresponding to the interface address can be found in the historical operation data and used as the return data.

[0070] In one embodiment of this application, when the application is a WeChat Mini Program, an interception processing method can be injected into the WeChat developer tools using a Mini Program automation script. This modifies the processing logic of the WeChat request function wx.request, allowing the interception of the Application Programming Interface (API) in the API request to directly return the return data corresponding to the API request recorded in the historical operation data. The following is the code logic implementation for the interception processing:

[0071]

[0072]

[0073] In one embodiment of this application, when the operation to be displayed is an operation that modifies a sub-component of the application in the client, the historical operation data may include the sub-component identifier corresponding to the sub-component modified in the application in the client, and the historical state data may include the historical sub-component state data generated by the application response to the modification of the sub-component in the client. The sub-component identifier in the historical operation data can be obtained, the sub-component state data corresponding to the sub-component identifier can be modified to the historical sub-component state data, and the sub-component can be re-rendered based on the historical sub-component state data, thereby modifying the page where the sub-component is located. The page where the sub-component is located before the modification and the page where the sub-component is located after the modification can both be used as the display page.

[0074] In one embodiment of this application, if the application is a WeChat mini program, the corresponding sub-component can be obtained by an automated script based on the sub-component identifier, and the state within the sub-component can be modified by calling the sub-component state setting function element.setData.

[0075] In one embodiment of this application, when the operation to be displayed is an operation that modifies the page state of the application in the client, the historical operation data may include the page identifier corresponding to the modification of the page state in the application in the client, and the historical state data may include the historical page state data generated by the application response to the modification of the page state in the client. The page identifier in the historical operation data can be obtained, and the page state data corresponding to the page identifier can be modified to the historical page state data. The page corresponding to the page state before the modification and the page corresponding to the page state after the modification can both be used as the display page, wherein modifying the page state means modifying the display content of the page.

[0076] Still refer to Figure 2 In step S230, a video is generated based on the display page, showing the operation of the application in the client.

[0077] In one embodiment of this application, a screenshot operation can be performed on the display page to obtain a display screenshot. Based on the operation time corresponding to the operation, the display duration of the display screenshot is obtained. According to the generation order of the display page, the display screenshots corresponding to the display page are combined according to the display duration to obtain a video, so as to display the process of operating the application in the client through video.

[0078] In one embodiment of this application, FFmpeg can be used to restore a screenshot to a backtracking video. FFmpeg is an open-source computer program that can be used to record, convert digital audio and video, and convert them into streams.

[0079] In one embodiment of this application, if the application is a WeChat mini-program, the source code of the mini-program can be imported into the mini-program developer tool through an automated script in the backtracking server, the WeChat developer tool can be invoked, and the backtracking can be achieved by combining historical operation data and historical status data through the automated script to control the WeChat developer tool, and the displayed page can be restored step by step according to the operation time sequence.

[0080] In one embodiment of this application, the automation tool may be miniprogram-automator, minium, or other automated testing tools provided by WeChat Mini Program.

[0081] In one embodiment of this application, screenshots can be taken of each generated display page in the WeChat developer tools.

[0082] exist Figure 2In this embodiment, by acquiring historical operation data generated by operating the application in the client and historical state data generated by the application's response to operations, the application is run according to the chronological order of the operations, based on the historical operation data and historical state data, to obtain a display page. Based on the display page, a video showing the operation of the application in the client is generated. Compared with storing images or videos in the prior art, storing historical operation data and historical state data requires less space. It can save a lot of storage space while restoring the client's operation process, reducing the network resources occupied, reducing the impact on the client's performance overhead, and enabling users to trace back operations without being noticing. It can provide proof for various fields, especially for financial transactions where disputes are frequent.

[0083] Figure 3 This illustration schematically depicts a video generation method according to an embodiment of this application. The execution entity of this video generation method can be a server, for example, a... Figure 1 The backtracking server 104 shown is illustrated.

[0084] Reference Figure 3 As shown, by monitoring global touch events, hijacking page and child custom component states, monitoring page and component scroll events, and monitoring interface requests, historical operation data and historical state data of the application can be collected. The historical operation data and historical state data are then sent to the developer tool as interaction data. The developer tool runs the application based on the interaction data, restores the user's operation, generates operation record snapshots, and thus generates a retrospective video, effectively realizing the retrospective of program operations.

[0085] The following describes an apparatus embodiment of this application, which can be used to execute the video generation method described above in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the embodiments of the video generation method described above in this application.

[0086] Figure 4 A block diagram of a video generation apparatus according to an embodiment of this application is shown schematically.

[0087] Reference Figure 4 As shown, a video generation apparatus 400 according to an embodiment of this application includes an acquisition module 401, an operation module 402, and a generation module 403.

[0088] In some embodiments of this application, based on the foregoing scheme, the acquisition module 401 is configured to acquire historical operation data generated by operating the application in the client, and acquire historical status data generated by the application responding to the operation; the running module 402 is configured to run the application according to the time sequence of the operation, based on the historical operation data and the historical status data, to obtain the display page; and the generation module 403 is configured to generate a video displaying the operation of the application in the client based on the display page.

[0089] In one embodiment of this application, based on the aforementioned scheme, historical operation data includes click positions of the application in the client, and historical state data includes historical click page state data generated by the application in response to the click operation at the click position; the running module 402 is configured to: pre-embed nodes in the application and set the style of the nodes to floating icons; control the floating icons to be displayed at the click positions; and modify the page state data of the page corresponding to the click position to the historical click page state data.

[0090] In one embodiment of this application, based on the aforementioned scheme, historical operation data includes the sliding position generated by sliding the application in the client, and the running module 402 is configured to slide the page corresponding to the sliding position according to the sliding position.

[0091] In one embodiment of this application, based on the aforementioned scheme, historical operation data includes interface requests generated by calling application interfaces in the client; historical state data includes return page state data obtained by the application responding to the interface requests; and the running module is configured to: control the application to send interface requests; intercept the interface requests sent by the application; search for historical return page state data corresponding to the interface requests in the historical state data based on the intercepted interface requests; and generate a return page based on the historical return page state data corresponding to the interface requests.

[0092] In one embodiment of this application, based on the aforementioned scheme, the running module 402 is configured to: obtain the interface address targeted by the interface request; and search for the return data corresponding to the interface address in the historical operation data, and use it as the return data.

[0093] In one embodiment of this application, based on the aforementioned scheme, the historical operation data includes the subcomponent identifier corresponding to the subcomponent in the application modified in the client, and the historical state data includes the historical subcomponent state data generated by the application response to the modification of the subcomponent in the client. The running module 402 is configured to: obtain the subcomponent identifier in the historical operation data; modify the subcomponent state data corresponding to the subcomponent identifier to the historical subcomponent state data; and modify the page where the subcomponent is located based on the historical subcomponent state data.

[0094] In one embodiment of this application, based on the aforementioned scheme, the historical operation data includes the page identifier corresponding to the page state in the application modified in the client, and the historical state data includes the historical page state data generated by the application response to the modification of the page state in the client. The running module 402 is configured to: obtain the page identifier in the historical operation data; and modify the page state data corresponding to the page identifier to the historical page state data.

[0095] In one embodiment of this application, based on the aforementioned scheme, the acquisition module 401 is configured to: before operating the application in the client, add monitoring code to the member function of the application code to monitor state change events; and collect the changed state data corresponding to the state change events as historical state data.

[0096] In one embodiment of this application, based on the aforementioned scheme, the generation module 403 is configured to: take a screenshot of the display page to obtain a display screenshot; obtain the display duration of the display screenshot based on the operation time corresponding to the operation; and combine the display screenshots corresponding to the display page according to the display duration in the order in which the display page is generated to obtain a video.

[0097] Those skilled in the art will understand that various aspects of this application can be implemented as a system, method, or program product. Therefore, various aspects of this application can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, collectively referred to herein as a "circuit," "module," or "system."

[0098] The following reference Figure 5 To describe an electronic device 50 according to this embodiment of the present application. Figure 5 The electronic device 50 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of this application.

[0099] like Figure 5 As shown, the electronic device 50 is presented in the form of a general-purpose computing device. The components of the electronic device 50 may include, but are not limited to: at least one processing unit 51, at least one storage unit 52, a bus 53 connecting different system components (including storage unit 52 and processing unit 51), and a display unit 54.

[0100] The storage unit stores program code that can be executed by the processing unit 51, causing the processing unit 51 to perform the steps described in the "Embodiment Method" section above according to various exemplary embodiments of this application.

[0101] Storage unit 52 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 521 and / or a cache memory unit 522, and may further include a read-only memory unit (ROM) 523.

[0102] Storage unit 52 may also include a program / utility 524 having a set (at least one) program module 525, such program module 525 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0103] Bus 53 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0104] Electronic device 50 can also communicate with one or more external devices (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 50, and / or with any device that enables electronic device 50 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 55. Furthermore, electronic device 50 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 56. As shown, network adapter 56 communicates with other modules of electronic device 50 via bus 53. It should be understood that, although not shown in the figure, other hardware and / or software modules can be used in conjunction with electronic device 50, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0105] Through the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the method according to the embodiments of this application.

[0106] According to one embodiment of this application, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above in this specification is stored. In some possible implementations, various aspects of this application may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of this application described in the "Exemplary Methods" section of this specification.

[0107] According to one embodiment of this application, the program product for implementing the above method can be a portable compact disc read-only memory (CD-ROM) and include program code, and can run on a terminal device, such as a personal computer. However, the program product of this application is not limited thereto. In this document, the readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0108] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0109] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0110] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0111] Program code for performing the operations of this application can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0112] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of this application, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0113] It should be understood that this application is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A method of video generation, the method comprising: The method is executed by the backtracking server, and the method includes: The system acquires historical operation data generated by operating the mini-program in the client, and acquires historical state data generated by the mini-program in response to the operation; the historical operation data includes the click position of clicking the mini-program in the client and the swipe position generated by swiping the mini-program in the client, and the historical state data includes historical click page state data generated by the mini-program in response to the click position. In the backtracking server, the source code of the mini-program is imported into the mini-program developer tool via an automated script and an automated tool. The developer tool is then invoked, and the mini-program is run according to the chronological order of the operations, based on the historical operation data and the historical state data. The automated script controls the developer tool to perform backtracking and obtain the displayed page. Running the mini-program based on the historical operation data and the historical state data includes: pre-embedding nodes in the mini-program and setting the style of the nodes to floating icons; controlling the floating icons to appear at the clicked location; modifying the page state data of the page corresponding to the clicked location to the historical clicked page state data; and sliding the page corresponding to the sliding location according to the sliding location. Based on the display page, a video is generated that shows the operation of the mini-program in the client.

2. The video generation method according to claim 1, characterized in that, The historical operation data includes interface requests generated by calling the mini-program interface in the client; the historical status data includes historical return page status data obtained by the mini-program in response to the interface requests, and running the mini-program based on the historical operation data and the historical status data includes: Control the mini-program to send the interface request; Intercept the interface request sent by the mini-program; Based on the intercepted interface request, search for the historical return page status data corresponding to the interface request in the historical status data; A return page is generated based on the historical return page status data corresponding to the interface request.

3. The video generation method according to claim 2, characterized in that, The intercepted interface request is used to search for the corresponding return data in the historical operation data, including: Obtain the interface address targeted by the interface request; In the historical operation data, find the return data corresponding to the interface address and use it as the return data.

4. The video generation method according to claim 1, characterized in that, The historical operation data includes the sub-component identifier corresponding to the sub-component modified in the client, and the historical state data includes the historical sub-component state data generated by the mini-program responding to the modification of the sub-component in the client. Running the mini-program based on the historical operation data and the historical state data includes: Obtain the sub-component identifier from the historical operation data; Modify the sub-component state data corresponding to the sub-component identifier to the historical sub-component state data; Based on the historical sub-component state data, modify the page containing the sub-component.

5. The video generation method according to claim 1, characterized in that, The historical operation data includes the page identifier corresponding to the modification of the page state in the mini-program in the client, and the historical state data includes the historical page state data generated by the mini-program responding to the modification of the page state in the client. Running the mini-program based on the historical operation data and the historical state data includes: Obtain the page identifier from the historical operation data; Modify the page status data corresponding to the page identifier to the historical page status data.

6. The video generation method according to claim 1, characterized in that, The step of obtaining historical state data generated by the mini-program in response to the historical operation data includes: Before operating the mini-program in the client, add monitoring code to the member function of the mini-program's code to monitor state change events; Collect the changed state data corresponding to the state change event, and use it as the historical state data.

7. The video generation method according to claim 1, characterized in that, The historical operation data includes the operation time corresponding to the operation. The step of generating a video displaying the operation of the mini-program on the client, based on the display page, includes: Take a screenshot of the displayed page to obtain a screenshot; Based on the operation time corresponding to the operation, the display duration of the screenshot is obtained; The video is obtained by combining the screenshots corresponding to the display pages according to the display duration, based on the order in which the display pages are generated.

8. A video generation apparatus, characterized in that, The device is used in a backtracking server, and the device includes: The acquisition module is configured to acquire historical operation data generated by the client's operation of the mini program, and to acquire historical state data generated by the mini program in response to the operation; the historical operation data includes the click position of clicking the mini program in the client and the swipe position generated by swiping the mini program in the client, and the historical state data includes historical click page state data generated by the mini program in response to the click position. The running module is configured to import the source code of the mini-program into the mini-program developer tool through the mini-program automation script in the backtracking server, invoke the developer tool, run the mini-program according to the time sequence of the operation, based on the historical operation data and the historical state data, and control the developer tool through the automation script to achieve backtracking and obtain the display page; The generation module is configured to generate a video based on the display page, which is then displayed on the client side showing the operation of the mini-program. The running module is configured to: pre-embed nodes in the mini-program and set the style of the nodes to floating icons; control the floating icons to be displayed at the click position; modify the page state data of the page corresponding to the click position to the historical click page state data; and slide the page corresponding to the sliding position according to the sliding position.

9. A computer-readable program medium storing computer program instructions, characterized in that, When the computer program instructions are executed by the computer, the computer performs the method as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: processor; A memory storing computer-readable instructions that, when executed by the processor, implement the method as described in any one of claims 1 to 7.

11. A computer program product, characterized in that, The computer program product includes computer instructions stored in a computer-readable storage medium, wherein a processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions to cause the computer device to perform the method as described in any one of claims 1 to 7.

Images