Model training method, application demonstration method, program product, electronic device, and storage medium

Abstract

The application discloses a model training method, an application demonstration method, a program product, an electronic device and a storage medium. The model training method comprises the following steps: obtaining a plurality of application videos corresponding to a plurality of user operation events of a target application, wherein the application video comprises a video from a user operation trigger to completion of an application interface response; constructing a plurality of data packets based on each user operation event and the corresponding application video; each data packet comprises at least an operation identifier of the corresponding user operation event, a data packet identifier, an application video identifier, an application page identifier of the user operation and an application element identifier; constructing a training packet based on the operation identifier, the data packet identifier, the application element identifier and the application page identifier in each data packet, thereby obtaining a training packet set, wherein the data packet identifier is a training label; training a first model based on the training packet set, wherein the output of the first model is the data packet identifier; and the data packet identifier is used to play an application video corresponding to the application video identifier in the corresponding data packet.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a model training method, an application demonstration method, a program product, an electronic device, and a storage medium. Background Technology

[0002] When demonstrating an application that has been taken offline, cannot be run directly, or has high redeployment costs, related technologies typically employ the following methods: static screenshots or presentation slides (PowerPoint), pre-recorded video demonstrations, and demonstrations in virtualized or sandboxed environments. However, static screenshots, presentation slides, and pre-recorded videos can only play fixed operations, lacking flexibility in the demonstration process; sandbox and virtualized deployments are costly, and become unusable once the original application is taken offline or the environment is missing. Summary of the Invention

[0003] In view of this, embodiments of the present invention provide a model training method, an application demonstration method, a program product, an electronic device, and a storage medium.

[0004] The technical solution of this invention is implemented as follows: On one hand, embodiments of the present invention provide a model training method, the method comprising: Acquire multiple application recordings corresponding to multiple user operation events of the target application, wherein the application recordings include video from the triggering of the user operation to the completion of the application interface response; Based on each user action event and the corresponding application recording, multiple data packets are constructed; each data packet includes at least the action identifier of the corresponding user action event, the data packet identifier, the application recording identifier, the application page identifier of the user action, and the application element identifier. A training package is constructed based on the operation identifier, data packet identifier, application element identifier, and application page identifier in each data packet, resulting in a training package set, wherein the data packet identifier is the training label; The first model is trained based on the training package set, and the output of the first model is a data packet identifier; the data packet identifier is used to play the application video corresponding to the application video identifier in the corresponding data packet.

[0005] In the above scheme, the data packet also includes one or more of the following: the location information of the application element corresponding to the user operation event in the application page, the page size of the application page, the time information of the application recording, and the application page identifier of the application page after the application interface response is completed.

[0006] In the above scheme, obtaining multiple application recordings corresponding to multiple user operation events of the target application includes: Record the application interface video in real time when each user action event is triggered, and stop recording when the application interface completes the response, resulting in multiple application recordings.

[0007] On the other hand, embodiments of this application also provide an application demonstration method, the method comprising: Obtain user operation information on the playback interface, wherein the playback interface displays the first page of the target application; Based on the user operation information, determine the operation identifier, the page identifier of the first page, and the application element identifier of the target application element in the first page; Input the operation identifier, the page identifier, and the application element identifier into the first model above to obtain the data packet identifier output by the first model; The application recording identifier in the corresponding data packet is determined based on the data packet identifier, and the corresponding application recording is played according to the application recording identifier.

[0008] In the above scheme, the user operation information includes the operation type and the coordinates of the user's operation area in the playback interface. The step of determining the operation identifier, the page identifier of the first page, and the application element identifier of the target application element in the first page based on the user operation information includes: Based on the size of the playback interface, the coordinates of the operation area, and the size of the first page, determine the application element identifier of the target application element in the first page; The operation identifier is determined according to the operation type, wherein each operation type corresponds to one operation identifier.

[0009] In the above scheme, after playing the corresponding application recording according to the application recording identifier, the method further includes: The second page of the target application is displayed on the playback interface. The second page is the application page after the application interface response indicated in the data packet is completed.

[0010] On the other hand, embodiments of this application also provide a computer program product, including a computer program that, when executed by a processor, implements the steps of the above-described method.

[0011] On the other hand, embodiments of the present invention provide an electronic device including a processor and a memory, the processor and the memory being interconnected, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to execute the steps of the method described above in the embodiments of the present invention.

[0012] On the other hand, embodiments of the present invention provide a computer-readable storage medium, comprising: the computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it implements the steps of the method described in the embodiments of the present invention.

[0013] The solution of this invention involves acquiring multiple application recordings corresponding to multiple user operation events of a target application. These recordings include video from the triggering of a user operation to the completion of the application interface response. Based on each user operation event and its corresponding application recording, multiple data packets are constructed. Each data packet includes at least an operation identifier corresponding to the user operation event, a data packet identifier, an application recording identifier, an application page identifier, and an application element identifier. A training package is constructed based on the operation identifier, data packet identifier, application element identifier, and application page identifier in each data packet, resulting in a training package set. The data packet identifier serves as the training label. A first model is trained based on the training package set, and the output of the first model is the data packet identifier. The data packet identifier is used to play the application recording corresponding to the application recording identifier in the corresponding data packet. This invention allows for the collection of user operation data and screen recordings during the actual operation of the target application, the construction of training packages to train the first model, and the output of the most matching data packet identifier from the first model after the target application goes offline. Based on the data packet identifier, the corresponding application recording can be found, enabling application demonstration. This allows users to obtain a dynamic demonstration experience with realistic interaction even without actually running the application. Attached Figure Description

[0014] Figure 1 This is a schematic diagram illustrating the implementation process of a model training method provided in an embodiment of the present invention; Figure 2 and Figure 3 This is a schematic diagram of a target application interface provided in an embodiment of the present invention; Figure 4 This is a schematic diagram of constructing a training package based on a step size, provided by an embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the implementation process of an application demonstration method provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the overall architecture of an application demonstration method provided in an embodiment of the present invention; Figure 7 This is a schematic diagram of an electronic device provided in an embodiment of the present invention. Detailed Implementation

[0015] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0016] Currently, when demonstrating an application that has been taken offline, cannot be run directly, or has high redeployment costs, the following technical methods are typically used: 1. Static screenshots or presentation (PPT): Showcase application interface screenshots with accompanying text descriptions to create slides; 2. Pre-recorded video demonstration: Record the pre-planned operation process as a video for later demonstration; 3. Virtualized or sandboxed runtime environment: Running a copy of the application in a controlled environment for interactive use; 4. High-fidelity prototyping tools (such as Axure, Figma, etc.): Use interface prototyping tools to recreate the interface interaction logic for demonstration purposes.

[0017] The above method has the following disadvantages: 1. Static screenshots, presentation slides, and pre-recorded video demonstrations can only play fixed operations, and users cannot interact in real time. The demonstration process lacks flexibility and responsiveness, cannot dynamically adjust content according to user behavior, and cannot re-record new scenes, resulting in high maintenance costs.

[0018] 2. Sandbox or virtualization operation methods rely on a complete runtime environment and backend services. Once the original application is taken offline or the environment is missing, it cannot be used. The deployment is complex and resource consumption is high, making it unsuitable for quickly reproducing the operation process of an already discontinued application.

[0019] 3. The interaction method of prototyping tools requires manual reconstruction of the interface and preset jump logic, which cannot reproduce real dynamic response, has limited interaction effect, cannot dynamically adjust the displayed content according to the actual operation of the user, and lacks scalability and realism.

[0020] To address the shortcomings of the aforementioned related technologies, embodiments of the present invention provide a model training method and an application demonstration method, enabling users to obtain a dynamic demonstration experience with realistic interactivity without actually running the application. To illustrate the technical solution described in this invention, specific embodiments are provided below.

[0021] Figure 1 This is a schematic diagram illustrating the implementation flow of a model training method provided in an embodiment of the present invention. The execution subject of the model training method is an electronic device. (Reference) Figure 1 Model training methods include: S101, acquire multiple application recordings corresponding to multiple user operation events of the target application, the application recordings including videos from the triggering of the user operation to the completion of the application interface response.

[0022] Here, the target application is the application that will be demonstrated in the following embodiments of the present invention. During the actual operation of the target application, user interaction behavior data and corresponding screen recordings are collected.

[0023] For example, during the actual operation of the target application, interactive elements can be marked on the target application's user interface. These marks include the current page, UI elements, UI element operations, etc., and each mark is assigned a unique ID.

[0024] When the target application is running, the system records the IDs of user interactions with UI elements on the page. It can also record the page size, the precise location of the UI element on the page, and the timestamp at the time of the user interaction.

[0025] For example, such as Figure 2 and Figure 3 As shown, the user clicked the "Add to Cart" button on the product details page of the target application, which yields an ID track " / product / detail / button123", with the action ID "click", and the page size is 1920. 1080, the coordinates of the page element (Add to Cart) are (440, 1820) to (640, 1900), the operation is triggered at T1: xxxx year xx month xx day xx hour xx minute xx second xx millisecond, and the time after the control is displayed after successfully adding to cart is T2: xxxx year xx month xx day xx hour xx minute xx second xx millisecond.

[0026] While capturing user actions, the system records video of the entire application screen in real time and synchronously. Each frame of the video is precisely timestamped, and this timestamp is precisely aligned with the timestamp of the user action event. For example, the recording service records the video segment from the moment the user action occurs until the application interface completes its response (e.g., the shopping cart animation finishes playing and the page redirects to the shopping cart and the item is successfully added).

[0027] S102, based on each user operation event and the corresponding application recording, construct multiple data packets; each data packet includes at least the operation identifier of the corresponding user operation event, the data packet identifier, the application recording identifier, the application page identifier of the user operation, and the application element identifier.

[0028] Application recording is performed simultaneously with user actions; therefore, each user action event corresponds to one application recording.

[0029] For each user action event, a data packet is constructed. The data packet includes at least the following: the action identifier of the user action event, the data packet identifier, the application recording identifier, the application page identifier of the user action, and the application element identifier.

[0030] Different operation types correspond to different operation identifiers. For example, ID1 is a click, ID2 is a long press, ID3 is a swipe left, ID4 is a swipe right, ID5 is a swipe down, and so on. Operation IDs only mark the enumerable operations and are not associated with UI elements.

[0031] Different data packets correspond to different data packet identifiers, different application recordings correspond to different application recording identifiers, different application pages correspond to different application page identifiers, and different application elements correspond to different application element identifiers.

[0032] S103, a training package is constructed based on the operation identifier, data packet identifier, application element identifier, and application page identifier in each data packet to obtain a training package set, wherein the data packet identifier is the training label.

[0033] The application page identifier, operation identifier, application element identifier, and data packet identifier in the data packet are repackaged into a "training packet," and the correspondence between each training packet and the data packet is recorded. The data packet identifier serves as the training label, used to determine the accuracy of the model's output.

[0034] Because data packets contain a lot of redundant information, directly using them as sequence input would increase the learning difficulty and risk of overfitting for the model. The training package simplifies this to discrete ID combinations (page ID, operation ID, UI element ID), making it easier to construct lightweight operation sequences and enabling the model to more easily capture user behavior patterns.

[0035] like Figure 4 As shown, the entire process from opening the application to closing it is recorded as a series of ordered "training package sets" based on the segmentation step size n. This is like storing the user's operation trajectory when using the app.

[0036] S104, a first model is trained based on the training package set, and the output of the first model is a data packet identifier; the data packet identifier is used to play the application video corresponding to the application video identifier in the corresponding data packet.

[0037] A training set (which can be a JSON string) is used to train a first model. The goal of this first model is to learn and predict the complex mapping between "user actions" and "data packets". The model input is the "training set", and the model output is the ID of the most matching data packet.

[0038] This embodiment does not specifically limit the training method of the first model. In actual applications, the model training method can be selected according to the actual situation. It can be a traditional method such as expert experience model or statistical features, or a machine learning method such as supervised learning, unsupervised learning or semi-supervised learning.

[0039] Once the first trained model is completed, it only needs to input "page identifier, operation identifier, and application element identifier" to obtain a matching data packet identifier. Based on the data packet identifier, it can find the corresponding data packet, retrieve the corresponding application recording identifier from the data packet, find the corresponding application recording based on the application recording identifier, play the application recording, and realize the demonstration of the application.

[0040] Specifically, by acquiring user operation information on the playback interface, the first page of the target application is displayed on the playback interface; based on the user operation information, the operation identifier, the page identifier of the first page, and the application element identifier of the target application element in the first page are determined; the operation identifier, page identifier, and application element identifier are input into the first model to obtain the data packet identifier output by the first model; based on the data packet identifier, the application recording identifier in the corresponding data packet is determined, and the corresponding application recording is played according to the application recording identifier.

[0041] Thus, for applications that have been taken offline, by collecting user behavior data and screen recordings during the actual operation of the application, and training the first model based on the data, users can still obtain a dynamic demonstration experience with a realistic sense of interaction without actually running the application.

[0042] The solution of this invention involves acquiring multiple application recordings corresponding to multiple user operation events of a target application. These recordings include video from the triggering of a user operation to the completion of the application interface response. Based on each user operation event and its corresponding application recording, multiple data packets are constructed. Each data packet includes at least an operation identifier corresponding to the user operation event, a data packet identifier, an application recording identifier, an application page identifier, and an application element identifier. A training package is constructed based on the operation identifier, data packet identifier, application element identifier, and application page identifier in each data packet, resulting in a training package set. The data packet identifier serves as the training label. A first model is trained based on the training package set, and the output of the first model is the data packet identifier. The data packet identifier is used to play the application recording corresponding to the application recording identifier in the corresponding data packet. This invention allows for the collection of user operation data and screen recordings during the actual operation of the target application, the construction of training packages to train the first model, and the output of the most matching data packet identifier from the first model after the target application goes offline. Based on the data packet identifier, the corresponding application recording can be found, enabling application demonstration. This allows users to obtain a dynamic demonstration experience with realistic interaction even without actually running the application.

[0043] In some embodiments, the data packet may further include one or more of the following: the location information of the application element corresponding to the user operation event in the application page, the page size of the application page, the time information of the application recording, and the application page identifier of the application page after the application interface response is completed.

[0044] Here, in addition to operation identifier, data packet identifier, application element identifier, and application page identifier, the data packet may also include the page size of the application page operated by the user, the coordinate range of the UI element operated by the user, the video timestamp range of the application recording, and the application page identifier of the application page after the application interface responds to the operation.

[0045] In some embodiments, acquiring multiple application recordings corresponding to multiple user operation events of the target application includes: Record the application interface video in real time when each user action event is triggered, and stop recording when the application interface completes the response, resulting in multiple application recordings.

[0046] While capturing user actions, the system records video of the entire application screen in real time and synchronously. Each frame of the video is precisely timestamped, and this timestamp is precisely aligned with the timestamp of the user action event. For example, the recording service records the video segment from the moment the user action occurs until the application interface completes its response (e.g., the shopping cart animation finishes playing and the page redirects to the shopping cart and the item is successfully added).

[0047] Figure 5 This is a schematic diagram illustrating the implementation flow of an application demonstration method provided in an embodiment of the present invention, wherein the execution subject of the application demonstration method is an electronic device. (Reference) Figure 5 The application demonstration methods include: S501, Obtain user operation information on the playback interface, wherein the first page of the target application is displayed in the playback interface.

[0048] Here, user operation information may include: operation type, screen coordinates of the operation, etc.

[0049] This embodiment of the invention demonstrates the target application through a "player application". The first page of the target application is displayed on the playback interface. When the "player application" is first launched, the first page can be the default starting page of the target application.

[0050] S502, based on the user operation information, determine the operation identifier, the page identifier of the first page, and the application element identifier of the target application element in the first page.

[0051] Users can perform actions such as clicking and swiping on the playback interface, just like they are actually operating the target application. The system will capture the type of these actions and the screen coordinates.

[0052] In response to user actions, the page identifier of the currently displayed first page can be obtained. Based on the coordinates of the user's interaction area, the page size, and the coordinate range of the UI element, the UI element ID (application element identifier) ​​of the user's interaction can be determined. The operation ID can be determined based on the type of user action (e.g., click, swipe, etc.). This forms an "action data package," which includes: page ID, operation ID, and UI element ID.

[0053] A series of user actions in the "player application" can be divided into an "action data package set" (which can be a piece of JSON) based on a preset step size.

[0054] S503, input the operation identifier, the page identifier, and the application element identifier into the first model to obtain the data packet identifier output by the first model.

[0055] Each user action generates an "action data packet," which is then input into the first model. The first model transforms the "action data packet" into a computable feature vector. It then searches its entire training dataset for the most similar recorded action to the current user's action. Based on this most similar historical action (for example, finding a previously known "click on the shopping cart"), the first model outputs the ID of the most matching data packet.

[0056] S504, determine the application recording identifier in the corresponding data packet according to the data packet identifier, and play the corresponding application recording according to the application recording identifier.

[0057] In the application recording stage, this embodiment of the invention pre-stores the mapping relationship between the application recording identifier, the application recording, and the data packet identifier.

[0058] Based on the application recording identifier output by the first model, the corresponding application recording can be found, and the application recording can be played to complete the application demonstration.

[0059] This invention can collect user operation data and screen recordings during the actual operation of the target application, construct a training package to train the first model, and output the most matching data packet identifier through the first model after the target application goes offline. Based on the data packet identifier, the corresponding application recording can be found, thereby demonstrating the application. This allows users to still obtain a dynamic demonstration experience with a realistic sense of interaction without actually running the application.

[0060] In embodiments of the present invention, when the application is not actually running, the first model can intelligently retrieve and play the most matching video clip from the pre-recorded video library based on the user's operation on the simulation interface, thereby providing a highly realistic, smooth and flexible simulation operation experience.

[0061] In some embodiments, the user operation information includes the operation type and the coordinates of the user's operation area in the playback interface. Determining the operation identifier, the page identifier of the first page, and the application element identifier of the target application element in the first page based on the user operation information includes: Based on the size of the playback interface, the coordinates of the operation area, and the size of the first page, determine the application element identifier of the target application element in the first page; The operation identifier is determined according to the operation type, wherein each operation type corresponds to one operation identifier.

[0062] Based on the user's operation coordinates on the playback interface, the size of the first page, and the coordinates of each application element on the first page, the application element identifier corresponding to the user's operation coordinates can be determined.

[0063] In some embodiments, after playing the corresponding application recording according to the application recording identifier, the method further includes: The second page of the target application is displayed on the playback interface. The second page is the application page after the application interface response indicated in the data packet is completed.

[0064] After the application recording demonstration is complete, the "Status" on the playback interface will be updated to the last frame of the application recording (i.e., the page ID after the operation), waiting for the user to perform the next operation.

[0065] In some embodiments, when the target application is first demonstrated in the playback interface, the first page is the starting page of the target application.

[0066] When a user opens the "Player app", the system will first load the pre-recorded video frames corresponding to the target app's start page as the first interface the user sees.

[0067] like Figure 6 As shown, Figure 6 This is a schematic diagram of the overall architecture of the application demonstration method provided in the embodiments of the present invention, including: Application: During application operation, collect user operation data and screen recordings.

[0068] Data acquisition module: responsible for capturing user interaction behavior and performing structured tagging during real application runtime, while also working with screen recording.

[0069] Screen recording module: Captures user actions while simultaneously recording the entire application screen in real time. Each frame of the video has a precise timestamp, which is precisely aligned with the timestamp of the user action event. For example, the recording service will record the video segment from the moment the user action occurs until the application interface completes its response (e.g., the shopping cart animation finishes playing and the page redirects to the shopping cart and the item is successfully added).

[0070] Data processing and storage platform: Receives and preprocesses (e.g., video slicing, compression) user behavior data packets and screen recording videos, and stores them after establishing precise associations.

[0071] Model training platform: Loads processed datasets to train and optimize large AI models.

[0072] Player application: Provides a user interface and captures user actions on the simulated page.

[0073] Data acquisition module: Collects user interaction operations, inputs them into the AI ​​model for real-time inference, returns the corresponding data packet identifier, and finds the corresponding video segment based on the data packet identifier.

[0074] Screen playback module: Based on the AI ​​inference results, the specified video clips are streamed to the front end for playback.

[0075] This invention employs a two-layer association architecture of structured data packets and training packets. User sessions are segmented into ordered JSON sequences by step size and input into the model. The mapping relationship is trained using data packet IDs as labels. This trains a powerful AI model. This AI model can intelligently retrieve and play the most suitable video clips from a pre-recorded video library based on user actions on a simulated interface, even when the application is not actually running, thus providing a highly realistic, smooth, and flexible simulated operating experience.

[0076] After loading the initial video frames in the simulation environment, the reverse analysis engine converts the user operation coordinates into UI element IDs to generate data packets. The AI ​​model matches similar historical operation sequences in real time and outputs video IDs. The streaming media service plays related videos and updates the interface state with the last frame. This engine achieves seamless simulation of real applications and significantly reduces the deployment cost of the demonstration environment.

[0077] It should be understood that the sequence number of each step in the above embodiments does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0078] It should be understood that, when used in this specification and the appended claims, the terms "comprising" and "including" indicate the presence of the described features, integrals, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integrals, steps, operations, elements, components and / or collections thereof.

[0079] It should be noted that the technical solutions described in the embodiments of the present invention can be combined arbitrarily without conflict.

[0080] In addition, in the embodiments of the present invention, "first," "second," etc. are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0081] In practical applications, the method steps of the above embodiments can be implemented by processors in electronic devices, such as central processing units (CPUs), digital signal processors (DSPs), microcontroller units (MCUs), or field-programmable gate arrays (FPGAs).

[0082] To implement the methods of the embodiments of this application, an electronic device is also provided. Figure 7 This is a schematic diagram of the hardware structure of the electronic device according to an embodiment of this application, as shown below. Figure 7 As shown, the electronic device includes: A communication interface enables information exchange with other devices, such as network devices. The processor, connected to the communication interface, enables information interaction with other devices and, when running a computer program, executes the methods provided by one or more technical solutions on the electronic device side. The computer program is stored in memory.

[0083] Of course, in practical applications, the various components in an electronic device are coupled together through a bus system. It can be understood that the bus system is used to achieve communication and connection between these components. In addition to the data bus, the bus system also includes a power bus, a control bus, and a status signal bus. However, for the sake of clarity, in... Figure 7 The general will label all buses as bus systems.

[0084] The memory in this application embodiment is used to store various types of data to support the operation of the electronic device. Examples of such data include any computer program used to operate on the electronic device.

[0085] It is understood that memory can be volatile or non-volatile, or both. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), ferromagnetic random access memory (FRAM), flash memory, magnetic surface memory, optical disc, or compact disc read-only memory (CD-ROM); magnetic surface memory can be disk storage or magnetic tape storage. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static Random Access Memory (SRAM), Synchronous Static Random Access Memory (SSRAM), Dynamic Random Access Memory (DRAM), Synchronous Dynamic Random Access Memory (SDRAM), Double Data Rate Synchronous Dynamic Random Access Memory (DDRSDRAM), Enhanced Synchronous Dynamic Random Access Memory (ESDRAM), SyncLink Dynamic Random Access Memory (SLDRAM), and Direct Rambus Random Access Memory (DRRAM).The memories described in the embodiments of this application are intended to include, but are not limited to, these and any other suitable types of memories.

[0086] The methods disclosed in the embodiments of this application can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. During implementation, each step of the above method can be completed by integrated logic circuits in the processor's hardware or by instructions in software form. The processor may be a general-purpose processor, a DSP, or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The processor can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. A general-purpose processor may be a microprocessor or any conventional processor, etc. The steps of the methods disclosed in the embodiments of this application can be directly manifested as execution by a hardware decoding processor, or execution by a combination of hardware and software modules in the decoding processor. The software modules may be located in a storage medium, which is located in memory. The processor reads the program from the memory and, in conjunction with its hardware, completes the steps of the aforementioned method.

[0087] Optionally, when the processor executes the program, it implements the corresponding processes implemented by the electronic device in the various methods of the embodiments of this application. For the sake of brevity, these will not be described in detail here.

[0088] In an exemplary embodiment, this application also provides a computer program product, including a computer program that can be executed by a processor of an electronic device to perform the steps described in the method of this application embodiment.

[0089] In an exemplary embodiment, this application also provides a storage medium, namely a computer storage medium, specifically a computer-readable storage medium, such as a first memory storing a computer program, which can be executed by a processor of an electronic device to complete the steps described in the aforementioned method. The computer-readable storage medium may be a memory such as FRAM, ROM, PROM, EPROM, EEPROM, Flash Memory, magnetic surface memory, optical disc, or CD-ROM.

[0090] In the several embodiments provided in this application, it should be understood that the disclosed electronic devices and methods can be implemented in other ways. The device embodiments described above are merely illustrative. For example, the division of units is only a logical functional division, and in actual implementation, there may be other division methods, such as: multiple units or components may be combined, or integrated into another system, or some features may be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection between devices or units may be electrical, mechanical, or other forms.

[0091] The units described above as separate components may or may not be physically separate. The components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the units may be selected to achieve the purpose of this embodiment according to actual needs.

[0092] In addition, each functional unit in the various embodiments of this application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be implemented in hardware or in the form of hardware plus software functional units.

[0093] Those skilled in the art will understand that all or part of the steps of the above method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When the program is executed, it performs the steps of the above method embodiments. The aforementioned storage medium includes various media that can store program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0094] Alternatively, if the integrated units described above are implemented as software functional modules and sold or used as independent products, they can also be stored in a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of this application, or the parts that contribute to related technologies, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as mobile storage devices, ROM, RAM, magnetic disks, or optical disks.

[0095] It should be noted that the technical solutions described in the embodiments of this application can be combined arbitrarily without conflict.

[0096] In addition, in this application example, terms such as "first" and "second" are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence.

[0097] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A model training method, characterized in that, The method includes: Acquire multiple application recordings corresponding to multiple user operation events of the target application, wherein the application recordings include video from the triggering of the user operation to the completion of the application interface response; Based on each user action event and the corresponding application recording, multiple data packets are constructed; each data packet includes at least the action identifier of the corresponding user action event, the data packet identifier, the application recording identifier, the application page identifier of the user action, and the application element identifier. A training package is constructed based on the operation identifier, data packet identifier, application element identifier, and application page identifier in each data packet, resulting in a training package set, wherein the data packet identifier is the training label; The first model is trained based on the training package set, and the output of the first model is a data packet identifier; the data packet identifier is used to play the application video corresponding to the application video identifier in the corresponding data packet.

2. The method according to claim 1, characterized in that, The data packet also includes one or more of the following: the location information of the application element corresponding to the user operation event in the application page, the page size of the application page, the time information of the application recording, and the application page identifier of the application page after the application interface response is completed.

3. The method according to claim 1, characterized in that, The acquisition of multiple application recordings corresponding to multiple user operation events of the target application includes: Record the application interface video in real time when each user action event is triggered, and stop recording when the application interface completes the response, resulting in multiple application recordings.

4. An application demonstration method, characterized in that, The method includes: Obtain user operation information on the playback interface, wherein the playback interface displays the first page of the target application; Based on the user operation information, determine the operation identifier, the page identifier of the first page, and the application element identifier of the target application element in the first page; Input the operation identifier, the page identifier, and the application element identifier into the first model according to any one of claims 1 to 3 to obtain the data packet identifier output by the first model; The application recording identifier in the corresponding data packet is determined based on the data packet identifier, and the corresponding application recording is played according to the application recording identifier.

5. The method according to claim 4, characterized in that, The user operation information includes the operation type and the coordinates of the user's operation area in the playback interface. Determining the operation identifier, the page identifier of the first page, and the application element identifier of the target application element in the first page based on the user operation information includes: Based on the size of the playback interface, the coordinates of the operation area, and the size of the first page, determine the application element identifier of the target application element in the first page; The operation identifier is determined according to the operation type, wherein each operation type corresponds to one operation identifier.

6. The method according to claim 4, characterized in that, After playing the corresponding application recording based on the application recording identifier, the method further includes: The second page of the target application is displayed on the playback interface. The second page is the application page after the application interface response indicated in the data packet is completed.

7. The method according to claim 4, characterized in that, When the target application is first demonstrated in the playback interface, the first page is the starting page of the target application.

8. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by the processor, it implements the steps of the model training method as described in any one of claims 1 to 3, or the steps of the application demonstration method as described in any one of claims 4 to 7.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the model training method as described in any one of claims 1 to 3, or the steps of the application demonstration method as described in any one of claims 4 to 7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, the computer program including program instructions that, when executed by a processor, cause the processor to perform the steps of the model training method as described in any one of claims 1 to 3, or the steps of the application demonstration method as described in any one of claims 4 to 7.

Images