Method and apparatus for labeling event in application program, device, and storage medium
By displaying the user interface of the event labeling tool in the browser, visual annotation of events in the application is solved, and the problem of high difficulty in event labeling and inability to obtain data in the existing technology is solved, improving analysis efficiency.
Patent Information
- Application Number
- PCT/CN2024/084299
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-23
- Filing Date
- 2024-03-28
- Publication Date
- 2024-11-28
AI Technical Summary
In the prior art, the event annotation of the application requires communication and coordination between multiple parties through code burying points, resulting in high communication costs and the inability to obtain data instantly for analysis, affecting operational optimization.
Provides an event annotation method within an application, displays the user interface of the event annotation tool in the browser, responds to the startup operation of the application experiencer, displays the user interface of the application experiencer, and displays related in element selection mode. event labeling information to realize visual annotation of events in the application.
Reducing the difficulty and communication cost of event annotation, analysts can independently complete event annotation, improve event annotation efficiency, and instantly obtain data for analysis, supporting application object behavior analysis.
Smart Images

Figure CN2024084299_28112024_PF_FP_ABST
Abstract
Description
Event annotation method, device, equipment and storage medium within application program
[0001] This application claims priority to Chinese patent application number 202310591425.3, filed on May 23, 2023, entitled “Event annotation method, apparatus, device and storage medium within an application program,” the entire contents of which are incorporated herein by reference. Technical Field
[0002] The embodiments of the present application relate to the field of application technology, and in particular to a method, apparatus, device, and storage medium for annotating events within an application. Background Art
[0003] After an application is released online, analysts usually need to collect statistics and analyze the application's object behavior to obtain user usage of the application and provide data support for the subsequent operation and optimization of the application.
[0004] Related technologies collect data through code tracking. Code tracking involves inserting code at the events that need to be counted within an application to measure behaviors such as clicks, number of visitors, page dwell time, and video playback duration. When using code tracking, an analyst first determines the tracking requirements, namely the events to be collected and the behaviors to be counted. The analyst then communicates the tracking requirements with the application developer, who then implements the tracking. The analyst, developer, or application tester may then test the accuracy and effectiveness of the tracking. Finally, the tracking takes effect after the application is re-released and the corresponding events are automatically collected and reported.
[0005] It can be seen that using code tracking to mark events requires communication and coordination among multiple parties, which has a high communication cost. In addition, tracking can only take effect after the application changes are released, and data cannot be obtained immediately for analysis.
[0006] Summary of the Invention
[0007] The present invention provides a method, apparatus, device, and storage medium for annotating events within an application. The technical solution is as follows:
[0008] According to one aspect of an embodiment of the present application, a method for annotating an event within an application is provided, the method comprising:
[0009] Display the event annotation tool's user interface in the browser;
[0010] In response to a start operation on an application experiencer, displaying a user interface of the application experiencer in a user interface of the event annotation tool, wherein the application experiencer refers to a runtime instance of an application for interactive use by an analyst;
[0011] After the application experiencer enters the element selection mode, in response to a selection operation on a first element in the user interface of the application experiencer, event annotation information related to the first element is displayed in the user interface of the event annotation tool, where the event annotation information is used to annotate the interaction type and display characteristics of the element related to the first element;
[0012] In response to a confirmation operation on the event annotation information, an operation event matching the event annotation information is marked within the application corresponding to the application experiencer.
[0013] According to one aspect of an embodiment of the present application, a device for marking events within an application is provided, the device comprising:
[0014] A tool interface display module, used to display the user interface of the event annotation tool in the browser;
[0015] An experiencer interface display module, configured to display a user interface of the application experiencer in the user interface of the event annotation tool in response to a startup operation for the application experiencer, wherein the application experiencer refers to a runtime instance of the application for interactive use by the analyst;
[0016] an annotation information display module for displaying event annotation information related to a first element in the user interface of the application experiencer after the application experiencer enters an element selection mode, in response to a selection operation on a first element in the user interface of the application experiencer, the event annotation information being used to annotate the interaction type and display characteristics of elements related to the first element;
[0017] An operation event marking module is used to mark an operation event matching the event marking information in the application corresponding to the application experiencer in response to a confirmation operation on the event marking information.
[0018] According to one aspect of an embodiment of the present application, a computer device is provided, comprising a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the event annotation method within the application as described above.
[0019] According to one aspect of an embodiment of the present application, a computer-readable storage medium is provided, in which a computer program is stored. The computer program is loaded and executed by a processor to implement the event annotation method within the application as described above.
[0020] According to one aspect of an embodiment of the present application, a computer program product is provided, which includes a computer program stored in a computer-readable storage medium, and a processor reads and executes the computer program from the computer-readable storage medium to implement the event annotation method within the application as described above.
[0021] The technical solutions provided by the embodiments of the present application include at least the following beneficial effects:
[0022] The event annotation tool enables visual annotation of events within the application, which reduces the difficulty of event annotation compared to code tracking. Analysts can then complete event annotation independently through the event annotation tool without having to communicate and coordinate with developers and testers, reducing communication costs and improving event annotation efficiency. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] FIG1 is a schematic diagram of an event annotation system provided by an embodiment of the present application;
[0024] FIG2 is a flow chart of an event annotation method within an application provided by an embodiment of the present application;
[0025] FIG3 is a schematic diagram of starting an application experiencer provided by one embodiment of the present application;
[0026] FIG4 is a schematic diagram of entering and exiting an element selection mode provided by an embodiment of the present application;
[0027] FIG5 is a schematic diagram showing event annotation information provided by an embodiment of the present application;
[0028] FIG6 is a flowchart of an event annotation method within an application provided by another embodiment of the present application;
[0029] FIG7 is a block diagram of an event annotation device within an application program according to an embodiment of the present application;
[0030] FIG8 is a block diagram of an event annotation device within an application provided by another embodiment of the present application;
[0031] FIG9 is a schematic diagram of the structure of a computer device provided in one embodiment of the present application. DETAILED DESCRIPTION
[0032] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.
[0033] Please refer to Figure 1, which shows a schematic diagram of an event annotation system provided by one embodiment of the present application. This event annotation system is used to annotate and extract events within an application to instantly obtain data for object behavior analysis of the application. As shown in Figure 1, the event annotation system includes: an application 10, an event annotation tool 20, and an application experiencer 30.
[0034] In the embodiment of the present application, the analyst annotates the events to be extracted in the application 10 through the event annotation tool 20. As shown in FIG1 , the event annotation tool 20 runs based on a browser. After receiving the relevant parameters of the application (such as scene parameters, path parameters, etc.) input by the analyst, the event annotation tool 20 loads and runs the online code package of the application 10 through the browser based on the relevant parameters to obtain an application experiencer 30, and the user interface of the application experiencer 30 is displayed in the user interface of the event annotation tool 20.
[0035] After the application experiencer 30 enters the element selection mode, in response to the analyst's selection operation on any element in the user interface of the application experiencer 30, the event annotation tool 20 displays the event annotation information related to the element in the user interface; thereafter, in response to the analyst's confirmation operation on the event annotation information, the event annotation tool 20 annotates the operation events in the application 10 that match the event annotation information, and these operation events will also be extracted and stored for the analyst to analyze the object behavior of the application 10.
[0036] Optionally, application 10 is a sub-application that depends on the target application to run. Among them, the sub-application can be called a mini-program. Although the mini-program cannot run independently, it can be used without downloading and installing. For example, the sub-application can be opened directly by scanning the QR code corresponding to the sub-application or searching for the name of the sub-application. In this implementation, application 10 serves as a sub-application, and the target application is the parent application of the sub-application; the parent application provides an environment for the implementation of the sub-application, and the sub-application can be conveniently obtained and disseminated in the parent application. Among them, the parent application is a native application, and the native application is an application that can run directly on the operating system. For example, the parent application can be a social application, a food application, a dedicated application that specifically supports the sub-application, a file management application, an email application, or a game application, etc.; social applications include but are not limited to instant messaging applications, SNS (Social Network Service, social networking site) applications, or live broadcast applications, etc.
[0037] In addition, as shown in Figure 1, the event annotation tool 20 and the browser that hosts it are both running on a physical computer device 40, which can access the network to obtain the online code package of the application 10, etc. The network can be a wide area network, a local area network, a metropolitan area network, a personal area network, or the Internet, etc. Optionally, the computer device 40 can be implemented as a server, or can be implemented as a terminal such as a tablet computer, a mobile phone, or a PC (Personal Computer). In the case of being implemented as a server, the computer device 40 can be a single server, a server cluster consisting of multiple servers, or a cloud computing center, which is not limited in the embodiments of the present application.
[0038] For further descriptions of the application experiencer and its startup method, the element selection mode and its entry and exit methods, the event annotation information and its acquisition and display methods in the embodiment of Figure 1, please refer to the following method embodiment, which will not be elaborated here.
[0039] Please refer to Figure 2, which shows a flow chart of an event annotation method within an application provided by one embodiment of the present application. This method can be applied to an event annotation tool, such as the event annotation tool 20 shown in Figure 1. The method is executed by a terminal device. As shown in Figure 2, the event annotation method includes at least one of the following steps (steps 210 to 240).
[0040] Step 210: Display the user interface of the event annotation tool in the browser.
[0041] In some embodiments, the event annotation tool runs on a browser, and after starting the event annotation tool, the user interface of the event annotation tool can be displayed in the browser. Optionally, the event annotation tool can be started by entering a URL in the browser, searching for a name, scanning a QR code, entering a voice command, etc.
[0042] In some embodiments, the event annotation tool can be a standalone client installed in a terminal device, for example, the event annotation tool runs in APP (Application) mode. In some embodiments, the event annotation tool can also be a plug-in integrated into other clients, or implemented as a functional module in other clients, which is not limited in this application.
[0043] The embodiment of the present application does not limit the display content of the initial user interface after the event annotation tool is started. Optionally, the initial user interface displays an interface for inputting authentication information such as account name, login password, and biometrics. In response to the corresponding information input by the analyst, the event annotation tool jumps from the initial user interface to the user interface of the normal working state; or, the initial user interface is directly displayed as the user interface of the normal working state. Among them, the user interface of the normal working state includes at least one of the following: a user interface for starting the application experiencer, a user interface for performing event annotation, and a user interface for setting event annotation properties (such as name, event storage location, etc.).
[0044] Step 220: In response to the start operation for the application experiencer, the user interface of the application experiencer is displayed in the user interface of the event annotation tool.
[0045] An application experiencer refers to a runtime instance of an application for interactive use by analysts. The application experiencer is obtained by running the online code package of the application loaded by the browser. After the application experiencer is started, the user interface of the application experiencer is directly displayed in the user interface of the event annotation tool. Optionally, the display level of the user interface of the application experiencer is equal to the display level of the user interface of the event annotation tool, and the user interface of the application experiencer can be displayed in the blank part of the content of the user interface of the event annotation tool; or, the display level of the user interface of the application experiencer is higher than the display level of the user interface of the event annotation tool, and the user interface of the application experiencer can be displayed above the user interface of the event annotation tool in the form of a floating window or the like. Optionally, the display of the user interface of the application experiencer in the user interface of the event annotation tool can be fixed or flexibly movable; when the user interface of the application experiencer is flexibly movable, the display position of each element in the user interface of the event annotation tool can be adaptively moved to avoid being blocked, or when the user interface of the application experiencer is displayed in the form of a floating window or the like, the user interface of the event annotation tool can remain unchanged.
[0046] The embodiments of the present application do not limit the method for starting the application experiencer. Optionally, the application experiencer can be started by entering a URL, searching a name, scanning a QR code, entering a voice command, entering identity verification information, etc. The following exemplifies several methods for starting the application experiencer.
[0047] In one example, the above-mentioned startup operation for the application experiencer includes the user interface of the event annotation tool receiving scene parameters; based on this, the above-mentioned step 220 includes: based on the scene parameters received in the user interface of the event annotation tool, loading the online code package of the application in the delivery channel corresponding to the scene parameters; running the online code package to obtain the application experiencer; and displaying the user interface of the application experiencer in the user interface of the event annotation tool.
[0048] In another example, the above-mentioned startup operation for the application experiencer includes the user interface of the event annotation tool receiving path parameters; based on this, the above-mentioned step 220 includes: loading the online code package of the application; running the online code package to obtain the application experiencer; based on the path parameters received in the user interface of the event annotation tool, displaying the user interface of the application experiencer in the user interface of the event annotation tool, and the initial user interface of the application experiencer matches the path parameters.
[0049] Among them, the above-mentioned scene parameters are used to locate the source channel of the online code package, that is, the delivery channel of the application corresponding to the application experiencer, such as the source of the application, the parent application that provides the environment for the operation of the application, etc.; the above-mentioned path parameters are used to locate the initial user interface of the application experiencer, such as the home page, settings page, login page, personal information page, etc. In actual applications, the application experiencer can also be started by combining the scene parameters and path parameters. Optionally, an experiencer start control can also be displayed in the user interface of the event annotation tool. If the analyst does not enter the scene parameters and / or path parameters but operates the experiencer start control, the event annotation tool responds to the operation on the experiencer start control and starts the application experiencer according to the default scene parameters and / or default path parameters.
[0050] For example, as shown in Figure 3, the user interface 310 of the event annotation tool displays an input interface 312 for scene parameters and an input interface 314 for path parameters. After the analyst enters the corresponding parameters in input interface 312 and input interface 314, the event annotation tool receives the scene parameters and path parameters. First, in response to the scene parameters, the online code package of the application in the delivery channel corresponding to the scene parameters is loaded; then the online code package is run to obtain the application experiencer; then, in response to the path parameters, the user interface 320 of the application experiencer is displayed in the user interface 310 of the event annotation tool. The initial user interface of the application experiencer matches the path parameters.
[0051] In another example, the above-mentioned startup operation for the application experiencer includes the authentication information received by the user interface of the event annotation tool; based on this, the above-mentioned step 220 includes: in response to the authentication information received by the user interface of the event annotation tool, obtaining the historical program parameters corresponding to the authentication information, the historical program parameters including the historical scene parameters and the historical path parameters; loading the online code package of the application in the delivery channel corresponding to the historical scene parameters; running the online code package to obtain the application experiencer; displaying the user interface of the application experiencer in the user interface of the event annotation tool, and the initial user interface of the application experiencer matches the historical path parameters. Optionally, the authentication information includes but is not limited to the account name, login password, biometrics, etc.
[0052] The historical program parameters may be the scene parameters and path parameters input by the analyst the last time the analyst annotated an event using the event annotation tool, or the scene parameters corresponding to the source channel of the loaded online code package and the path parameters corresponding to the source page of the annotated event. Of course, the historical program parameters may also be the scene parameters and path parameters input most frequently when the analyst annotates an event using the event annotation tool for the first N times (N is an integer greater than 2), or the scene parameters corresponding to the source channel of the most frequently loaded online code package and the path parameters corresponding to the source page with the most annotated events. This embodiment of the present application does not limit this.
[0053] The event annotation tool can record and store the correspondence between authentication information and historical program parameters. Each time the analyst enters authentication information and program parameters, or each time the analyst completes event annotation, the event annotation tool updates this correspondence, replacing some or all of the stored historical program parameters with the current program parameters. Optionally, a parameter update control can be displayed in the user interface of the event annotation tool. After completing event annotation, the analyst can use the parameter update control to freely select whether to replace the historical program parameters with the current program parameters, thereby updating the aforementioned correspondence.
[0054] Step 230: After the application experiencer enters the element selection mode, in response to the selection operation of the first element in the user interface of the application experiencer, the event annotation information related to the first element is displayed in the user interface of the event annotation tool.
[0055] In some embodiments, the event annotation information is used to annotate the interaction type and display characteristics of elements related to the first element.
[0056] The element selection mode refers to a functional mode provided by the event annotation tool for operating in the user interface of the application experiencer. In this mode, the application experiencer does not respond to the interactive operations received by the user interface of the application experiencer, but the event annotation tool responds to the interactive operations.
[0057] After the application experiencer is started, the analyst can interact with the application experiencer like an application user. For each interactive operation of the analyst, the application experiencer will provide corresponding feedback according to the operating logic of the application. When the application experiencer is in element selection mode, the application experiencer does not respond to the interactive operations received by the application experiencer's user interface. Instead, the event annotation tool responds to the interactive operations, avoiding the impact of the application experiencer's user interface response on the analysis process. After exiting the element selection mode, the currently selected element and the current event annotation information are cleared in the event annotation tool to avoid the event annotation information affecting subsequent operations.
[0058] Among them, when the application experiencer is in element selection mode, the analyst's interactive operations on the user interface of the application experiencer include selection operations on any element in the user interface of the application experiencer, and the element refers to a visual element in the user interface, and the type of the element includes but is not limited to text, pictures, buttons, symbols, icons, sliders, videos, progress bars, audio, labels, etc. At this time, the event annotation tool responds to the selection operation and displays the event annotation information related to the element in the user interface of the event annotation tool. For the specific content, acquisition method and display method of the event annotation information, please refer to the following method embodiment.
[0059] The embodiment of the present application does not limit the method of entering and exiting the element selection mode. Optionally, the element selection mode can be entered and exited by operating on a specific control, inputting voice commands, inputting gestures, etc. In one example, taking entering and exiting the element selection mode by operating on a specific control as an example, the above step 220 also includes: in response to the operation of the mode selection control in the user interface of the event annotation tool, controlling the application experiencer to enter the element selection mode; after the above control of the application experiencer to enter the element selection mode, it also includes: in response to the operation of the element reset control in the user interface of the event annotation tool, clearing the currently selected element and the current event annotation information in the event annotation tool, and controlling the application experiencer to exit the element selection mode. Among them, the mode selection control and the element reset control can be displayed in the form of buttons, icons, sliders, etc., and the operations on the mode selection control and the element reset control include but are not limited to click operations, double-click operations, press operations, slide operations, etc.
[0060] For example, as shown in Figure 4, a mode selection control 412 and an element reset control 414 are displayed in the user interface 410 of the event annotation tool. After the analyst clicks the mode selection control 412, the application experiencer enters the element selection mode and no longer responds to interactive operations on the user interface 420 of the application experiencer; when the application experiencer is in the element selection mode, the analyst clicks on any element 422 in the user interface 420 of the application experiencer, and the event annotation tool responds to the analyst's operation and displays the event annotation information 416 related to the element 422 in the user interface 410 of the event annotation tool; after the analyst clicks the element reset control 414, the currently selected element 422 and the current event annotation information 416 are cleared in the event annotation tool, and the application experiencer also exits the element selection mode and resumes responding to the analyst's interactive operations on the user interface 420 of the application experiencer.
[0061] In some embodiments, the mode selection control may not be displayed in the user interface of the event annotation tool. For example, in response to a first operation on the user interface of the event annotation tool, the application experiencer is controlled to enter the element selection mode. The first operation may be preset when the event annotation tool is developed, or it may be customized by the user of the event annotation tool, and this application does not limit this. The first operation includes but is not limited to a click operation, a double-click operation, a press operation, a slide operation, etc.
[0062] Step 240: In response to the confirmation operation for the event annotation information, an operation event matching the event annotation information is marked within the application corresponding to the application experiencer.
[0063] The analyst can modify the event annotation information. For the modification method of the event annotation information, please refer to the following method embodiment. If the analyst does not have the modification requirement or has completed the modification, the event annotation can be completed by confirming the event annotation information. The embodiment of the present application does not limit the confirmation method of the event annotation information. Optionally, the event annotation information can be confirmed by operating a specific control, inputting a voice command, inputting a gesture, etc. For example, as shown in Figure 4, an information confirmation control 418 is displayed in the user interface 410 of the event annotation tool, and the analyst clicks the information confirmation control 418 to confirm the event annotation information.
[0064] In response to the analyst's confirmation operation on the event annotation information, the event annotation tool annotates all operation events in the application corresponding to the application experiencer that match the event annotation information. In this embodiment of the present application, the event type of the operation event annotated by the event annotation tool in step 240 may be consistent with or inconsistent with the type of selection operation performed by the analyst in step 230. For example, if the selection operation performed by the analyst is a click operation, the operation events annotated by the event annotation tool may include only click events, or may include click events, press events, slide events, double-click events, and the like.
[0065] In an embodiment of the present application, all operation events of an application user on the application are collected and stored in real time, such as on the application server. When performing event annotation, the event annotation tool can extract operation events that match the event annotation information from the storage location of the operation events and export or store them to a default storage location or a storage location specified by the analyst, so that the analyst can immediately obtain relevant data and conduct object behavior analysis of the application.
[0066] In summary, the embodiment of the present application starts the application experiencer corresponding to the application through the event annotation tool, and when the application experiencer is in element selection mode, in response to the analyst's selection operation on any element in the user interface of the application experiencer, obtains and displays the event annotation information related to the element, and then in response to the analyst's confirmation operation on the event annotation information, annotates all operation events in the application that match the event annotation information. The embodiment of the present application realizes the visual annotation of events in the application through the event annotation tool, which reduces the difficulty of event annotation compared to code burial, so that the analyst can independently complete the event annotation through the event annotation tool without the need for multi-party communication and coordination with developers and testers, thereby reducing communication costs and improving event annotation efficiency. In addition, all operation events of the application by users of the application are collected and stored in real time, so that the event annotation tool can immediately extract and store the corresponding operation events when executing event annotation. Without the need for application changes and releases, the analyst can immediately obtain relevant data and conduct analysis, which helps to improve the efficiency of object behavior analysis of the application.
[0067] In addition, the embodiments of the present application provide a variety of flexible ways to start the application experiencer. By starting the application experiencer through scenario parameters, you can select the application's delivery channel and conduct targeted analysis on the object behavior of the application under the corresponding delivery channel; by starting the application experiencer through path parameters, you can select the initial user interface displayed after the application experiencer is started, so as to directly locate the source page of the event that the analyst needs to annotate; by starting the application experiencer through historical program parameters corresponding to identity authentication information, you can match the analyst's annotation habits so that the analyst can quickly carry out event annotation.
[0068] The following describes how to obtain, display, and modify event annotation information.
[0069] In one example, the above step 230 includes the following sub-steps (steps 232 to 234 ).
[0070] Step 232: In response to the selection operation on the first element, obtain a first element set from the user interface of the application experiencer.
[0071] The first element set includes the first element and similar elements of the first element. Optionally, the similar elements of the first element have at least one of the following features: the same interaction type as the first element, the same or partially the same display features as the first element, the same text content as the first element, and the same display area as the first element. Exemplarily, the similar elements of the first element have the same interaction type (i.e., tag label) as the first element and have at least one identical display feature (i.e., class label).
[0072] Optionally, the embodiment of the present application obtains similar elements through a DOM (Document Object Model) tree; based on this, the above step 232 includes: obtaining a DOM tree corresponding to the user interface of the application experiencer; searching for similar elements of the first element based on the DOM tree, and constructing a first element set based on the search results. Among them, the DOM tree is used to indicate each element in the user interface of the application experiencer, as well as the hierarchical relationship between each element; each node in the DOM tree corresponds to an element in the user interface of the application experiencer, and each node includes the identification, hierarchy, instance, child node and other information of the corresponding element. When obtaining similar elements through the DOM tree, the event annotation tool can reduce the search scope based on heuristic search to improve search efficiency.
[0073] Based on this, optionally, the above-mentioned searching for similar elements of the first element based on the DOM tree and constructing the first element set based on the search results include: obtaining the sibling elements of the first element; searching for similar elements of the first element from the sibling elements of the first element; if similar elements of the first element exist among the sibling elements of the first element, combining the first element and the similar elements of the first element to obtain the first element set; if similar elements of the first element do not exist among the sibling elements of the first element, recursively searching upward in the DOM tree; when the upward recursive search reaches the recursive level threshold and no similar elements of the first element are found, constructing the first element set with the first element.
[0074] The nodes corresponding to the sibling elements of the first element are located at the same level in the DOM tree as the node corresponding to the first element. By searching for similar elements among the sibling elements of the first element, the search scope is effectively reduced. Interaction types include but are not limited to click, press, and slide, and display features include but are not limited to display color, display shape, display position, and display element.
[0075] If there is no similar element of the first element among the sibling elements of the first element, the event annotation tool recursively searches upward in the DOM tree. The upward recursive search refers to replacing the first element with the parent element of the first element and re-executes the search for similar elements by obtaining the sibling elements of the first element, that is, re-executes the search for similar elements with the parent element of the first element. By searching upward recursively, the situation where elements are nested in the list rendering method can be taken into account, thereby improving the accuracy of searching for similar elements. However, considering the number of layers of list rendering and the efficiency of searching for similar elements, this example sets a recursive level threshold for upward recursive search, such as two or three layers; when the upward recursive search does not reach the recursive level threshold and similar elements of the first element are found, the first element set is constructed with the first element and its similar elements; when the upward recursive search reaches the recursive level threshold and similar elements of the first element are not found, the event annotation tool abandons the search for similar elements of the first element and directly constructs the first element set with the first element. In some embodiments, the recursive level threshold can be set by the analyst (user of the event annotation tool) or by the developer of the event annotation tool, and this application does not limit this.
[0076] Step 234: Display the selectors of each element in the first element set and its parent element in the user interface of the event annotation tool.
[0077] In some embodiments, the parent element of the first element is the upper element of the first element, or in other words, the parent element of the first element and the first element are in a containment relationship. In this embodiment of the present application, the parent element of the first element is at a higher level in the DOM tree than the first element. For example, the parent element of the first element is a text paragraph element in the user interface of the application experiencer, and the first element is one or more text elements in the text paragraph.
[0078] The event annotation information includes a selector for at least one element, and the selector is used to annotate the interaction type and display characteristics of the element. After obtaining the event annotation information, the event annotation tool can display the event annotation information in its user interface for the analyst to modify and confirm. Optionally, in order to reduce the difficulty of understanding the event annotation information and give the analyst positive feedback, while the event annotation tool displays the event annotation information, it can also enhance the display of each element in the first element set and its parent element in the user interface of the application experiencer. Optionally, the enhanced display method includes but is not limited to highlighting text and pictures, bolding text and lines, additional display of text and icons, etc.
[0079] When there is a need for modification, the analyst can achieve precise selection of the event source element by operating the selector. Based on this, after the event annotation tool displays the event annotation information, in response to the removal operation of the selector for the second element in the user interface of the event annotation tool, the enhanced display of the second element is canceled in the user interface of the application experiencer. Wherein, the second element refers to any element in the first element set and its parent element, and the removal operation of the selector for the second element includes but is not limited to click operation, sliding operation, pressing operation, etc. When the analyst removes the selector of the second element, in addition to canceling the enhanced display of the second element in the user interface of the application experiencer, the display style of the selector of the second element can also be changed in the user interface of the event annotation tool to remind the analyst whether the second element is selected. Optionally, the change in display style includes but is not limited to at least one of the following: display color change, icon state change, display content change, etc.
[0080] For example, as shown in Figure 5, assuming that the analyst clicks on element 522 in the user interface 520 of the application experiencer, the event annotation tool automatically infers similar elements of element 522 to construct a first element set, and displays event annotation information in the user interface 510 of the event annotation tool. The event annotation information includes selectors 512 for each element in the first element set and its parent element; at the same time, in the user interface 520 of the application experiencer, the element 522 and its similar elements 524 are enhanced by bolding the text, and the parent elements of these elements are enhanced by highlighting the area.
[0081] In one example, in order to meet the analyst's needs for richer and more accurate event annotation, after the above step 232, it also includes: displaying at least one supplementary annotation condition in the user interface of the event annotation tool; wherein, in response to the selection operation of the first supplementary annotation condition in the at least one supplementary annotation condition, the event annotation information also includes the first supplementary annotation condition.
[0082] Optionally, the supplementary annotation conditions include at least one of the following: page routing parameters, element text content, and operation event type. Among them, the page routing parameters are used to limit the operation events to the current user interface of the application experiencer; the element text content is used to limit the operation events to the specified text content; the operation event type is used to limit the operation events to the specified event type. Optionally, the event type includes but is not limited to click events, press events, double-click events, sliding events, etc. It should be understood that when the analyst specifies the operation event type, the event type of the operation event annotated and extracted by the event annotation tool may be inconsistent with the analyst's selection operation for the event source element. For example, the analyst selects the event source element through a click operation, but subsequently selects the two supplementary annotation conditions of sliding events and pressing events. In this case, the event annotation tool annotates and extracts sliding events and pressing events for the event source element and its related elements.
[0083] For example, as shown in FIG5 , the user interface 510 of the event annotation tool also displays three types of supplementary annotation conditions, namely, page routing parameters 514, element text content 516, and operation event type 518. By clicking on the content or control corresponding to page routing parameters 514, the analyst can add a restriction to the operation event originating from the current user interface of the application experiencer in the event annotation information; by clicking on the content or control corresponding to element text content 516, the analyst can add a restriction to the operation event originating from the specified text content in the event annotation information; and by clicking on the content or control corresponding to operation event type 518, the analyst can add a restriction to the operation event originating from the specified event type in the event annotation information.
[0084] In summary, in the embodiment of the present application, after the analyst selects the event source element from the user interface of the application experiencer, the event annotation tool automatically infers the similar elements of the event source element, and based on the event source element and its similar elements, obtains and displays the event annotation information, which helps to achieve rapid annotation of events corresponding to a class of elements. In addition, when inferring similar elements, the event annotation tool considers that the user interface rendering may be in the form of list rendering, and if necessary, recursively searches upward for similar elements based on the parent element of the event source element, thereby improving the accuracy and comprehensiveness of the similar element search. In addition, the embodiment of the present application also meets the analyst's needs for richer and more accurate event annotation by adding restrictions on event annotation information through page routing parameters, element text content, and operation event types.
[0085] Below, an embodiment is used to describe the overall process of the event annotation method provided by the present application. The method can be applied to an event annotation tool, such as the event annotation tool 20 shown in Figure 1. In one example, the event annotation tool includes a runtime control layer, an event annotation editor, and an event definition manager. As shown in Figure 6, the event annotation method includes at least one of the following steps (steps 610 to 670).
[0086] Step 610, runtime initialization.
[0087] The runtime control layer performs runtime initialization, which includes: loading the online code package of the application, running the online code package, and displaying the user interface of the application experiencer. After loading the online code package of the application, the event annotation tool can automatically inject a custom script to wrap addEventListener to intercept and record operation events. In addition, through pauseInnerCaptureMouseListeners and resumeInnerCaptureMouseListeners, dynamic closing response and dynamic resumption response for interactive operations on the user interface of the application experiencer are respectively implemented. In addition, by registering lifecycle hooks such as closing the application experiencer and route changes, the normal operation of the application experiencer is ensured; packaging methods such as entering element selection mode, exiting element selection mode, and enhancing display, and attaching custom logic by injecting scripts into the corresponding levels (native, webview) in the application experiencer iframe (inline frame).
[0088] Step 620: Control the application experiencer to enter element selection mode.
[0089] In response to the analyst's corresponding operations, the event definition manager sends corresponding instructions to the runtime control layer to control the application experiencer to enter element selection mode, prohibit the application experiencer from responding to interactive operations on its user interface, and can add overlay events such as hover (mouseover) and click (click).
[0090] Step 630: Calculate similar elements to the event source element and display event annotation information.
[0091] In response to the analyst's selection operation on the event source element, the event definition manager returns the event source element to the event annotation editor; the event annotation editor further infers similar elements of the event source element, determines the selector of at least one element in the event annotation information based on the event source element and its similar elements, and initializes the display of the event annotation information.
[0092] Step 640: Enhance the display of corresponding elements based on the event annotation information.
[0093] The event definition manager coordinates the runtime control layer to enhance the display of corresponding elements in the user interface of the application experiencer based on the selector in the event annotation information and the analyst's selection operation of the supplementary annotation conditions in the event annotation information set by the event annotation editor.
[0094] Step 650: Determine whether the event annotation information has been modified.
[0095] In response to the analyst's actions, the event definition manager determines whether the event annotation information has been modified. If so, step 640 is re-executed using the modified event annotation information, indicating that the event annotation editor needs to respond to the element change. If not, step 660 is executed.
[0096] Step 660: Determine whether the application experiencer has exited the element selection mode.
[0097] In response to the analyst's operation, the event definition manager determines whether the application experiencer has exited element selection mode, that is, whether the event annotation information has been reset. If the application experiencer has exited element selection mode, the analysis will restart from step 620 after the analyst restarts element selection mode. If the application experiencer has not exited element selection mode, the analysis will continue to step 670.
[0098] Step 670: Mark the operation event that matches the event marking information.
[0099] In response to the analyst's confirmation operation on the event annotation information, the event definition manager annotates all operation events in the application that match the event annotation information, and stores these operation events for the analyst to perform object behavior analysis, etc.
[0100] For other introductions and explanations about how to enter and exit the element selection mode in the embodiment of Figure 6, how to calculate similar elements, how to start the application experiencer, the specific content of event annotation information, how to enhance the display of elements, etc., please refer to the above method embodiment and no further details will be given here.
[0101] The following are device embodiments of the present application, which can be used to implement the method embodiments of the present application. For details not disclosed in the device embodiments of the present application, please refer to the method embodiments of the present application.
[0102] Please refer to Figure 7, which shows a block diagram of an event annotation device within an application provided by an embodiment of the present application. The device has the function of implementing the example of the event annotation method within the above-mentioned application, and the function can be implemented by hardware or by hardware executing corresponding software. The device can be a computer device or can be set in a computer device. The device 700 can include a tool interface display module 710, an experiencer interface display module 720, a annotation information display module 730 and an operation event annotation module 740.
[0103] The tool interface display module 710 is used to display the user interface of the event annotation tool in the browser.
[0104] The experiencer interface display module 720 is used to display the user interface of the application experiencer in the user interface of the event annotation tool in response to the startup operation of the application experiencer. The application experiencer refers to the application runtime instance used for analyst interaction.
[0105] The annotation information display module 730 is used to display event annotation information related to the first element in the user interface of the event annotation tool in response to a selection operation on the first element in the user interface of the application experiencer after the application experiencer enters the element selection mode, and the event annotation information is used to annotate the interaction type and display characteristics of the elements related to the first element.
[0106] The operation event marking module 740 is used to mark the operation event matching the event marking information in the application corresponding to the application experiencer in response to the confirmation operation of the event marking information.
[0107] In one example, as shown in Figure 8, the annotation information display module 730 includes: an element set acquisition unit 732, which is used to obtain a first element set from the user interface of the application experiencer in response to a selection operation on the first element, wherein the first element set includes the first element and similar elements of the first element; an element selector display unit 734, which is used to display the selectors of each element in the first element set and its parent element in the user interface of the event annotation tool; wherein the event annotation information includes a selector for at least one element, and the selector is used to annotate the interaction type and display characteristics of the element, and the parent element of the first element is the upper element of the first element.
[0108] In one example, as shown in FIG8 , the element set acquisition unit 732 is used to: obtain a DOM tree corresponding to the user interface of the application experiencer, the DOM tree being used to indicate each element in the user interface of the application experiencer, and the hierarchical relationship between each of the elements; search for similar elements of the first element based on the DOM tree, and construct the first element set based on the search results.
[0109] In one example, as shown in FIG8 , the element set acquisition unit 732 is further used to: acquire sibling elements of the first element, wherein the nodes corresponding to the sibling elements of the first element and the nodes corresponding to the first element are located at the same level in the DOM tree; search for similar elements of the first element from the sibling elements of the first element; if similar elements of the first element exist among the sibling elements of the first element, combine the first element and the similar elements of the first element to obtain the first element set; if similar elements of the first element do not exist among the sibling elements of the first element, recursively search upward in the DOM tree; wherein the upward recursive search refers to replacing the first element with the parent element of the first element and executing again from acquiring the sibling elements of the first element; when the upward recursive search reaches a recursive level threshold and no similar elements of the first element are found, construct the first element set with the first element; wherein the similar elements of the first element have at least one of the following characteristics: the same interaction type as the first element, the same or partially the same display characteristics as the first element, the same text content as the first element, and being located in the same display area as the first element.
[0110] In one example, as shown in Figure 8, the device 700 also includes an element enhancement display module 750, which is used to: enhance the display of each element in the first element set and its parent element in the user interface of the application experiencer; in response to the removal operation of the selector of the second element in the user interface of the event annotation tool, cancel the enhanced display of the second element in the user interface of the application experiencer; the second element refers to any element among the elements in the first element set and their parent elements.
[0111] In one example, as shown in Figure 8, the annotation information display module 730 also includes a supplementary condition display unit 736, which is used to: display at least one supplementary annotation condition in the user interface of the event annotation tool, the supplementary annotation condition including page routing parameters and element text content, the page routing parameters are used to limit the operation event to originate from the current user interface of the application experiencer, and the element text content is used to limit the operation event to a specified text content; wherein, in response to a selection operation for a first supplementary annotation condition among the at least one supplementary annotation condition, the event annotation information also includes the first supplementary annotation condition.
[0112] In one example, as shown in Figure 8, the device 700 also includes a selection mode control module 760, which is used to: control the application experiencer to enter the element selection mode in response to the operation of the mode selection control in the user interface of the event annotation tool; wherein, when the application experiencer is in the element selection mode, the application experiencer does not respond to the interactive operations received by the user interface of the application experiencer.
[0113] In one example, as shown in Figure 8, the selection mode control module 760 is also used to: clear the currently selected element and the current event annotation information in the event annotation tool in response to the operation of the element reset control in the user interface of the event annotation tool; and control the application experiencer to exit the element selection mode.
[0114] In one example, the experiencer interface display module 720 is used to: load the online code package of the application in the delivery channel corresponding to the scene parameters based on the scene parameters received in the user interface of the event annotation tool; run the online code package to obtain the application experiencer; display the user interface of the application experiencer in the user interface of the event annotation tool; wherein, the startup operation for the application experiencer includes receiving the scene parameters in the user interface of the event annotation tool.
[0115] In one example, the experiencer interface display module 720 is used to: load the online code package of the application; run the online code package to obtain the application experiencer; based on the path parameters received in the user interface of the event annotation tool, display the user interface of the application experiencer in the user interface of the event annotation tool, and the initial user interface of the application experiencer matches the path parameters; wherein the startup operation for the application experiencer includes the user interface of the event annotation tool receiving the path parameters.
[0116] In summary, the embodiment of the present application starts the application experiencer corresponding to the application through the event annotation tool, and when the application experiencer is in element selection mode, in response to the analyst's selection operation on any element in the user interface of the application experiencer, obtains and displays the event annotation information related to the element, and then in response to the analyst's confirmation operation on the event annotation information, annotates all operation events in the application that match the event annotation information. The embodiment of the present application realizes the visual annotation of events in the application through the event annotation tool, which reduces the difficulty of event annotation compared to code burial, so that the analyst can independently complete the event annotation through the event annotation tool without the need for multi-party communication and coordination with developers and testers, thereby reducing communication costs and improving event annotation efficiency. In addition, all operation events of the application by users of the application are collected and stored in real time, so that the event annotation tool can immediately extract and store the corresponding operation events when executing event annotation. Without the need for application changes and releases, the analyst can immediately obtain relevant data and conduct analysis, which helps to improve the efficiency of object behavior analysis of the application.
[0117] It should be noted that the apparatus provided in the above embodiments, when implementing its functions, is only illustrated by the division of the above functional modules. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the apparatus and method embodiments provided in the above embodiments are based on the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.
[0118] Please refer to Figure 9, which shows a schematic diagram of the structure of a computer device provided in an embodiment of the present application, which can be used to implement the functions of the example of the event annotation method within the above application. Specifically:
[0119] The computer device 900 includes a processing unit (such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array)) 901, a system memory 904 including RAM (Random-Access Memory) 902 and ROM (Read-Only Memory) 903, and a system bus 905 connecting the system memory 904 and the CPU 901. The computer device 900 also includes an I / O system (Basic Input / Output System) 906 that facilitates information transmission between various components within the computer device, and a mass storage device 907 for storing an operating system 913, application programs 914, and other program modules 915.
[0120] The basic input / output system 906 includes a display 908 for displaying information and an input device 909 such as a mouse and a keyboard for user input. The display 908 and the input device 909 are connected to the central processing unit 901 via an input / output controller 910 connected to the system bus 905. The basic input / output system 906 may also include an input / output controller 910 for receiving and processing input from a variety of other devices such as a keyboard, a mouse, or an electronic stylus. Similarly, the input / output controller 910 also provides output to a display screen, a printer, or other types of output devices.
[0121] The mass storage device 907 is connected to the central processing unit 901 via a mass storage controller (not shown) connected to the system bus 905. The mass storage device 907 and its associated computer-readable media provide non-volatile storage for the computer device 900. In other words, the mass storage device 907 may include a computer-readable medium (not shown) such as a hard disk or a CD-ROM (Compact Disc Read-Only Memory) drive.
[0122] Without loss of generality, the computer-readable medium may include computer storage media and communication media. Computer storage media include volatile and non-volatile, removable and non-removable media implemented by any method or technology for storing information such as computer-readable instructions, data structures, program modules or other data. Computer storage media include RAM, ROM, EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-Only Memory), flash memory or other solid-state storage technologies, CD-ROM, DVD (Digital Video Disc) or other optical storage, tape cassettes, magnetic tape, disk storage or other magnetic storage devices. Of course, those skilled in the art will appreciate that the computer storage media is not limited to the above-mentioned ones. The above-mentioned system memory 904 and mass storage device 907 can be collectively referred to as memory.
[0123] According to an embodiment of the present application, the computer device 900 can also be connected to a remote computer on a network such as the Internet for operation. That is, the computer device 900 can be connected to a network 912 via a network interface unit 911 connected to the system bus 905. Alternatively, the network interface unit 911 can be used to connect to other types of networks or remote computer systems (not shown).
[0124] The memory further includes a computer program, which is stored in the memory and configured to be executed by one or more processors to implement the event marking method in the above application program.
[0125] In an exemplary embodiment, a computer-readable storage medium is further provided, wherein a computer program is stored in the storage medium. The computer program is loaded and executed by a processor to implement the event annotation method within the above application program.
[0126] Optionally, the computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a solid-state drive (SSD), or an optical disk. Among them, the random access memory may include a resistance random access memory (ReRAM) and a dynamic random access memory (DRAM).
[0127] In an exemplary embodiment, a computer program product is also provided, the computer program product including a computer program stored in a computer-readable storage medium. A processor of a computer device reads the computer program from the computer-readable storage medium and executes the computer program, causing the computer device to implement the above-mentioned event annotation method within an application.
[0128] It should be understood that the "multiple" mentioned in this article refers to two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can represent three situations: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the previous and subsequent associated objects are in an "or" relationship. In addition, the step numbers described in this article only illustrate a possible execution sequence between the steps. In some other embodiments, the above steps may not be executed in the order of the numbers, such as two steps with different numbers are executed at the same time, or two steps with different numbers are executed in the opposite order to the diagram. The embodiments of the present application do not limit this.
[0129] The above description is merely an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.
[0130] In this application, relevant data and event processing (such as collection, recording, storage, etc.) should be strictly in accordance with the requirements of relevant national laws and regulations when applied in examples, and the informed consent or separate consent of the personal information subject (or other legal basis) should be obtained. Subsequent data use and processing (such as extraction, statistics, analysis, etc.) should be carried out within the scope of authorization of laws and regulations and the personal information subject.
[0131] In this application, when the facial (or other biometric) recognition technology involved is applied to specific products or technologies, the relevant data collection, use and processing processes should comply with the requirements of national laws and regulations. Before collecting facial information, the information processing rules should be informed and the informed consent or separate consent of the personal information subject (or other legal basis) should be obtained. Facial information should be processed strictly in accordance with the requirements of laws and regulations and personal information processing rules, and technical measures should be taken to ensure the security of relevant data.
Claims
1. A method for annotating an event in an application, the method being executed by a terminal device, the method comprising: Display the event annotation tool's user interface in the browser; In response to a start operation on an application experiencer, displaying a user interface of the application experiencer in a user interface of the event annotation tool, wherein the application experiencer refers to a runtime instance of an application for interactive use by an analyst; After the application experiencer enters the element selection mode, in response to a selection operation on a first element in the user interface of the application experiencer, event annotation information related to the first element is displayed in the user interface of the event annotation tool, wherein the event annotation information is used to annotate the interaction type and display characteristics of the element related to the first element; In response to a confirmation operation on the event annotation information, an operation event matching the event annotation information is marked within the application corresponding to the application experiencer.
2. The method according to claim 1, wherein: In response to a selection operation on a first element in a user interface of the application experiencer, displaying event annotation information related to the first element in a user interface of the event annotation tool includes: In response to a selection operation on the first element, obtaining a first element set from a user interface of the application experiencer, wherein the first element set includes the first element and elements similar to the first element; In the user interface of the event annotation tool, selectors of each element in the first element set and its parent element are displayed, wherein the event annotation information includes a selector of at least one element, and the selector is used to annotate the interaction type and display characteristics of the element.
3. The method according to claim 1 or 2, wherein: The obtaining a first element set from the user interface of the application experiencer includes: Obtain a document object model DOM tree corresponding to the user interface of the application experiencer, wherein the DOM tree is used to indicate each element in the user interface of the application experiencer and the hierarchical relationship between each of the elements; Similar elements of the first element are searched based on the DOM tree, and the first element set is constructed based on the search results.
4. The method according to any one of claims 1 to 3, wherein: The searching for similar elements of the first element based on the DOM tree and constructing the first element set based on the search results includes: Acquire a sibling element of the first element, where a node corresponding to the sibling element of the first element and a node corresponding to the first element are located at the same level in the DOM tree; Search for similar elements to the first element from among the sibling elements of the first element; If there are similar elements to the first element among the sibling elements of the first element, then the first element and the similar elements to the first element are combined to obtain the first element set; If there is no similar element to the first element among the sibling elements of the first element, recursively search upward in the DOM tree; wherein the upward recursive search means replacing the first element with the parent element of the first element and performing the search again from obtaining the sibling elements of the first element; When the upward recursive search reaches a recursive level threshold and no similar element of the first element is found, constructing the first element set with the first element; Among them, the similar element of the first element has at least one of the following characteristics: the same interaction type as the first element, the same or partially the same display characteristics as the first element, the same text content as the first element, and the same display area as the first element.
5. The method according to any one of claims 1 to 4, wherein: After obtaining the first element set from the user interface of the application experiencer, the method further includes: In the user interface of the application experiencer, each element in the first element set and its parent element are enhanced and displayed; In response to a removal operation of a selector for a second element in the user interface of the event annotation tool, the enhanced display of the second element is canceled in the user interface of the application experiencer; the second element refers to any element in the first element set and its parent element.
6. The method according to any one of claims 1 to 5, wherein: After obtaining the first element set from the user interface of the application experiencer, the method further includes: Displaying at least one supplementary annotation condition in the user interface of the event annotation tool, the supplementary annotation condition including a page routing parameter and an element text content, the page routing parameter being used to limit the operation event to originate from the current user interface of the application experiencer, and the element text content being used to limit the operation event to a specified text content; In response to a selection operation on a first supplementary marking condition among the at least one supplementary marking condition, the event marking information further includes the first supplementary marking condition.
7. The method according to any one of claims 1 to 6, wherein: After the user interface of the application experiencer is displayed in the user interface of the event annotation tool, the method further includes: In response to an operation on a mode selection control in a user interface of the event annotation tool, controlling the application experiencer to enter the element selection mode; Wherein, when the application experiencer is in the element selection mode, the application experiencer does not respond to interactive operations received by the user interface of the application experiencer.
8. The method according to any one of claims 1 to 7, wherein: After controlling the application experiencer to enter the element selection mode, the method further includes: In response to an operation of resetting a control element in a user interface of the event annotation tool, clearing the currently selected element and the current event annotation information in the event annotation tool; Control the application experiencer to exit the element selection mode.
9. The method according to any one of claims 1 to 8, wherein: In response to the start operation on the application experiencer, displaying the user interface of the application experiencer in the user interface of the event annotation tool includes: Based on the scene parameters received in the user interface of the event annotation tool, loading the online code package of the application in the delivery channel corresponding to the scene parameters; Running the online code package to obtain the application experiencer; Displaying the user interface of the application experiencer in the user interface of the event annotation tool; Among them, the startup operation for the application experiencer includes the user interface of the event annotation tool receiving scene parameters.
10. The method according to any one of claims 1 to 9, wherein: In response to the start operation on the application experiencer, displaying the user interface of the application experiencer in the user interface of the event annotation tool includes: Loading the online code package of the application; Running the online code package to obtain the application experiencer; Based on the path parameters received in the user interface of the event annotation tool, displaying the user interface of the application experiencer in the user interface of the event annotation tool, the initial user interface of the application experiencer matching the path parameters; Among them, the startup operation for the application experiencer includes the user interface receiving path of the event annotation tool parameter.
11. An event marking device in an application, the device comprising: A tool interface display module, used to display the user interface of the event annotation tool in the browser; An experiencer interface display module, for displaying the user interface of the application experiencer in the user interface of the event annotation tool in response to a start-up operation for the application experiencer, wherein the application experiencer refers to a runtime instance of the application for interactive use by the analyst; An annotation information display module is used to display event annotation information related to the first element in the user interface of the event annotation tool in response to a selection operation on the first element in the user interface of the application experiencer after the application experiencer enters the element selection mode, wherein the event annotation information is used to annotate the interaction type and display characteristics of the element related to the first element; An operation event marking module is used to mark the operation events matching the event marking information in the application corresponding to the application experiencer in response to a confirmation operation on the event marking information.
12. A computer device, comprising a processor and a memory, wherein a computer program is stored in the memory, and the computer program is loaded and executed by the processor to implement the event annotation method in an application according to any one of claims 1 to 10.
13. A computer-readable storage medium, wherein a computer program is stored in the storage medium, and the computer program is loaded and executed by a processor to implement the event annotation method in an application according to any one of claims 1 to 10.
14. A computer program product, comprising a computer program, wherein the computer program is stored in a computer-readable storage medium, and a processor reads and executes the computer program from the computer-readable storage medium to implement the event annotation method within an application as described in any one of claims 1 to 10.
Citation Information
Patent Citations
Debugging method and device of application program, computer equipment and storage medium
CN113032273A
Method and device for labeling data of information interaction interface
CN113093974A
Behavior event labeling method and device, computer equipment and storage medium
CN113095049A
Accurate and efficient recording of user experience, GUI changes and user interaction events on a remote web document
US20160188548A1
