Interface data processing method, computer equipment and computer program product
By formatting the interface operation track model, the problem of converting screen content to other file formats in the prior art requires dependence on a specific environment, and the flow of the operation track model and compatibility of multiple applications is achieved.
Patent Information
- Application Number
- CN202411571403.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-05
- Publication Date
- 2025-02-11
AI Technical Summary
The prior art requires relying on a specific environment when converting screen content to other file formats, the process is complex, and the flow of the operation track model is limited.
By converting the first operation track model in format, converting it into a second operation track model in at least one data format, the conversion of information formats, rather than the conversion of special data such as codes. This method does not need to rely on a special computer environment and is implemented directly through the application of the computer device.
It solves the problem that converting screen content to other file formats requires dependence on a specific environment, and realizes the flow of the operation track model and can be shared to at least one application for processing.
Smart Images
Figure CN119415380A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of artificial intelligence technology, and in particular to a method for processing interface data, a computer device, and a computer program product. Background Art
[0002] When users encounter a favorite or interesting interface that needs to be saved during daily use of the terminal, they can connect to the terminal through other terminals with an SDK development environment, then use the element positioning tool (uiautomatorviewer) on the other terminal to dump (dumpsys) the terminal screen content and save the current interface elements as a uix type file. Summary of the invention
[0003] The embodiment of the present application provides a method for processing interface data, a computer device and a computer program product. The technical solution is as follows:
[0004] On the one hand, an embodiment of the present application provides a method for processing interface data, the method comprising:
[0005] Determine a first operation trajectory model, wherein the first operation trajectory model includes node information of a node corresponding to an interface during an operation and edge information of a behavior edge corresponding to an operation behavior in the interface, wherein the node information includes information characterizing interface features of the interface from different dimensions, and the edge information includes information for characterizing the operation behavior and node jump conditions before and after the operation behavior;
[0006] Determining at least one data format, wherein the at least one data format is a data format supported by at least one application;
[0007] The first operation trajectory models are converted into second operation trajectory models in the at least one data format respectively, and the second operation trajectory model in any data format is used to be shared with its corresponding application for processing.
[0008] On the other hand, an embodiment of the present application provides a device for processing interface data, the device comprising:
[0009] A first determination module is used to determine a first operation trajectory model, wherein the first operation trajectory model includes node information of nodes corresponding to the interface during the operation and edge information of behavior edges corresponding to the operation behavior in the interface, wherein the node information includes information characterizing interface features of the interface from different dimensions, and the edge information includes information used to characterize the operation behavior and node jump conditions before and after the operation behavior;
[0010] A second determination module, configured to determine at least one data format, wherein the at least one data format is a data format supported by at least one application program;
[0011] The conversion module is used to convert the first operation trajectory model into a second operation trajectory model in the at least one data format, and the second operation trajectory model in any data format is used to be shared with its corresponding application for processing.
[0012] On the other hand, an embodiment of the present application provides a computer device, which includes a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the method for processing interface data as described in the above aspects.
[0013] On the other hand, an embodiment of the present application provides a computer-readable storage medium, which stores at least one computer instruction, and the at least one computer instruction is used to be executed by a processor to implement the method for processing interface data as described in the above aspects.
[0014] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the method for processing interface data as described in the above aspects.
[0015] In the embodiment of the present application, the first operation trajectory model is converted into a second operation trajectory model in at least one data format by format conversion; since the first operation trajectory model includes node information of the corresponding node of the interface during the operation and edge information of the corresponding behavior edge of the operation behavior in the interface; thus, the embodiment of the present application realizes the conversion of the format of information, rather than the conversion of special data such as code; therefore, the format conversion of the embodiment of the present application does not need to rely on a special computer environment, and can be realized directly through the application of the computer device, solving the problem that the conversion of screen content into other file formats requires dependence on a specific environment. Moreover, since at least one data format is a data format supported by at least one application; therefore, the second operation trajectory model can be shared with at least one application for processing, thereby realizing the circulation of the operation trajectory model. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] Figure 1 is a schematic diagram of an implementation environment of a method for processing interface data provided by an exemplary embodiment of the present application;
[0017] Figure 2 is a flow chart of a method for processing interface data provided by an exemplary embodiment of the present application;
[0018] Figure 3is a flow chart of a method for processing interface data shown in an exemplary embodiment of the present application;
[0019] Figure 4 is a schematic diagram of a method for processing interface data shown in an exemplary embodiment of the present application;
[0020] Figure 5 is a schematic diagram of a method for processing interface data shown in an exemplary embodiment of the present application;
[0021] Figure 6 is a schematic diagram of a method for processing interface data shown in an exemplary embodiment of the present application;
[0022] Figure 7 is a schematic diagram of a method for processing interface data shown in an exemplary embodiment of the present application;
[0023] Figure 8 is a schematic diagram of a method for processing interface data shown in an exemplary embodiment of the present application;
[0024] Fig. 9 A structural block diagram of an interface data processing device provided by another exemplary embodiment of the present application is shown;
[0025] Fig.10 A structural block diagram of a computer device provided by an exemplary embodiment of the present application is shown. DETAILED DESCRIPTION
[0026] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.
[0027] The term "multiple" as used herein refers to two or more than two. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects are in an "or" relationship.
[0028] It should be noted that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions. For example, the node information and edge information involved in this application are all obtained with full authorization.
[0029] In the related art, users usually use the uiautomatorviewer tool to transfer screen content into a uix type file; and transferring into a uix type file not only requires configuring the SDK development environment, but also requires each interface to refresh the page and then export the interface elements during the transfer process, which makes the process of obtaining the uix type file complicated.
[0030] In the embodiment of the present application, the first operation trajectory model is converted into a second operation trajectory model in at least one data format by format conversion; since the first operation trajectory model includes node information of the corresponding node of the interface during the operation and edge information of the corresponding behavior edge of the operation behavior in the interface; thus, the embodiment of the present application realizes the conversion of the format of information, rather than the conversion of special data such as code; therefore, the format conversion of the embodiment of the present application does not need to rely on a special computer environment, and can be realized directly through the application of the computer device, solving the problem that the conversion of screen content into other file formats requires dependence on a specific environment. Moreover, since at least one data format is a data format supported by at least one application; therefore, the second operation trajectory model can be shared with at least one application for processing, thereby realizing the circulation of the operation trajectory model.
[0031] For example, if the application is a browser, the data format may be a web page format, that is, the first operation trajectory model is converted into a second operation trajectory model in a web page format, so that the second operation trajectory model can be transferred to the browser for display. For another example, if the application is a text editor, the data format may be a text format or a json format, that is, the first operation trajectory model is converted into a second operation trajectory model in a text format or a json format, so that the second operation trajectory model can be transferred to the text editor for editing and display. For another example, if the application is notion, Youdao Cloud, typora and other applications, the data format may be a lightweight markup language format, that is, the first operation trajectory model is converted into a second operation trajectory model in a lightweight markup language format, so that the second operation trajectory module can be transferred in notion, Youdao Cloud, typora and other applications. For another example, if the application is a picture display, the application is a mind map format, that is, the first operation trajectory model is converted into a second operation trajectory model in a mind map format, so that the second operation trajectory model can be edited and displayed in the picture display.
[0032] Optionally, before converting the first operation trajectory model into at least one second operation trajectory model in a data format, the first operation trajectory model may be compressed, and the compressed third operation trajectory model may be converted into at least one second operation trajectory model in a data format, and then the second operation trajectory model may be circulated. Since when the first operation trajectory model is compressed, the second nested relationship between the multiple interfaces involved in the operation process deletes the node information corresponding to some interfaces and the edge information of the behavior edges corresponding to the operation behaviors in some interfaces, thus reducing the page level nesting; therefore, when performing data query based on the third operation trajectory model (or the second operation trajectory model obtained by format conversion), the data query efficiency can be improved; and the readability of the data is also improved.
[0033] The method for processing interface data provided in the embodiment of the present application can be executed by a computer device with an interface display function. The computer device can be a smart phone, a tablet computer, a smart wearable device, a personal computer, etc. In the following embodiments, the execution subject of the method for processing interface data is a computer device as an example for explanation. Please refer to Figure 1 , which shows a schematic diagram of an implementation environment of a method for processing interface data provided by an exemplary embodiment of the present application. The implementation environment includes a computer device, the computer device displays an interface, the interface includes operation buttons (button1, button2 and button3), and the operation buttons are used to trigger operation behaviors; during the user operation of the interface, the computer device determines the node information of the corresponding nodes (node1, node2 and node3) of the interface during the operation and the edge information of the corresponding behavior edges of the operation behaviors (action1 and cation2) in the interface, generates a first operation trajectory model, compresses the first operation trajectory model and converts it into a second operation trajectory model in other data formats for circulation, and multiple second operation trajectory models in other data formats can be called circulation models, and the circulation models include a second operation trajectory model in a web page format, a second operation trajectory model in a text format, a second operation trajectory model in a lightweight markup language format, a second operation trajectory model in a json format, and a second operation trajectory model in a mind map format.
[0034] Please refer to Figure 2 , which shows a flowchart of a method for processing interface data provided by an exemplary embodiment of the present application. This embodiment is described by taking the method used in a computer device as an example, and the method may include the following steps:
[0035] Step 201: Determine a first operation trajectory model, where the first operation trajectory model includes node information of nodes corresponding to the interface during the operation and edge information of behavior edges corresponding to the operation behaviors in the interface. The node information includes information characterizing interface features of the interface from different dimensions, and the edge information includes information used to characterize the operation behavior and the node jump situation before and after the operation behavior.
[0036] Introduction to the interface: The interface is a visual user interface, such as a desktop, an application interface, a system interface, and the like. In the embodiment of the present application, different interfaces during the operation process correspond to respective nodes (Node), and different nodes characterize the interface features of their respective corresponding interfaces through their respective node information, that is, different nodes can be distinguished based on the node information. Optionally, the dimensions characterizing the interface features may include interface content dimensions, interface layout dimensions, interface function dimensions, interface description dimensions, the window to which the interface belongs, and the like.
[0037] The interface in this embodiment is not limited to the visible range of the screen, but includes the completed interface within and outside the visible range of the screen. Accordingly, before and after the interface content is scrolled, even if the displayed content within the visible range of the screen changes significantly, the interface before and after the scrolling will not be judged as a different interface.
[0038] The different interfaces in the embodiments of the present application refer to interfaces with large differences in interface layout, content, and style. Usually, when an interface switch occurs, the interfaces before and after the switch are determined to be different interfaces. For example, when the text or images in the interface change dynamically, or when the list elements in the interface change, they are not determined to be different interfaces; when clicking on a control in the interface to trigger an interface jump, the interfaces before and after the jump are determined to be different interfaces. The process of determining different interfaces will be described in detail in the following embodiments.
[0039] Introduction to node information: The node information of a node at least includes the node identifier and the node factor of the node.
[0040] Accordingly, in a possible implementation, determining the node information of the node corresponding to the interface during the operation process may include: determining the node factor of the node corresponding to the interface during the operation process, the node factor including node sub-factors that characterize the interface characteristics of the interface from different dimensions; determining the node identifier of the node; and determining the node factor and the node identifier as the node information of the node.
[0041] The node identifier is a unique identifier assigned to a node (with no actual meaning), which can be represented in the form of a string. Different nodes have different node identifiers, while the same node has the same node identifier.
[0042] Optionally, a node factor is composed of several fields that characterize interface features from different dimensions, wherein the fields that characterize interface features can be referred to as node sub-factors. When determining whether two nodes are the same node, the computer device matches the field contents with the same field name in the two node factors, that is, matches the node sub-factors in different node factors, thereby determining whether the two nodes are the same node based on the matching results of multiple dimensions.
[0043] Introduction to edge information: The behavior edge (Edge) corresponding to the operation behavior is a directed edge, and the direction of the behavior edge is the interface jump direction, that is, the behavior edge points from the interface before the jump to the interface after the jump. Optionally, the operation behavior refers to a manual operation behavior of the user, such as a trigger operation on a control in the interface; or, the operation behavior refers to a system behavior, such as an automatic interface jump operation after reaching a preset time. In some embodiments, a variety of operation behavior types are pre-set, and the edge information of the behavior edge includes the operation behavior type to which the operation behavior belongs. Regarding the recording method of the node jump situation before and after the operation behavior, in a possible implementation, the edge information includes information about the corresponding nodes of the interface before and after the operation behavior. For example, the edge information includes the node identifier of the corresponding node of the interface before the operation behavior is executed, and the node identifier of the corresponding node of the interface after the operation behavior is executed. Among them, the interface identifiers corresponding to the interfaces before and after the operation behavior can be different interface identifiers (that is, the interface jump occurs after the operation), or they can be the same interface identifier (that is, the operation does not occur an interface jump).
[0044] In some embodiments, the edge information includes at least: behavior information of the operation behavior, a starting node identifier (fromNodeID) of the starting node corresponding to the starting interface before the operation behavior, an ending node identifier (toNodeID) of the ending node corresponding to the ending interface after the operation behavior, and an edge identifier (a unique identifier similar to a node identifier and a node element identifier).
[0045] Correspondingly, when determining the edge information of the behavior edge corresponding to the operation behavior in the interface during the operation process, the computer device determines the behavior information of the operation behavior in the interface during the operation process, and determines the starting node identifier of the starting node corresponding to the starting interface before the operation behavior, and the ending node identifier of the ending node corresponding to the ending interface after the operation behavior, thereby determining the behavior information, the starting node identifier and the ending node identifier as the edge information of the behavior edge corresponding to the operation behavior.
[0046] Among them, when the interface before and after the operation behavior does not switch, the starting node identifier and the ending node identifier in the side information corresponding to the operation behavior are the same; when the interface before and after the operation behavior switches, the starting node identifier and the ending node identifier in the side information corresponding to the operation behavior are different.
[0047] As an important content in the side information, the behavior information needs to be able to clearly indicate the operation behavior. In a possible implementation, the behavior information includes the action type (ActionType) of the operation behavior.
[0048] Step 202: Determine at least one data format, where the at least one data format is a data format supported by at least one application.
[0049] Optionally, at least one application includes at least one of a browser, a text editor, notion, Youdao Cloud, typora, and a picture display; for example, if the application is a browser, the data format supported by the browser is a web page format; for another example, if the application is a text editor, the data format supported by the text editor is a text format or a json format; for another example, if the application is notion, Youdao Cloud, typora, etc., the data format supported by notion, Youdao Cloud, and typora may be a lightweight markup language format; for another example, if the application is a picture display, the data format supported by the picture display is a mind map format.
[0050] In a possible implementation, there are multiple data formats, and the computer device converts the first operation trajectory model into multiple data formats of the second operation trajectory model by default, so that the user can subsequently process the second operation trajectory model in one data format or multiple data formats based on needs. The first operation trajectory model supports a wide range of data format conversions for connecting with other applications to achieve ecological layout.
[0051] In another possible implementation, when the user wants to transfer the first operation trajectory model through an application, the user triggers the data format to the computer device, and the computer device determines the triggered data format; in this case, the number of data formats is one. Accordingly, the step of determining the data format by the computer device may be: in response to the first sharing instruction, displaying a first format selection interface, the first format selection interface includes multiple data formats; determining the selected data format from the multiple data formats; for example, the first format selection interface includes a web page format, a text format, a lightweight language format, a json format, and a mind map format; if the user selects a web page format, the computer determines that the converted data format is a web page format.
[0052] In another possible implementation, when the user wants to transfer the first operation trajectory model through an application, the user triggers the application to the computer device, and the computer device determines the data format supported by the triggered application based on the triggered application; in this case, the number of data formats is one. Accordingly, the step of determining the data format by the computer device may be: in response to the second sharing instruction, displaying a second format selection interface, the second format selection interface including application identifiers of multiple applications; determining the application identifier of the selected application from the multiple application identifiers; and determining the data format supported by the application. In the case where the application supports multiple data formats, the computer device can determine the multiple data formats supported by the application, and subsequently convert the first operation trajectory model into a second operation trajectory model in multiple data formats; or, in the case where the application supports multiple data formats, the computer device selects one data format from the multiple data formats supported by the application; wherein the step of the computer device selecting a data format from the multiple data formats supported by the application may be: the computer device selects the most frequently used data format from the multiple data formats supported by the application; or, the computer device selects the data format that has been used historically from the multiple data formats supported by the application; or, the computer device displays the multiple data formats supported by the application, and the user selects a data format from the multiple data formats supported by the application, and the computer device determines the selected data format from the multiple data formats supported by the application.
[0053] Step 203: Convert the first operation trajectory model into a second operation trajectory model in at least one data format. The second operation trajectory model in any data format is used to be shared with its corresponding application for processing.
[0054] The at least one data format includes at least one of a web page format, a text format, a lightweight markup language format, a mind map format, and a json format. After the first operation trajectory model is converted into a second operation trajectory model in a certain data format, the second operation trajectory model is shared with an application corresponding to the data format for processing, so as to realize the circulation of the second operation trajectory model in a third-party application.
[0055] In the embodiment of the present application, the first operation trajectory model is converted into a second operation trajectory model in at least one data format by format conversion; since the first operation trajectory model includes node information of the corresponding node of the interface during the operation and edge information of the corresponding behavior edge of the operation behavior in the interface; thus, the embodiment of the present application realizes the conversion of the format of information, rather than the conversion of special data such as code; therefore, the format conversion of the embodiment of the present application does not need to rely on a special computer environment, and can be realized directly through the application of the computer device, solving the problem that the conversion of screen content into other file formats requires dependence on a specific environment. Moreover, since at least one data format is a data format supported by at least one application; therefore, the second operation trajectory model can be shared with at least one application for processing, thereby realizing the circulation of the operation trajectory model.
[0056] Please refer to Figure 3 , which shows a flow chart of a method for processing interface data provided by an exemplary embodiment of the present application. The method may include the following steps:
[0057] Step 301: The computer device determines a first operation trajectory model, which includes node information of nodes corresponding to the interface during the operation and edge information of behavior edges corresponding to the operation behaviors in the interface. The node information includes information that characterizes interface features from different dimensions, and the edge information includes information used to characterize the operation behavior and the node jump situation before and after the operation behavior.
[0058] In some embodiments, this step is the same as step 201 and will not be repeated here.
[0059] Step 302: The computer device determines at least one data format, where the at least one data format is a data format supported by at least one application.
[0060] In some embodiments, this step is the same as step 202 and will not be repeated here.
[0061] Step 303: The computer device compresses the first operation trajectory model to obtain a third operation trajectory model.
[0062] The data volume of the third operation trajectory model is smaller than that of the first operation trajectory model, but the third operation trajectory model follows minimum integrity, that is, the third operation trajectory model only applies basic data to the interface involved in the corresponding operation process.
[0063] In a possible implementation, the computer device deletes relevant information of some interfaces from the operation process to achieve compression of the first operation trajectory model; accordingly, this step may be: the computer device determines a second nested relationship between multiple interfaces involved in the operation process based on behavior edge information corresponding to the operation behavior in the interface in the first operation trajectory model; based on the second nested relationship, deletes node information of nodes corresponding to some interfaces and edge information of behavior edges corresponding to the operation behavior in some interfaces from the first operation trajectory model to obtain a third operation trajectory model.
[0064] Optionally, the computer device deletes the node information of the nodes corresponding to some interfaces and the edge information of the behavior edges corresponding to some operation behaviors in the interface from the first operation trajectory model based on the second nested relationship, and the step of obtaining the third operation trajectory model can be: the computer device deletes the node information of the nodes corresponding to the interface located in the middle layer and the edge information of the behavior edges corresponding to the operation behaviors in the interface in the middle layer from the first operation trajectory model based on the second nested relationship, and obtains the third operation trajectory model. In an embodiment of the present application, since the interface operated by the user jumps from the first interface to the second interface and then jumps to the third interface, the user may pay more attention to the first interface and the third interface, and the second interface is equivalent to a springboard; therefore, the node information of the nodes corresponding to the second interface and the edge information of the behavior edges corresponding to the operation behaviors in the second interface are deleted in the first operation trajectory model to reduce the amount of data in the operation trajectory model. In addition, since the node information corresponding to some nodes is deleted; therefore, when performing data query based on the third operation trajectory model, the data query efficiency can be improved.
[0065] In another possible implementation, the computer device extracts relevant information of the key interface from the first operation trajectory model to achieve compression of the first operation trajectory model; accordingly, this step may be: the computer device extracts the key nodes from the first operation model based on the node information of the interface corresponding nodes in the first operation trajectory model, and determines the key behavior edges from the first operation data type based on the key nodes, where the key behavior edges are edges corresponding to the operation behaviors in the interface corresponding to the key nodes; and generates a third operation trajectory model using the node information of the key nodes and the edge information of the key behavior edges.
[0066] Optionally, the step of extracting key nodes from the first operation model based on the node information of the nodes corresponding to the interfaces in the first operation trajectory model may be as follows: the computer device determines, from each node, key nodes whose information amount exceeds a preset information amount based on the information amount of the nodes corresponding to each interface in the first operation trajectory model; or, the computer device determines, from each node, key nodes whose browsing time exceeds a preset time based on the browsing time corresponding to each interface in the first operation trajectory model.
[0067] In another possible implementation, the computer device deletes part of the information of the interface from the first operation trajectory model to achieve compression of the first operation trajectory model; accordingly, this step may be: for the node information of each node corresponding to the interface in the first operation trajectory model, the computer device deletes the node information whose importance is less than a preset importance from the node information to obtain the second operation trajectory model.
[0068] In another possible implementation, the computer device includes a tool model, and the tool model is used to compress the first operation trajectory model into a third operation trajectory model; accordingly, please refer to Figure 4 The computer device includes a data model, the data model includes a first operation trajectory model (outs), and outs includes edge information (action) of the behavior edge corresponding to the operation behavior in the interface, or is called outs depends on action; the tool model includes out-compress, and out-compress is used to compress the first operation trajectory model into a third operation trajectory model.
[0069] In an embodiment of the present application, by extracting and compressing the first operation trajectory model, page hierarchical nesting is reduced, but key information of the node is retained; therefore, when data query is performed based on the compressed third operation trajectory model, data query efficiency can be improved; or when data query is performed based on the second operation trajectory model converted from the compressed third operation trajectory model into another format, data query efficiency can be improved; and, since the information included in the compressed third operation trajectory model is more simplified, the readability of the data is improved.
[0070] Step 304: The computer device converts the third operation trajectory model into a second operation trajectory model in at least one data format, and the second operation trajectory model in any data format is used to be shared with its corresponding application for processing.
[0071] The at least one data format includes at least one of a web page format, a text format, a lightweight markup language format, a mind map format, and a json format. For example, please continue to refer to Figure 4 The computer device includes a circulation model, and the circulation model includes a third operation track model in a web page format (outs-html), a third operation track model in a text format (outs-txt), and a third operation track model in a json format (outs-json).
[0072] The computer device includes a data conversion layer, and the data conversion layer is used to convert the format of the third operation trajectory model; for example, please refer to Figure 5The computer device converts the third operation trajectory model into a second operation trajectory model in a web page format, a second operation trajectory model in a text format, a second operation trajectory model in a json format, a second operation trajectory model in a lightweight markup language format, and a second operation trajectory model in a mind map format through a data conversion layer; and the computer device can also export the converted second operation trajectory model.
[0073] In a possible implementation, at least one data format includes a web page format; in this step, the computer device may convert the third operation trajectory model into a second operation trajectory model in a web page format; optionally, the step of converting the third operation trajectory model into a second operation trajectory model in a web page format may be: inputting node information and edge information in the third operation trajectory model into a web page editor, outputting the second operation trajectory model in a web page format, and the web page editor is used to convert data into data in a web page format. In an embodiment of the present application, the first operation trajectory model is converted into a second operation trajectory model in a web page format, so that the second operation trajectory model can be transferred to a browser for opening.
[0074] At least one data format includes a text format; in this step, the computer device may convert the third operation trajectory model into a second operation trajectory model in a text format; optionally, the step of converting the third operation trajectory model into a second operation trajectory model in a text format may be: extracting key interface information from node information of the interface corresponding node in the third operation trajectory model, and generating a second operation trajectory model in a text format from the key interface information. In an embodiment of the present application, the first operation trajectory model is converted into a second operation trajectory model in a text format, so that the second operation trajectory model can be transferred to a text editor for editing and circulation.
[0075] At least one data format includes a lightweight markup language format; in this step, it can be: the computer device converts the third operation trajectory model into a second operation trajectory model in a lightweight markup language format; optionally, the step of converting the third operation trajectory model into a second operation trajectory model in a lightweight markup language format can be: extracting the image and the button information of the operation button from the node information of each interface corresponding node in the third operation trajectory model, storing the image in a preset directory, and generating a second operation trajectory model in a lightweight markup language format from the path information of the preset directory and the button information of the operation button. In an embodiment of the present application, the first operation trajectory model is converted into a second operation trajectory model in a lightweight markup language format, so that the second operation trajectory model can be transferred to notion, Youdao Cloud, and typora for editing and circulation.
[0076] For example, the second operation trajectory model in the lightweight markup language format is as follows:
[0077] ###Page Title
[0078] >Long text content...
[0079] [Picture]( / sdcard / Download / ……)
[0080] **Button 1**
[0081] **Button 2**
[0082] At least one data format includes a mind map format; in this step, it can be: the computer device converts the third operation trajectory model into a second operation trajectory model in a mind map format; optionally, the step of converting the third operation trajectory model into a second operation trajectory model in a mind map format can be: based on the behavior edge information corresponding to the operation behavior in the interface in the third operation trajectory model, determine the first nested relationship between multiple interfaces involved in the operation process; based on the first nested relationship between multiple interfaces, generate a second operation trajectory model in a mind map format. In an embodiment of the present application, the first operation trajectory model is converted into a second operation trajectory model in a mind map format, so that the second operation trajectory model can be edited and circulated in a picture display. Among them, the second operation trajectory model in the mind map format is as follows: Figure 6 shown.
[0083] In some embodiments, the above data format is only an example, and does not specifically limit the data format of the embodiment of the present application. Other types can also be extended; for example, please refer to Figure 7 The computer device includes a data model layer, a tool model layer and a data conversion layer. The data model layer includes a first operation trajectory model. The first operation trajectory model may also have other auxiliary capabilities. For example, the first operation trajectory model can generate a node identifier including a node, and the node identifier is used to uniquely identify and locate the node. The first operation trajectory model may also perform model expansion. For example, in addition to node information and edge information, the first operation trajectory model may also include path information corresponding to the operation path of the operation process, and the path information is used to characterize the timing between operation behaviors. Of course, the first operation trajectory model may also include other functions. The tool model layer includes outs-compress, and outs-compress is used to compress the first operation trajectory model to obtain a third operation trajectory model; the data conversion layer is used to convert the third operation trajectory model into a second operation trajectory model in a web page format, convert the third operation trajectory model into a second operation trajectory model in a text format, convert the third operation trajectory model into a second operation trajectory model in a mind map format, convert the third operation trajectory model into a second operation trajectory model in a lightweight markup language format, and convert the third operation trajectory model into a second operation trajectory model in a json format.
[0084] In some embodiments, the first operation trajectory model also supports conversion to a first operation data model, and then the first operation data model is converted into other data formats for ecological circulation. Accordingly, the step of the computer device converting the first operation trajectory model into at least one second operation trajectory model in a data format may be: the computer device stores the node information and the edge information in the information type corresponding to the data layer of the first operation data model based on the information type of the node information and the information type of the edge information included in the first operation trajectory model, the first operation data model includes a data layer and a conversion layer, the conversion layer is used to convert the data of the data layer into data of other multiple data types, and the other multiple data types include the target data type; through the conversion layer in the first operation data model, the node information and the edge information stored in the data layer are converted into node information and edge information of at least one data format; the node information and edge information of at least one data format are used to generate a second operation trajectory model of at least one data format.
[0085] For example, see Figure 8 The computer device includes a data model layer, a transfer layer and an export layer; the data model layer includes a first operation trajectory model (OUTS), the transfer layer is used to convert the first operation trajectory model into a first operation data model (OUDS), and the export layer is used to export the first operation data model into a second operation trajectory model in at least one data format; at least one data format can be excel or other formats.
[0086] In some embodiments, the computer device may further compress the first operation trajectory model to obtain a third operation trajectory model, then convert the third operation trajectory model into a second operation data model, and then convert the second operation trajectory model based on the second operation data model to obtain at least one second operation trajectory model in a data format. The process in which the computer device converts the third operation trajectory model into the second operation data model, and then converts the second operation trajectory model based on the second operation data model to obtain at least one second operation trajectory model in a data format is similar to the process in which the computer device converts the third operation trajectory model into the first operation data model, and then converts the second operation trajectory model based on the first operation data model to obtain at least one second operation trajectory model in a data format, and will not be repeated here.
[0087] In the embodiment of the present application, the computer device converts the first operation trajectory model into a second operation trajectory model in another format, which is convenient for the user to perform page data analysis and other operations on the second operation trajectory model based on different data formats, and also convenient for the user to export the second operation trajectory model in a desired format. In addition, the second operation trajectory model can be circulated and displayed in a third-party application.
[0088] Please refer to Fig. 9, which shows a structural block diagram of an interface data processing device provided by an exemplary embodiment of the present application. The device includes:
[0089] A first determination module 901 is used to determine a first operation trajectory model, wherein the first operation trajectory model includes node information of nodes corresponding to the interface during the operation and edge information of behavior edges corresponding to the operation behavior in the interface, wherein the node information includes information characterizing interface features of the interface from different dimensions, and the edge information includes information characterizing the operation behavior and node jump conditions before and after the operation behavior;
[0090] A first determining module 902, configured to determine at least one data format, wherein the at least one data format is a data format supported by at least one application program;
[0091] The conversion module 903 is used to convert the first operation trajectory model into a second operation trajectory model in the at least one data format, and the second operation trajectory model in any data format is used to be shared with its corresponding application for processing.
[0092] Optionally, the conversion module 903 is configured to compress the first operation trajectory model to obtain a third operation trajectory model; and convert the third operation trajectory model into a second operation trajectory model in the at least one data format.
[0093] Optionally, the at least one data format includes a web page format; the conversion module 903 is used to convert the third operation trajectory model into a second operation trajectory model in a web page format;
[0094] The at least one data format includes a text format; the conversion module 903 is used to convert the third operation trajectory model into a second operation trajectory model in a text format;
[0095] The at least one data format includes a lightweight markup language format; the conversion module 903 is used to convert the third operation trajectory model into a second operation trajectory model in a lightweight markup language format;
[0096] The at least one data format includes a mind map format; the conversion module 903 is used to convert the third operation trajectory model into a second operation trajectory model in a mind map format.
[0097] Optionally, the conversion module 903 is used to input the node information and edge information in the third operation trajectory model into a webpage editor, and output the second operation trajectory model in the webpage format, wherein the webpage editor is used to convert data into data in the webpage format.
[0098] Optionally, the conversion module 903 is used to extract key interface information from node information of the interface corresponding node in the third operation trajectory model, and generate a second operation trajectory model in text format with the key interface information.
[0099] Optionally, the conversion module 903 is used to extract pictures and button information of operation buttons from the node information of each interface corresponding node in the third operation trajectory model, store the pictures in a preset directory, and generate a second operation trajectory model in a lightweight markup language format using the path information of the preset directory and the button information of the operation button.
[0100] Optionally, the conversion module 903 is used to determine the first nested relationship between multiple interfaces involved in the operation process based on the behavior edge information corresponding to the operation behavior in the interface in the third operation trajectory model; and generate a second operation trajectory model in a mind map format based on the first nested relationship between the multiple interfaces.
[0101] Optionally, the conversion module 903 is used to determine a second nested relationship between multiple interfaces involved in the operation process based on the behavior edge information corresponding to the operation behavior in the interface in the first operation trajectory model; based on the second nested relationship, delete the node information of the nodes corresponding to some interfaces and the edge information of the behavior edges corresponding to the operation behavior in some interfaces from the first operation trajectory model to obtain the third operation trajectory model.
[0102] Optionally, the conversion module 903 is used to extract key nodes from the first operation model based on the node information of the interface corresponding to the node in the first operation trajectory model, and determine key behavior edges from the first operation data type based on the key nodes, wherein the key behavior edges are edges corresponding to the operation behaviors in the interface corresponding to the key nodes; and generate the third operation trajectory model by combining the node information of the key nodes and the edge information of the key behavior edges.
[0103] Optionally, the first determining module 902 is configured to display a first format selection interface in response to the first sharing instruction, wherein the first format selection interface includes a plurality of data formats; and determine at least one data format selected from the plurality of data formats;
[0104] The device also includes:
[0105] The first sharing module is used to share the second operation trajectory model in any data format to the corresponding application program for processing.
[0106] Optionally, the first determining module 902 is configured to display a second format selection interface in response to the second sharing instruction, wherein the second format selection interface includes application identifiers of multiple applications; and determine an application identifier of at least one application selected from the application identifiers of the multiple applications;
[0107] The device also includes:
[0108] The second sharing module is used to share the second operation trajectory model of any data format to its corresponding application program for processing.
[0109] Optionally, the conversion module 903 is used to store the node information and edge information in the information type corresponding to the data layer of the operation data model based on the information type of the node information and the information type of the edge information included in the first operation trajectory model, the operation data model includes a data layer and a conversion layer, the conversion layer is used to convert the data in the data layer into data of other multiple data types, the other multiple data types include the target data type; through the conversion layer in the operation data model, the node information and edge information stored in the data layer are converted into node information and edge information in the at least one data format; the node information and edge information in the at least one data format are used to generate a second operation trajectory model in the at least one data format.
[0110] In the embodiment of the present application, the first operation trajectory model is converted into a second operation trajectory model in at least one data format by format conversion; since the first operation trajectory model includes node information of the corresponding node of the interface during the operation and edge information of the corresponding behavior edge of the operation behavior in the interface; thus, the embodiment of the present application realizes the conversion of the format of information, rather than the conversion of special data such as code; therefore, the format conversion of the embodiment of the present application does not need to rely on a special computer environment, and can be realized directly through the application of the computer device, solving the problem that the conversion of screen content into other file formats requires dependence on a specific environment. Moreover, since at least one data format is a data format supported by at least one application; therefore, the second operation trajectory model can be shared with at least one application for processing, thereby realizing the circulation of the operation trajectory model.
[0111] It should be noted that the interface data processing device provided in the above embodiment only uses the division of the above functional modules as an example when processing interface data. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the computer device is divided into different functional modules to complete all or part of the functions described above. In addition, the interface data processing device provided in the above embodiment and the interface data processing method embodiment belong to the same concept, and the specific implementation process is detailed in the method embodiment, which will not be repeated here.
[0112] See also Fig.10 , Fig.10 10 is a schematic diagram of a computer device provided by an exemplary embodiment of the present application. The computer device may also include one or more of the following components: a processor 1010 , a memory 1020 , and a display screen 1030 .
[0113] The processor 1010 uses various interfaces and lines to connect various parts of the entire computer device 1000, and executes various functions and processes data of the computer device 1000 by running or executing instructions, programs, code sets or instruction sets stored in the memory 1020, and calling data stored in the memory 1020. Optionally, the processor 1010 can be implemented in at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), and programmable logic array (PLA). The processor 1010 can integrate one or a combination of a central processing unit (CPU), a graphics processing unit (GPU), a neural network processor (NPU), and a modem. Among them, the CPU mainly processes the operating system, user interface, and application programs; the GPU is responsible for rendering and drawing the content to be displayed on the display screen 1030; the NPU is used to implement artificial intelligence (AI) functions; and the modem is used to process wireless communications. It is understandable that the above-mentioned modem may not be integrated into the processor 1010, but may be implemented separately through a computer program product.
[0114] The memory 1020 may include a random access memory (RAM) or a read-only memory (ROM). Optionally, the memory 1020 includes a non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, codes, code sets, or instruction sets. The memory 1020 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc.; the data storage area may store data (such as audio data, a phone book), etc., created according to the use of the computer device 1000.
[0115] The display screen 1030 is a display component for displaying a user interface. Optionally, the display screen 1030 is a display screen with a touch function, through which the user can use any suitable object such as a finger, a touch pen, etc. to perform touch operations on the display screen 1030.
[0116] The display screen 1030 is usually arranged on the front panel of the computer device 1000. The display screen 1030 can be designed as a full screen, a curved screen, a special-shaped screen, a double-sided screen or a folding screen. The display screen 1030 can also be designed as a combination of a full screen and a curved screen, a combination of a special-shaped screen and a curved screen, etc., which is not limited in this embodiment.
[0117] In addition, those skilled in the art can understand that the structure of the computer device 1000 shown in the above figures does not constitute a limitation on the computer device 1000, and the computer device 1000 may include more or fewer components than shown, or combine certain components, or arrange the components differently. For example, the computer device 1000 also includes a Wifi module, an audio acquisition device, a speaker, a radio frequency circuit, an input unit, a sensor, an audio circuit, a Bluetooth module, a power supply, and other components, which will not be described in detail here.
[0118] An embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores at least one computer instruction, and the at least one computer instruction is used to be executed by a processor to implement the method for processing interface data as described in the above embodiment.
[0119] On the other hand, an embodiment of the present application provides a computer program product, which includes computer instructions, and the computer instructions are stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the interface data processing method as described in the above embodiment.
[0120] Those skilled in the art should be aware that in one or more of the above examples, the functions described in the embodiments of the present application can be implemented with hardware, software, firmware, or any combination thereof. When implemented using software, these functions can be stored in a computer-readable medium or transmitted as one or more instructions or codes on a computer-readable medium. Computer-readable media include computer storage media and communication media, wherein the communication media include any media that facilitates the transmission of a computer program from one place to another. The storage medium can be any available medium that a general or special-purpose computer can access.
[0121] The above description is only an optional embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.
Claims
1. A method for processing interface data, characterized in that: The method comprises: Determine a first operation trajectory model, wherein the first operation trajectory model includes node information of a node corresponding to an interface during an operation and edge information of a behavior edge corresponding to an operation behavior in the interface, wherein the node information includes information characterizing interface features of the interface from different dimensions, and the edge information includes information for characterizing the operation behavior and node jump conditions before and after the operation behavior; Determining at least one data format, wherein the at least one data format is a data format supported by at least one application; The first operation trajectory models are converted into second operation trajectory models in the at least one data format respectively, and the second operation trajectory model in any data format is used to be shared with its corresponding application for processing.
2. The method according to claim 1, characterized in that The converting the first operation trajectory model into a second operation trajectory model in the at least one data format comprises: compressing the first operation trajectory model to obtain a third operation trajectory model; The third operation trajectory models are converted into second operation trajectory models in the at least one data format respectively.
3. The method according to claim 2, characterized in that The converting the third operation trajectory model into a second operation trajectory model in the at least one data format comprises: The at least one data format includes a web page format; converting the third operation trajectory model into a second operation trajectory model in a web page format; The at least one data format includes a text format; converting the third operation trajectory model into a second operation trajectory model in a text format; The at least one data format includes a lightweight markup language format; converting the third operation trajectory model into a second operation trajectory model in a lightweight markup language format; The at least one data format includes a mind map format; and the third operation trajectory model is converted into a second operation trajectory model in a mind map format.
4. The method according to claim 3, characterized in that The converting the third operation trajectory model into a second operation trajectory model in a webpage format includes: The node information and edge information in the third operation trajectory model are input into a webpage editor, and the second operation trajectory model in the webpage format is output. The webpage editor is used to convert data into data in the webpage format.
5. The method according to claim 3, characterized in that: The converting the third operation trajectory model into a second operation trajectory model in a text format includes: Extract key interface information from the node information of the interface corresponding node in the third operation trajectory model, and generate a second operation trajectory model in text format with the key interface information.
6. The method according to claim 3, characterized in that The converting the third operation track model into a second operation track model in a lightweight markup language format includes: Extracting pictures and button information of operation buttons from node information of each interface corresponding node in the third operation trajectory model, storing the pictures in a preset directory, and generating a second operation trajectory model in a lightweight markup language format using the path information of the preset directory and the button information of the operation buttons.
7. The method according to claim 3, characterized in that The converting the third operation trajectory model into a second operation trajectory model in a mind map format includes: Determining a first nested relationship between multiple interfaces involved in the operation process based on the behavior edge information corresponding to the operation behavior in the interface in the third operation trajectory model; Based on the first nested relationship between the multiple interfaces, a second operation trajectory model in a mind map format is generated.
8. The method according to claim 2, characterized in that: The compressing the first operation trajectory model to obtain a third operation trajectory model includes: Determining a second nested relationship between multiple interfaces involved in the operation process based on the behavior edge information corresponding to the operation behavior in the interface in the first operation trajectory model; Based on the second nested relationship, the node information of the nodes corresponding to the partial interface and the edge information of the behavior edges corresponding to the operation behaviors in the partial interface are deleted from the first operation trajectory model to obtain the third operation trajectory model.
9. The method according to claim 2, characterized in that: The compressing the first operation trajectory model to obtain a third operation trajectory model includes: Extract key nodes from the first operation model based on node information of the interface corresponding to the first operation trajectory model, and determine key behavior edges from the first operation data type based on the key nodes, where the key behavior edges are edges corresponding to the operation behaviors in the interface corresponding to the key nodes; The node information of the key nodes and the edge information of the key behavior edges are used to generate the third operation trajectory model.
10. The method according to claim 1, characterized in that The determining of at least one data format comprises: In response to the first sharing instruction, displaying a first format selection interface, wherein the first format selection interface includes a plurality of data formats; determining at least one data format selected from the plurality of data formats; After converting the first operation trajectory model into a second operation trajectory model in at least one data format, the method further includes: For any data format, the second operation trajectory model in any data format is shared to its corresponding application for processing.
11. The method according to claim 1, characterized in that: The determining of at least one data format comprises: In response to the second sharing instruction, displaying a second format selection interface, wherein the second format selection interface includes application identifiers of a plurality of application programs; Determining an application identifier of at least one application selected from the application identifiers of the multiple applications; determining at least one data format supported by the at least one application; After converting the first operation trajectory model into a second operation trajectory model in at least one data format, the method further includes: For any data format, the second operation trajectory model in any data format is shared to its corresponding application for processing.
12. The method according to claim 1, characterized in that The converting the first operation trajectory model into a second operation trajectory model in the at least one data format comprises: Based on the information type of the node information and the information type of the edge information included in the first operation trajectory model, the node information and the edge information are stored in the information type corresponding to the data layer of the operation data model, the operation data model includes a data layer and a conversion layer, the conversion layer is used to convert the data of the data layer into data of other multiple data types, and the other multiple data types include the target data type; Converting the node information and edge information stored in the data layer into the node information and edge information in the at least one data format through a conversion layer in the operation data model; The node information and edge information in the at least one data format are used to generate a second operation trajectory model in the at least one data format.
13. A device for processing interface data, characterized in that: The device comprises: A first determination module is used to determine a first operation trajectory model, wherein the first operation trajectory model includes node information of nodes corresponding to the interface during the operation and edge information of behavior edges corresponding to the operation behavior in the interface, wherein the node information includes information characterizing interface features of the interface from different dimensions, and the edge information includes information used to characterize the operation behavior and node jump conditions before and after the operation behavior; A second determination module, configured to determine at least one data format, wherein the at least one data format is a data format supported by at least one application program; The conversion module is used to convert the first operation trajectory model into a second operation trajectory model in the at least one data format, and the second operation trajectory model in any data format is used to be shared with its corresponding application for processing.
14. A computer device, characterized in that: The computer device includes a processor and a memory, wherein the memory stores at least one computer instruction, and the at least one computer instruction is loaded and executed by the processor to implement the method for processing interface data as described in any one of claims 1 to 12.
15. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer instruction, and the at least one computer instruction is used to be executed by a processor to implement the method for processing interface data according to any one of claims 1 to 12.
16. A computer program product, characterized in that The computer program product includes computer instructions, which are stored in a computer-readable storage medium; a processor reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions to implement the method for processing interface data as described in any one of claims 1 to 12.