Page information acquisition method, system and equipment based on barrier-free service and medium

Abstract

The invention belongs to the technical field of artificial intelligence, and relates to a method, a system and equipment for obtaining page information based on a barrier-free service and a medium, and the method comprises the following steps: 1) obtaining the barrier-free service node information Accession Service NodeInfo of a page based on an Accession Service; (2), a node Node in the accessible service node information Accession ServiceNodeInfo is analyzed, and an analysis result is converted into a variable character sequence StringBuilder (StringBuilder), and the StringBuilder (StringBuilder) is converted into the accessible service node information Accession ServiceNodeInfo, and the accessible service node information Accession ServiceNodeInfo; (3) adjusting the structure of the variable character sequence StringBuilder, and carrying out parallel analysis on a child node ChildNode of a node Node corresponding to the variable character sequence StringBuilder; 4) obtaining an analysis result of each child node ChildNod, splicing the analysis result into a variable character sequence StringBuilder, and adjusting the structure of the spliced variable character sequence StringBuilder again; and 5) obtaining a final XML (Extensible Markup Language) of the page on the basis of the variable character sequence StringBuilder which is adjusted again. According to the method and the device, the page information acquisition efficiency is improved, and the integrity of the acquired page information is also ensured.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and relates to a method, system, device and medium for obtaining page information, and in particular to a method, system, device and medium for obtaining page information based on barrier-free services. Background Art

[0002] With the rapid development of artificial intelligence, mobile smart assistants have also become the main field and battlefield for AI capabilities output. Mobile smart assistants can complete some complex and tedious tasks according to the instructions input by users, bringing more convenience to users.

[0003] The execution of the smart assistant task is based on the current interface of the mobile phone. It is essential to obtain the page information of the mobile phone (including the XML and / or screenshots of the page) for model judgment and analysis, and then output instructions. Therefore, the speed of obtaining page information is particularly important.

[0004] The conventional method of obtaining page information is to obtain the accessibility service node information AccessibilityServiceNodeInfo of the mobile page through the mobile accessibility service AccessibilityService, and then obtain the XML and screenshot information of the page based on the current node. The speed of obtaining page information directly determines the speed of task execution and has become one of the most concerned issues.

[0005] AccessibilityServiceNodeInfo is a node tree NodeTree composed of page information, which often contains a lot of complex information. Nodes have strict order and logic. Although serial parsing can ensure the correct order, it is relatively slow. Although the parallel parsing solution is more efficient, it often destroys the node tree NodeTree structure of AccessibilityServiceNodeInfo itself, which will disrupt the page information and affect the model judgment.

[0006] There are two main methods for parsing page information:

[0007] 1. Recursively parse page information. Recursive algorithm is a programming technique of calling the function itself. This algorithm has a very good effect on traversing maps and trees. Recursive parsing uses recursive algorithms to parse XML, and is also the most commonly used parsing algorithm for AccessibilityServiceNodeInfo. Recursive parsing can well ensure the sequential relationship between parent nodes, child nodes, and sibling nodes, which greatly guarantees the integrity of page information. However, recursive parsing is based on a serial parsing algorithm, so it is time-consuming and slows down significantly when dealing with complex tree structures.

[0008] 2. Recursively parse page information. Recursive algorithm is an algorithm relative to recursion, and usually uses loop structure to solve problems instead of function self-call. Recursive algorithm is often more efficient because it avoids the function overhead and stack overflow risk of recursive call. Recursive parsing uses recursive algorithm to parse XML. Compared with recursive parsing, it has an additional List to save node Node. It traverses the List through a while loop, obtains the node Node information, and also continues to store the node Node’s ChileNode into the List, parsing in a storage-while-retrieval manner. Although recursive parsing is more efficient than recursive parsing, it is essentially a serial parsing method. It does not solve the performance and time-consuming problems inherent in recursive parsing, and is equally time-consuming for complex tree structures.

[0009] Therefore, in view of the defects existing in the above-mentioned prior art, it is necessary to develop a new method for obtaining page information. Summary of the invention

[0010] In order to overcome the defects of the prior art, the present invention proposes a method, system, device and medium for obtaining page information based on accessibility services, which improves the efficiency of obtaining page information while ensuring the integrity of the obtained page information.

[0011] In order to achieve the above object, the present invention provides the following technical solutions:

[0012] A method for obtaining page information based on accessibility service, characterized by comprising the following steps:

[0013] 1) Get the accessibility service node information AccessibilityServiceNodeInfo of the page based on the accessibility service AccessibilityService;

[0014] 2) Parse the node Node in the accessibility service node information AccessibilityServiceNodeInfo, and convert the parsing result into a variable character sequence StringBuilder;

[0015] 3) Adjust the structure of the variable character sequence StringBuilder, and parse the child nodes ChildNod of the corresponding node Node in parallel;

[0016] 4) Obtain the parsing results of each child node ChildNod and assemble them into a variable character sequence StringBuilder, and adjust the structure of the assembled variable character sequence StringBuilder again;

[0017] 5) Obtain the final XML of the page based on the mutable character sequence StringBuilder that has been adjusted again.

[0018] Preferably, the parsing of the node Node in the accessibility service node information AccessibilityServiceNodeInfo in step 2) is performed in a RecursiveTask and the step 2) specifically includes:

[0019] 21) Create an XmlSerializer object, and set the StringWriter object and the startDocument property and the startTag property;

[0020] 22) Get the key fields of the node Node in the accessibility service node information AccessibilityServiceNodeInfo;

[0021] 23) Set the end tag endTag property and the end document endDocument property of the XmlSerialize object, insert the key field of the obtained node Node into the XmlSerializer object, and use the StringWriter object to convert it into a mutable character sequence StringBuilder.

[0022] Preferably, the key fields of the acquired node Node include className, contentDescription, text, viewIdResourceName, isClickable, isEnabled, isChecked, isFocused, isFocusable, isScrollable, isSelected, isLongClickable and boundsInScreen.

[0023] Preferably, the step 3) specifically includes:

[0024] 31) Remove the version, encoding and standalone information of the XmlSerializer object in the mutable character sequence StringBuilder;

[0025] 32) If the node Node corresponding to the variable character sequence StringBuilder contains a child node ChildNode, remove the end tag endTag of the Node node in the variable character sequence StringBuilder;

[0026] 33) Acquire the key fields of each child node ChildNod in parallel and set a depth parameter for each child node ChildNod to record the depth of each child node ChildNod, and save the key fields and depth parameter of each child node ChildNod acquired into RecursiveTaskList.

[0027] Preferably, the step 4) specifically includes:

[0028] 41) Traverse RecursiveTaskList and obtain the key fields and depth parameters of each child node ChildNod;

[0029] 42) splicing the key fields of each child node ChildNod obtained into a variable character sequence StringBuilder, and setting the number of indentation spaces according to the depth parameter to complete the format indentation;

[0030] 43) After the key fields of all child nodes ChildNod are spliced ​​into the variable character sequence StringBuilder, the end tag endTag of the variable character sequence StringBuilder is completed.

[0031] Preferably, in step 5), ForkJoinPool.invoke() is used to obtain the final XML of the page based on the variable character sequence StringBuilder that has been adjusted again.

[0032] Preferably, in the step 1), a screenshot Screenshot of the page is obtained based on the accessibility service AccessibilityService; and the step further includes the following steps:

[0033] 6) The screenshot Screenshot is compressed in quality without changing the original resolution of the screenshot Screenshot to obtain a final screenshot of the page.

[0034] In addition, the present invention also provides a page information acquisition system based on accessibility service, which is characterized by comprising:

[0035] An accessibility service node information AccessibilityServiceNodeInfo acquisition module is used to obtain the accessibility service node information AccessibilityServiceNodeInfo of a page based on the accessibility service AccessibilityService;

[0036] The node parsing module is used to parse the node Node in the accessibility service node information AccessibilityServiceNodeInfo and convert the parsing result into a variable character sequence StringBuilder;

[0037] The adjustment and child node parsing module is used to adjust the structure of the variable character sequence StringBuilder and perform parallel parsing of the child nodes ChildNod of the corresponding node Node;

[0038] The assembly module is used to obtain the parsing results of each child node ChildNod and assemble them into a variable character sequence StringBuilder, and adjust the structure of the assembled variable character sequence StringBuilder again;

[0039] The XML acquisition module is used to obtain the final XML of the page based on the variable character sequence StringBuilder that has been adjusted again.

[0040] Moreover, the present invention also provides a page information acquisition device based on accessibility service, characterized in that it includes:

[0041] one or more processors;

[0042] A memory for storing one or more programs;

[0043] When the one or more programs are executed by the one or more processors, the one or more processors implement the page information acquisition method based on accessibility service as described above.

[0044] Finally, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the method for obtaining page information based on accessibility services as described above are implemented.

[0045] Compared with the prior art, the method, system, device and medium for obtaining page information based on accessibility services of the present invention have one or more of the following beneficial technical effects:

[0046] 1. The present invention obtains child node ChildNode information based on accessibility service node information AccessibilityServiceNodeInfo in parallel parsing, obtains information of multiple child nodes ChildNode at the same time, and then sequentially combines the obtained child node ChildNode information to obtain complete XML content, which greatly improves the time.

[0047] 2. The present invention retains the information of the corresponding node Node at the end of parsing each node Node, and then reversely completes the node tree NodeTree to obtain a complete XML, thereby improving efficiency and ensuring the integrity of the page information.

[0048] 3. Through a large number of experiments, it is found that under the same mobile phone interface, the speed of obtaining page information by the present invention saves nearly half of the time compared with recursive parsing and recursive parsing, and the integrity of XML is also well guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] Figure 1 It is a flow chart of the method for obtaining page information based on barrier-free service of the present invention.

[0050] Figure 2 It is a schematic diagram of the structure of the page information acquisition system based on barrier-free service of the present invention. DETAILED DESCRIPTION

[0051] Before describing in detail any embodiment of the present invention, it should be understood that the present invention is not limited in its application to the construction and arrangement details of the components described below or illustrated in the following figures. The present invention can have other embodiments and can be practiced or carried out in various ways. In addition, it should be understood that the words and terms used here are for descriptive purposes and should not be considered restrictive. The use of "including" or "having" and its variations herein is intended to cover the items and their equivalents and additional items displayed below. Unless otherwise specified or limited, the terms "install", "connect", "support" and "couple" and their variations are widely used and cover direct installation and indirect installation, connection, support and connection. In addition, "connect" and "couple" are not limited to physical or mechanical connections or connections.

[0052] Furthermore, on the first aspect, in the disclosure of the present invention, the orientation or positional relationship indicated by terms such as "longitudinal", "transverse", "upper", "lower", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside" and "outside" are based on the orientation or positional relationship shown in the accompanying drawings, which are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation, and therefore the above terms cannot be understood as limitations on the present invention; on the second aspect, the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" cannot be understood as a limitation on the quantity.

[0053] Before introducing the specific content of the present invention, some technical terms involved in the present invention are briefly introduced to help those skilled in the art to better understand the present invention.

[0054] AccessibilityService: Android's accessibility service to help users with visual, hearing or other disabilities better use devices and applications.

[0055] AccessibilityServiceNodeInfo: Accessibility service node information, which is a key class in the AccessibilityService, and represents a node in the user interface (UI). Each AccessibilityServiceNodeInfo instance contains information related to the UI element, which can be read and operated by the AccessibilityService.

[0056] XmlSerializer: used to serialize objects into XML format output.

[0057] ForkJoinPool: A parallel computing framework introduced in Java7, which is specifically used for the efficient execution of recursive tasks. It is based on the CPU and executes a large number of small tasks in parallel.

[0058] Screenshot: A screenshot containing information about what is currently displayed on the device screen.

[0059] Figure 1 A flow chart of a method for acquiring page information based on barrier-free service of the present invention is shown.

[0060] like Figure 1 As shown, the page information acquisition method based on accessibility service of the present invention comprises the following steps:

[0061] 1. Get the accessibility service node information AccessibilityServiceNodeInfo of the page based on the accessibility service AccessibilityService.

[0062] Of course, when obtaining page information, it is sometimes necessary to obtain a screenshot of the page. Therefore, a screenshot of the page can also be obtained based on the Accessibility Service. The screenshot can be compressed without changing the original resolution of the screenshot to speed up the image transmission efficiency and reduce traffic consumption, so as to obtain the final screenshot of the page.

[0063] In the present invention, the Bitmap.compress() method can be used to compress the Bitmap image data after the screenshot, and the method signature is: public boolean compress(Bitmap.CompressFormat format, int quality, OutputStream stream). Among them, the parameter format is the format of the compressed image, and the Bitmap.CompressFormat enumeration is used to specify the format, such as JPEG, PNG and WEBP; the parameter quality is the quality of the compressed image, and the integer range is 0 to 100, where 0 represents the lowest quality (minimum compression) and 100 represents the highest quality (minimum compression). If the format is PNG, this parameter will be ignored because PNG is lossless compression; the parameter stream is the output stream to which the image compression data will be written.

[0064] 2. Parse the node Node in the accessibility service node information AccessibilityServiceNodeInfo, and convert the parsing result into a variable character sequence StringBuilder.

[0065] In the present invention, the accessibility service node information AccessibilityServiceNodeInfo is passed into RecursiveTask (a subclass of ForkJoinPool, specifically used for recursive tasks with return values) to perform node Node parsing. RecursiveTask parses the passed-in node Node to obtain key fields, and then converts it into StringBuilder. In addition, when parsing the node Node, it is parsed from the root node RootNode first.

[0066] The RecursiveTask code logic is:

[0067] 1. Parameter passing. The parameters passing include: Node, which is the Node object to be parsed; index, which is the order of the current node Node in the parent node FatherNode. When assembling the results, it is spliced ​​according to the index order to prevent the XML structure from being disordered; depth, which is the depth of the current node Node in the node tree NodeTree. When the format is indented, it is used to fill in the number of spaces.

[0068] 2. Method. The method is compute(), which is the core method of RecursiveTask. If the current task needs to be split, the task is split into more subtasks, and the fork() method is used to process these subtasks in parallel. After the parsing is completed, the join() method is called to get the result of the corresponding task. The node parsing logic is executed asynchronously in this method.

[0069] In the compute() method, the key fields of the current node Node are obtained first, and then the child node ChildNode of the current node Node is obtained. Then, according to the number of child nodes ChildNode, it is split into a fixed number of subtasks, and each child node ChildNode and the sequential index of the current node Node are passed to the new RecursiveTask, and fork() is called to execute the parsing task of each child node ChildNode, and the parsing result of the child node ChildNode is obtained through join().

[0070] In the present invention, the node Node in the accessibility service node information AccessibilityServiceNodeInfo is parsed, and the parsing result is converted into a variable character sequence StringBuilder, which specifically includes:

[0071] 1. Create an XmlSerializer object (the present invention uses the XmlSerializer object to ensure that the data complies with the XML specification), and set the StringWriter object and the startDocument attribute and the startTag attribute of the start document. The sample code for its specific settings is as follows:

[0072] valstringWriter=StringWriter()

[0073] valserializer=Xml.newSerializer()

[0074] serializer.setOutput(stringWriter)

[0075] serializer.startDocument("UTF-8",true)

[0076] serializer.startTag("","node").

[0077] 2. Get the key fields of the node Node in the accessibility service node information AccessibilityServiceNodeInfo.

[0078] The node itself carries some information fields, and the key fields of the node can be obtained directly through method calls. The following is an example of how the XmlSerializer object loads the key fields of the node:

[0079] serializer.attribute("","class",node.className.toString());

[0080] serializer.attribute("","text",node.text.toString()).

[0081] Among them, node.className can get the information of the className field of the node Node, and node.text can get the information of the text field of the node Node.

[0082] In the present invention, the key fields of the node Node include: className, contentDescription, text, viewIdResourceName, isClickable, isEnabled, isChecked, isFocused, isFocusable, isScrollable, isSelected, isLongClickable and boundsInScreen.

[0083] 3. Set the end tag endTag property and the end document endDocument property of the XmlSerialize object, insert the key field of the obtained node Node into the XmlSerializer object, and use the StringWriter object to convert it into a variable character sequence StringBuilder. The sample code for the specific settings is as follows:

[0084] serializer.endTag("","node")

[0085] serializer.endDocument()

[0086] valstringBuilder=StringBuilder()

[0087] stringBuilder.append(stringWriter.toString()).

[0088] Among them, stringWriter is the StringWriter object set in step 1.

[0089] Thus, an exemplary StringBuilder is obtained as follows:

[0090] <? xml version='1.0'encoding='UTF-8'standalone='yes'? > <node NAF="true" index="0"text=""resource-id=""

[0091] class="android.view.ViewGroup" package="com.zhipu.agent"content-desc="" checkable="false" checked="false" clickable="false" enabledd="true" focusable="false" focused="false" scrollable="false"long-clickable="false" password="false" selected="false"

[0092] bounds="[0,148][1224,2776]" / >.

[0093] 3. Adjust the structure of the variable character sequence StringBuilder, and perform parallel parsing of the child node ChildNod of the corresponding node Node, which specifically includes:

[0094] 1. Remove the version, encoding and standalone information of the XmlSerializer object in the variable character sequence StringBuilder, and only keep the version, encoding and standalone information of the outermost node Node to avoid duplication. The sample code is as follows:

[0095] StringBuilder.toString().replace("<?xml version='1.0'encoding='UTF-8'

[0096] standalone='yes'? >","").

[0097] 2. Determine whether the node Node corresponding to the mutable character sequence StringBuilder contains a child node ChildNode. If the node Node corresponding to the mutable character sequence StringBuilder contains a child node ChildNode, remove the end tag endTag of the node Node in the mutable character sequence StringBuilder, otherwise a structural error will occur when assembling the child node ChildNode. The sample code is as follows:

[0098] StringBuilder.toString().replace(" / >",">").

[0099] 3. Obtain the key fields of each child node ChildNod in parallel and set a depth parameter for each child node ChildNod to record the depth of each child node ChildNod. Save the key fields and depth parameters of each child node ChildNod obtained into RecursiveTaskList.

[0100] The key fields of each child node ChildNod are obtained in the same manner as the key fields of the node Node in step 2, and will not be described for simplicity. However, the key fields of multiple child nodes ChildNod of a node Node can be obtained in parallel to improve parsing efficiency.

[0101] Of course, in order to facilitate subsequent assembly, the index of each child node ChildNod also needs to be saved in RecursiveTaskList.

[0102] 4. Obtain the parsing results of each child node ChildNod and assemble them into a variable character sequence StringBuilder, and adjust the structure of the assembled variable character sequence StringBuilder again, which specifically includes:

[0103] 1. Traverse RecursiveTaskList and obtain the key fields and depth parameters of each child node ChildNod. Of course, if there are multiple child nodes ChildNod, you also need to obtain the index of each child node ChildNod.

[0104] 2. Splice the key fields of each child node ChildNod obtained into a variable character sequence StringBuilder, and set the number of indentation spaces according to the depth parameter to complete the format indentation. The sample code for format indentation is as follows:

[0105] StringBuilder.append("\r\n").append("".repeat(depth)).

[0106] Among them, "\r\n" is a line break character, and depth is the depth of the current child node ChildNod.

[0107] The sample code for concatenating the key fields of each child node ChildNod obtained into a variable character sequence StringBuilder is as follows:

[0108] StringBuilder.append(childNodeStr).

[0109] Among them, childNodeStr is the result of parsing the child node ChildNode. If there are multiple child nodes ChildNode, childNodeStr needs to be assembled according to the index order to prevent the overall structure from being chaotic.

[0110] 3. When the key fields of all child nodes ChildNod are concatenated into the variable character sequence StringBuilder, complete the end tag endTag of the variable character sequence StringBuilder. The sample code is as follows:

[0111] StringBuilder.append("\r\n").append("".repeat(depth-1)).append("").

[0112] Thus, starting from the parsing of the root node RootNode, through the gradual parsing and splicing of its child node ChildNod and the child node ChildNod of the child node ChildNod, all the nodes Node in the accessibility service node information AccessibilityServiceNodeInfo of the page will be assembled into a StringBuilder.

[0113] 5. Get the final XML of the page based on the mutable character sequence StringBuilder that has been adjusted again.

[0114] The final StringBuilder, which is the final XML of the current page, can be obtained through the ForkJoinPool.invoke() method.

[0115] An example final XML is as follows:

[0116] <? xml version='1.0'encoding='UTF-8'standalone='yes'? >

[0117] <hierarchy rotation="0">

[0118] <node index="0"text=""resource-id=""

[0119] class="android.widget.FrameLayout" package="com.autonavi.minimap" content-desc="" checkable="false" checked="false" clickable="false" enabledd="true" focusable="true" focused="false" scrollable="false" long-clickable="false" password="false" selected="false"

[0120] bounds="[0,0][1224,2776]">

[0121] <node index="0"text=""resource-id=""

[0122] class="android.widget.LinearLayout"package="com.autonavi.minimap"content-desc=""checkable="false"checked="false"clickable="false"enabled="true"focusable="false"focused="false"scrollable="false"long-clickable="false"password="false"selected="false"

[0123] bounds="[0,0][1224,2776]">

[0124] <node index="0"text=""resource-id="android:id / content"

[0125] class="android.widget.FrameLayout"package="com.autonavi.minimap"content-desc=""checkable="false"checked="false"clickable="false"enabled="true"focusable="false"focused="false"scrollable="false"long-clickable="false"password="false"selected="false"

[0126] bounds="[0,0][1224,2776]">

[0127] <node index="0"text=""

[0128] resource-id="com.autonavi.minimap:id / root_view"

[0129] class="android.widget.LinearLayout"package="com.autonavi.minimap"content-desc=""checkable="false"checked="false"clickable="false"enabled="true"focusable="false"focused="false"scrollable="false"long-clickable="false"password="false"selected="false"

[0130] bounds="[0,0][1224,2776]">

[0131] <node index="0"text=""

[0132] resource-id="com.autonavi.minimap:id / fl_content_view"

[0133] class="android.widget.FrameLayout"package="com.autonavi.minimap"content-desc=""checkable="false"checked="false"clickable="false"enabled="true"focusable="false"focused="false"scrollable="false"long-clickable="false"password="false"selected="false"

[0134] bounds="[0,0][1224,2776]">

[0135] <node index="0"text=""

[0136] resource-id="com.autonavi.minimap:id / map_container"

[0137] class="android.widget.FrameLayout"package="com.autonavi.minimap"content-desc=""checkable="false"checked="false"clickable="false"enabled="true"focusable="false"focused="false"scrollable="false"long-clickable="false"password="false"selected="false"

[0138] bounds="[0,0][1224,2776]">

[0139] <node index="0"text=""

[0140] resource-id="com.autonavi.minimap:id / atmapsView"

[0141] class="android.widget.FrameLayout"package="com.autonavi.minimap"content-desc=""checkable="false"checked="false"clickable="false"enabled="true"focusable="false"focused="false"scrollable="false"long-clickable="false"password="false"selected="false"

[0142] bounds="[0,0][1224,2776]">

[0143] <node index="0"text=""resource-id=""class="android.view.View"package="com.autonavi.minimap"content-desc=""checkable="false"checked="false"clickable="false"enabled="true"focusable="false"focused="false"scrollable="false"long-clickable="false"password="false"selected="false"bounds="[0,0][1224,2776]" / >

[0144]

[0145]

[0146] <node index="1"text=""

[0147] resource-id="com.autonavi.minimap:id / temporary_layer"

[0148] class="android.widget.FrameLayout"package="com.autonavi.minimap"content-desc=""checkable="false"checked="false"clickable="false"enabled="true"focusable="false"focused="false"scrollable="false"long-clickable="false"password="false"selected="false"

[0149] bounds="[0,0][1224,2776]" / >

[0150] <node index="2"text=""resource-id=""

[0151] class="android.widget.RelativeLayout"package="com.autonavi.minimap"content-desc=""checkable="false"checked="false"clickable="false"enabled="true"focusable="false"focused="false"scrollable="false"long-clickable="false"password="false"selected="false"

[0152] bounds="[0,0][1224,2776]">

[0153] <node index="0"text=""

[0154] resource-id="com.autonavi.minimap:id / notification_layer"

[0155] class="android.widget.FrameLayout"package="com.autonavi.minimap"content-desc=""checkable="false"checked="false"clickable="false"enabled="true"focusable="false"focused="false"scrollable="false"long-clickable="false"password="false"selected="false"

[0156] bounds="[0,0][1224,2776]" / >

[0157]

[0158]

[0159]

[0160]

[0161]

[0162]

[0163] 。

[0164] Figure 2The schematic diagram of the structure of the page information acquisition system based on barrier-free service of the present invention is shown. Figure 2 As shown, the page information acquisition system based on accessibility service of the present invention includes:

[0165] 1. AccessibilityServiceNodeInfo acquisition module.

[0166] The accessibility service node information AccessibilityServiceNodeInfo acquisition module is used to acquire the accessibility service node information AccessibilityServiceNodeInfo of the page based on the accessibility service AccessibilityService.

[0167] 2. Node parsing module.

[0168] The node parsing module is used to parse the node Node in the accessibility service node information AccessibilityServiceNodeInfo, and convert the parsing result into a variable character sequence StringBuilder.

[0169] 3. Adjustment and sub-node parsing module.

[0170] The adjustment and child node parsing module is used to adjust the structure of the variable character sequence StringBuilder, and perform parallel parsing of the child nodes ChildNod of the corresponding node Node.

[0171] 4. Assemble modules.

[0172] The assembling module is used to obtain the parsing results of each child node ChildNod and assemble them into a variable character sequence StringBuilder, and adjust the structure of the assembled variable character sequence StringBuilder again.

[0173] 5. XML acquisition module.

[0174] The XML acquisition module is used to obtain the final XML of the page based on the variable character sequence StringBuilder that has been adjusted again.

[0175] In addition, the present invention also provides a page information acquisition device based on accessibility service, which includes: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the page information acquisition method based on accessibility service as described above.

[0176] Finally, the present invention provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the program is executed by a processor, the steps of the method for obtaining page information based on accessibility services as described above are implemented.

[0177] Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of the present invention, rather than to limit the scope of protection of the present invention. Those skilled in the art can modify or replace the technical solution of the present invention according to the idea of ​​the present invention without departing from the essence and scope of the technical solution of the present invention.

Claims

1. A method for obtaining page information based on accessibility services, characterized in that: The following steps are involved: 1) Get the accessibility service node information AccessibilityServiceNodeInfo of the page based on the accessibility service AccessibilityService; 2) Parse the node Node in the accessibility service node information AccessibilityServiceNodeInfo, and convert the parsing result into a variable character sequence StringBuilder; 3) Adjust the structure of the variable character sequence StringBuilder, and parse the child nodes ChildNod of the corresponding node Node in parallel; 4) Obtain the parsing results of each child node ChildNod and assemble them into a variable character sequence StringBuilder, and adjust the structure of the assembled variable character sequence StringBuilder again; 5) Obtain the final XML of the page based on the mutable character sequence StringBuilder that has been adjusted again.

2. The method for obtaining page information based on barrier-free service according to claim 1, characterized in that: In step 2), parsing the node Node in the accessibility service node information AccessibilityServiceNodeInfo is performed in a RecursiveTask and the step 2) specifically includes: 21) Create an XmlSerializer object, and set the StringWriter object and the startDocument property and the startTag property; 22) Get the key fields of the node Node in the accessibility service node information AccessibilityServiceNodeInfo; 23) Set the end tag endTag property and the end document endDocument property of the XmlSerialize object, insert the key field of the obtained node Node into the XmlSerializer object, and use the StringWriter object to convert it into a mutable character sequence StringBuilder.

3. The method for obtaining page information based on accessibility service according to claim 2, characterized in that: The key fields of the obtained node Node include className, contentDescription, text, viewIdResourceName, isClickable, isEnabled, isChecked, isFocused, isFocusable, isScrollable, isSelected, isLongClickable, and boundsInScreen.

4. The method for obtaining page information based on barrier-free service according to claim 3, characterized in that: The step 3) specifically includes: 31) Remove the version, encoding and standalone information of the XmlSerializer object in the mutable character sequence StringBuilder; 32) If the node Node corresponding to the variable character sequence StringBuilder contains a child node ChildNode, remove the end tag endTag of the Node node in the variable character sequence StringBuilder; 33) Acquire the key fields of each child node ChildNod in parallel and set a depth parameter for each child node ChildNod to record the depth of each child node ChildNod, and save the key fields and depth parameter of each child node ChildNod acquired into RecursiveTaskList.

5. The method for obtaining page information based on barrier-free service according to claim 4, characterized in that: The step 4) specifically includes: 41) Traverse RecursiveTaskList and obtain the key fields and depth parameters of each child node ChildNod; 42) splicing the key fields of each child node ChildNod obtained into a variable character sequence StringBuilder, and setting the number of indentation spaces according to the depth parameter to complete the format indentation; 43) After the key fields of all child nodes ChildNod are spliced ​​into the variable character sequence StringBuilder, the end tag endTag of the variable character sequence StringBuilder is completed.

6. The method for obtaining page information based on barrier-free service according to claim 5, characterized in that: In the step 5), ForkJoinPool.invoke() is used to obtain the final XML of the page based on the variable character sequence StringBuilder that has been adjusted again.

7. The method for obtaining page information based on accessibility service according to any one of claims 1 to 6, characterized in that: In the step 1), a screenshot of the page is obtained based on the accessibility service AccessibilityService; and the step further includes the following steps: 6) The screenshot Screenshot is compressed in quality without changing the original resolution of the screenshot Screenshot to obtain a final screenshot of the page.

8. A page information acquisition system based on accessibility service, characterized in that: include: An accessibility service node information AccessibilityServiceNodeInfo acquisition module is used to obtain the accessibility service node information AccessibilityServiceNodeInfo of a page based on the accessibility service AccessibilityService; The node parsing module is used to parse the node Node in the accessibility service node information AccessibilityServiceNodeInfo and convert the parsing result into a variable character sequence StringBuilder; The adjustment and child node parsing module is used to adjust the structure of the variable character sequence StringBuilder and perform parallel parsing of the child nodes ChildNod of the corresponding node Node; The assembly module is used to obtain the parsing results of each child node ChildNod and assemble them into a variable character sequence StringBuilder, and adjust the structure of the assembled variable character sequence StringBuilder again; The XML acquisition module is used to obtain the final XML of the page based on the variable character sequence StringBuilder that has been adjusted again.

9. A device for acquiring page information based on barrier-free service, characterized in that: include: one or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the page information acquisition method based on accessibility service as described in any one of claims 1 to 7.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the method for obtaining page information based on accessibility services as described in any one of claims 1 to 7 are implemented.

Images