Multi-level fuzzy search method, device and equipment

Abstract

The invention relates to the field of robot process automation, and provides a multi-level fuzzy search method, device and equipment, and the method comprises the steps: converting a pickup expression into an element feature list based on defined element features and element feature attribute classes; obtaining all elements of the target type as to-be-judged elements, and constructing a hierarchical relationship list of the to-be-judged elements; comparing the hierarchical relationship list of the to-be-judged elements with the element feature list corresponding to the pickup expression, and scoring to obtain a scoring result; and performing screening and sorting according to a scoring result to obtain an optimal matching element. According to the method, the defect that a traditional fuzzy retrieval method is poor in fault tolerance in the prior art is overcome, and the adaptation capacity and retrieval efficiency of robot process automation software to interface element changes are remarkably improved.

Description

Technical Field

[0001] The present invention relates to the field of robotic process automation technology, and in particular to a multi-level fuzzy search method, device and equipment. Background Art

[0002] With the acceleration of digital transformation of enterprises, Robotic Process Automation (RPA) technology has received widespread attention and application due to its ability to simulate humans performing repetitive tasks on computers and automate business processes. RPA software interacts with various applications by identifying and operating interface elements, and can execute complex business logic according to preset rules and logic, thereby helping enterprises improve work efficiency, reduce labor costs, reduce errors, and enhance compliance. In the implementation of RPA technology, how to accurately and efficiently pick up and locate interface elements is one of the key technologies.

[0003] In RPA software, picking and locating interface elements usually depends on the picking expression, which is a string expression that describes the hierarchical structure of interface elements. When RPA software searches for elements in the interface element tree, it traverses and matches according to the path defined by the picking expression. However, since the software interface may change due to demand adjustments, resulting in changes in the hierarchical structure of interface elements, the traditional picking method may fail to find the target element. To solve this problem, the industry generally adopts fuzzy retrieval technology. RPA's multi-level fuzzy retrieval method starts from the root element according to the picking expression, and uses the breadth-first or depth-first traversal algorithm to determine whether the element meets the expression requirements layer by layer. If a wildcard is encountered, it is considered that the element and the hierarchy match successfully, and the matching continues downward. This method improves the adaptability of RPA software to changes in interface elements to a certain extent.

[0004] Although fuzzy retrieval technology has solved the problem of picking up elements caused by changes in interface elements to a certain extent, its fault tolerance is not ideal. In practical applications, due to the complexity and diversity of the interface element hierarchy, traditional fuzzy retrieval methods may still fail to accurately find the target elements in many cases. For example, when the interface hierarchy changes significantly, such as adding, deleting, or rearranging elements, traditional fuzzy retrieval methods may fail. Therefore, how to further improve the fault tolerance and efficiency of RPA software in picking up and locating interface elements has become an important issue that needs to be solved in the industry. Summary of the invention

[0005] The present invention provides a multi-level fuzzy search method, device and equipment, which solves the defect of poor fault tolerance of traditional fuzzy retrieval methods in the prior art, and achieves a significant improvement in the adaptability of RPA software to changes in interface elements and retrieval efficiency.

[0006] The present invention provides a multi-level fuzzy search method, comprising the following steps:

[0007] Based on the defined element features and element feature attribute classes, the pick expression is converted into an element feature list; the element feature list includes the type of element, the type of technology used, the name and the matching type; the element feature list is used to represent each element level in the pick expression;

[0008] Obtain all elements of the target type as elements to be judged, and construct a hierarchical relationship list of the elements to be judged; the hierarchical relationship list includes the parent-child hierarchical relationship from the top-level root element to each element found; the hierarchical relationship list is used to compare with the element feature list corresponding to the picking expression;

[0009] Compare and score the hierarchical relationship list of the elements to be judged and the element feature list corresponding to the picking expression to obtain a scoring result; the scoring result is determined based on the matching degree between the element feature list and the element feature list corresponding to the picking expression, and the higher the matching degree, the higher the score;

[0010] Screening and sorting are performed according to the scoring results to obtain the best matching element.

[0011] According to a multi-level fuzzy search method provided by the present invention, all elements of the target type are obtained as elements to be judged, and a hierarchical relationship list of the elements to be judged is constructed, which specifically includes: calling an application programming interface provided by a target user interface scene to retrieve all elements of a specified type as elements to be judged; for each retrieved element, obtaining its hierarchical relationship in the interface element tree; and determining the hierarchical relationship list of the elements to be judged according to the hierarchical relationship of each element.

[0012] According to a multi-level fuzzy search method provided by the present invention, the hierarchical relationship list of the element to be judged and the element feature list corresponding to the picking expression are compared and scored to obtain a scoring result, which specifically includes: traversing the hierarchical relationship list of the element to be judged; comparing the hierarchical relationship list of the element to be judged currently traversed with the element feature list corresponding to the picking expression; according to the comparison result, obtaining a score based on a preset scoring rule; for each element to be judged, after traversing its hierarchical relationship list, accumulating its score at each level to obtain the total score of the element to be judged; recording the total score of each element to be judged to form a scoring result.

[0013] According to a multi-level fuzzy search method provided by the present invention, the screening and sorting according to the scoring results to obtain the best matching element specifically includes: screening out elements with non-zero scores from all elements to be judged; sorting the screened out elements with non-zero scores according to their scores; and selecting the element with the highest score in the sorted element list as the best matching element.

[0014] According to a multi-level fuzzy search method provided by the present invention, the scoring is obtained based on a preset scoring rule, specifically including: traversing the hierarchical relationship list of elements to be judged, and obtaining the index of each element to be judged in the element feature list corresponding to the picking expression in turn, to obtain an index list; starting a corresponding number of loops according to the length of the element feature list corresponding to the picking expression; matching the current loop variable in the index list, and when the current loop variable is matched in the index list, if the match is successful, scoring and recording the index position; otherwise, no scoring; each loop starts matching from the index position of the last successful match plus one, until all elements of the picking expression are processed.

[0015] According to a multi-level fuzzy search method provided by the present invention, after sorting the screened elements with non-zero scores according to their scores, the method also includes: if there are multiple elements with the same score and all of them are the highest scores, then selecting one of the multiple elements as the best matching element according to other conditions; the other conditions include the stability of the element and the historical matching success rate.

[0016] The present invention also provides a multi-level fuzzy search device, comprising the following modules:

[0017] A pick expression conversion module, used to convert the pick expression into an element feature list based on defined element features and element feature attribute classes; the element feature list includes the type of element, the type of technology used, the name and the matching type; the element feature list is used to represent each element level in the pick expression;

[0018] A hierarchical relationship list construction module is used to obtain all elements of the target type as elements to be judged, and to construct a hierarchical relationship list of the elements to be judged; the hierarchical relationship list includes the parent-child hierarchical relationship from the top-level root element to each element found; the hierarchical relationship list is used to compare with the element feature list corresponding to the picking expression;

[0019] A comparison and scoring module is used to compare and score the hierarchical relationship list of the element to be judged and the element feature list corresponding to the picking expression to obtain a scoring result; the scoring result is determined based on the matching degree between the element feature list and the element feature list corresponding to the picking expression, and the higher the matching degree, the higher the score;

[0020] The screening and sorting module is used to screen and sort according to the scoring results to obtain the best matching element.

[0021] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the multi-level fuzzy search method described above is implemented.

[0022] The present invention also provides a non-transitory computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the multi-level fuzzy search method as described above is implemented.

[0023] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the multi-level fuzzy search method described above is implemented.

[0024] The present invention provides a multi-level fuzzy search method, device and equipment, including the following beneficial effects: by defining element features and element feature attribute classes, it can adapt to different types of elements and complex search requirements. By converting the picking expression into an element feature list, a hierarchical relationship list is constructed for the elements to be judged, and through a comparison and scoring mechanism, efficient screening of the elements to be judged is achieved. This screening mechanism can quickly identify the elements that best match the picking expression. Through scoring, screening and sorting, the elements that best match the picking expression can be accurately found, and the target elements can be quickly located. Compared with traditional search methods, the present invention can find matching elements in a shorter time through multi-level fuzzy search, thereby improving search efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is a schematic diagram of the format of the UIFeature class and the UIFeatureAttribute class provided by the present invention.

[0027] Figure 2 It is a flow chart of the multi-level fuzzy search method provided by the present invention.

[0028] Figure 3 It is a schematic diagram of a comparison process of a hierarchical relationship list of elements to be judged and an element feature list corresponding to a picking expression provided by the present invention.

[0029] Figure 4 It is a schematic diagram of the scoring algorithm provided by the present invention.

[0030] Figure 5 It is a structural schematic diagram of the multi-level fuzzy search device provided by the present invention.

[0031] Figure 6 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0033] Some concepts involved in the present invention are described below.

[0034] GTK scenario under Linux: mainly refers to the scenario of using GTK (GIMP Toolkit) toolkit for graphical user interface (GUI) development and application in Linux operating system. GTK is a widely used cross-platform GUI toolkit, especially in Linux system, it is an important tool for creating graphical interface applications.

[0035] UIFeature class (element feature class): used to express each element level in the picking expression. This class defines the element type, the type of technology used, the name, the matching type, and some necessary properties. The UIFeature class is also used for comparison between two objects, that is, the compare method in the figure.

[0036] UIFeatureAttribute class (element feature attribute class): defines the attributes of UIFeature. UIFeature and UIFeatureAttribute are in a one-to-many relationship, and the UIFeatureAttribute class also has a compare method.

[0037] like Figure 1 As shown, UIFeature defines the type of element, the type of technology used, the name, the matching type and some necessary attributes. UIFeature is used to express each element level in the picking expression. Within the scope of the present invention, UIFeature is also used for comparison between two objects, namely the compare method in the figure.

[0038] There is another class UIFeatureAttribute that defines the attributes of UIFeature. UIFeature and UIFeatureAttribute have a one-to-many relationship. Similarly, UIFeatureAtttribute also has a compare method.

[0039] On the one hand, these two classes are used to represent the picking expression, and on the other hand, they are also used to represent the element hierarchy relationship to be judged in multi-level fuzzy search, that is, the problem of matching the picking expression to find the element hierarchy is converted into a comparison between two UIFeature lists, which makes the judgment logic simpler.

[0040] The present invention relates to a new method for multi-level fuzzy search in picking scenarios under multiple operating systems and different technologies. The technology is a pure code algorithm and has nothing to do with the operating system used and the interface technology adopted by the specific picking target software. It has been verified that the technology discussed in the present invention has been tested in different picking scenarios such as UIA (Windows operating system), JAB (Windows system), AT-SPI (Linux operating system) and browser (Windows system and Linux system), and has good applicability.

[0041] RPA is a type of Robotic Process Automation (RPA) technology that can automate business processes by simulating the behavior of humans performing repetitive tasks on computers. RPA software can interact with various applications by identifying and operating interface elements, and can execute complex business logic based on preset rules and logic. RPA software can help companies improve work efficiency, reduce labor costs, reduce errors, and enhance compliance.

[0042] As we all know, the operating system uses auxiliary technology to store each application's interface elements in the memory as a single-rooted tree.

[0043] "Interface elements" refer to the various visual components that make up the user interface of an application. These components are the interface for users to interact with the application, including but not limited to the following common elements:

[0044] Buttons: Controls that users click to perform specific actions, such as "Save", "Submit", "Cancel", etc.

[0045] Text Fields: Areas that allow users to enter text, such as input boxes for user names, passwords, search keywords, etc.

[0046] Labels: Used to display text or images to provide information or guidance to users.

[0047] Dropdown Lists: A control that allows the user to select an item from a predefined list of options.

[0048] Checkboxes and Radio Buttons: Controls that allow the user to select one or more options.

[0049] List Boxes: Display a list of selectable items and allow the user to select one or more items.

[0050] Sliders: Allow users to select a value within a range by sliding.

[0051] Menus: Provides a series of commands or options for users to choose from, usually accessed by clicking items on the menu bar.

[0052] Links: In a web application or help system, links allow users to navigate to different pages or get more information.

[0053] Icons and Images: Graphic elements that represent functionality or provide visual information.

[0054] Progress Bars: Visual indicators that show the progress of a task.

[0055] Tables and Grids: Controls that organize and display data in rows and columns.

[0056] Toolbars and Tool Palettes: Areas that contain multiple tools or functions, where users can perform specific operations by clicking icons or buttons.

[0057] Interface elements are UI components that software can recognize and interact with. RPA tools automate tasks such as clicking a button, filling out a form, or reading data on an interface by recognizing the characteristics of these elements (such as text, images, location, etc.).

[0058] Each application contains different types of interface elements according to its function and design. For example, a web browser contains an address bar (text box), forward and back buttons, a bookmark bar, etc., while a spreadsheet application contains a menu bar, a toolbar, cells (which can be regarded as special text boxes), a function bar, etc. These elements together constitute the interface for users to interact with the application.

[0059] The RPA software also searches for interface elements on this interface element tree when picking up elements. On this interface element tree, except for the root node, each node has a unique parent node, and each parent node has at least one child node. Based on this principle, as long as each parent node of the interface element is described in a structure, the RPA software can locate a certain element according to this structure. This structure is also called a picking expression.

[0060] Once the pick expression is defined, it remains unchanged. However, if the interface of the tested software is adjusted due to demand adjustments, the runtime interface element tree will also change accordingly, so the previously defined pick expression may not be able to find the element. In response to this situation, RPA software generally uses fuzzy search to improve fault tolerance.

[0061] As the name suggests, RPA's multi-level fuzzy search means that when searching for a matching element, it starts from the root element and uses a breadth-first or depth-first traversal algorithm to determine the element layer by layer according to the pick expression. If it is determined to meet the expression requirements, it continues to search downwards, otherwise it tries another path. If a wildcard is encountered, it is considered that the element and the level match successfully, and it continues to match downwards. This fuzzy search method is a way to improve the adaptability of RPA software to changes in interface elements, which is also a common practice of RPA software. How to make the pick expression adapt to the changing page is one of the core technologies of RPA software.

[0062] However, the fault tolerance of this fuzzy search method is not ideal. There are still many cases where elements cannot be found. Take the element hierarchy of a web page as an example:

[0063] Assume that the hierarchy of interface elements is:

[0064] / div / div / div / span / div / button

[0065] This is a concrete element-level path that specifies a specific path starting from the root element. In this path:

[0066] The first / div means to find the first div element starting from the root element.

[0067] The second / div means to find the second div child element inside the previous div element.

[0068] The third / div means to find the third div child element inside the previous div element.

[0069] / span means to find the first span child element inside the previous div element.

[0070] The fourth / div means to find the first div child element inside the span element.

[0071] / button means to find the first button child element inside the previous div element.

[0072] The corresponding picking expression is:

[0073] / div / * / span / *button

[0074] This is an element-level path with a wildcard (*) to provide greater flexibility. In this path:

[0075] The first / div also means to find the first div element starting from the root element.

[0076] / * means to find any number of child elements of any type inside the previous div element.

[0077] / span means to find the first span element inside any child element ("any child element" here refers to any path matched by the previous / *).

[0078] The second / * also means to find any number of child elements of any type inside the span element.

[0079] / button means to find the first button element inside any child element.

[0080] Compared with the first path, the wildcard in this path allows it to skip any number of intermediate elements in the interface element tree, as long as it can eventually match the button element after the span element. This flexibility is very useful when dealing with scenarios where interface elements change frequently.

[0081] If the interface hierarchy changes, the new element hierarchy is:

[0082] / div / div / div / button

[0083] In this path, the first / div: Starting from the root element, the path indicates to find the first div element. In HTML or similar markup languages, div is a commonly used container element for organizing and laying out page content.

[0084] Second / div: Inside the first div element found, the path continues to indicate to find the second div child element. This means to locate another div that is a direct child of the first div.

[0085] Third / div: Similarly, inside the second div element, the path directs to find the third div child element. This step goes further into the nested div structure.

[0086] / button: Finally, inside the third div element, the path directs you to find the first button element. A button element is a clickable control that is typically used to submit a form, trigger an action, or navigate to another page.

[0087] Here, the span layer is missing in the middle, so the previous fuzzy search technology is not applicable. In view of this problem, the present invention proposes a matching algorithm with better fault tolerance.

[0088] The present invention relates to an improved retrieval scheme for multi-element hierarchical fuzzy retrieval technology in the RPA picking and positioning stage, which enhances the fault tolerance of fault picking by introducing a scoring method. The technology is independent of the operating system and the technology adopted by the picked software. This scoring method for multi-element hierarchical fuzzy retrieval has better fault tolerance than the traditional fuzzy retrieval method.

[0089] And in this patent scenario, there are two innovations:

[0090] Innovation 1: A bottom-up screening-then-matching algorithm based on the interface element tree. This new algorithm is more efficient than the traditional top-down element search.

[0091] Innovation 2: Supports fault-tolerant scoring algorithm when fuzzy searching the interface element tree.

[0092] The design goal of the present invention is to introduce a scoring method when performing fuzzy search in the interface element tree. Through this scoring method, the fault tolerance of the picking expression can be significantly improved, and the RPA process can still run stably to a certain extent after the interface changes. In addition, the search and matching algorithm discussed in the present invention can provide better performance and significantly reduce screen freezes during picking.

[0093] The software provided by the present invention realizes the following functions:

[0094] Algorithm of screening first and matching later: Generally speaking, RPA software searches for elements in the interface element tree in depth-first or breadth-first order, and the algorithm retrieval efficiency is low. In this patent, the algorithm of screening first and matching later is used to avoid traversing the entire tree, so it is faster. According to calculations, the speed of the new bidirectional retrieval algorithm is 10 times that of the traditional layer-by-layer search.

[0095] Scoring matching elements during multi-level fuzzy search: By picking up expressions to fuzzily search for elements, each matched element will be scored according to the algorithm. The more it matches the matching algorithm, the higher the score. Finally, the elements are sorted from high to low according to the score, and the element with the highest score is the best matching element.

[0096] The following is a detailed description of the embodiment of the present invention using the GTK scenario under Linux (a scenario in which the GTK (GIMP Toolkit) toolkit is used in the Linux operating system to develop and apply a graphical user interface). GTK is a graphics library developed specifically for GIMP (GNU Image Manipulation Program).

[0097] Figure 2 is one of the flow charts of the multi-level fuzzy search method provided by the present invention, such as Figure 2 As shown, the method comprises the following steps:

[0098] S210. Based on the defined element features and element feature attribute classes, the picked expression is converted into an element feature list.

[0099] The element feature list includes the element type, the technology type used, the name and the matching type; the element feature list is used to represent each element level in the picking expression.

[0100] Specifically, Figure 3 As shown, after defining the UIFeature and UIFeatureAttribute classes, first convert the pick expression into a UIFeature list, where each UIFeature instance is used to describe a hierarchical object. The pick expression is a string that describes the path from the root element to the target element, such as " / div / div / span / button". Parse this string and decompose it into individual element parts, such as ["div","div","span","button"]. For each element part in the pick expression, create a UIFeature instance.

[0101] Set the type property of UIFeature to the type of the current element part (such as div, span, etc.).

[0102] Set other necessary properties, such as the type of technology used (if any) and the match type (e.g., exact match, partial match, etc.).

[0103] Put all created UIFeature instances into a list, and this list is the final element feature list.

[0104] Then parse this UIFeature list to get the type of the lowest-level element.

[0105] S220: Acquire all elements of the target type as elements to be judged, and construct a hierarchical relationship list of the elements to be judged.

[0106] The hierarchical relationship list includes the parent-child hierarchical relationship from the top-level root element to each element found; the hierarchical relationship list is used to compare with the element feature list corresponding to the picking expression.

[0107] According to a multi-level fuzzy search method provided by the present invention, all elements of the target type are obtained as elements to be judged, and a hierarchical relationship list of the elements to be judged is constructed, which specifically includes: calling an application programming interface provided by the target user interface scene to retrieve all elements of a specified type as elements to be judged; for each retrieved element, obtaining its hierarchical relationship in the interface element tree; and determining the hierarchical relationship list of the elements to be judged according to the hierarchical relationship of each element.

[0108] Specifically, the application programming interface API provided by the target user interface UI scene is used to retrieve all elements of the same type in the target program. Different UI scenes call different APIs. For the browser scene, the document.querySelectorAll() method can be called, and for the AT-SPI2 scene, atspi_collection_get_matches can be called. The purpose of different scenes is the same, that is, to retrieve all elements of the specified type from the interface element tree. Because this step directly calls the underlying API, many intermediate processing links and steps are reduced in the middle, and high performance can be maintained even if there are many elements to be found.

[0109] The interface element tree is the tree structure in which the interface elements of each application are organized into a tree structure in memory. Each element is a node in the tree. Except for the root node, each node has a unique parent node, and each parent node can have multiple child nodes.

[0110] In the found element list, first build a parent-child hierarchical relationship list from the top-level root element to the found element using the UIFeature list according to the hierarchical relationship of each element (the position of an element in the interface element tree, that is, the relationship between its parent node and child node). Then compare this parent-child hierarchical relationship list with the UIFeature list corresponding to the picked expression.

[0111] By calling the API provided by the target user interface scenario, all elements of the specified type can be efficiently retrieved. The hierarchical relationship of each element is obtained and a hierarchical relationship list is constructed, which provides the necessary data structure for subsequent multi-level fuzzy search. This method is universal and can be applied to different user interface scenarios. It only needs to be adjusted according to the specific API.

[0112] S230: Compare and score the hierarchical relationship list of the element to be judged and the element feature list corresponding to the picking expression to obtain a scoring result.

[0113] The scoring result is determined based on the degree of matching between the element feature list and the element feature list corresponding to the pick expression. The higher the degree of matching, the higher the score.

[0114] According to a multi-level fuzzy search method provided by the present invention, the hierarchical relationship list of the element to be judged and the element feature list corresponding to the picking expression are compared and scored to obtain a scoring result, which specifically includes: traversing the hierarchical relationship list of the element to be judged; comparing the hierarchical relationship list of the element to be judged currently traversed with the element feature list corresponding to the picking expression; according to the comparison result, obtaining a score based on a preset scoring rule; for each element to be judged, after traversing its hierarchical relationship list, accumulating its score at each level to obtain the total score of the element to be judged; recording the total score of each element to be judged to form a scoring result.

[0115] Specifically, suppose there is a pick expression / div / span / button, which is converted into an element feature list as [UIFeature(div), UIFeature(span), UIFeature(button)]. All elements of type button are retrieved through the API of the target UI scene, and a hierarchical relationship list is constructed for each element.

[0116] For each retrieved button element, compare its hierarchical list with the element feature list corresponding to the pick expression. For example, the hierarchical list of a button element might be [UIFeature(root), UIFeature(div), UIFeature(span), UIFeature(button)].

[0117] When comparing, check whether the element features at each level match the corresponding positions in the picked expression.

[0118] If all levels match, the score is full; if some levels do not match, the corresponding score is given according to the number of matching levels.

[0119] Finally, the total score of each button element is recorded to form a scoring result.

[0120] According to a multi-level fuzzy search method provided by the present invention, a score is obtained based on a preset scoring rule, specifically including: traversing the hierarchical relationship list of elements to be judged, and obtaining the index of each element to be judged in the element feature list corresponding to the picking expression in turn to obtain an index list; starting a corresponding number of loops according to the length of the element feature list corresponding to the picking expression; matching the current loop variable in the index list, and when the current loop variable is matched in the index list, if the match is successful, scoring and recording the index position; otherwise, no scoring; each loop starts matching from the index position of the last successful match plus one, until all elements of the picking expression are processed.

[0121] Specifically, suppose there is an element ABCDEF, the pick expression is XYZ, and the list length is 3. So currently there is a UIFeature list representing ABCDEF and a UIFeature list representing the pick expression XYZ. Figure 4 The figure shows the scoring algorithm flow chart, which shows the logical flow of the scoring algorithm. It is inferred by looking up the index list of the picking expression corresponding to the element, and the following rules are met:

[0122] (1) Multiple values ​​may be found in the generated list T, and each value will participate in the judgment and scoring

[0123] (2) Each loop starts from the position where the last successful match was made in T + 1 and searches backwards for the value of K. If the value is found, 1 point is awarded; otherwise, no point is awarded.

[0124] S240: Screen and sort according to the scoring results to obtain the best matching element.

[0125] According to a multi-level fuzzy search method provided by the present invention, screening and sorting are performed according to the scoring results to obtain the best matching element, which specifically includes: screening out elements with non-zero scores from all elements to be judged; sorting the screened out elements with non-zero scores according to their scores; and selecting the element with the highest score in the sorted element list as the best matching element.

[0126] Specifically, after the above scoring algorithm, a score can be obtained for each UIFeature list corresponding to the found element, and then the elements with a score of 0 are excluded. Among all the elements to be judged, the elements with non-zero scores are first screened out. The non-zero score means that these elements have a matching relationship with the picking expression at least at some levels, so they are eligible to become potential matching elements.

[0127] And sort the remaining scores in descending order. For the filtered elements with non-zero scores, sort them from high to low according to their scores. The higher the score, the higher the matching degree of the element with the picking expression, so it should be ranked first.

[0128] In the sorted list of elements, the element with the highest score is selected as the best matching element, because the element with the highest score best meets the description of the picking expression.

[0129] According to a multi-level fuzzy search method provided by the present invention, after the screened elements with non-zero scores are sorted according to their scores, if there are multiple elements with the same score and all of them are the highest scores, one of the multiple elements is selected as the best matching element according to other conditions; the other conditions include the stability of the element and the historical matching success rate.

[0130] Specifically, if there are multiple elements with the same score and all of them are the highest score, the element that is scored first is obtained as the best matching element.

[0131] In this way, it can be ensured that in multi-level fuzzy search, the most reliable element that best meets the picking expression is selected as the optimal matching element.

[0132] The method of the present invention makes the retrieval and positioning speed faster. According to the measurement, the method of the present invention is 10 times faster than the traditional top-down retrieval algorithm. Secondly, the scoring mechanism of multi-level fuzzy search introduces a fault-tolerant mechanism for the fuzzy retrieval mechanism by scoring the matching elements. It can reduce the situation where the picking expression is not applicable due to interface modification and improve the adaptability of the picking expression.

[0133] The multi-level fuzzy search device provided by the present invention is described below. The multi-level fuzzy search device described below and the multi-level fuzzy search method described above can be referred to each other.

[0134] like Figure 5 The present invention provides a multi-level fuzzy search device, comprising:

[0135] The picking expression conversion module 510 is used to convert the picking expression into an element feature list based on the defined element features and element feature attribute classes; the element feature list includes the type of element, the type of technology used, the name and the matching type; the element feature list is used to represent each element level in the picking expression;

[0136] A hierarchical relationship list building module 520 is used to obtain all elements of the target type as elements to be judged, and build a hierarchical relationship list of elements to be judged; the hierarchical relationship list includes the parent-child hierarchical relationship from the top-level root element to each element found; the hierarchical relationship list is used to compare with the element feature list corresponding to the picking expression;

[0137] The comparison and scoring module 530 is used to compare and score the hierarchical relationship list of the element to be judged and the element feature list corresponding to the picking expression to obtain a scoring result; the scoring result is determined based on the matching degree between the element feature list and the element feature list corresponding to the picking expression, and the higher the matching degree, the higher the score;

[0138] The screening and sorting module 540 is used to screen and sort according to the scoring results to obtain the best matching element.

[0139] Figure 6 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 6 As shown, the electronic device may include: a processor (processor) 610, a communication interface (Communications Interface) 620, a memory (memory) 630 and a communication bus 640, wherein the processor 610, the communication interface 620, and the memory 630 communicate with each other through the communication bus 640. The processor 610 can call the logic instructions in the memory 630 to execute a multi-level fuzzy search method, which includes: based on the defined element features and element feature attribute classes, converting the picking expression into an element feature list; the element feature list includes the element type, the technology type used, the name and the matching type; the element feature list is used to represent each element level in the picking expression; obtaining all elements of the target type as elements to be judged, and constructing a hierarchical relationship list of the elements to be judged; the hierarchical relationship list includes the parent-child hierarchical relationship from the top-level root element to each element found; the hierarchical relationship list is used to compare with the element feature list corresponding to the picking expression; comparing and scoring the hierarchical relationship list of the elements to be judged and the element feature list corresponding to the picking expression to obtain a scoring result; the scoring result is determined based on the degree of matching between the element feature list and the element feature list corresponding to the picking expression, and the higher the matching degree, the higher the score; filtering and sorting are performed according to the scoring result to obtain the best matching element.

[0140] In addition, the logic instructions in the above-mentioned memory 630 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0141] On the other hand, the present invention also provides a computer program product, which includes a computer program. The computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the multi-level fuzzy search method provided by the above methods, which includes: based on the defined element features and element feature attribute classes, converting the picking expression into an element feature list; the element feature list includes the type of element, the type of technology used, the name and the matching type; the element feature list is used to represent each element level in the picking expression; all elements of the target type are obtained as elements to be judged, and a hierarchical relationship list of the elements to be judged is constructed; the hierarchical relationship list includes the parent-child hierarchical relationship from the top-level root element to each element found; the hierarchical relationship list is used to compare with the element feature list corresponding to the picking expression; the hierarchical relationship list of the elements to be judged and the element feature list corresponding to the picking expression are compared and scored to obtain a scoring result; the scoring result is determined based on the degree of matching between the element feature list and the element feature list corresponding to the picking expression, and the higher the matching degree, the higher the score; screening and sorting are performed according to the scoring result to obtain the best matching element.

[0142] On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, it is implemented to execute the multi-level fuzzy search method provided by the above-mentioned methods, the method comprising: based on the defined element features and element feature attribute classes, converting the picking expression into an element feature list; the element feature list includes the element type, the technology type used, the name and the matching type; the element feature list is used to represent each element level in the picking expression; obtaining all elements of the target type as elements to be judged, and constructing a hierarchical relationship list of the elements to be judged; the hierarchical relationship list includes the parent-child hierarchical relationship from the top-level root element to each element found; the hierarchical relationship list is used to compare with the element feature list corresponding to the picking expression; the hierarchical relationship list of the elements to be judged and the element feature list corresponding to the picking expression are compared and scored to obtain a scoring result; the scoring result is determined based on the degree of matching between the element feature list and the element feature list corresponding to the picking expression, and the higher the matching degree, the higher the score; screening and sorting are performed according to the scoring result to obtain the best matching element.

[0143] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0144] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods of each embodiment or some parts of the embodiment.

[0145] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A multi-level fuzzy search method, characterized in that: include: Based on the defined element features and element feature attribute classes, the pick expression is converted into an element feature list; the element feature list includes the type of element, the type of technology used, the name and the matching type; the element feature list is used to represent each element level in the pick expression; Obtain all elements of the target type as elements to be judged, and construct a hierarchical relationship list of the elements to be judged; The hierarchical relationship list includes the parent-child hierarchical relationship from the top-level root element to each element found; The hierarchical relationship list is used to compare with the element feature list corresponding to the picking expression; Compare and score the hierarchical relationship list of the elements to be judged and the element feature list corresponding to the picking expression to obtain a scoring result; the scoring result is determined based on the matching degree between the element feature list and the element feature list corresponding to the picking expression, and the higher the matching degree, the higher the score; Screening and sorting are performed according to the scoring results to obtain the best matching element.

2. The multi-level fuzzy search method according to claim 1, characterized in that: The step of obtaining all elements of the target type as elements to be determined and constructing a hierarchical relationship list of the elements to be determined specifically includes: Calling the application programming interface provided by the target user interface scene to retrieve all elements of the specified type as elements to be judged; For each retrieved element, obtain its hierarchical relationship in the interface element tree; A hierarchical relationship list of the elements to be determined is determined according to the hierarchical relationship of each element.

3. The multi-level fuzzy search method according to claim 1, characterized in that: The step of comparing and scoring the hierarchical relationship list of the elements to be judged and the element feature list corresponding to the picking expression to obtain a scoring result specifically includes: Traverse the hierarchical relationship list of elements to be judged; Compare the hierarchical relationship list of the currently traversed element to be judged with the element feature list corresponding to the picking expression; According to the comparison results, scores are obtained based on preset scoring rules; For each element to be judged, after traversing its hierarchical relationship list, accumulate its score at each level to obtain the total score of the element to be judged; The total score of each element to be judged is recorded to form a scoring result.

4. The multi-level fuzzy search method according to claim 1, characterized in that: The screening and sorting according to the scoring results to obtain the best matching element specifically includes: Filter out the elements with non-zero scores from all the elements to be judged; Sort the filtered elements with non-zero scores according to their scores; In the sorted list of elements, select the element with the highest score as the best matching element.

5. The multi-level fuzzy search method according to claim 3, characterized in that: The step of obtaining a score based on a preset scoring rule specifically includes: Traversing the hierarchical relationship list of the elements to be judged, obtaining the index of each element to be judged in the element feature list corresponding to the picking expression in turn, and obtaining an index list; Start the corresponding number of loops according to the length of the element feature list corresponding to the picking expression; Match the current loop variable in the index list. When the current loop variable is matched in the index list, if the match is successful, score and record the index position; otherwise, no score is scored; Each loop starts matching from the index position of the last successful match plus one until all elements of the picked expression have been processed.

6. The multi-level fuzzy search method according to claim 4, characterized in that: After sorting the filtered elements with non-zero scores according to their scores, the method further includes: If there are multiple elements with the same score and they are all the highest score, then one of the multiple elements is selected as the best matching element according to other conditions; The other conditions include the stability of the elements and the historical matching success rate.

7. A multi-level fuzzy search device, characterized in that: include: A pick expression conversion module, used to convert the pick expression into an element feature list based on defined element features and element feature attribute classes; the element feature list includes the type of element, the type of technology used, the name and the matching type; the element feature list is used to represent each element level in the pick expression; A hierarchical relationship list building module is used to obtain all elements of the target type as elements to be judged, and build a hierarchical relationship list of the elements to be judged; The hierarchical relationship list includes the parent-child hierarchical relationship from the top-level root element to each element found; The hierarchical relationship list is used to compare with the element feature list corresponding to the picking expression; A comparison and scoring module is used to compare and score the hierarchical relationship list of the element to be judged and the element feature list corresponding to the picking expression to obtain a scoring result; the scoring result is determined based on the matching degree between the element feature list and the element feature list corresponding to the picking expression, and the higher the matching degree, the higher the score; The screening and sorting module is used to screen and sort according to the scoring results to obtain the best matching element.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the multi-level fuzzy search method according to any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the multi-level fuzzy search method according to any one of claims 1 to 6 is implemented.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the multi-level fuzzy search method according to any one of claims 1 to 6 is implemented.

Images