A method and device for intelligent search of a page, a storage medium and an electronic device
By recognizing controls on the APP page and generating operation instructions, combined with an image classification model, intelligent page search is achieved, solving the problems of high efficiency and low cost in existing APP inspection technologies, improving search efficiency and reducing costs.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-12-09
- Publication Date
- 2026-03-31
AI Technical Summary
In the existing technology, there is no effective solution for how to intelligently perform page searches to inspect applications (APPs), especially when third-party functions are provided or authorized to ensure that they fulfill the agreement.
By acquiring the image of the current page, recognizing controls, generating operation instructions to switch pages, and judging page matching based on preset target features until the target page is found, intelligent search is performed using control detection algorithms and image classification models.
It improves search efficiency, reduces search costs, and enables intelligent page search by eliminating the need to obtain page code or manually input commands.
Smart Images

Figure CN115982495B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The present specification relates to the technical field of computer, and particularly relates to a page intelligent search method and device, a storage medium and an electronic equipment. BACKGROUND
[0002] With the development of science and technology, application programs (APPs) are applied more and more widely. With the development of application programs, the application programs can provide more and more functions, and some functions may need to be provided or authorized by a third party.
[0003] When the third party provides or authorizes the APP to implement the function, the third party needs to sign an agreement with the developer of the APP to ensure that the APP can perform the agreement when providing or authorizing the function. Meanwhile, during the use of the APP, the third party can perform an inspection on the function provided or authorized by the third party in the APP. For example, the third party provides a payment tool required for a payment function in an APP, and according to the agreement signed by the two parties, the payment tool provided by the third party should be the default tool or the top tool when a user uses the payment function in the APP. Therefore, the payment function in the APP can be inspected, and the page implementing the payment function in the APP can be searched through a screen recording manner to check whether the APP performs the agreement when providing the payment function.
[0004] Therefore, how to intelligently perform page search to perform an inspection on the APP is a problem to be solved. SUMMARY
[0005] The present specification provides a page intelligent search method and device, a storage medium and an electronic equipment to partially solve the above problems in the prior art.
[0006] The present specification adopts the following technical solutions:
[0007] The present specification provides a page intelligent search method, comprising:
[0008] obtaining an image corresponding to a current page, and recognizing a control in the image;
[0009] generating an operation instruction according to each recognized control;
[0010] performing page switching on the current page according to the operation instruction;
[0011] determining whether the switched page matches a target feature according to the target feature, if yes, determining that the switched page is a target page, and if not, re-determining the switched page as the current page and continuing to perform page switching according to the re-determined current page until the switched page is the target page.
[0012] Optionally, for each identified control, an operation instruction is generated based on that control, specifically including:
[0013] Sort the controls according to the number of preset keywords they contain;
[0014] In accordance with the sorting order, operation instructions are generated for each control in turn.
[0015] Optionally, after switching the current page according to the operation instruction, the method further includes:
[0016] Update the control to a previously manipulated control;
[0017] For each identified control, an operation instruction is generated based on that control, specifically including:
[0018] For each identified inactive control, an operation instruction is generated based on that inactive control;
[0019] When all controls are the already operated controls and the current page does not match the target feature, the method further includes:
[0020] Generate the first rollback instruction;
[0021] According to the first rollback instruction, a rollback operation is performed on the current page to obtain a rollback page. It is determined whether the rollback page contains an untouched control. If so, the rollback page is re-identified as the current page. Otherwise, the rollback operation is continued according to the first rollback instruction until the rollback page contains an untouched control.
[0022] Optionally, the page that was returned to the previous page can be redefined as the current page, specifically including:
[0023] Determine the page similarity between the current page and the page before the switch and the page to go back;
[0024] Based on the page similarity, determine whether the page to be returned to is the page before the switch;
[0025] When the page to be returned to is the page before the switch, the page to be returned to be the current page.
[0026] Optionally, after generating the operation instructions, the method further includes:
[0027] The operation instructions are stored in an instruction sequence;
[0028] When the page to be returned to is not the page before the switch, the method further includes:
[0029] According to the instruction sequence, the back page is switched to the page before the switch.
[0030] Optionally, according to the instruction sequence, switching the back page to the page before the switch specifically includes:
[0031] Based on the instruction sequence, determine the operation instructions required to switch from the back page to the page before the switch;
[0032] Based on the determined operation instructions, the back page is switched to the page before the switch.
[0033] Optionally, after obtaining the image corresponding to the current page, the method further includes:
[0034] The image is input into a pre-trained image classification model to determine whether the current page contains any anomalies.
[0035] When an anomaly exists on the current page, the type of the anomaly is determined, and the anomaly is processed according to a preset anomaly handling instruction corresponding to the type.
[0036] Optionally, when no exception handling instruction corresponding to the type is found, the method further includes:
[0037] Generate a second rollback instruction;
[0038] According to the second rollback instruction, a rollback operation is performed on the current page to return the current page to the state before the exception occurred.
[0039] Optionally, after switching the current page according to the operation instruction, the method further includes:
[0040] Update the control to a previously manipulated control;
[0041] For each identified control, an operation instruction is generated based on that control, specifically including:
[0042] For each identified inactive control, an operation instruction is generated based on that inactive control;
[0043] After switching the current page, the method further includes:
[0044] When the switched page has preset signpost features, all controls in the previous page corresponding to the switched page are updated to the operated controls.
[0045] This specification provides a device for intelligent page search, comprising:
[0046] The parsing module is used to obtain the image corresponding to the current page and identify the controls in the image;
[0047] The instruction module is used to generate operation instructions for each identified control.
[0048] The switching module is used to switch the current page according to the operation instruction;
[0049] The determination module is used to determine whether the switched page matches the preset target features. If so, the switched page is determined to be the target page. Otherwise, the switched page is re-determined as the current page, and the page switching continues according to the re-determined current page until the switched page is the target page.
[0050] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described method for intelligent page search.
[0051] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the above-described method for intelligent page search.
[0052] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:
[0053] The intelligent page search method provided in this manual identifies controls in the image corresponding to the current page. For each identified control, an operation command is generated based on the control, and the current page is switched according to the operation command. It is then determined whether the switched page matches the preset target features. When the switched page matches the preset target features, the switched page is determined as the target page. When the switched page does not match the preset target features, the switched page is re-determined as the current page, and the page switching continues according to the re-determined current page until the switched page is the target page.
[0054] As can be seen from the above method, this method does not require obtaining the page code or manually inputting search commands. It only needs to obtain the image corresponding to the current page, identify the controls in the image, and perform a page switching operation based on the operation commands generated by the identified controls. It then determines whether the switched page matches the preset target features. If so, the switched page is determined as the target page; otherwise, the switched page is re-determined as the current page, and the page switching continues based on the re-determined current page until the switched page is the target page. By intelligently performing page search, the search efficiency is improved and the search cost is reduced. Attached Figure Description
[0055] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and their descriptions, serving to explain this specification and do not constitute an undue limitation thereof.
[0056] In the picture:
[0057] Figure 1 This is a flowchart illustrating a method for intelligent page search provided in this specification;
[0058] Figure 2 This is a schematic diagram of an image corresponding to the current page provided in this specification;
[0059] Figure 3 This is a graph that is performing a depth-first search, as described in this specification.
[0060] Figure 4 This is a schematic diagram of one search path provided in this specification;
[0061] Figure 5 This is a schematic diagram of a page where a product category selection pop-up window appears, as provided in this manual.
[0062] Figure 6 This is a schematic diagram of an APP page provided in this description;
[0063] Figure 7 A schematic diagram of a device for intelligent page search provided in this specification;
[0064] Figure 8 The corresponding information provided in this specification Figure 1 A schematic diagram of an electronic device. Detailed Implementation
[0065] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.
[0066] The embodiments of this specification provide a method, apparatus, storage medium, and electronic device for intelligent page search. The technical solutions provided by the embodiments of this specification are described in detail below with reference to the accompanying drawings.
[0067] Figure 1 This is a flowchart illustrating a method for intelligent page search provided in this specification, which specifically includes the following steps:
[0068] S100: Obtain the image corresponding to the current page and identify the controls in the image.
[0069] The device used for page searching acquires the image corresponding to the current page and identifies controls within the image. This device can be a page search system, which includes at least one server and one terminal. The terminal can be a mobile phone or emulator capable of running the app. For clarity, the following explanation uses a system consisting of one server and one terminal as the execution entity. The current page is the page currently searched when performing a page search on the app, and it is displayed on the terminal. The current page can be any page within the app. Controls are all elements contained on the page, such as text, input boxes, images, and buttons.
[0070] Specifically, the terminal sends the image of the currently searched and displayed page to the server. After receiving the image corresponding to the current page from the terminal, the server identifies the controls in the image. The terminal can obtain the image of the current page by taking a screenshot and sending it to the server, or it can obtain the image of the current page by scanning and sending it to the server. This specification does not specify the exact method by which the terminal obtains the image of the current page. When identifying controls in the image, a pre-stored control detection algorithm can be used. This control detection algorithm can be a pre-defined rule algorithm or a pre-trained machine learning model; this specification does not specify the exact method. For example, suppose that when searching for pages in an app, the currently searched and displayed page on the terminal is page 1, i.e., the current page is page 1. The controls in image 1 corresponding to page 1 are identified, and the identified controls are assumed to be control A, control B, and control C.
[0071] When using control detection algorithms to identify controls in an image, errors or over-identification may occur. For example, icons related to mobile phone functions at the top of the image might be identified. Therefore, the server can use a pre-defined strategy to filter the identified controls. For instance, if a terminal obtains an image of the current page by taking a screenshot, such as... Figure 2 As shown, Figure 2 This is a schematic diagram of an image corresponding to the current page provided in this specification. The top of the image may contain controls such as the phone's remaining battery power, time, phone signal strength, and notification information. Figure 2 The controls marked with a black box will be identified during image recognition. Therefore, the server can filter these identified icons related to mobile phone functions, rather than controls on the APP page, according to a pre-set strategy.
[0072] Because control detection algorithms identify all controls in an image, the number of identified controls is enormous, resulting in a very large search space when searching for these controls. To reduce the page search space, the server can merge the identified controls according to pre-defined rules. Specifically, when there is a high degree of overlap between identified controls, the overlapping parts of the controls can be used as the merging result, and the type of the control with the higher priority among the highly overlapping controls can be used as the type of the merged control. For example... Figure 2Control 1 and Control 2 in the image are two highly overlapping controls, with Control 1 having a higher priority than Control 2. The intersection of these two controls can be considered the result of merging them. Figure 2 In the "Control 2" section, the type of "Control 1" is used as the type of the merged control. The pre-defined priority of controls can be set according to their type; for example, the priority order can be set to: button, input box, image, and text. When a containment relationship is identified between controls, and the contained control has a lower priority than the containing control, the server can merge the containment controls and use the type of the control with the higher priority as the type of the merged control. For example... Figure 2 Control 3 contains Control 4, and Control 3 has a higher priority than Control 4. The server can merge Control 3 and Control 4 according to pre-set rules and use the type of Control 3 as the type of the merged control.
[0073] For the identified text, the server can merge multiple lines of text with each line spaced less than a preset height, for example... Figure 2 Text 4, Text 5, Text 6, and Text 7 in the text are examples of text where the distance between each line is less than a preset height. The server can merge these three lines of text. The server can also merge text controls where each line has the same font height and the distance between controls is less than a preset distance, for example... Figure 2 In the example text 5 and text 6, the two text controls have the same font height and the distance between them is less than a preset distance. The server can merge text 5 and text 6. The server can also merge identified controls based on their layout relationships; for example, a bottom navigation bar typically consists of an icon and a piece of text. Figure 2 The bottom navigation bar consists of icons 1-3 and text 1-3. The recognized icons and text can be merged into a single control. In other words, icon 1 and text 1 can be merged into a single control.
[0074] When a server recognizes controls in an image, inaccurate recognition may occur, especially when recognizing text. The recognized text control may not contain all of the text content, resulting in omissions. To address this, the server can use a pre-stored text recognition algorithm to identify the text in the image. Based on the recognized text information, it can determine whether the text control contains all the text content. If the text control does not contain all the text content, it can correct the text content based on the identified information. For example... Figure 2In the text 8, the text control marked by the black box recognized by the server only contains the text content of the part "文" and does not contain the entire text content, that is, "文本8". Therefore, the text recognition algorithm can be used to perform text recognition on the image, and according to the recognized text information, the text content in the recognized text control can be corrected.
[0075] S102: For each recognized control, generate an operation instruction according to the control.
[0076] For each recognized control, the server generates an operation instruction according to the control. Among them, the operation instruction is an instruction used to operate the control. Specifically, the server can directly generate an operation instruction for each recognized control according to the control, without prior sorting of each control, and can randomly generate the operation instruction corresponding to each control. Continuing with the above example, for the three controls (i.e., control A, control B, and control C) on page 1, assuming that control A is a button, the operation instruction 1 generated according to control A is a click instruction to click on control A on page 1, and the page can be switched by clicking on control A on page 1.
[0077] The server can also sort each control according to the number of preset keywords included in each control, and in the order of sorting, sequentially generate an operation instruction for each control according to the control. Continuing with the above example, assuming that the preset keyword is "会员", the number of "会员" in control A on page 1 is 3, the number of "会员" in control B is 1, and the number of "会员" in control C is 2. According to the number of "会员" included in the three controls of control A, control B, and control C, sort control A, control B, and control C. The sorting result is control A, control C, control B. In the order of sorting, sequentially generate an operation instruction for each control (i.e., control A, control C, and control B). The operation instruction 1 generated according to control A is a click instruction to click on control A on page 1, and the page can be switched by clicking on control A on page 1.
[0078] The server can also sort each control according to the preset priority of the control, and in the order of sorting, sequentially generate an operation instruction for each control according to the control. Continuing with the above example, assuming that the three controls on page 1 are sorted according to the preset priority of the control, the sorting result is control A, control B, control C. In the order of sorting, sequentially generate an operation instruction for each control (i.e., control A, control B, and control C). The operation instruction 1 generated according to control A is a click instruction to click on control A on page 1, and the page can be switched by clicking on control A on page 1.
[0079] S104: Switch the current page according to the operation instruction.
[0080] The server sends the operation instruction generated in step S102 to the terminal. The terminal switches the current page according to the operation instruction. Continuing with the previous example, the terminal clicks control A on page 1 to switch pages according to operation instruction 1 (i.e. click instruction) sent by the server. Assume that after clicking control A on page 1, page 1 switches to page 2.
[0081] S106: Based on the preset target features, determine whether the switched page matches the target features. If yes, proceed to step S108; otherwise, proceed to step S110.
[0082] The terminal sends the image corresponding to the switched page to the server. The server determines whether the switched page matches the target features based on preset target features. If yes, step S108 is executed; otherwise, step S110 is executed. The target features are pre-defined characteristics of the target page, which can be text or images. For example, for a shopping app, if the target page to be searched is the payment page, the pre-defined target features could be text such as "Please select payment tool" or an icon of a payment tool. This manual does not impose specific limitations, as long as the preset target features can be used to determine whether the switched page is the target page.
[0083] When determining whether the switched page matches the preset target features, a control detection algorithm can be used to identify the image corresponding to the switched page. Based on the identified controls and the preset target features, it is determined whether the identified controls contain the target features. If the identified controls contain the target features, the switched page matches the target features; if none of the identified controls contain the target features, the switched page does not match the target features. When the target feature is text, a pre-stored text recognition algorithm can also be used to recognize text in the image corresponding to the switched page. If the recognized text information contains text features, the switched page matches the target features; otherwise, the switched page does not match the target features.
[0084] S108: Determine that the switched page is the target page.
[0085] When the switched page matches the target features, the server determines that the switched page is the target page, and the page search for the APP ends.
[0086] S110: Re-establish the switched page as the current page and return to step S100.
[0087] If the switched page does not match the target characteristics, the server will re-determine the switched page as the current page and continue to switch pages based on the re-determined current page. That is, it returns to step S100 and repeats the above steps S100 to S110 until the switched page is the target page.
[0088] Continuing with the previous example, assuming the preset target feature is the text feature of "Please select payment tool", the terminal sends image 2 corresponding to page 2 to the server. The server determines whether page 2 matches the target feature based on the preset target feature. If page 2 matches the target feature, step S108 is executed to determine that page 2 is the target page, and the page search for the APP ends. Otherwise, step S110 is executed to re-determine page 2 as the current page, and page switching continues based on the re-determined current page (i.e., page 2), that is, returning to step S100 and repeating the above steps S100 to S110 until the switched page is the target page.
[0089] When determining whether page 2 matches the target features based on the preset target features, a control detection algorithm can be used to identify the controls in image 2. Assuming that after identifying image 2, the identified controls are controls D, E, and F, it is determined whether controls D, E, and F contain the target features based on the target features. Assuming that control D contains the target feature "Please select payment tool", then page 2 matches the target features.
[0090] As can be seen from the above method, this method does not require obtaining the page code or manually inputting search commands. The server can identify the controls in the image corresponding to the current page sent by the terminal, generate operation commands based on the identified controls, and send the operation commands to the terminal. The terminal switches pages according to the operation commands, and then judges whether the switched page matches the target features. If so, the switched page is the target page to be searched, and the page search for the APP ends. Otherwise, the switched page is re-determined as the current page, and the page switching continues according to the re-determined current page until the switched page is the target page. The method of generating operation commands by the server and sending them to the terminal for execution can perform page searches for the APP more intelligently, improve search efficiency, and reduce search costs.
[0091] When performing intelligent page search, the image corresponding to the current page may be an image that the server is recognizing for the first time, or it may be an image that the server has already recognized. When the image corresponding to the current page is an image that the server has already recognized, some of the recognized controls may have already generated operation instructions and sent them to the terminal. The terminal has already used these operation instructions to switch pages. For these already operated controls, the server does not need to generate corresponding operation instructions again. The server should generate corresponding operation instructions based on the controls that have not been operated.
[0092] Therefore, in step S104 above, after the terminal switches the current page according to the operation instruction sent by the server, the server updates the control corresponding to the operation instruction to an operated control. An operated control is a control for which the server has already generated an operation instruction and sent it to the terminal, and which the terminal has already switched pages according to the operation instruction. Therefore, in step S102 above, when the server generates an operation instruction for each identified control, it can also generate an operation instruction for each identified unoperated control. An unoperated control is a control for which the server has not yet generated an operation instruction and sent it to the terminal. Continuing with the example above, for controls A, B, and C on page 1, since control A has already been operated by the terminal in step S104, control A is updated to an operated control. Controls B and C are unoperated controls. Therefore, in step S102 above, the server can, for each identified unoperated control (i.e., control B and control C), assuming an operation instruction 2 is generated based on control B, send the operation instruction 2 generated based on control B to the terminal.
[0093] In step S106 above, when all controls are already operated and the current page does not match the target characteristics, the server generates a first rollback instruction. The terminal performs a rollback operation on the current page according to the first rollback instruction, obtaining a rollback page. The server determines whether the rollback page contains any unoperated controls. If so, the rollback page is redefined as the current page; otherwise, the terminal continues to perform a rollback operation on the rollback page according to the first rollback instruction until the rollback page contains any unoperated controls. The first rollback instruction is the operation instruction for performing page rollback. Specifically, when all controls are already manipulated and the current page does not match the target features, the server can generate a first rollback instruction and send it to the terminal. The terminal performs a rollback operation on the current page according to the first rollback instruction to obtain a rollback page. The server determines whether the rollback page contains unmanaged controls based on the image corresponding to the rollback page sent by the terminal. If so, the rollback page is redefined as the current page, and steps S100 to S110 are repeated. Otherwise, the terminal continues to perform a rollback operation on the rollback page according to the first rollback instruction until the server determines that the rollback page contains unmanaged controls.
[0094] Continuing with the previous example, suppose the current page is page 2. Page 2 does not match the target features, and controls D, E, and F in page 2 are all manipulated controls. The server generates a first rollback instruction. The terminal performs a rollback operation on page 2 according to the first rollback instruction sent by the server, and obtains the rollback page. The server determines whether the rollback page contains unmanaged controls based on the image corresponding to the rollback page sent by the terminal. Suppose the rollback page is page 1. Page 1 contains unmanaged controls B and C. The rollback page (i.e., page 1) can be redefined as the current page.
[0095] This method, which involves backtracking when all controls are already manipulated and the current page does not match the target characteristics, and then continuing the search for other pages, is similar to Depth-First Search (DFS). DFS searches downwards according to pre-defined conditions or randomly, aiming to find the target node or search all nodes. However, when the target node cannot be found and further downward searching is impossible, it backtracks to the previous node to search other paths until the target node is found or all nodes have been searched. For example... Figure 3 As shown, Figure 3 This specification describes a graph undergoing a depth-first search, assuming that all nodes in the graph need to be searched. Figure 3Node 1 is the currently searched node. A node is randomly selected from nodes 2 and 3 connected to node 1 for further searching (let's say node 2). Then, a node is selected from nodes 4 and 5 connected to node 2 for further searching (let's say node 5). When node 5 is found, since there are no nodes connected to it, the search cannot continue downwards and all nodes in the graph have not yet been searched. Therefore, it is necessary to backtrack to the node preceding node 5, which is node 2, and then search the other nodes connected to node 2 until all nodes in the graph have been searched.
[0096] For different types of apps, when going back to the current page based on the first back button, the resulting back page might be the page before the current page was switched, another page within the app, or even the app itself. In these cases, when re-identifying the back page as the current page, the server can determine the page similarity between the current page and the page before the switch. Based on the page similarity, it determines whether the back page is the page before the switch. If the back page is the page before the switch, it is re-identified as the current page. Specifically, the server can determine the structural similarity index Measure (SSIM) between the image corresponding to the current page and the image corresponding to the back page, using this as the page similarity. Based on the determined page similarity, it determines whether the back page is the page before the switch. If the back page is the page before the switch, it is re-identified as the current page. Continuing with the previous example, the page before switching to page 2 is page 1. Determine the structural similarity between the image corresponding to page 1 and the image corresponding to the back page, and use it as the page similarity. Based on the determined page similarity, determine whether the back page is page 1. Since it is assumed that the back page of page 2 is page 1, the back page is page 1, and page 1 can be directly re-determined as the current page.
[0097] Since there might be a situation where the page to be returned to is not the page before the current page, after generating the operation command in step S102, the server stores the operation command in the command sequence. When the page to be returned to in step S106 is not the page before the current page, the server switches the page to the page before the current page according to the command sequence. Specifically, after generating the operation command in step S102, the server stores the operation command in the command sequence. When the page to be returned to in step S106 is not the page before the current page, the server can determine the operation command required to switch from the page to the page before the current page according to the command sequence, so that the terminal can switch the page step by step from the page to the page before the current page according to the determined operation command. Continuing with the example above, after generating operation command 1 in step S102, the generated operation command 1 is stored in the command sequence. Assuming that when returning to page 2, the page to be returned to is the initial page of the APP, and the page before the current page is page 1, the search process from the initial page to page 2 is as follows. Figure 4 As shown, Figure 4 This is a schematic diagram of a search path provided in this specification. According to the instruction sequence, the operation instructions required to switch from the initial page to page 1 are determined. It is assumed that it takes operation instructions 3 to 5 to switch from the initial page to page 1. The terminal switches to page 1 step by step from the initial page according to the determined operation instructions 3 to 5.
[0098] In step S100 above, after acquiring the image corresponding to the current page, the server can input the image into a pre-trained image classification model to determine whether the current page has any anomalies. If an anomaly exists, the server determines the type of the anomaly and processes it according to preset anomaly handling instructions corresponding to the type. The image classification model is a model capable of determining whether an anomaly exists in the input image. There are many types of anomalies, such as pop-ups, lag, and incomplete page loading. For ease of explanation, the following description uses pop-ups as an example. Specifically, after acquiring the image corresponding to the current page, the server can input the image into the pre-trained image classification model. Based on the output of the image classification model, the server determines whether the current page has any anomalies. When an anomaly exists, the server determines the type of the anomaly. Based on the type of the anomaly, the server selects the corresponding anomaly handling instruction from preset anomaly handling instructions and sends it to the terminal. The terminal then processes the anomaly according to the anomaly handling instruction. Continuing with the previous example, suppose page 1 has an anomaly 1. After the server obtains the image corresponding to page 1, it can input the image corresponding to page 1 into a pre-trained image classification model. Based on the output of the image classification model, it determines that page 1 has an anomaly 1. Based on the type of anomaly 1, it determines the anomaly handling instruction 1 corresponding to the type of anomaly 1 from the preset anomaly handling instructions and sends it to the terminal. The terminal processes anomaly 1 according to the anomaly handling instruction 1.
[0099] Because there are many types of pop-ups, such as update pop-ups, marketing pop-ups, or product category selection pop-ups, and the design of pop-ups may differ between different apps, we have preset exception handling instructions for different types of pop-ups, and also preset different exception handling instructions for pop-ups of the same type. For example, if the exception type on the current page is a product category selection pop-up, such as... Figure 5 As shown, Figure 5 This is a schematic diagram of a page displaying a product category selection pop-up window, as provided in this instruction manual. Figure 5 In the product category selection pop-up, each row of options is aligned along the y-axis, and each column of options is aligned along the x-axis. For this type of pop-up, the exception handling command could be to click the last option in each row of options aligned along the y-axis. The selected option would then be... Figure 5 Options 3, 5, and 6 are highlighted in black boxes. The exception handling command can also randomly click on one option in each row aligned along the y-axis. The option clicked according to the exception handling command can be... Figure 5 Options 1, 4, and 6 are highlighted in the middle.
[0100] If no exception handling instruction corresponding to the type is found, the server can generate a second rollback instruction. The terminal, based on this instruction, rolls back the current page to its state before the exception occurred. Specifically, the second rollback instruction rolls back the current page to its state before the exception occurred. The server generates the second rollback instruction and sends it to the terminal. The terminal, based on this instruction, rolls back the current page to obtain the exception rollback page. It then determines whether the exception rollback page contains an exception. If it does, it means that rolling back the current page did not return it to the state where the exception occurred; the exception was not handled, and the current page cannot be searched further. The server then generates the first rollback instruction and sends it to the terminal, causing the terminal to roll back the current page. The subsequent page search process is similar to the rollback process in step S106 above, and will not be repeated here. Otherwise, the exception rollback page is determined to be the state of the current page before the exception occurred.
[0101] Continuing with the previous example Figure 5 Assuming Figure 5 This is a schematic diagram of the page after anomaly 1 occurs on page 1. Based on the type of anomaly 1 on page 1 (i.e., a product category selection pop-up), no corresponding anomaly handling instruction is found in the preset anomaly handling instructions. The server generates a second rollback instruction and sends it to the terminal. The terminal performs a rollback operation on page 1 according to the second rollback instruction, obtaining the anomaly rollback page. It then determines whether anomaly 1 exists on the anomaly rollback page. If so, a first rollback instruction is generated, and page 1 is rolled back according to the first rollback instruction. The subsequent page search process is similar to the example of rolling back the current page in step S106 above. Otherwise, the anomaly rollback page is determined to be page 1 before anomaly 1 occurred. Figure 6 As shown, Figure 6 This is a schematic diagram of an APP page provided in this description.
[0102] When determining whether there are anomalies on the current page by examining the corresponding image, the server can use a control detection algorithm to determine whether there are anomalies based on the identified controls. Alternatively, the server can also determine whether there are anomalies based on image features obtained from a pre-trained HSV (Hue, Saturation, Value) color model.
[0103] After switching the current page in step S104 above, the server can also determine whether the switched page matches the road sign features based on preset road sign features. If the switched page matches the road sign features, all controls in the previous page corresponding to the switched page are updated to operated controls. The road sign features can be set keywords or key controls, and there is a correlation between the road sign features and the target features. For example, if the target feature is "Please select payment tool," then the road sign feature could be "shopping cart." This specification does not specify a particular limitation. The method for determining whether the switched page matches the road sign features is similar to the method for determining whether the switched page matches the target features in step S106 above, and will not be repeated here. Specifically, the server can also determine whether the switched page matches the road sign features based on preset road sign features. If so, the switched page is determined to be the road sign page, and all controls in the previous page corresponding to the switched page are updated to operated controls. That is, the APP performs a page search with the switched page as the new starting page, and pages that have been searched before the switched page are not searched again.
[0104] Continuing with the example in step S106 above, assuming the preset signpost feature is "shopping cart", the server determines whether page 2 matches the signpost feature based on the preset signpost feature. If page 2 matches the signpost feature, then page 2 is the signpost page. The controls (i.e., controls A, B, and C) in the page before the switch (i.e., page 1) corresponding to page 2 are updated to the operated controls. Page 2 is the new starting page for page search. Pages that have been searched before page 2, such as page 1, will no longer be explored.
[0105] However, when starting the page search from the road sign page and not searching pages before the road sign page, there may be cases where no page matching the target features is found, that is, the target page cannot be found. Therefore, the server can preset the number of searches n. When searching the page of the APP, if the number of searches reaches the preset number of searches n and no page matching the target features is found, that is, the target page, it means that the search has failed and the page search of the APP needs to be performed again, where n is a positive integer.
[0106] based on Figure 1The page intelligent search method shown in this specification also provides a page intelligent search method. In this method, the server can perform page search on the APP according to a pre-set search path until a complete page search path is found. Continuing with the above steps S100 to S104, in step S106, based on the preset characteristics of the node page, it is determined whether the characteristics of the switched page match those of the node page. If so, the switched page is determined to be a node page, and the node page is re-determined as the current page. Page switching continues based on the re-determined current page until the current page is the end node page. Otherwise, the switched page is re-determined as the current page, and page switching continues based on the re-determined current page until the current page is a node page. Specifically, the server determines whether the characteristics of the switched page match those of the preset node page. If they match, the switched page is identified as the node page, and the current page is redefined. Page switching continues based on this redefined current page until the current page is the next node page in the preset search path. Otherwise, the switched page is redefined as the current page, and page switching continues until the current page is a node page. Once the current page is the next node page in the preset search path, the page search continues, following the same method, until the current page is the end node page.
[0107] For example, if the pre-defined search path for an app is page 1, page 2, and page 3, where pages 1 and 2 are node pages and page 3 is the ending node page, when searching for pages in the app, the server determines whether the characteristics of the switched page match those of the node page based on the pre-defined characteristics of page 1. If they match, the switched page is determined to be the node page, i.e., page 1, and page 1 is redefined as the current page. The page switching continues based on page 1 until the current page is the next node page in the pre-defined search path, i.e., page 2. Otherwise, the switched page is redefined as the current page, and the page switching continues based on the redefined current page until the current page is the node page, i.e., page 1. After the current page is page 2, the page search continues according to the above method until the current page is the ending node page, i.e., page 3.
[0108] Regarding the above method of searching for pages in an app based on a pre-defined search path, node pages can be used as landmark pages. Correspondingly, the server-preset landmark features are the features possessed by landmark pages, i.e., the features of node pages. These landmark features can be text features or control features of the landmark page, or simply the landmark page itself. This specification does not specify a particular feature, as long as the landmark features can determine whether the switched page is a landmark page.
[0109] When the signpost feature is a signpost page, the server can determine whether the switched page is a signpost page based on the similarity between the switched page and the previous page. The specific process is the same as described above. Figure 1 The process of determining whether the page to go back to is the same as the page before the switch is similar in the method shown, so it will not be described again here.
[0110] exist Figure 1 Based on the intelligent page search method described above, this specification provides another intelligent page search method. This method searches all pages of the APP as comprehensively as possible, rather than stopping the search once a target page is found. Therefore, in step S106 above, the server should determine whether the switched page contains any unoperated controls. If so, the switched page is redefined as the current page, and page switching continues based on the redefined current page until the search end condition is met. Otherwise, a first rollback instruction is generated to roll back the switched page, and the rolled-back page is redefined as the switched page. The server then continues to determine whether the redefined switched page contains any unoperated controls. Specifically, if all controls in the searched pages are operated controls and there are no unoperated controls to search for, then the page search for the APP is considered complete. Alternatively, a preset search count n can be used. When the preset search count n is reached, the page search for the APP is considered complete, where n is a positive integer.
[0111] This specification can also include multiple terminals. For each identified control, the server generates an operation instruction based on that control and sends the generated operation instruction to different terminals so that different terminals can operate simultaneously. This method can improve the efficiency of page search. For example, for controls A, B, and C on page 1, the operation instruction generated based on control A can be sent to terminal 1, the operation instruction generated based on control B can be sent to terminal 2, and the operation instruction generated based on control C can be sent to terminal 3, so that terminals 1 to 3 can operate simultaneously according to the corresponding operation instructions.
[0112] This manual describes a system that can be implemented using a server and a terminal. Figure 1 The intelligent page search method shown can also be implemented using the same device. Figure 1 The page intelligent search method shown is, for example, implemented solely by the server. Figure 1 The page intelligent search method shown is implemented solely by the server. Figure 1When performing the intelligent page search method shown, it is not necessary to obtain the image corresponding to the current page and then identify the controls in the image. The server can directly analyze the page code of the APP to determine the controls in the page. The subsequent process is similar to the above steps S102 to S110.
[0113] After the server performs an intelligent page search on the app to obtain the target page, it can determine whether the target page is legal according to preset inspection rules. If it is, the target page is determined to be a legal page; otherwise, based on the pre-saved app developer information, the server determines the identifier of the app developer corresponding to the target page and sends a prompt message to the app developer. This prompt message carries information about the illegal parts of the target page, prompting the app developer to improve the target page accordingly. The inspection rules can be rules set according to laws and regulations or rules set according to pre-signed relevant provisions. Specifically, the server can check the target page according to the preset inspection rules to determine whether the target page is legal, that is, to determine whether there are any parts of the target page that do not comply with relevant regulations or laws and regulations. If there are no parts of the target page that do not comply with relevant regulations or laws and regulations, then the target page is a legal page. Otherwise, the server can determine the identifier of the app developer corresponding to the target page from the pre-saved app developer information and send a prompt message to the app developer corresponding to the identifier, so that the app developer can improve the target page for any parts that do not comply with relevant regulations or laws and regulations. For example, if the target page is a payment page, the preset inspection rules are set according to the agreement signed between the app developer and the payment tool provider. This means the payment tool provided by the provider is set as the default tool. Therefore, the payment page can be checked according to these rules to determine if the payment tool is the default tool. If it is, the payment page complies with the relevant regulations; otherwise, it does not. The server uses pre-saved app developer information to determine the app developer's identifier for the payment page. Based on this identifier, the server sends a message to the app developer stating "Payment tool not set as default tool," prompting the app developer to improve the payment page accordingly.
[0114] The above describes one or more embodiments of the intelligent page search method provided in this specification. Based on the same idea, this specification also provides a corresponding intelligent page search device, such as... Figure 7 As shown.
[0115] Figure 7This specification provides a schematic diagram of a device for intelligent page search, specifically including:
[0116] The parsing module 200 is used to obtain the image corresponding to the current page and identify the controls in the image;
[0117] Instruction module 202 is used to generate operation instructions for each identified control based on that control;
[0118] The switching module 204 is used to switch the current page according to the operation instruction;
[0119] The determination module 206 is used to determine whether the switched page matches the preset target features. If so, the switched page is determined to be the target page. Otherwise, the switched page is re-determined as the current page, and the page switching continues according to the re-determined current page until the switched page is the target page.
[0120] Optionally, the instruction module 202 is specifically used to sort each control according to the number of preset keywords contained in each control; and generate operation instructions for each control in turn according to the sorting order.
[0121] Optionally, after switching the current page according to the operation instruction, the switching module 204 is further configured to update the control to an operated control;
[0122] The instruction module 202 is specifically used to generate an operation instruction for each identified non-operated control;
[0123] The device further includes:
[0124] The rollback module 208 is used to generate a first rollback instruction when all controls are the operated controls and the current page does not match the target feature; according to the first rollback instruction, the current page is rolled back to obtain a rollback page; it is determined whether the rollback page contains unoperated controls; if so, the rollback page is re-determined as the current page; otherwise, the rollback operation continues according to the first rollback instruction until the rollback page contains unoperated controls.
[0125] Optionally, the fallback module 208 is specifically used to: determine the page similarity between the page before the switch and the fallback page corresponding to the current page; determine whether the fallback page is the page before the switch based on the page similarity; and when the fallback page is the page before the switch, re-determine the fallback page as the current page.
[0126] Optionally, after generating the operation instruction, the instruction module 202 is further configured to store the operation instruction into an instruction sequence;
[0127] When the fallback page is not the page before the switch, the fallback module 208 is further configured to switch the fallback page to the page before the switch according to the instruction sequence.
[0128] Optionally, the rollback module 208 is specifically used to: determine the operation instructions required to switch from the rollback page to the page before the switch based on the instruction sequence; and switch the rollback page to the page before the switch based on the determined operation instructions.
[0129] Optionally, after the parsing module 200 obtains the image corresponding to the current page, the device further includes:
[0130] The anomaly handling module 210 is used to input the image into a pre-trained image classification model to determine whether there is an anomaly on the current page through the image classification model; when there is an anomaly on the current page, the type of the anomaly is determined, and the anomaly is processed according to a preset anomaly handling instruction corresponding to the type.
[0131] Optionally, when no exception handling instruction corresponding to the type is found, the exception handling module 210 is further configured to generate a second rollback instruction; and according to the second rollback instruction, perform a rollback operation on the current page to return the current page to the state before the exception occurred.
[0132] Optionally, after switching the current page according to the operation instruction, the switching module 204 is further configured to update the control to an operated control;
[0133] The instruction module 202 is specifically used to generate an operation instruction for each identified non-operated control;
[0134] After switching the current page, the determining module 206 is further configured to update all controls in the previous page corresponding to the switched page to the operated controls when the switched page has preset signpost features.
[0135] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The page shown illustrates the method for intelligent search.
[0136] This instruction manual also provides Figure 8 The diagram shows a schematic structural representation of the electronic device. Figure 8At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The method for intelligent page search is shown. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.
[0137] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.
[0138] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, ASICs, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.
[0139] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.
[0140] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.
[0141] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0142] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.
[0143] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.
[0144] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes The steps of the function specified in one or more boxes.
[0145] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.
[0146] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.
[0147] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.
[0148] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.
[0149] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.
[0150] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.
[0151] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.
[0152] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.
Claims
1. A method for intelligent search of a page, comprising: obtaining an image corresponding to a current page and identifying a control in the image; generating an operation instruction according to each identified control, wherein the controls are multiple, and the operation instruction is generated according to each control in turn; switching the current page according to the operation instruction; determining whether the switched page matches a target feature according to the target feature, and if yes, determining that the switched page is a target page, and if not, re-determining the switched page as the current page and continuing to switch the page according to the re-determined current page until the switched page is the target page. 2.The method of claim 1, wherein generating the operation instruction according to each identified control comprises: sorting the controls according to a number of preset keywords contained in each control; and generating the operation instruction according to each control in turn according to the sorting order. 3.The method of claim 1, wherein after switching the current page according to the operation instruction, the method further comprises: updating the control as an operated control; and generating the operation instruction according to each identified control comprises: generating the operation instruction according to each un-operated control; and when all the controls are the operated controls and the current page does not match the target feature, the method further comprises: generating a first rollback instruction; and performing a rollback operation on the current page according to the first rollback instruction to obtain a rollback page, determining whether the rollback page contains an un-operated control, and if yes, re-determining the rollback page as the current page, and if not, continuing to perform the rollback operation on the rollback page according to the first rollback instruction until the rollback page contains the un-operated control. 4.The method of claim 3, wherein re-determining the rollback page as the current page comprises: determining a page similarity between a page before switching corresponding to the current page and the rollback page; determining whether the rollback page is the page before switching according to the page similarity; and re-determining the rollback page as the current page when the rollback page is the page before switching. 5.The method of claim 4, wherein after generating the operation instruction, the method further comprises: storing the operation instruction in an instruction sequence; and when the rollback page is not the page before switching, the method further comprises: switching the rollback page to the page before switching according to the instruction sequence. 6.The method of claim 5, wherein switching the rollback page to the page before switching according to the instruction sequence comprises: determining operation instructions required for switching the rollback page to the page before switching according to the instruction sequence; and switching the rollback page to the page before switching according to the determined operation instructions. 7.The method of claim 1, wherein after obtaining the image corresponding to the current page, the method further comprises: inputting the image into a pre-trained image classification model to determine whether the current page has an abnormality through the image classification model; when the current page has an abnormality, determining a type of the abnormality existing in the current page, and processing the abnormality according to a preset abnormality processing instruction corresponding to the type.
8. The method of claim 7, when the abnormality processing instruction corresponding to the type is not found, the method further comprises: generating a second rollback instruction; performing a rollback operation on the current page according to the second rollback instruction, so that the current page is rolled back to a state before the abnormality occurs.
9. The method of claim 1, after performing page switching on the current page according to the operation instruction, the method further comprises: updating the control to an operated control; for each identified control, generating an operation instruction according to the control, specifically comprising: for each identified unoperated control, generating an operation instruction according to the unoperated control; after performing page switching on the current page, the method further comprises: when the switched page has a preset landmark feature, updating each control of the page before switching corresponding to the switched page to an operated control.
10. An apparatus for intelligent page searching, comprising: a parsing module configured to obtain an image corresponding to a current page and identify controls in the image; an instruction module configured to generate an operation instruction according to each identified control; wherein the controls are multiple, and for each control, an operation instruction is generated according to the control; a switching module configured to perform page switching on the current page according to the operation instruction; a determination module configured to determine whether a switched page matches a preset target feature, if yes, determine that the switched page is a target page, otherwise, determine the switched page as a current page again, and continue to perform page switching according to the current page determined again until the switched page is the target page.
11. A computer readable storage medium, the storage medium storing a computer program, the computer program being executed by a processor to implement the method of any one of claims 1-9.
12. An electronic device, comprising a memory, a processor, and a computer program stored on the memory and executable on the processor, the processor executing the program to implement the method of any one of claims 1-9.
Citation Information
Patent Citations
Page analysis method and device
CN111460355A
Page skip test method and device, equipment and storage medium
CN112256562A