Application program traversal method, device, electronic device and storage medium

Abstract

The present disclosure discloses an application traversal method, device, electronic device and storage medium, and relates to the field of artificial intelligence and application detection. The specific implementation scheme is: identify the to-be-processed page of the target application, and obtain the functional information of the to-be-processed page; if it is determined based on the functional information that the function of the to-be-processed page does not belong to the traversed function, detect the to-be-processed page; in response to completing the detection of the to-be-processed page, mark the function of the to-be-processed page as a traversed function, and traverse at least one sub-page of the to-be-processed page, and determine the traversed sub-page as a new to-be-processed page. The embodiment of the present disclosure can improve the traversal efficiency.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to the field of artificial intelligence and application detection. Background Art

[0002] In the process of detecting an application (Application, App), it is necessary to traverse the application pages as much as possible to detect the pages. Generally, the traversal of the App pages is based on the page coverage as the traversal target, and a comprehensive recursive traversal is performed. Summary of the invention

[0003] The present disclosure provides a method, device, electronic device and storage medium for traversing an application program.

[0004] According to one aspect of the present disclosure, a method for traversing an application is provided, comprising:

[0005] Identify the to-be-processed page of the target application and obtain function information of the to-be-processed page;

[0006] In the case where it is determined based on the function information that the function of the page to be processed does not belong to the traversed functions, detecting the page to be processed;

[0007] In response to completing the detection of the page to be processed, marking the function of the page to be processed as a traversed function, traversing at least one subpage of the page to be processed, and determining the traversed subpage as a new page to be processed.

[0008] According to another aspect of the present disclosure, a traversal device for an application is provided, comprising:

[0009] A first identification module is used to identify the to-be-processed page of the target application program and obtain function information of the to-be-processed page;

[0010] A detection module, configured to detect the page to be processed when it is determined based on the function information that the function of the page to be processed does not belong to the traversed functions;

[0011] The recursive traversal module is used to mark the function of the page to be processed as a traversed function in response to completing the detection of the page to be processed, and traverse at least one subpage of the page to be processed, and determine the traversed subpage as a new page to be processed.

[0012] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0013] at least one processor; and

[0014] a memory communicatively connected to the at least one processor; wherein,

[0015] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method provided by the embodiment of the present disclosure.

[0016] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable a computer to execute the method provided by an embodiment of the present disclosure.

[0017] The embodiment of the present disclosure further provides a computer program product, including a computer program, which implements the above method when executed by a processor.

[0018] According to the technical solution disclosed in the present invention, for the pages that have been detected in the application, their functions will be marked as traversed functions; for the traversed pages to be processed, they will be identified and the functions of the pages will be determined. Only when the functions do not belong to the traversed functions, the sub-pages of the pages will be traversed. Based on this, the function coverage can be ensured during the traversal detection process of the application, and the number of traversals of pages with the same functions can be reduced, thereby improving the traversal efficiency.

[0019] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0021] Figure 1 is a schematic diagram of a traversal method of an application provided by an embodiment of the present disclosure;

[0022] Figure 2 is a schematic diagram of traversal in an embodiment of the present disclosure;

[0023] Figure 3 is a schematic diagram of control identification in an embodiment of the present disclosure;

[0024] Figure 4 is a schematic diagram of an application example of the traversal method of the disclosed application program;

[0025] Figure 5 is a schematic diagram of a recognition engine in an embodiment of the present disclosure;

[0026] Figure 6 is a schematic diagram of a traversal device for an application provided by an embodiment of the present disclosure;

[0027] Figure 7 is a schematic diagram of a traversal device for an application provided by another embodiment of the present disclosure;

[0028] Figure 8 The block diagram is a block diagram of an electronic device used to implement the application traversal method of the embodiment of the present disclosure. DETAILED DESCRIPTION

[0029] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0030] Figure 1 FIG. 1 is a schematic diagram showing a method for traversing an application program provided by an embodiment of the present disclosure. Figure 1 As shown, the method may include:

[0031] Step S11, identifying the to-be-processed page of the target application and obtaining function information of the to-be-processed page;

[0032] Step S12, when it is determined based on the function information that the function of the page to be processed does not belong to the traversed functions, the page to be processed is detected;

[0033] Step S13, when the detection of the page to be processed is completed, the function of the page to be processed is marked as a traversed function, and at least one subpage of the page to be processed is traversed, and the traversed subpage is determined as a new page to be processed.

[0034] Among them, the target application is the application (App) to be detected, such as social apps, shopping apps, video apps, etc.

[0035] For example, AI (Artificial Intelligence) technology can be used to obtain a recognition engine, and the page to be processed can be recognized based on the recognition engine. The recognition engine can be used to output corresponding function information for an input page.

[0036] Exemplarily, the function information may include identification information such as the function code and name. In practical applications, a traversed function list may be created, and when a function is marked as a traversed function, the corresponding function information is added to the traversed function list. For the identified function information, it may be determined whether the function belongs to the traversed function by judging whether the function information is in the traversed function list.

[0037] In the disclosed embodiment, when the function of the page to be processed does not belong to the traversed functions, the page to be processed is detected, and the sub-pages of the page to be processed are traversed after the detection is completed. Correspondingly, when the function of the page to be processed belongs to the traversed functions, the page to be processed may not be detected, and the traversal of the page to be processed is ended. That is to say, the traversal method of the above application may also include: when the page to be processed is not detected and the function of the page to be processed belongs to the traversed functions, the processing of the page to be processed is ended.

[0038] Based on this, during the traversal detection process of the application, for pages with the same function, only one detection and sub-page traversal is performed. Therefore, it is possible to avoid meaningless multiple detections and sub-page traversals for pages with the same function. For example, it is possible to avoid performing one-by-one detections and sub-page traversals on a large number of pages with the same function, such as a large number of product browsing pages in a shopping app or a large number of video pages in a video app. For another example, it is possible to avoid performing unlimited detections and sub-page traversals on pages that can be automatically refreshed dynamically.

[0039] Furthermore, since after the page to be processed is detected, its sub-pages will be traversed, therefore, according to the method of the embodiment of the present disclosure, pages with different functions can be fully traversed to ensure the functional coverage of the traversal detection of the application program.

[0040] For example, the target application may be a social app. Figure 2 As shown, first, the homepage of the social App is determined as the page to be processed. Since the homepage does not belong to the traversed function at this time, after the homepage is detected and processed, multiple sub-pages under the homepage are traversed, including the first conversation page, the second conversation page, and the settings page.

[0041] When traversing to the first conversation page, the first conversation page is used as a page to be processed. At this time, the function of the page to be processed is conversation. Since the conversation function does not belong to the traversed functions, after detecting and processing the first conversation page, traverse multiple subpages of the first conversation page, including the first subpage and the second subpage. If the functions of the first subpage and the second subpage do not belong to the traversed functions and neither has a subpage, when the detection of the first subpage is completed, the traversal process of the first subpage is completed, and then the second subpage is traversed; when the detection of the second subpage is completed, the traversal process of the second subpage is completed. At this point, the traversal process of the first conversation page is completed.

[0042] Then, the second conversation page is traversed. Since the function of the second conversation page is conversation, which is a traversed function, the second conversation page is not detected, and the subpages of the second conversation page are not traversed. The processing of the second conversation page is ended, and the setting page is traversed.

[0043] When traversing to the setting page, since the function of the setting page is setting, which is not a traversed function, the setting page is detected and its sub-pages are traversed.

[0044] By analogy, you can deeply recursively traverse various pages with different functions in the social app.

[0045] It can be seen that according to the technical solution disclosed in the present invention, for the pages that have been detected in the application, their functions will be marked as traversed functions; for the traversed pages to be processed, they will be identified and the functions of the pages will be determined. Only when the functions do not belong to the traversed functions, the sub-pages of the page will be traversed. Based on this, the function coverage can be ensured during the traversal detection process of the application, and the number of traversals of pages with the same functions can be reduced, thereby improving the traversal efficiency.

[0046] Exemplarily, in the above step S11, the to-be-processed page of the target application is identified to obtain the function information of the to-be-processed page, which may include:

[0047] Based on the image recognition module in the recognition engine, the image information in the page to be processed is recognized to obtain the image recognition result of the page to be processed;

[0048] Based on the natural language understanding module in the recognition engine, the text information in the page to be processed is recognized to obtain the semantic recognition result of the page to be processed;

[0049] Based on the image recognition result and the semantic recognition result of the page to be processed, the functional information of the page to be processed is obtained.

[0050] According to the above implementation, the recognition engine includes an image recognition module and a natural language understanding module. Therefore, the functional information of the page to be processed can be accurately determined by combining image recognition and semantic recognition. Among them, the image recognition module and the natural language processing module can both be deep learning models.

[0051] For example, an automated testing framework such as UiAutomator2 may be used to obtain screenshots of the page to be processed, image information such as control icons, and text information such as text in the control.

[0052] The image recognition module can obtain information such as the function category of the page to be processed and the possible functions of its sub-pages by identifying the entire screenshot of the page to be processed and the control icons, and output the image recognition result based on this information.

[0053] The natural language understanding module can understand the semantics of the text in the control by recognizing the text in the control, such as understanding the control function described by the text in the control, and output the semantic recognition result based on this.

[0054] Exemplarily, the image recognition result may include at least one candidate function and its corresponding probability score, etc.; the semantic recognition result may also include at least one candidate function and its corresponding probability score, etc. Based on the image recognition result, the semantic recognition result, and information such as pre-configured weights or priorities, one may be selected from the at least one candidate function as the final recognition result, i.e., the function information of the page to be processed.

[0055] By utilizing the capabilities of AI technology such as image recognition and natural language understanding, it is possible to accurately identify the functions of the page to be processed, thereby accurately maintaining the information of traversed functions, and accurately identifying untraversed functions and detecting their corresponding pages, thereby ensuring the functional coverage of traversal detection of the application.

[0056] In some implementations, the traversal method of the above application may further include:

[0057] Identify at least one control in the page to be processed and obtain a priority order of the at least one control;

[0058] Based on the priority order of at least one control, a traversal order of at least one sub-page corresponding to the at least one control of the page to be processed is obtained.

[0059] Accordingly, the at least one sub-page is traversed according to the traversal order.

[0060] Exemplarily, a recognition engine with image recognition capability and natural language understanding capability may also be used to recognize at least one control in the page to be processed. Figure 3 By identifying the control icons shown in the figure, it can be identified that the functions they represent are "Home", "Favorites", "Browse", "Messages" and "Personal Settings". The image recognition module can sort these functions based on their usage frequency, usage duration and other information, and output the sorting result, which is the priority order of these controls. The priority order of the controls is determined as the traversal order of the corresponding sub-pages. For example, Figure 3 The priority order of each control in is determined as: "Home", "Message", "Browse", "Personal Settings" and "Favorites". Based on this, the sub-page corresponding to the control "Home" is traversed first, and then the sub-page corresponding to the control "Message" is traversed, and so on, until the sub-pages corresponding to each space are traversed.

[0061] According to the above method, by identifying controls, important sub-pages can be traversed preferentially, so that important detection information can be obtained as quickly as possible, and the traversal procedure and effect can be optimized.

[0062] In practical applications, after determining the traversal order of at least one subpage, the subpages in the top N order can be traversed one by one, and the traversal of the subpages not in the top N order can be abandoned, where N is an integer greater than or equal to 1. Based on this, the efficiency of traversal detection can be improved.

[0063] As an exemplary implementation, the traversal method of the above application may further include:

[0064] When a pop-up window appears in the page to be processed, the pop-up window is identified, an operation method for closing the pop-up window is obtained, and the pop-up window is closed using the operation method.

[0065] Exemplarily, pop-ups may include advertisements, prompt information, and dialog boxes. Since the application is generally connected to the Internet during the detection process, and it is necessary to enter the page to be processed to detect it, pop-ups may appear. In the disclosed embodiment, the pop-up window can be identified to determine the operation method for closing the pop-up window. For example, the type of pop-up window can be identified first, such as an advertisement, prompt information, or dialog box; if the type of pop-up window is an advertisement, it can be identified as a full-screen advertisement or a countdown advertisement. Based on different pop-up window types, different operation methods can be determined and implemented, such as clicking the close button in the pop-up window page, clicking the "Agree" option in the dialog box, waiting for the countdown, etc.

[0066] Exemplarily, the above-mentioned recognition engine can also be used to recognize pop-up windows. For example, the dialog box that pops up when the App is running often contains a lot of text information, such as the dialogue text "Please read and agree to the "App Terms of Service" before using the App" and the option text "Agree" and "Disagree". Using the natural language understanding module in the recognition engine, the semantics can be understood, and it can be determined that only when "Agree" is clicked will the pending page continue to be displayed, so it is determined that the way to close the pop-up window is to click the "Agree" option.

[0067] Since the pop-up windows are automatically identified and processed, interruption of the traversal detection process of the application program due to the pop-up windows can be avoided, thereby improving the traversal detection efficiency.

[0068] Exemplarily, in some implementations, detecting the page to be processed may include:

[0069] Trigger the privacy behavior in the page to be processed and perform compliance detection on the privacy behavior.

[0070] For example, entering the pending page causes the application to perform privacy behaviors in the pending page, such as calling the camera to take pictures or scan codes, connecting to the network and calling the camera for live broadcasts, calling GPS (Global Positioning System) for map navigation or booking a taxi, calling payment software for online payments, etc.

[0071] In the disclosed embodiment, by triggering the privacy behavior in the page to be processed, the privacy behavior can be checked for compliance. Since the function coverage of the traversal detection of the application is high, the type coverage of the privacy behavior is also high, and a more comprehensive compliance detection can be performed to protect the personal privacy information in the App.

[0072] Figure 4 A schematic diagram showing an application example of the traversal method of the disclosed application program is shown. Figure 4 As shown, the execution subject first obtains the recognition object of the recognition engine. The recognition object can be a page to be processed or a pop-up window in the App. Then, a screenshot of the recognition object is extracted, and the control image and text in the recognition object are extracted. The screenshot of the recognition object, the spatial image and the text are input into the recognition engine. The recognition engine contains an image recognition module and a natural language understanding module, which perform image recognition and semantic understanding on the input information respectively, and output the recognition result.

[0073] If the identified object is a page to be processed, the function of the page to be processed is identified. If the function is a traversed function, the processing of the page to be processed is terminated. If the function is an untraversed function, the priority ranking of each control in the page to be processed is obtained, the traversal order of each sub-page is determined based on the priority ranking, and then the sub-pages are traversed based on the traversal order.

[0074] If the identified object is a pop-up window, an operation method for closing the pop-up window is found and the pop-up window is processed using the operation method.

[0075] Figure 5 Figure 2 shows a functional schematic diagram of the recognition engine. Figure 5 As shown, the recognition objects of the recognition engine may include pages to be processed and pop-up windows. The pages to be processed may include the homepage of the App and / or the business function pages that may invoke privacy behaviors. Pop-up windows may include dialog boxes and advertisements.

[0076] Among them, for the App homepage, the various business function types contained in the App and the main function entrances in the homepage can be identified, and the importance of the function entrances can be sorted. According to the various business function types contained in the App and the importance of the function entrances, a sorted function entrance list is obtained and output as the identification result. Here, the function entrance can be expressed in the form of a control in the page.

[0077] For business function pages that call for privacy behaviors, it is possible to identify function entrances involving privacy behaviors in the page, such as photo taking or code scanning functions, live broadcast functions, map navigation functions, taxi booking functions, online payment functions, etc., and sort these functions to obtain the function order, which is then output as the recognition result.

[0078] For dialog boxes, the type of dialog box can be identified, such as a permission application dialog box, a terms of use display box, an App upgrade dialog box, an App login dialog box, etc. For different dialog boxes, different operation methods and the information required to perform the operation method are determined, and they are output as recognition results. Among them, the operation method corresponding to the permission application dialog box and the terms of use display box can be clicking an agree button, and the recognition result can include the location information of the agree button; the operation method corresponding to the App upgrade dialog box can be clicking an agree or cancel button, and the recognition result can include the location information of the agree button or the cancel button; the operation method corresponding to the App login dialog box can be clicking a login button, etc., and the recognition result can include the location information of the login button.

[0079] For advertisements, the types of advertisements can be identified, such as full-screen advertisements, countdown advertisements, pop-up advertisements, embedded advertisements, and native advertisements. Different operation methods and the information required to perform the operation methods are determined according to different advertisement types, and are output as identification results. Among them, the operation method corresponding to full-screen advertisements can be swiping right to close; the operation method corresponding to countdown advertisements can be waiting for timeout to automatically close; the operation method corresponding to pop-up advertisements can be clicking the close button, and the identification result may include the location information of the close button; embedded advertisements and native advertisements do not affect the operation of the page to be processed, and the advertising area range can be marked and used as the identification result.

[0080] It can be seen that the recognition engine helps to determine the traversal order of the sub-pages of the page to be processed and to handle pop-up windows in a timely manner, thereby helping to improve the traversal efficiency of the application.

[0081] In summary, the technical solution of the embodiment of the present disclosure marks the functions of the pages that have been detected in the application as traversed functions; the traversed pages to be processed are identified and the functions of the pages are determined. Only when the functions do not belong to the traversed functions, the sub-pages of the pages are traversed. Based on this, the function coverage can be ensured during the traversal detection process of the application, and the number of traversals of pages with the same functions can be reduced, thereby improving the traversal efficiency.

[0082] As an implementation of the above method, the embodiment of the present disclosure also provides a traversal device for an application program, such as Figure 6 As shown, the device comprises:

[0083] The first identification module 610 is used to identify the to-be-processed page of the target application program and obtain function information of the to-be-processed page;

[0084] A detection module 620, configured to detect the page to be processed if it is determined based on the function information that the function of the page to be processed does not belong to the traversed functions;

[0085] The recursive traversal module 630 is used to mark the function of the page to be processed as a traversed function after completing the detection of the page to be processed, and traverse at least one subpage of the page to be processed, and determine the traversed subpage as a new page to be processed.

[0086] For example, Figure 7 As shown, the first identification module 610 may include:

[0087] An image recognition unit 611 is used to recognize the image information in the page to be processed based on the image recognition module in the recognition engine, and obtain the image recognition result of the page to be processed;

[0088] The semantic recognition unit 612 is used to recognize text information in the page to be processed based on the natural language understanding module in the recognition engine to obtain a semantic recognition result of the page to be processed;

[0089] The function recognition unit 613 is used to obtain function information of the page to be processed based on the image recognition result and the semantic recognition result of the page to be processed.

[0090] For example, Figure 7 As shown, the device may also include:

[0091] The second identification module 710 is used to identify at least one control in the page to be processed and obtain a priority order of the at least one control;

[0092] The sorting module 720 is used to obtain a traversal order of at least one sub-page corresponding to at least one control of the page to be processed based on the priority order of at least one control.

[0093] For example, Figure 7 As shown, the device may also include:

[0094] The pop-up window processing module 730 includes identifying the pop-up window when a pop-up window appears in the page to be processed, obtaining an operation method for closing the pop-up window, and closing the pop-up window using the operation method.

[0095] For example, Figure 7 As shown, the device may also include:

[0096] The end processing module 740 is used to end the processing of the page to be processed when the page to be processed has not been detected and the function of the page to be processed belongs to the traversed functions.

[0097] Exemplarily, the detection module 620 may be used to trigger privacy behaviors in the page to be processed and perform compliance detection on the privacy behaviors.

[0098] The device provided by the embodiment of the present disclosure can implement the method provided by the embodiment of the present disclosure and has corresponding beneficial effects.

[0099] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0100] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0101] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. In the RAM 803, various programs and data required for the operation of the device 800 can also be stored. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0102] A number of components in the device 800 are connected to the I / O interface 805, including: an input unit 806, such as a keyboard, a mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a disk, an optical disk, etc.; and a communication unit 809, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 809 allows the device 800 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0103] The computing unit 801 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the traversal method of the application. For example, in some embodiments, the traversal method of the application may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program may be loaded and / or installed on the device 800 via ROM 802 and / or communication unit 809. When the computer program is loaded into RAM 803 and executed by the computing unit 801, one or more steps of the traversal method of the application described above may be performed. Alternatively, in other embodiments, the computing unit 801 may be configured to execute the traversal method of the application by any other appropriate means (e.g., by means of firmware).

[0104] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0106] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0108] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0109] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0110] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0111] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A traversal method of an application, comprising: Identify the to-be-processed page of the target application program, and obtain function information of the to-be-processed page, wherein the function information at least includes a code and name identification information of the function; In the case where it is determined based on the function information that the function of the page to be processed does not belong to the traversed functions, detecting the page to be processed; When the detection of the page to be processed is completed, marking the function of the page to be processed as a traversed function, traversing at least one subpage of the page to be processed, and determining the traversed subpage as a new page to be processed; Among them, identifying the to-be-processed page of the target application to obtain the functional information of the to-be-processed page includes: obtaining the image recognition result and the semantic recognition result of the to-be-processed page; obtaining the functional information of the to-be-processed page based on the image recognition result and the semantic recognition result; When the page to be processed is not detected and the function of the page to be processed belongs to a traversed function, the processing of the page to be processed is terminated, wherein the traversed function is used to represent different pages with the same function.

2. The method according to claim 1, wherein: Obtaining the image recognition result and semantic recognition result of the page to be processed, including: Recognize the image information in the page to be processed based on the image recognition module in the recognition engine to obtain the image recognition result of the page to be processed; Based on the natural language understanding module in the recognition engine, the text information in the page to be processed is recognized to obtain the semantic recognition result of the page to be processed.

3. The method according to claim 1 or 2, further comprising: Identify at least one control in the page to be processed, and obtain a priority order of the at least one control; Based on the priority order of the at least one control, a traversal order of at least one sub-page of the page to be processed corresponding to the at least one control is obtained.

4. The method according to claim 1 or 2, further comprising: When a pop-up window appears in the page to be processed, the pop-up window is identified, an operation method for closing the pop-up window is obtained, and the pop-up window is closed using the operation method.

5. The method according to claim 1 or 2, wherein: The detecting the page to be processed includes: Trigger the privacy behavior in the page to be processed, and perform compliance detection on the privacy behavior.

6. A traversal device for an application, comprising: A first identification module is used to identify the to-be-processed page of the target application program and obtain function information of the to-be-processed page, wherein the function information at least includes a code and name identification information of the function; A detection module, configured to detect the page to be processed when it is determined based on the function information that the function of the page to be processed does not belong to the traversed functions; A recursive traversal module, used for marking the function of the page to be processed as a traversed function after completing the detection of the page to be processed, and traversing at least one subpage of the page to be processed, and determining the traversed subpage as a new page to be processed; The first recognition module includes a function recognition unit; the function recognition unit is used to obtain the image recognition result and the semantic recognition result of the page to be processed; based on the image recognition result and the semantic recognition result, the function information of the page to be processed is obtained; The end processing module is used to end the processing of the page to be processed if the page to be processed is not detected and the function of the page to be processed belongs to the traversed functions, wherein the situation of belonging to the traversed functions is used to represent the situation of different pages with the same function.

7. The device according to claim 6, wherein: The first identification module further includes: An image recognition unit, used to recognize the image information in the page to be processed based on the image recognition module in the recognition engine, and obtain the image recognition result of the page to be processed; The semantic recognition unit is used to recognize the text information in the page to be processed based on the natural language understanding module in the recognition engine to obtain the semantic recognition result of the page to be processed.

8. The device according to claim 6 or 7, further comprising: A second identification module, used for identifying at least one control in the page to be processed, and obtaining a priority order of the at least one control; The sorting module is used to obtain the traversal order of at least one sub-page of the page to be processed corresponding to the at least one control based on the priority order of the at least one control.

9. The device according to claim 6 or 7, further comprising: The pop-up window processing module includes identifying the pop-up window when it appears in the page to be processed, obtaining an operation method for closing the pop-up window, and closing the pop-up window using the operation method.

10. The device according to claim 6 or 7, wherein: The detection module is used to trigger the privacy behavior in the page to be processed and perform compliance detection on the privacy behavior.

11. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 5.

13. A computer program product, comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images