How to support gathering information about screen elements

Abstract

According to an embodiment of the present disclosure, a method for supporting collection of information about screen components, executed by a computing device including at least one processor, is disclosed. The method for supporting collection of information about screen components may include: accessing a pre-specified library in response to execution of a target application; executing a command to detect a hierarchical structure of parent user interfaces (UIs) included in the library; determining whether or not at least one child UI type exists in a lower hierarchy determined by the type of the parent UI; and, if the at least one child UI exists, generating report information for displaying at least one child object representing the at least one child UI and a parent object representing the parent UI in a pre-specified visualization manner.

Description

[Technical Field]

[0001] The present disclosure relates to a method for supporting collection of information about screen components, and more particularly to a method for supporting collection of information about screen components executed by a computing device for collecting screen components on a remotely located computing device. [Background technology]

[0002] The process of distributing apps for the Android operating system can be briefly summarized as follows:

[0003] Application Development: First, you need to develop the app that will run on the user's device. This process can include designing the user interface, implementing the app's logic, and testing.

[0004] Building the Application: Once you have completed developing your app, you need to build it. Building can refer to the process of compiling the source code and packaging it into an APK (Android Package) file.

[0005] Signing: Your app must be signed. Adding a signature to your APK file identifies your app as a trusted source. You can sign using a debug keystore for development or a release keystore.

[0006] Google Play Console Registration: To distribute your app to the Google Play Store, you need to register it with the Google Play Console, which allows you to set up app information, pricing, description, screenshots, version control, etc.

[0007] Upload APK: Upload your APK file in the Google Play Console. You can upload your APK file for release or set different release tracks such as Alpha, Beta, Production, etc.

[0008] Configure resources and store page: Configure resources such as app icon, graphics, description, and app store page.

[0009] Pricing and Country Settings: Set the price of your app and specify the countries it will be available in. You can choose whether to make it a free or paid app and manage your pricing policy.

[0010] Release and Review: Release your app on the Google Play Console and undergo a Google review process. The review checks the app's content, compliance with advertising policies, etc.

[0011] Distributing the application: Once the review is complete, distribute the app to the Google Play Store, where users can search for and download the app.

[0012] Manage application updates: When you need to update your app or release a new version, repeat the steps above to upload the new APK file and release it.

[0013] User support and feedback management: Managing communication with users who use the App, processing feedback and problem reports, and helping to improve the App.

[0014] Marketing and Promotion: After the app is successfully released, marketing and promotional activities are carried out to attract more users and promote the app.

[0015] Android applications developed as native apps usually need to follow the distribution process described above to update the application. The process of development, building, testing, and distribution requires a lot of time and resources, and even when updating the application, the development, building, and testing processes are still required. Furthermore, remote resource patching has limitations on changes, as it is not possible to change the executing entity. Summary of the Invention [Problem to be solved by the invention]

[0016] The present disclosure has been devised in response to the background art described above, and provides a method that can support the collection of information about screen components without rebuilding or redistributing an application.

[0017] The technical problems in the present disclosure are not limited to the above-mentioned technical problems, and a person skilled in the art can clearly understand problems other than the above-mentioned technical problems based on the following description. [Means for solving the problem]

[0018] According to an embodiment of the present disclosure for solving the above-mentioned problem, a method for supporting collection of information about screen components, executed by a computing device including at least one processor, is disclosed. The method for supporting collection of information about screen components may include the steps of: accessing a pre-specified library in response to execution of a target application; executing a command to detect a hierarchical structure of parent UIs (User Interfaces) included in the library; determining whether or not at least one child UI type exists in a lower hierarchy determined by the type of the parent UI; and, if the at least one child UI exists, generating report information for displaying at least one child object representing the at least one child UI and a parent object representing the parent UI in a pre-specified visualization manner.

[0019] The commands may also include a first command that defines a type of at least one child UI to be detected based on the type of the parent UI, and a second command that detects the type of the at least one child UI.

[0020] In addition, the step of determining whether or not at least one child UI type exists, which is a lower hierarchy determined by the type of the parent UI, may include a step of repeatedly determining whether or not the at least one child UI type exists until the lowest child UI located at the bottom of the hierarchy is determined.

[0021] Furthermore, the step of iteratively determining the presence or absence of at least one child UI type until the lowest child UI located at the bottom of the hierarchy is determined may include a step of determining an activity of the parent UI corresponding to the command; a step of determining at least one deco view that is a sub-hierarchy of the activity; a step of determining at least one child view that is a sub-hierarchy of each of the at least one deco view; and a step of iteratively determining a sub-hierarchy of each of the at least one child view until the lowest child view of each of the at least one child view is determined.

[0022] In addition, the at least one child object may include at least one of depth information indicating the depth in the visualization technique, an index of the at least one child UI, a class indicating the type of the at least one child UI, text in which the at least one child UI is displayed on the front end, an ID of the at least one child UI, and a tag assigned to the at least one child UI.

[0023] In addition, the step of generating the report information for display may include a step of generating, as the report information, a command that supports generating a plurality of graphic objects indicating at least one of: a first indicator indicating whether an ID is assigned to the at least one child UI; a second indicator indicating whether an action corresponding to user interaction exists in the at least one child UI; a third indicator indicating whether a trigger is assigned to the at least one child UI; and a fourth indicator indicating whether the at least one child UI is a UI displayed to a user.

[0024] The technical solutions obtained from the present disclosure are not limited to the solutions described above, and a person having ordinary skill in the technical field to which the present disclosure pertains can clearly understand solutions other than the solutions described above from the following description. [Effects of the Invention]

[0025] According to multiple embodiments of the present disclosure, a method is provided that can support the collection of information about screen components without rebuilding or redistributing the application.

[0026] The effects obtained from the present disclosure are not limited to the effects described above, and a person having ordinary knowledge in the technical field to which the present disclosure pertains can clearly understand effects other than those described above based on the following description.

[0027] Various aspects are described with reference to the drawings. In the following description of the drawings, like reference numerals are used to generally refer to like components. In the following examples, for purposes of explanation, numerous specific details are provided to facilitate an overall understanding of one or more aspects. However, it will be apparent that such aspect(s) may be practiced without such specific details. In other instances, well-known structures and devices are shown in block diagram form to facilitate describing one or more aspects. [Brief explanation of the drawings]

[0028] [Figure 1] FIG. 1 is a block diagram illustrating an example computing device in accordance with embodiments of the present disclosure. [Figure 2] FIG. 2 is a flowchart illustrating an example method by which a computing device can support collection of information about screen components, according to embodiments of the present disclosure. [Figure 3] FIG. 3 is a flowchart illustrating an example method for a computing device to determine a lowest-level child UI located at the lowest level in a hierarchy, according to embodiments of the present disclosure. [Figure 4] FIG. 4 is a diagram illustrating an example of components of a screen according to embodiments of the present disclosure. [Figure 5]FIG. 5 is a diagram illustrating at least one child object according to embodiments of the present disclosure. [Figure 6] FIG. 6 is a flowchart illustrating an example method for collecting screen components between a user, a computing device, and a server, according to embodiments of the present disclosure. DETAILED DESCRIPTION OF THE INVENTION

[0029] The present invention is susceptible to various modifications and can have multiple embodiments. Specific embodiments are shown in the drawings and will be described in detail in the Detailed Description of the Invention. However, this does not limit the present invention to the specific embodiments, and it should be understood that the present invention includes all modifications, equivalents, and alternatives within the spirit and technical scope of the present invention. In describing the various drawings, similar reference symbols are used to refer to similar components.

[0030] Terms such as first, second, A, B, etc. are used to describe various components, but the components should not be limited by these terms. These terms are used only to distinguish one component from another. Thus, without departing from the scope of the present invention, a first component can be termed a second component, and similarly, a second component can be termed a first component. The term "and / or" is intended to include a combination of multiple associated listed items or any one of multiple associated listed items.

[0031] When a component is described as being "connected," "coupled," or "connected" to another component, it should be understood that the component may be directly connected, coupled, or connected to the other component, but that there may be other components between them. On the other hand, when a component is described as being "directly connected," "directly coupled," or "directly connected" to another component, it should be understood that there are no other components between them.

[0032] The terms used in this application are used to describe particular embodiments and are not intended to limit the present invention. The singular term "a," "an," "an," or "an" includes the plural term unless the context clearly dictates otherwise. In this application, the terms "comprise," "include," "includes," "have," or "having" indicate the presence of features, numbers, steps, operations, components, parts, or combinations thereof described in the specification, but are not intended to preclude the possibility that one or more other features, numbers, steps, operations, components, parts, or combinations thereof may be present or added.

[0033] Unless otherwise defined, all terms used in this disclosure, including technical and scientific terms, have the same meaning as commonly understood by those having ordinary skill in the art to which this invention belongs, and terms defined in common dictionaries should be interpreted as having the same meaning in the context of the relevant art, and should not be interpreted ideally or in an overly formal sense unless otherwise defined in this specification.

[0034] In the present disclosure, a computing device can transmit information about screen elements to a server. Specifically, the computing device can access a pre-specified library in response to execution of a target application. The target application can be an application pre-distributed to the computing device. The target application can be an application for which a software development kit (SDK) is installed in the form of a library. The computing device can execute a command to detect the hierarchical structure of the screen elements included in the library. Here, the command can be a script, etc. The script can be a programming language that can be executed without compiling source code. Then, the computing device can send the collected information about the screen elements to a server. Therefore, a user can conveniently collect information about the screen elements without rebuilding or redistributing an application. Below, an example of a method for supporting collection of information about screen elements based on the present disclosure is described with reference to FIGS. 1 to 6.

[0035] FIG. 1 is a block diagram illustrating an example of a computing device according to several embodiments of the present disclosure.

[0036] 1, the computing device 100 may include a control unit 110, a storage unit 120, and a communication unit 130. However, the above components are not required to implement the computing device 100, and the computing device 100 may include more or fewer components than the above components.

[0037] The computing device (100) may include a user-owned personal computer (PC), notebook computer, mobile terminal, smartphone, tablet PC, etc., and may include all types of terminals that can be connected to a wired / wireless network.

[0038] The computing device (100) may use a combination of typical computer hardware (e.g., devices that may include a computer processor, memory, storage, input and output devices, and other conventional computing device components; electronic communication devices such as routers and switches; and electronic information storage systems such as network-attached storage (NAS) and storage area networks (SAN)) and computer software (i.e., commands that cause the computing device to function in a particular manner) to achieve desired system performance.

[0039] The control unit 110 is generally capable of processing the overall operation of the computing device 100. The control unit 110 processes signals, data, information, etc. input or output through components of the computing device 100, and can provide or process appropriate information or functions to a user by running application programs stored in the storage unit 120.

[0040] The control unit (110) may be configured with one or more cores and may include a processor for data analysis, such as a central processing unit (CPU), a general purpose graphics processing unit (GPGPU), or a tensor processing unit (TPU).

[0041] In the present disclosure, the control unit 110 can detect a hierarchical structure of parent user interfaces (UIs). Here, the parent UI can be the user interface located at the highest level in the hierarchy. For example, it can be assumed that an application such as an "integrated patent document creation tool" exists. The parent UI can be assumed to be an activity class. In this case, the home screen or login screen of the "integrated patent document creation tool" application can be the parent UI. Alternatively, the parent UI can be assumed to be a DecorView class. In this case, the parent UI can be the top view of the activity. The control unit 110 can determine whether a child UI type exists, which is a lower level determined by the type of the parent UI. For example, if the parent UI is an activity class, the control unit 110 can determine whether a child UI of the DecorView type exists. As another example, if the parent UI is a DecorView class, the control unit 110 can determine whether a child UI of the LinearLayout type exists. If a child UI exists, the control unit 110 can generate report information. The report information may be information for displaying the parent UI and the child UI in a pre-specified visualization method. The computing device 100 may transmit the report information to a server or the like, thereby causing the server to display the child object representing the child UI and the parent object representing the parent UI in a pre-specified visualization method. An example of a method for the control unit 110 to generate the report information will be described below with reference to FIG. 2.

[0042] The storage unit 120 may include a memory and / or a permanent storage medium, and the memory may include at least one type of storage medium selected from the group consisting of flash memory, hard disk, micro multimedia card, card-type memory (e.g., SD or XD memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, and optical disk.

[0043] The communications unit (130) may include one or more modules that enable communications between the computing device (100) and a communications system, between the computing device (100) and a server, between the computing device (100) and other computing devices, or between the computing device (100) and a network.

[0044] The following describes a specific example of how the computing device (100) supports the collection of information about screen components.

[0045] FIG. 2 is a flowchart illustrating an example method by which a computing device can support collection of information about screen components, according to embodiments of the present disclosure.

[0046] Referring to FIG. 2, the control unit 110 of the computing device 100 may access a pre-designated library in response to the execution of a target application (S110). Here, the target application may be an application pre-installed on the computing device 100. The target application may be an application in which a software development kit (SDK) is installed in the form of a library. The library may be a program that is compiled to execute only a specific partial function and exists in the form of a machine code. The execution of the target application may be performed by user interaction.

[0047] The control unit (110) can execute a command to detect the hierarchical structure of parent UIs (User Interfaces) included in the library (S120). The parent UI can be the user interface located at the top of the hierarchy. The command can be a script or the like. The script can be a programming language that can be executed without compiling source code. The control unit (110) can detect the hierarchical structure of the parent UI by executing the command.

[0048] In one embodiment, the commands may include a first command and a second command.

[0049] The first command may be a first command that defines a type of at least one child UI to be detected by a type of parent UI.

[0050] For example, it is possible to assume that an application such as an "integrated patent document creation tool" exists. The parent UI can be assumed to be an activity class. In this case, the home screen or login screen of the "integrated patent document creation tool" application can be a parent UI that is an activity class. If the parent UI type is an activity class, a deco view class can exist as a child UI. The first command can be a command that defines at least one child UI type as a deco view, depending on the type of the parent UI that is an activity class.

[0051] The second command can be a second command for tracking at least one child UI.

[0052] For example, the first command can define that the type of at least one child UI is a deco view, and the second command can be a command to locate at least one child UI of the deco view type.

[0053] The control unit (110) can determine whether or not there is at least one child UI type, which is a lower layer determined by the type of the parent UI (S130).

[0054] For example, the control unit (110) may determine that there is at least one child UI that is a lower hierarchy of the parent UI if at least one child UI of a deco view type is detected by the second command.

[0055] If at least one child UI exists, the control unit (110) can generate report information for displaying at least one child object representing the at least one child UI and a parent object representing the parent UI in a pre-specified visualization method (S140). Then, the control unit (110) can transmit the generated report information to a server or other computing device via the communication unit (130). Here, the pre-specified visualization method can be, for example, a tree structure. For ease of explanation, Figure 4 can be used.

[0056] 4 is a diagram illustrating an example of components of a screen according to embodiments of the present disclosure, which may be a screen (200) displayed on a server that receives report information from a computing device (100).

[0057] 4, the screen 200 can display parent objects 210 and child objects 220 in a pre-specified visualization manner by receiving report information from the computing device 100. For example, the screen 200 can display the parent objects 210 and child objects 220 in a tree structure.

[0058] Based on the above configuration, the computing device (100) can execute a command for detecting the hierarchical structure of a parent UI included in a pre-specified library in response to execution of a target application. The command for detecting the hierarchical structure of a parent UI can be a script or the like that does not require rebuilding or redistribution of the target application. In other words, the command can be a programming language that can be executed without compiling source code. By executing the command, the computing device (100) can generate report information that causes a parent object representing the parent UI and at least one child object representing at least one child UI to be displayed on a server in a pre-specified visualization manner. The computing device (100) can then transmit the generated report information to the server. Thus, a user can conveniently collect information about screen components.

[0059] Meanwhile, in several embodiments of the present disclosure, when determining whether at least one child UI type exists, the computing device (100) may iteratively determine whether at least one child UI type exists until the lowest child UI located at the bottom of the hierarchy is determined. Hereinafter, an example of a method by which the computing device (100) of the present disclosure determines the lowest child UI located at the bottom of the hierarchy will be described with reference to FIG. 3 .

[0060] FIG. 3 is a flowchart illustrating an example method for a computing device to determine a lowest-level child UI located at the lowest level in a hierarchy, according to embodiments of the present disclosure.

[0061] Referring to FIG. 3, the control unit (110) of the computing device (100) can determine the activity of the parent UI corresponding to the command (S131).

[0062] Specifically, the control unit 110 can determine the parent UI activity corresponding to the command by executing a command that detects the hierarchical structure of the parent UI.

[0063] The control unit (110) can determine at least one deco view that is a lower level of the activity (S132). The at least one deco view can be a child UI type determined because the parent UI is an activity type.

[0064] The control unit (110) can determine at least one child view that is a lower layer of each of the at least one deco view (S133). The at least one child view can be a LinearLayout or the like.

[0065] In one embodiment, the commands may include a third command defining at least one type of second child UI to be detected based on the type of the first child UI, and a fourth command for detecting the type of the at least one second child UI.

[0066] For example, if the type of the first child UI is a DecorView class, a LinearLayout may exist as a child UI. The third command may be a command to define the type of at least one second child UI as LinearLayout according to the type of the first child UI, which is a DecorView class. The fourth command may be a command to find at least one second child UI of the LinearLayout type.

[0067] The control unit 110 may iteratively determine the lower hierarchy of each of the at least one child view until the lowest child view of each of the at least one child view is determined (S134). The lowest child view may be the lowest child UI located at the lowest level in the hierarchy.

[0068] For example, the control unit 110 can detect the lowest child UI by repeatedly determining the presence or absence of at least one child UI type, such as ViewStub, FrameLayout, ActionBarOverlayLayout, etc., through command execution.

[0069] On the other hand, if the lowest child UI is detected, the control unit 110 can generate report information for displaying at least one child object representing at least one child UI, including the lowest child UI, and a parent object representing the parent UI in a pre-specified visualization manner. The control unit 110 can then transmit the generated report information to a server or other computing device via the communication unit 130. Thus, the server or other computing device can display the screen components in a pre-specified visualization manner.

[0070] 4 is a diagram illustrating an example of components of a screen according to embodiments of the present disclosure, which may be a screen (200) displayed on a server that receives report information from a computing device (100).

[0071] Referring to FIG. 4, the screen (200) can display a parent object (210) representing a parent UI, a child object (220) representing a child UI, and a lowest-level child object (230) representing a lowest-level child UI in a pre-specified visualization manner by receiving report information from the computing device (100).

[0072] Meanwhile, in several embodiments of the present disclosure, the at least one child object may include at least one of depth information, an index, a class, text, an ID (Identifier), and a tag. Hereinafter, the at least one child object in the present disclosure will be described with reference to FIG. 5.

[0073] 5 is a diagram illustrating at least one child object according to embodiments of the present disclosure, in which a lowest-level child object (230) is used to illustrate an example of at least one child object.

[0074] Referring to FIG. 5, a lowest-level child object (230) representing a lowest-level child UI may include at least one of depth information (231), index (232), class (233), text (234), ID (235), and tag (236).

[0075] The depth information (231) may refer to the depth in the visualization. The depth information (231) may indicate the depth between at least one child object and a parent object. The depth information (231) may indicate how far the application must search to reach related data.

[0076] The index (232) can be an index for quickly searching for an object from a large amount of data. For example, a user can quickly find the lowest-level child object (230) by using an index (232) such as 8-12.

[0077] The class (233) can indicate the type of at least one child UI. The class (233) can indicate the type of UI, such as button, text, image, etc.

[0078] The text (234) can be the text that at least one child UI displays in the front end, i.e., the text (234) can represent text that the user sees directly on the user side of the application.

[0079] The ID (235) can indicate the ID of at least one child UI. The ID (235) can be an internal name for at least one child object, or a name specified by a client developer.

[0080] The tag 236 can refer to a tag attached to at least one child UI, and can indicate additional information or related data if a customer developer sets tag information for the UI.

[0081] Meanwhile, in some embodiments of the present disclosure, at least one child object may include at least one indicator, which may be a graphic object for displaying information to a user.

[0082] For example, the at least one child object may include a first indicator (241), a second indicator (242), a third indicator (243), a fourth indicator (not shown), and so on.

[0083] The first indicator 241 may indicate whether an ID is assigned to at least one child UI. In one embodiment, the first indicator 241 may be colored to indicate whether an ID is assigned to at least one child UI. The first indicator 241 may be colored blue.

[0084] The second indicator 242 may indicate whether at least one child UI element has an action in response to a user interaction. In one embodiment, the second indicator 242 may be colored to indicate whether at least one child UI element has an action in response to a user interaction. The second indicator 242 may be colored yellow.

[0085] The third indicator 243 may indicate whether a trigger is assigned to at least one child UI. In one embodiment, the third indicator 243 may be colored to indicate whether a trigger is assigned to at least one child UI. The third indicator 243 may be colored green.

[0086] The fourth indicator (244) may indicate whether at least one child UI is a UI that is displayed to the user. In one embodiment, the fourth indicator (244) may be colored to indicate whether at least one child UI is a UI that is displayed to the user. The fourth indicator (244) may be colored black.

[0087] The control unit 110 can generate, as report information, commands supporting the generation of a plurality of graphic objects representing at least one indicator, such as the first indicator 241 through the fourth indicator, and can transmit the generated report information to a server or other computing device via the communication unit 130. Thus, a plurality of graphic objects, such as the first indicator 241 through the fourth indicator, can be displayed on the server or other computing device.

[0088] FIG. 6 is a flowchart illustrating an example method for collecting screen components between a user, a computing device, and a server, according to embodiments of the present disclosure.

[0089] Referring to FIG. 6, the computing device 100 may execute a session and detect a library in response to a first user interaction (S210). The first user interaction (S210) may be an interaction to execute a target application. The session may be a logical connection period for communication between a user, a computing device, and a server. The computing device 100 may access a pre-specified library in response to the execution of the target application. The computing device 100 may execute a command to detect a hierarchical structure of parent user interfaces (UIs) included in the library. The computing device 100 may determine whether at least one child UI type exists, which is a lower hierarchy determined by the type of the parent UI. If the at least one child UI exists, the computing device 100 may generate report information for displaying at least one child object representing the at least one child UI and a parent object representing the parent UI in a pre-specified visualization manner.

[0090] The computing device (100) can report the UI hierarchy structure (S230), i.e., the computing device (100) can transmit the generated report information to a server.

[0091] Meanwhile, the computing device 100 may receive a remote collection command from the server (S240). The remote collection command may be a command for collecting data related to the computing device 100. The remote collection command may include a tag item, a trigger item, a variable item, and the like.

[0092] In response to the second user interaction (S250), the computing device (100) may activate a trigger condition and collect target data (S260).

[0093] The second user interaction can be an interaction that is the target of the trigger condition. For example, the trigger condition can be an interaction such as clicking a button, touching a button, clicking a specific key, touching a specific key, clicking a specific item, or touching a specific item. The target data can be data specified as a tag item.

[0094] The computing device (100) may report the collected data to a server (S270).

[0095] The description of the embodiments set forth herein is provided to enable any person skilled in the art to use or practice the present disclosure. Various modifications to the embodiments will be apparent to those skilled in the art, and the general principles defined herein may be applied to other embodiments without departing from the scope of the present disclosure. Thus, the present disclosure is not intended to be limited by the embodiments set forth herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. 1. A method for supporting collection of information about screen components, the method being performed by a computing device including at least one processor, the method comprising: accessing a pre-specified library in response to execution of the target application; Executing a command to detect a hierarchical structure of a parent user interface (UI) included in the library; Determining whether or not there is at least one child UI type that is a lower layer determined by the parent UI type; and generating report information for displaying at least one child object representing the at least one child UI and a parent object representing the parent UI in a pre-specified visualization manner when the at least one child UI exists; Including, A method for collecting screen components.

2. In claim 1, The command a first command for defining a type of at least one child UI to be detected according to a type of the parent UI; and a second command for detecting the type of the at least one child UI. A method for collecting screen components.

3. In claim 1, The step of determining whether or not at least one child UI type exists, which is a lower layer determined by the parent UI type, includes: repeatedly determining whether or not the at least one child UI type exists until a lowest child UI located at the lowest position in the hierarchy is determined; Including, A method for collecting screen components.

4. In claim 3, The step of repeatedly determining whether or not at least one child UI type exists until a lowest child UI located at the lowest position in the hierarchy is determined includes: determining an activity of the parent UI corresponding to the command; determining at least one deco view that is a sub-hierarchy of the activity; determining at least one child view that is a descendant of each of the at least one deco view; and iteratively determining a sub-hierarchy of each of the at least one child views until a lowest child view of each of the at least one child view is determined. Including, A method for collecting screen components.

5. In claim 1, The at least one child object is The at least one of the following information is included: depth information indicating a depth in the visualization method; an index of the at least one child UI; a class indicating a type of the at least one child UI; text in which the at least one child UI is displayed on a front end; an ID of the at least one child UI; and a tag assigned to the at least one child UI. A method for collecting screen components.

6. In claim 1, The step of generating report information for display includes: generating, as the report information, a command supporting generation of a plurality of graphic objects indicating at least one of: a first indicator indicating whether an ID is assigned to the at least one child UI; a second indicator indicating whether an action corresponding to a user interaction exists in the at least one child UI; a third indicator indicating whether a trigger is assigned to the at least one child UI; and a fourth indicator indicating whether the at least one child UI is a UI displayed to a user. Including, A method for collecting screen components.

Images