Page filling method and device, electronic equipment and storage medium

Abstract

The embodiment of the invention discloses a page filling method and device, electronic equipment and a storage medium. The method comprises the steps that a filling rule corresponding to a to-be-filled page is obtained, the filling rule is configured based on the page type of a form page and page view node information of a page form, and the filling rule comprises a determination rule of the type of an input box included in the form page and a filling frame; the page view node information is attribute information of each input box in the form page; and based on the filling rule, filling an input box of the to-be-filled page with target content. Through the method, the corresponding filling rule is configured for the form page in advance according to the page type of the form page and the page view node information of the form page, so that the form pages of different page types can have different filling rules, an automatic filling scene is expanded, and more reliable support is provided for automatic filling of target data.

Description

Technical Field

[0001] The present application belongs to the field of data processing technology, and specifically relates to a page filling method, device, electronic device and storage medium. Background Art

[0002] When users use various applications on mobile terminals, they often encounter various form filling pages, such as real-name authentication pages, delivery address input pages, bank card information filling pages, etc. The relevant page filling methods support automatic filling of form pages, but the scenarios covered by the relevant automatic filling methods are not comprehensive. Summary of the invention

[0003] In view of the above problems, the present application proposes a page filling method, device, electronic device and storage medium to improve the above problems.

[0004] In a first aspect, an embodiment of the present application provides a page filling method, the method comprising: obtaining filling rules corresponding to a page to be filled, the filling rules being configured based on a page type of a form page and page view node information of a page form, the filling rules comprising a determination rule and a filling framework for the type of input boxes included in the form page, the page view node information being attribute information of each input box in the form page; based on the filling rules, filling target content into the input box of the page to be filled.

[0005] In the second aspect, an embodiment of the present application provides a page filling device, which includes: a rule acquisition unit, used to obtain the filling rules corresponding to the page to be filled, the filling rules are configured based on the page type of the form page and the page view node information of the page form, the filling rules include the determination rules and filling framework of the type of input boxes included in the form page, and the page view node information is the attribute information of each input box in the form page; a filling unit, used to fill the target content into the input box of the page to be filled based on the filling rules.

[0006] In a third aspect, an embodiment of the present application provides an electronic device, comprising one or more processors and a memory; one or more programs, wherein the one or more programs are stored in the memory and configured to be executed by the one or more processors, and the one or more programs are configured to execute the above-mentioned method.

[0007] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a program code is stored, wherein the above method is executed when the program code is run.

[0008] The embodiments of the present application provide a page filling method, device, electronic device and storage medium. First, the filling rules corresponding to the page to be filled are obtained, and the filling rules are configured based on the page type of the form page and the page view node information of the form page. The filling rules include the determination rules and the filling framework of the type of input boxes included in the form page, and the page view node information is the attribute information of each input box in the form page. Then, based on the filling rules, the target data is filled into the page to be filled. Through the above method, the corresponding filling rules are configured for the form page in advance according to the page type of the form page and the page view node information of the form page, so that form pages of different page types can have different filling rules, which expands the automatic filling scenario and provides more reliable support for the automatic filling of target data. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0010] Figure 1 A flow chart of a page filling method proposed in an embodiment of the present application is shown;

[0011] Figure 2 A flow chart of a page filling method proposed in another embodiment of the present application is shown;

[0012] Figure 3 A schematic diagram showing a tree structure in another embodiment of the present application is shown;

[0013] Figure 4 A flow chart of a page filling method proposed in another embodiment of the present application is shown;

[0014] Figure 5 A structural block diagram of a page filling device proposed in an embodiment of the present application is shown;

[0015] Figure 6 A structural block diagram of a page filling device proposed in an embodiment of the present application is shown;

[0016] Figure 7 A structural block diagram of an electronic device for executing a page filling method according to an embodiment of the present application is shown;

[0017] Figure 8 A storage unit in an embodiment of the present application is shown for storing or carrying program codes for implementing a page filling method according to an embodiment of the present application. DETAILED DESCRIPTION

[0018] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0019] As 5G technology matures and costs decrease, 5G mobile phones and other mobile terminals have become widely popular among users, and using mobile terminals to communicate and trade on the Internet has become a common practice for users. For example, when purchasing a product, requesting information, or submitting a question, users are often required to fill out many forms (some of which are not similar in format). Users must repeatedly enter the same information on various forms, such as user name, password, home address, work address, email address, chat software account number and / or phone number, etc.

[0020] In order to solve the trouble caused by repeatedly filling in the same information in different forms, the automatic filling of the form can be achieved through the automatic filling function set in the mobile terminal.

[0021] The inventor found in the study of the relevant page filling method that the relevant page filling method supports automatic filling of form pages, but the scenarios covered by the relevant automatic filling method are not comprehensive. For example, on the mobile platform, some existing automatic filling functions are mainly aimed at filling in accounts and passwords. On the account and password input page of a certain application, after the user enters the account and password, the account and password information can be saved through the automatic filling solution. When the account and password need to be entered next time, the user can select the stored account and password information to complete the automatic filling of the page. This implementation scheme is implemented through the automatic filling framework provided by Android natively. This function is only used for automatic filling of account and password information, and the filling information is too limited.

[0022] For example, in some application ecosystems, there are also some automatic filling functions. As long as the application has recorded certain personal information of the user, within the scope of the application ecosystem, when the user goes to a new form input page, the user's recorded data will be retrieved for filling. However, this solution is also relatively limited and can only be used within the scope of the ecosystem.

[0023] Therefore, the inventors proposed the page filling method, device, electronic device and storage medium in the present application. First, the filling rules corresponding to the page to be filled are obtained, and the filling rules are configured based on the page type of the form page and the page view node information of the form page. The filling rules include the determination rules and the filling framework of the type of input boxes included in the form page, and the page view node information is the attribute information of each input box in the form page. Then, based on the filling rules, the target data is filled into the page to be filled. Through the above method, the corresponding filling rules are configured for the form page in advance according to the page type of the form page and the page view node information of the form page, so that form pages of different page types can have different filling rules, which expands the automatic filling scenario and provides more reliable support for the automatic filling of target data.

[0024] The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0025] See also Figure 1 , a page filling method provided in an embodiment of the present application, the method comprising:

[0026] Step S110: Obtain the filling rules corresponding to the page to be filled, wherein the filling rules are configured based on the page type of the form page and the page view node information of the page form, wherein the filling rules include the determination rules and the filling framework of the type of input boxes included in the form page, and the page view node information is the attribute information of each input box in the form page.

[0027] In an embodiment of the present application, the page to be filled is a form page that needs to be filled with information. Among them, the form page can also be directly called a form, and the form is mainly responsible for the data collection function in the web page. A form has three basic components, which can specifically include: a form tag, a form field, and a form button. Among them, the form tag: it contains the URL (Uniform Resource Locator) of the CGI (Common Gateway Interface) program used to process the form data and the method of submitting data to the server. Form field: includes text boxes, password boxes, hidden fields, multi-line text boxes, check boxes, radio buttons, drop-down selection boxes, and file upload boxes. Form buttons: include submit buttons, reset buttons, and general buttons; used to transmit data to the CGI script on the server or cancel input, and form buttons can also be used to control other processing tasks that define processing scripts.

[0028] In an embodiment of the present application, the filling rule is a pre-set rule for automatically filling the input boxes included in the form page. Each filling rule can be composed of two parts of rules. One is a rule for determining the type of input box included in the form page. Among them, the types of input boxes may include: password, password field, which is similar to a text box, but dots are displayed for the input content; radio, radio button, such as male or female; checkbox, check box; file, upload field (this type can be solved depending on the difficulty level); color, defines a color palette; tel, defines an input field for a phone number; email, defines an input field for an email; url, defines an input field containing a URl address; search, defines a search field; number, defines an input field containing a numerical value; range, defines an input containing a numeric value within a certain range; date, defines an input field for selecting a day, month, and year; month, defines an input field for selecting a month and year; week, defines an input field for selecting a week and year; time, defines an input field for selecting an hour and a second; datetime, defines an input field for selecting a time, day, month, and year (UTC time); datatime-local, defines an input field for selecting a time, day, month, and year (local time), etc., and no specific limitations are given here.

[0029] The other is the filling framework of the form page. Among them, the filling framework can include two frameworks: the automatic filling framework and the barrier-free service filling framework. When the two frameworks automatically fill in the form page, the underlying logic of filling is similar. Both of them first determine the type of the input box in the form to be filled, and then automatically fill the corresponding type of target content into the corresponding type of input box. However, the calling interfaces of the two filling frameworks are different. The calling interface of the automatic filling framework is the calling interface of Google's automatic filling service, and the calling interface of the barrier-free service filling framework is the barrier-free service calling interface.

[0030] Furthermore, each filling rule is configured based on the page type of the form page and the page view node information of the form page. Among them, the page type of the form page can be divided into two page types, one page type is the native control page, which can be understood as a form page developed using the native system kernel and can be directly operated on the system; the other page type is the self-rendering page, which can be understood as a form page designed based on the rendering engine.

[0031] Page view node information refers to the attribute information of each view node in the form page. Among them, after obtaining the form page, the tree structure information of the form page can be obtained through the interface provided by the automatic filling framework and the accessibility service, so that the view nodes constituting the form page and the attribute information of each view node can be determined according to the tree structure information. Among them, the view node can be understood as an element in the form page; the attribute information can be understood as information that can define an input box, such as the type attribute of the input box. In an embodiment of the present application, the attribute information of the input box can be at least one of the isEditable (editable) attribute, Hint (hint) attribute, Text (text) attribute, TextInputType (text input type) attribute, type (type) attribute, maxlength (maximum length) attribute and value (value) attribute, etc., and of course it can also be other attribute information, which is not specifically limited here.

[0032] In the embodiment of the present application, a correspondence between form pages and filling rules is pre-established. When a page to be filled is obtained, the filling rule corresponding to the page to be filled can be obtained according to the correspondence between the form page and the filling rule. The correspondence between the form page and the filling rule can be pre-stored in a preset storage area or a cloud server, and when it is needed, it can be directly obtained from the preset storage area or the cloud server.

[0033] Further, in the embodiment of the present application, the page to be filled can be a form page generated in response to a specified application or a specified event, or can be a form page generated in real time according to a preset generation rule. The specified application can be a pre-set application for triggering the acquisition of the filling rule, for example, the specified application can be a browser; the specified event can be a pre-set click event or page jump event for triggering the acquisition of the filling rule, etc., which is not specifically limited here.

[0034] Optionally, there may be multiple pages to be filled. When obtaining the filling rules corresponding to the pages to be filled, some of the multiple pages to be filled may have the same filling rules. Of course, some of the pages to be filled may have different filling rules. The filling rules corresponding to each of the multiple pages to be filled may be determined based on the correspondence between the form page and the filling rules.

[0035] Optionally, the page to be filled may be generated by the electronic device itself or sent by other electronic devices, which is not specifically limited here. The electronic device may be a laptop, tablet computer, desktop computer, smart phone, smart wearable device or other terminal, which is not specifically limited here.

[0036] Step S120: Based on the filling rule, fill the target content into the input box of the page to be filled.

[0037] In the embodiment of the present application, the target content is the content used to fill in the corresponding input box in the page to be filled in. The target content can be the information filled in by the user when registering, or it can be specific content pre-stored in a preset storage location or cloud server, which is not specifically limited here.

[0038] As a method, filling the target content into the input box of the page to be filled based on the filling rules means determining the types corresponding to each input box included in the page to be filled based on the determination rules of the input box types included in the filling rules, and then filling the corresponding type of target content into the corresponding type of input box in the page to be filled based on the filling framework included in the filling rules.

[0039] Specifically, when filling the target content into the input box of the page to be filled, it is necessary to first determine whether the type of each input box of the page to be filled matches the type of the target content. Only when the two match can automatic filling be performed. For example, there is an input box of "phone number" on a page to be filled, and its type is the input field of the phone number. If there is a preset information type that matches the "phone number" input box in the target content, it will be automatically filled; if there is no preset information type that matches the "phone number" input box, it cannot be automatically filled.

[0040] Optionally, in an embodiment of the present application, the target content may be acquired after the type of input box in the page to be filled is determined based on the filling rule, or after the filling rule is acquired. Of course, it may also be acquired at other times, which is not specifically limited here.

[0041] Among them, if the target content is obtained after the filling rules are obtained, in this case, the correspondence between the form page, the filling rules and the target content can be established in advance. After the filling rules corresponding to the page to be filled are obtained, the corresponding target content can be obtained based on the correspondence between the form page, the filling rules and the target content. Of course, the target content can also be obtained without the correspondence. In this case, multiple reference contents can be obtained. After the type of the input box in the page to be filled is determined by the filling rules, the matching reference content is determined from multiple reference contents based on the type of the input box in the page to be filled, and the matching reference content is used as the target content. Among them, when multiple reference contents are obtained, the relevance of each reference content to the page to be filled can be calculated, and the reference contents can be sorted in order from high to low based on the relevance, and then the reference content with the relevance ranked in the front specified position is used as the reference content to be obtained. Among them, when calculating the relevance, it can be calculated based on a pre-trained model, and of course it can also be calculated by other pre-set algorithms, which are not specifically limited here.

[0042] The present application provides a page filling method, which first obtains the filling rules corresponding to the page to be filled, and the filling rules are configured based on the page type of the form page and the page view node information of the form page. The filling rules include the determination rules and filling framework of the type of input boxes included in the form page, and the page view node information is the attribute information of each input box in the form page. Then, based on the filling rules, the target data is filled into the page to be filled. Through the above method, the corresponding filling rules are configured for the form page in advance according to the page type of the form page and the page view node information of the form page, so that form pages of different page types can have different filling rules, which expands the automatic filling scenario and provides more reliable support for the automatic filling of target data.

[0043] See also Figure 2 , a page filling method provided in an embodiment of the present application, the method comprising:

[0044] Step S210: Based on the page type of the form page and the page view node information, a corresponding filling rule is configured for the form page.

[0045] In an embodiment of the present application, by analyzing the form page that is ranked by the number of times used by users in the application store, corresponding filling rules can be configured for the form page based on the page type and page view node information.

[0046] As a way, after obtaining the form page, two aspects of the form page will be identified. One is page type identification, the main purpose of which is to identify whether the form page is an Android native control page or an application self-rendering page (including webview page, RN (ReactNative, a cross-platform mobile application development framework) page, Flutter page, etc.), and the purpose of identifying the page type is to match different filling frameworks. Android native control pages can be filled using the automatic filling framework provided by Google, while self-rendering pages can be filled using Android's barrier-free service area.

[0047] Furthermore, when identifying the page type of a form page, the page type of the form page can be determined based on whether the form page can display data when it is disconnected from the Internet, or based on whether the form page has paste and copy functions, or based on the way the form page is loaded.

[0048] As one of the ways, when determining the page type of a form page based on whether the form page can display data when the network is disconnected, if the form page cannot display data when the network is disconnected, then the page type of the form page can be determined to be a self-rendering page; if the form page can display data when the network is disconnected, then the page type of the form page can be determined to be a native control page.

[0049] As another method, when determining the page type of a form page based on whether the form page has the paste and copy function, if the form page has the paste and copy function, then the page type of the form page can be determined to be a self-rendering page; if the form page does not have the paste and copy function, then the page type of the form page can be determined to be a native control page.

[0050] As another method, when determining the page type of a form page based on the loading method of the form page, if there is a loading bar under the navigation bar of the form page, it can be determined that the page type of the form page is a self-rendering page; if there is no loading bar under the navigation bar of the form page, it can be determined that the page type of the form page is a native control page.

[0051] The second is the recognition of page view node information, because the Google AutoFill Framework can recognize the input box filling type by itself only when the third party actively adapts. Specifically, the third party specifies the type of data to be filled in for this input box, so the framework can recognize the type of input box and fill in the data. This is reflected in the fact that the autofill node (ViewNode) object has a hint attribute that indicates the input box type, while the input box node of the page that is not adapted to the framework has no type hint in this attribute, so it is necessary to identify these input boxes and specify the type of the input box through rules. For self-rendering pages, accessibility services are required to fill in. Accessibility services also have page view nodes, which will also have attributes that specify the type of the input box, but like autofill nodes, there will also be cases where there is no type attribute, so corresponding rule configuration is also required to indicate the input box type.

[0052] Therefore, after obtaining the form page, you can first use the interface provided by the auto-fill framework and the accessibility service to obtain the tree structure information of the form page in two ways, so that you can obtain the view nodes that make up the form page, and then you can obtain the attribute information included in the view nodes that make up the form page in the two schemes. Among them, the tree structure can be as follows Figure 3 shown.

[0053] By analyzing the attribute information contained in the view nodes that make up the form page under these two schemes, it can be obtained that in most cases, the input box type that can be filled in on the form page is of the EditText type, and its isEditable attribute is true. The type of content that needs to be filled in the input box type can be obtained through the Hint, Text and other attributes of the node. However, not all page node information is as above. In the case where the form page is a self-rendering page, the type of the input box node of the auto-fill framework is empty. In this case, the content cannot be filled in through the auto-fill framework. Therefore, accessibility services are needed to handle this situation as a backup when the auto-fill framework cannot fill in. From the analysis of the EditText node, it can also be obtained that not all such nodes contain valid fill-in type information. Therefore, at this stage, it is also necessary to manually identify the input box type for these situations in order to prepare for the production of rules. In the process of analyzing many pages, there are also filling scenarios that cannot be handled by the auto-fill framework and accessibility services. In this case, either both services cannot fill normally, or there are too many identical page identifiers (the application package name, application version number and form page name are used to uniquely determine the applicable page of the rule), and the situation cannot be handled by the rules. In this case, you can consider using OCR (Optical Character Recognition) + NER (Named Entity Recognition) as a backup for the filling solution.

[0054] After the page type and page view node information of the form page are identified in the above manner, corresponding filling rules can be configured for the form page based on the page type and page view node information of the form page.

[0055] Step S220: establishing a correspondence between the filling rule and the page information of the form page.

[0056] In the embodiment of the present application, the page information of the form page may include the application package name, application version number, form page name, and page name of the application to which the form page belongs, which are not limited here. Among them, the application package name, application version number, and form page name are used to uniquely determine the applicable page of the filling rule, and the page name is used by the operation and maintenance personnel to identify the specific page.

[0057] After the corresponding filling rules are configured for the form page in the above manner, a correspondence between the filling rules and the page information of the form page can be established, so that after the page to be filled is obtained, the filling rules corresponding to the page to be filled can be obtained through the correspondence.

[0058] Step S230: Based on the corresponding relationship and the page information of the page to be filled, a filling rule corresponding to the page to be filled is obtained.

[0059] In an embodiment of the present application, the page information of the page to be filled may include the application package name, application version number and form page name of the application to which the page to be filled belongs.

[0060] After obtaining the application package name, application version number and form page name of the application to which the page to be filled belongs, the filling rules configured locally or in the cloud can be matched based on the application package name, application version number and form page name of the application to which the page to be filled belongs.

[0061] Step S240: Based on the filling rule, fill the target content into the input box of the page to be filled.

[0062] The present application provides a page filling method, firstly, based on the page type and page view node information of the form page, the corresponding filling rules are configured for the form page, the corresponding relationship between the filling rules and the form page is established, and then based on the corresponding relationship and the page information of the page to be filled, the filling rules corresponding to the page to be filled are obtained, and based on the filling rules, the target content is filled into the input box of the page to be filled. Through the above method, the corresponding multiple filling rules can be configured based on the page type and page view node information of the form page, thereby providing reliable guarantee for the automatic filling of different types of form pages, and expanding the filling range supported by it.

[0063] See also Figure 4 , a page filling method provided in an embodiment of the present application, the method comprising:

[0064] Step S310: Based on the page view node information of the form page, it is determined whether the type of the input box included in the form page can be determined to obtain a determination result.

[0065] In an embodiment of the present application, after obtaining the page view node information of the form page, the type of the input box included in the form page can be determined based on the page view node information (HintText, Text, TextInputType and other information). When determining the type of the input box based on the page view node information, due to incomplete attribute information of some view nodes, the types of some input boxes may not be determined. Therefore, the judgment result may include one of the following: the type of the input box can be completely determined, the type of some input boxes can be determined, and the type of all input boxes cannot be determined.

[0066] Step S320: Based on the judgment result and the page type, a corresponding filling rule is configured for the form page.

[0067] In the embodiment of the present application, corresponding filling rules can be configured for the form page based on the judgment result obtained in the aforementioned manner and the page type of the form page.

[0068] As a method, if the judgment result represents that the types of all input boxes included in the form page can be determined, and the page type of the form page is a native control page, a first filling rule is configured for the form page, and the first filling rule includes determining the type of the input box included in the form page through the page view node information, and the filling frame is a first filling frame.

[0069] The first filling frame is an automatic filling frame. When the types of all input boxes included in the form page can be determined by the page view node information, and the page type of the form page is a native control page, the type of the input box included in the form page can be directly determined by the page view node information, and then the target content can be filled into the input box of the form page through the automatic filling frame.

[0070] As another way, if the judgment result represents that the types of some input boxes included in the form page can be determined, and the page type of the form page is a native control page, a second filling rule and / or a third filling rule is configured for the form page, wherein the second filling rule includes determining the type of the input box included in the form page in the form of a key-value pair, and the filling frame is a first filling frame, and the third filling rule includes determining the type of the input box included in the form page in the form of a key-value pair, and the filling frame is a second filling frame.

[0071] Among them, determining the type of the input box included in the form page in the form of a key-value pair means first filtering out the nodes included in the form page whose attribute information is isEditable=true, then manually identifying the type whose attribute information is isEditable=true, and matching the type through a key-value pair, where the key is the filtered serial number value and the value is the node type. The second filling framework is an accessibility service.

[0072] When the input box of the page to be filled has no obvious mark and the type of the input box cannot be determined, if the page type of the page to be filled is a native control page, then the type of input box included in the page to be filled can be determined by the input box type determination rule in the second filling rule, and then the target content can be filled into the input box of the page to be filled through the automatic filling framework in the second filling rule; if the input box of the page to be filled still cannot be filled through the second filling rule, the target content can be filled into the input box of the page to be filled through the accessibility service in the third filling rule.

[0073] As another way, if the judgment result indicates that the types of all input boxes included in the form page can be determined, and the page type of the form page is a self-rendering page, a fourth filling rule is configured for the form page, and the fourth filling rule includes determining the type of the input box included in the form page through the page view node information, and the filling frame is the second filling frame.

[0074] Among them, the second filling framework is an accessibility service. When the types of all input boxes included in the form page can be determined by the page view node information, and the page type of the form page is a self-rendering page, the type of the input box included in the form page can be directly determined by the page view node information (HintText, Text, TextInputType and other information), and then the target content can be filled into the input box of the form page through the accessibility service.

[0075] Optionally, if the judgment result indicates that the types of at least some of the input boxes included in the form page cannot be determined, and the page type of the form page is a self-rendering page, a fifth filling rule is configured for the form page, and the fifth filling rule includes determining the types of the input boxes included in the form page in the form of key-value pairs, and the filling frame is a second filling frame.

[0076] Among them, if the type of an input box in the input box included in the form page cannot be determined, and the page type of the form page is a self-rendering page, you can first filter out the nodes whose attribute information is isEditable=true included in the form page, and then manually identify the type whose attribute information is isEditable=true, and match the type through key-value pairs to determine the type of each input box in the form page.

[0077] That is, if the type of an input box included in the page to be filled cannot be determined, and the page type of the form page is a self-rendering page, the type of the input box included in the page to be filled can be determined by the input box type determination rule in the fifth filling rule, and then the target content can be filled into the input box of the page to be filled through the accessibility service in the fifth filling rule.

[0078] Optionally, if the judgment result indicates that the types of at least some of the input boxes included in the form page cannot be determined, the page type of the form page is a self-rendering page, and the page names of multiple form pages are the same, a sixth filling rule is configured for the multiple form pages, the sixth filling rule including determining a page identifier of each of the form pages based on page view node information, determining the type of the input box included in each of the form pages in the form of a key-value pair, and the filling frame is a second filling frame.

[0079] Determining the page ID of each form page based on the page view node information means first filtering out nodes with attribute information of isEditable=true based on the page view node information of each form page, recording the sequence number of each filtered node, and then taking the sum of the sequence numbers of nodes with attribute information of isEditable=true corresponding to each form page as the page ID of the form page. Figure 3 The label in the tree structure, that is, each node corresponds to a sequence number.

[0080] If there are multiple input boxes with the same name on the pages to be filled without obvious identification and the type of the input boxes cannot be determined, it is necessary to distinguish different form pages. At this time, it is necessary to first filter out the nodes with the attribute information of isEditable=true through the page view node information of each page to be filled, record the serial number of each filtered node, and then use the sum of the serial numbers of the nodes with the attribute information of isEditable=true corresponding to each page to be filled as the page identifier of the page to be filled, and then identify the type of input box included in each page to be filled based on manual recognition, match the type through key-value pairs to determine the type of input box included in each page to be filled, and then fill the target content into the input box of the page to be filled through accessibility services.

[0081] Optionally, if the judgment result indicates that the types of at least some of the input boxes included in the form page cannot be determined, a seventh filling rule is configured for the form page, the seventh filling rule including determining the types of the input boxes included in the form page through an input box type recognition model, and the filling frame is a second filling frame.

[0082] The recognition model may be an OCR+NER model. When the type of at least some input boxes included in the form page cannot be determined, the position and type of the input boxes in the form page may be determined directly through the OCR+NER model, and then the target content may be filled into the input boxes of the form page through the accessibility service. Optionally, the seventh filling rule may be used as a fallback solution for the first six filling rules.

[0083] The above filling rules can be shown in Table 1:

[0084] Table 1

[0085]

[0086]

[0087]

[0088] "1, 2, 3, 4, 5, 6, 7" in the above table refer to the aforementioned first filling rule, second filling rule, third filling rule, fourth filling rule, fifth filling rule, sixth filling rule, and seventh filling rule; the type in the above table refers to the aforementioned judgment result and page type; the usage method in the above table refers to the aforementioned method of determining the type of the input box in the form page; the receiving rule party processing flow in the above table refers to the aforementioned filling framework.

[0089] Step S330: establishing a correspondence between the filling rule and the page information of the form page.

[0090] In the embodiment of the present application, the correspondence between the filling rules and the page information of the form page can be expressed in the form of a json configuration file. The json configuration file is a configuration file made in a common json format. For example, the form of the json configuration file can be as follows:

[0091]

[0092]

[0093] Among them, the rule type is the specific rule serial number in the aforementioned table, the application package name, application version number and form page name are used to uniquely determine the applicable page of the rule, the page name is used to identify the specific page for operation and maintenance personnel, the page id is used in rule 6 to distinguish between the same pages, and the page rule parameters are used to specifically indicate the input type of a certain input box.

[0094] Step S340: Based on the corresponding relationship and the page information of the page to be filled, the filling rule corresponding to the page to be filled is obtained.

[0095] In an embodiment of the present application, when entering a form page to be filled, the application package name, application version number and form page name corresponding to the page to be filled can be used to match the filling rules configured locally or in the cloud. If no corresponding filling rule is matched, the filling is performed by default according to the first filling rule. If the corresponding filling rule is matched, the view nodes under the automatic filling framework or barrier-free service are traversed according to the filling rule, and then the required input box type nodes are filtered out. First, the input box type is judged according to the attribute information of the node itself. If the judgment cannot be made, these nodes are matched and bound to the configured type parameters one by one, so as to know the specific input type, and then the target content can be filled into the corresponding input box. In the case that neither the automatic filling framework nor the barrier-free service can handle it, the seventh filling rule can be used to determine the type of the input box in the page to be filled, so that the filling can be completed by barrier-free simulated click.

[0096] Step S350: Based on the filling rule, fill the target content into the input box of the page to be filled.

[0097] The present application provides a page filling method, which first determines whether the type of the input box included in the form page can be determined based on the page view node information of the form page to obtain a judgment result, and then configures the corresponding filling rule for the form page based on the judgment result and the page type of the form page, establishes a corresponding relationship between the filling rule and the page information of the form page, and then obtains the filling rule corresponding to the page to be filled based on the corresponding relationship and the page information of the page to be filled, and fills the target content into the input box of the page to be filled based on the corresponding filling rule. Through the above method, the corresponding filling rule is configured for the form page in advance according to the page type of the form page and the page view node information of the form page, so that form pages of different page types can have different filling rules, which expands the automatic filling scenario and provides more reliable support for the automatic filling of target data.

[0098] See also Figure 5 , an embodiment of the present application provides a page filling device 400, the device 400 comprising:

[0099] The rule acquisition unit 410 is used to obtain the filling rules corresponding to the page to be filled, and the filling rules are configured based on the page type of the form page and the page view node information of the page form. The filling rules include the determination rules and the filling framework of the type of input boxes included in the form page, and the page view node information is the attribute information of each input box in the form page.

[0100] As a method, the rule acquisition unit 410 is specifically used to configure corresponding filling rules for the form page based on the page type and page view node information of the form page; and establish a corresponding relationship between the filling rules and the page information of the form page.

[0101] The filling unit 420 is used to fill the target content into the input box of the page to be filled based on the filling rule.

[0102] See also Figure 6 , the device 400 further includes:

[0103] The relationship establishing unit 430 is used to configure corresponding filling rules for the form page based on the page type and page view node information of the form page; and establish a corresponding relationship between the filling rules and the page information of the form page.

[0104] As a method, the relationship establishing unit 430 is specifically used to determine whether the type of the input box included in the form page can be determined based on the page view node information to obtain a judgment result; based on the judgment result and the page type, configure corresponding filling rules for the form page.

[0105] As one of the methods, the relationship establishing unit 430 is specifically used to configure a first filling rule for the form page if the judgment result indicates that the types of all input boxes included in the form page can be determined, and the page type of the form page is a native control page, the first filling rule includes determining the type of the input box included in the form page through the page view node information, and the filling frame is a first filling frame; if the judgment result indicates that the types of some input boxes included in the form page can be determined, and the page type of the form page is a native control page, the second filling rule and / or the third filling rule is configured for the form page, wherein the second filling rule includes determining the type of the input box included in the form page in the form of a key-value pair, and the filling frame is a first filling frame, and the third filling rule includes determining the type of the input box included in the form page in the form of a key-value pair, and the filling frame is a second filling frame.

[0106] As another embodiment, the relationship establishing unit 430 is specifically used to configure a fourth filling rule for the form page if the judgment result indicates that the types of all input boxes included in the form page can be determined, and the page type of the form page is a self-rendering page, the fourth filling rule includes determining the types of the input boxes included in the form page through the page view node information, and the filling frame is the second filling frame; if the judgment result indicates that the types of at least some of the input boxes included in the form page cannot be determined, and the page type of the form page is a self-rendering page, the fifth filling rule is configured for the form page, the fifth filling rule includes determining the types of the input boxes included in the form page through key-value pairs, and the filling frame is the second filling frame.

[0107] As another method, the relationship establishing unit 430 is specifically used to configure a sixth filling rule for the multiple form pages if the judgment result indicates that the type of at least some of the input boxes included in the form page cannot be determined, the page type of the form page is a self-rendering page, and the page names of multiple form pages are the same, the sixth filling rule includes determining the page identifier of each of the form pages based on the page view node information, determining the type of the input box included in each of the form pages in the form of a key-value pair, and the filling frame is the second filling frame.

[0108] Optionally, the relationship establishing unit 430 is specifically used to configure a seventh filling rule for the form page if the judgment result indicates that the types of at least some of the input boxes included in the form page cannot be determined, the seventh filling rule including determining the types of the input boxes included in the form page through an input box type recognition model, and the filling frame is a second filling frame.

[0109] It should be noted that the device embodiment in the present application corresponds to the aforementioned method embodiment. The specific principles in the device embodiment can be found in the contents of the aforementioned method embodiment and will not be repeated here.

[0110] The following will be combined Figure 7 An electronic device provided by the present application is described.

[0111] See also Figure 7 Based on the above-mentioned page filling method and device, the embodiment of the present application also provides another electronic device 800 that can execute the above-mentioned page filling method. The electronic device 800 includes one or more (only one is shown in the figure) processors 802, a memory 804 and a network module 806 that are coupled to each other. Among them, the memory 804 stores a program that can execute the content of the above-mentioned embodiment, and the processor 802 can execute the program stored in the memory 804.

[0112] Among them, the processor 802 may include one or more processing cores. The processor 802 uses various interfaces and lines to connect various parts of the entire electronic device 800, and executes various functions and processes data of the electronic device 800 by running or executing instructions, programs, code sets or instruction sets stored in the memory 804, and calling data stored in the memory 804. Optionally, the processor 802 can be implemented in at least one hardware form of digital signal processing (Digital Signal Processing, DSP), field programmable gate array (Field-Programmable Gate Array, FPGA), and programmable logic array (Programmable Logic Array, PLA). The processor 802 can integrate one or a combination of a central processing unit (Central Processing Unit, CPU), a graphics processing unit (Graphics Processing Unit, GPU) and a modem. Among them, the CPU mainly processes the operating system, user interface and application programs; the GPU is responsible for rendering and drawing display content; and the modem is used to process wireless communications. It can be understood that the above-mentioned modem may not be integrated into the processor 802, but may be implemented separately through a communication chip.

[0113] The memory 804 may include a random access memory (RAM) or a read-only memory (ROM). The memory 804 may be used to store instructions, programs, codes, code sets or instruction sets. The memory 804 may include a program storage area and a data storage area, wherein the program storage area may store instructions for implementing an operating system, instructions for implementing at least one function (such as a touch function, a sound playback function, an image playback function, etc.), instructions for implementing the following various method embodiments, etc. The data storage area may also store data (such as a phone book, audio and video data, chat record data) created by the electronic device 800 during use.

[0114] The network module 806 is used to receive and send electromagnetic waves, realize the mutual conversion between electromagnetic waves and electrical signals, so as to communicate with a communication network or other devices, such as communicating with electronic devices. The network module 806 may include various existing circuit elements for performing these functions, such as antennas, radio frequency transceivers, digital signal processors, encryption / decryption chips, user identity modules (SIM) cards, memories, etc. The network module 806 may communicate with various networks such as the Internet, corporate intranets, wireless networks, or communicate with other devices via wireless networks. The above-mentioned wireless networks may include cellular telephone networks, wireless local area networks, or metropolitan area networks. For example, the network module 806 may exchange information with a base station.

[0115] Please refer to Figure 8 , which shows a structural block diagram of a computer-readable storage medium provided in an embodiment of the present application. The computer-readable storage medium 900 stores program codes, which can be called by a processor to execute the method described in the above method embodiment.

[0116] The computer-readable storage medium 900 may be an electronic memory such as a flash memory, an EEPROM (electrically erasable programmable read-only memory), an EPROM, a hard disk, or a ROM. Optionally, the computer-readable storage medium 900 includes a non-transitory computer-readable storage medium. The computer-readable storage medium 900 has storage space for program code 910 that performs any method step in the above method. These program codes can be read from or written to one or more computer program products. The program code 910 can be compressed, for example, in an appropriate form.

[0117] The present application provides a page filling method, device, electronic device and storage medium, which first obtains the filling rules corresponding to the page to be filled, and the filling rules are configured based on the page type of the form page and the page view node information of the form page. The filling rules include the determination rules and filling framework of the type of input boxes included in the form page, and the page view node information is the attribute information of each input box in the form page. Then, based on the filling rules, the target data is filled into the page to be filled. Through the above method, the corresponding filling rules are configured for the form page in advance according to the page type of the form page and the page view node information of the form page, so that form pages of different page types can have different filling rules, which expands the automatic filling scenario and provides more reliable support for the automatic filling of target data.

[0118] The embodiments of the present invention are described above in conjunction with the accompanying drawings, but the present invention is not limited to the above-mentioned specific implementation modes, which are merely illustrative rather than restrictive. Under the guidance of the present invention, ordinary technicians in this field can also make many forms without departing from the scope of protection of the present invention and the claims, all of which are within the protection of the present invention.

Claims

1. A page filling method, characterized in that: The method comprises: Obtaining a filling rule corresponding to the page to be filled, wherein the filling rule is configured based on the page type of the form page and the page view node information of the page form, wherein the filling rule includes a determination rule and a filling framework for the type of the input box included in the form page, and the page view node information is the attribute information of each input box in the form page; Based on the filling rule, the target content is filled into the input box of the page to be filled.

2. The method according to claim 1, characterized in that Before obtaining the filling rule corresponding to the page to be filled, the following step is also included: Based on the page type of the form page and the page view node information, configure corresponding filling rules for the form page; Establishing a correspondence between the filling rule and the page information of the form page; The step of obtaining the filling rule corresponding to the page to be filled includes: Based on the corresponding relationship and the page information of the page to be filled, a filling rule corresponding to the page to be filled is obtained.

3. The method according to claim 2, characterized in that The configuring corresponding filling rules for the form page based on the page type and page view node information of the form page includes: Based on the page view node information, determining whether the type of the input box included in the form page can be determined to obtain a determination result; Based on the judgment result and the page type, a corresponding filling rule is configured for the form page.

4. The method according to claim 3, characterized in that The configuring corresponding filling rules for the form page based on the judgment result and the page type includes: If the judgment result indicates that the types of all input boxes included in the form page can be determined, and the page type of the form page is a native control page, a first filling rule is configured for the form page, wherein the first filling rule includes determining the type of the input box included in the form page through the page view node information, and a filling frame is a first filling frame; If the judgment result indicates that the types of some input boxes included in the form page can be determined, and the page type of the form page is a native control page, a second filling rule and / or a third filling rule is configured for the form page, wherein the second filling rule includes determining the type of the input box included in the form page in the form of a key-value pair, and the filling frame is a first filling frame, and the third filling rule includes determining the type of the input box included in the form page in the form of a key-value pair, and the filling frame is a second filling frame.

5. The method according to claim 4, characterized in that The method further comprises: If the judgment result indicates that the types of all input boxes included in the form page can be determined, and the page type of the form page is a self-rendering page, a fourth filling rule is configured for the form page, the fourth filling rule including determining the type of the input box included in the form page through the page view node information, and a filling frame is a second filling frame; If the judgment result indicates that the types of at least some of the input boxes included in the form page cannot be determined, and the page type of the form page is a self-rendering page, a fifth filling rule is configured for the form page, and the fifth filling rule includes determining the types of the input boxes included in the form page in the form of key-value pairs, and the filling frame is a second filling frame.

6. The method according to claim 4, characterized in that The method further comprises: If the judgment result indicates that the types of at least some of the input boxes included in the form page cannot be determined, the page type of the form page is a self-rendering page, and the page names of multiple form pages are the same, a sixth filling rule is configured for the multiple form pages, the sixth filling rule including determining a page identifier of each of the form pages based on page view node information, determining the type of the input box included in each of the form pages in the form of a key-value pair, and a filling frame being a second filling frame.

7. The method according to claim 4, characterized in that The method further comprises: If the judgment result indicates that the types of at least some of the input boxes included in the form page cannot be determined, a seventh filling rule is configured for the form page, wherein the seventh filling rule includes determining the types of the input boxes included in the form page through an input box type recognition model, and the filling frame is a second filling frame.

8. A page filling device, characterized in that: The device comprises: A rule acquisition unit, used to acquire a filling rule corresponding to a page to be filled, wherein the filling rule is configured based on a page type of a form page and page view node information of a page form, wherein the filling rule includes a determination rule and a filling framework for a type of an input box included in the form page, and the page view node information is attribute information of each input box in the form page; A filling unit is used to fill the target content into the input box of the page to be filled based on the filling rule.

9. An electronic device, characterized in that: The method comprises one or more processors; one or more programs are stored in the memory and configured to execute the method according to any one of claims 1 to 7 by the one or more processors.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program code, wherein when the program code is executed by a processor, the method according to any one of claims 1 to 7 is executed.

Images