Page element identification method and device, equipment and storage medium

Abstract

The embodiment of the invention provides a page element recognition method and device, equipment and a storage medium. The method comprises the steps that at least one piece of element attribute information corresponding to a to-be-recognized page element in a target page is obtained; for each piece of element attribute information, a first conditional probability and a second conditional probability corresponding to the element attribute information are determined through an attribute statistical model, and the first conditional probability is used for representing the probability that the page element of the target element type in the target corpus has the element attribute information; the second conditional probability is used for representing the probability that the page elements of the non-target element type in the target corpus have the element attribute information, and the target corpus comprises a plurality of preset page elements of the target element type and a plurality of page elements of the non-target element type; and according to the first conditional probability and the second conditional probability respectively corresponding to the at least one piece of element attribute information, identifying an element type corresponding to the page element. According to the method, the accuracy of identifying the element types of the page elements can be improved.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the field of page search technology, and in particular, to a page element recognition method, device, equipment and storage medium. Background Art

[0002] In order to facilitate users to find page elements of a certain element type in a timely manner, it is necessary to first determine the element types corresponding to the multiple page elements included in each page. The user can enter the search term corresponding to the element type (for example, function point) to find the page element corresponding to the element type.

[0003] In the related art, a machine crawling method can be used to determine the element types corresponding to the multiple page elements included in each page. The specific steps are: according to the element type, multiple position information corresponding to the element type is determined, and for each position information, the text corresponding to the position information is crawled through a script, and the text is used as the page element corresponding to the element type.

[0004] However, the inventors have discovered that the related art has at least the following technical problems: the accuracy of the element types obtained by machine crawling is low. Summary of the invention

[0005] The embodiments of the present disclosure provide a method, apparatus, device and storage medium for identifying page elements to improve the recognition accuracy of page elements.

[0006] In a first aspect, an embodiment of the present disclosure provides a method for identifying page elements, including:

[0007] Obtaining at least one element attribute information corresponding to the page element to be identified in the target page;

[0008] For each element attribute information, a first conditional probability and a second conditional probability corresponding to the element attribute information are determined by an attribute statistical model, wherein the first conditional probability is used to represent the probability that a page element of a target element type in a target corpus has the element attribute information, and the second conditional probability is used to represent the probability that a page element of a non-target element type in the target corpus has the element attribute information, wherein the target corpus includes a plurality of preset page elements of target element types and a plurality of non-target element types, wherein the non-target element type is different from the target element type;

[0009] The element type corresponding to the page element is identified according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information.

[0010] In a second aspect, an embodiment of the present disclosure provides a device for identifying page elements, including:

[0011] An acquisition module, used to acquire at least one element attribute information corresponding to a page element to be identified in a target page;

[0012] A determination module, configured to determine, for each element attribute information, a first conditional probability and a second conditional probability corresponding to the element attribute information by using an attribute statistical model, wherein the first conditional probability is used to represent the probability that a page element of a target element type in a target corpus has the element attribute information, and the second conditional probability is used to represent the probability that a page element of a non-target element type in the target corpus has the element attribute information, wherein the target corpus includes a plurality of preset page elements of target element types and a plurality of non-target element types, wherein the non-target element type is different from the target element type;

[0013] The identification module is used to identify the element type corresponding to the page element according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information.

[0014] In a third aspect, an embodiment of the present disclosure provides an electronic device, including: a processor and a memory;

[0015] The memory stores computer-executable instructions;

[0016] The processor executes the computer-executable instructions stored in the memory, so that the at least one processor executes the page element identification method described in the first aspect and various possible designs of the first aspect.

[0017] In a fourth aspect, an embodiment of the present disclosure provides a computer-readable storage medium, in which computer execution instructions are stored. When a processor executes the computer execution instructions, the page element recognition method described in the first aspect and various possible designs of the first aspect is implemented.

[0018] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the page element recognition method as described in the first aspect and various possible designs of the first aspect.

[0019] The present embodiment provides a method, device, equipment and storage medium for identifying page elements. The method includes: obtaining at least one element attribute information corresponding to a page element to be identified in a target page; for each element attribute information, determining a first conditional probability and a second conditional probability corresponding to the element attribute information through an attribute statistical model, wherein the first conditional probability is used to represent the probability that a page element of a target element type in a target corpus has element attribute information, and the second conditional probability is used to represent the probability that a page element of a non-target element type in a target corpus has element attribute information, and the target corpus includes a plurality of preset page elements of target element types and a plurality of page elements of non-target element types, wherein the non-target element type is different from the target element type; identifying the element type corresponding to the page element according to the first conditional probability and the second conditional probability respectively corresponding to at least one element attribute information. In the disclosed embodiment, since the element type corresponding to the page element is identified from the perspective of the conditional probability of the element attribute information corresponding to the page element through the attribute statistical model, and the accuracy of the attribute statistical model is high, the accuracy of the determined element type is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0021] Figure 1 A schematic diagram of an application scenario of a page element recognition method provided by an embodiment of the present disclosure;

[0022] Figure 2 A flow chart of a page element recognition method provided by an embodiment of the present disclosure;

[0023] Figure 3 A schematic diagram of a page element recognition method provided by an embodiment of the present disclosure;

[0024] Figure 4 A flowchart of another page element identification method provided by an embodiment of the present disclosure;

[0025] Figure 5 A structural block diagram of a page element recognition device provided by an embodiment of the present disclosure;

[0026] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0028] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0029] On the page of a data product, there are many business-related page elements, such as the indicator names of data indicators (transaction amount, number of transactions, transaction order volume, transaction amount of goods, etc.), and the names of function points (live broadcast, live broadcast calendar, live broadcast room details, traffic source analysis, etc.). When the business volume becomes larger, the names of the function points involved in the entire product will be thousands, and the names of these function points will be scattered across multiple pages. When users use data products and need to find a data indicator, they cannot quickly locate the data that contains the data indicator.

[0030] In order to facilitate users to find page elements of a certain element type in a timely manner, it is necessary to first determine the element types corresponding to the multiple page elements included in each page. The user can enter the search term corresponding to the element type (for example, the indicator name) to find the page element corresponding to the element type.

[0031] In the related art, a machine crawling method can be used to determine the element types corresponding to each of the multiple page elements included in each page. The specific steps are: according to the element type, multiple position information corresponding to the element type is determined, and for each position information, the text corresponding to the position information is crawled through a script, and the text is used as the page element corresponding to the element type. However, the accuracy of the element type obtained by machine crawling is low.

[0032] It can be seen that how to improve the accuracy of identifying the element types corresponding to each of the multiple page elements included in the page is a problem that needs to be solved urgently.

[0033] In order to solve the above problems, the present embodiment provides the following technical concepts: first, obtain at least one element attribute information corresponding to the page element to be identified in the target page; then, for each element attribute information, determine the first conditional probability and the second conditional probability corresponding to the element attribute information through the attribute statistical model, wherein the first conditional probability is used to represent the probability that the page element of the target element type has the element attribute information, and the second conditional probability is used to represent the probability that the page element of the non-target element type has the element attribute information; finally, according to the first conditional probability and the second conditional probability respectively corresponding to at least one element attribute information, identify the element type corresponding to the page element.

[0034] In this case, the element type corresponding to the page element is identified from the perspective of conditional probability of the element attribute information corresponding to the page element through the attribute statistical model, and the accuracy of the attribute statistical model is high, thereby improving the accuracy of the determined element type.

[0035] The application scenarios of the embodiments of the present disclosure are explained below:

[0036] The page element identification method provided by the embodiment of the present disclosure can be applied in multiple scenarios. For example, the target element type can be "function point name", in the scenario of maintaining the "function point name" in the page. Figure 1 A schematic diagram of an application scenario of a page element recognition method provided by an embodiment of the present disclosure. Figure 1 As shown, the page includes multiple page elements. Each page element has a different type. Among them, through the page element identification method, the page element corresponding to the "function point name" can be identified from multiple page elements. Among them, the page elements corresponding to the "function point name" are: "live broadcast", "live broadcast calendar", "live broadcast room details". Among them, other page elements such as "home page" and "short video" do not belong to the "function point name". The data processing method provided by the embodiment of the present disclosure is described in detail using a detailed embodiment below.

[0037] Figure 2 The following is a flow chart of a method for identifying page elements provided in an embodiment of the present disclosure. The execution subject of the method for identifying page elements may be a terminal or a server. In the embodiment of the present disclosure, the execution subject is taken as an example to illustrate. Figure 2 , the page element identification method includes:

[0038] S201: Obtain at least one element attribute information corresponding to a page element to be identified in a target page.

[0039] In the embodiments of the present disclosure, a target page usually includes multiple page elements. The page element to be recognized can be any one or any number of the multiple page elements. Among them, the page elements can be presented in various forms. Optionally, the page elements can include images, questions, trend charts, etc. Exemplarily, the page elements can include XX trend charts, function point names, product information, serial numbers, avatars, function controls, etc. The acquisition method can be scraping by using a crawler or a script, etc.

[0040] In an embodiment of the present disclosure, the element attribute information includes one or more of text length, font size, font weight, text line height, text color, interaction type, and label type.

[0041] Among them, the interaction type is used to indicate whether the element can be clicked, and the interaction type can include an interactive type and a non-interactive type. Among them, the label type indicates the element label defined by the element in HTML (Hyper Text Markup Language). Among them, HTML is a language used to describe web pages. The label type of the element can include: , , .

[0042] In an embodiment of the present disclosure, as Figure 3 shown, in order to improve the recognition accuracy and efficiency, the obtained raw data can be preprocessed first. Correspondingly, obtaining at least one element attribute information corresponding to the page element to be recognized in the target page can include: obtaining at least one raw element attribute information corresponding to the page element to be recognized in the target page; filtering the raw element attributes that do not conform to the preset format in the at least one raw element attribute information to obtain the element attribute information.

[0043] Specifically, the raw element attributes that do not conform to the preset format can be null values or illegal values. Exemplarily, the raw element attribute information is: "&￥%@". Among them, "&￥%@" is an illegal value that does not conform to the preset format, so the raw element attribute information is deleted. Exemplarily, the raw element attribute information is: "font % bold". Among them, "%" is an illegal value that does not conform to the preset format, and after filtering, the element attribute information obtained is: "font bold".

[0044] Since the raw data (i.e., the raw element attribute information) is cleaned, the above null values and illegal values are filtered out, improving the effectiveness of the obtained element attribute information.

[0045] S202. For each element attribute information, determine a first conditional probability and a second conditional probability corresponding to the element attribute information through an attribute statistical model, wherein the first conditional probability is used to represent the probability that a page element of a target element type in a target corpus has the element attribute information, and the second conditional probability is used to represent the probability that a page element of a non-target element type in the target corpus has the element attribute information, and the target corpus includes a preset plurality of page elements of target element types and a plurality of page elements of non-target element types, wherein the non-target element type is different from the target element type.

[0046] In the disclosed embodiments, the element type can be identified from the perspective of element attribute information. The target element type may be the element type that is intended to be maintained. The non-target element type may be an element type other than the target element type. Exemplarily, the target element type may be a "function point name". The non-target element type may be an element type other than a "function point name"; for example, element types such as "product information", "avatar", and "function control". None of the above-mentioned element types are "function point names" and are non-target element types.

[0047] In the disclosed embodiment, the target corpus includes a plurality of page elements of preset target element types and a plurality of page elements of non-target element types. In order to facilitate the determination of whether a certain page element includes the sample attribute information, the target corpus may also include at least one sample attribute information corresponding to each page element. For example, a page element may be "live broadcast". The at least one sample attribute information corresponding to "live broadcast" may include: Songti, bold, size 4 font, etc. Optionally, the target corpus may be obtained by discovering page elements from the original data of multiple pages, annotating the element type of the page element, and adding it to the corpus.

[0048] Optionally, the attribute statistical model includes conditional probabilities corresponding to each of the multiple sample attribute information; accordingly, for each element attribute information, the first conditional probability and the second conditional probability corresponding to the element attribute information are determined by the attribute statistical model, including: according to the element attribute information, querying the target attribute information that is the same as the element attribute information from the multiple sample attribute information; determining the first conditional probability and the second conditional probability corresponding to the target attribute information as the first conditional probability and the second conditional probability corresponding to the element attribute information. The first conditional probability corresponding to the target attribute information is determined based on the first number of multiple target page elements including the target attribute information in the target corpus and the second number of first target page elements belonging to the target element type among the multiple target page elements. The second conditional probability corresponding to the target attribute information is determined based on the first number and the third number of second target page elements belonging to the non-target element type among the multiple target page elements.

[0049] Exemplarily, the first conditional probability can be represented by P(F|S), where S represents the first quantity and F represents the second quantity. Exemplarily, the second conditional probability can be represented by P(F'|S), where S represents the first quantity and F' represents the third quantity.

[0050] S203: Identify the element type corresponding to the page element according to the first conditional probability and the second conditional probability respectively corresponding to at least one element attribute information.

[0051] In the embodiment of the present disclosure, this step may include step S2031 to step S2032:

[0052] Step S2031: determine the first joint probability that the page element of the target element type has at least one element attribute information based on the first conditional probability respectively corresponding to the at least one element attribute information, and determine the second joint probability that the page element of the non-target element type has at least one element attribute information based on the second conditional probability respectively corresponding to the at least one element attribute information.

[0053] Among them, according to the first conditional probabilities respectively corresponding to at least one element attribute information, determining the first joint probability that the page element of the target element type has at least one element attribute information includes: determining the first product of the first conditional probabilities respectively corresponding to at least one element attribute information, and determining the first product as the first joint probability that the page element of the target element type has at least one element attribute information.

[0054] In some embodiments, the first joint probability is equal to the first product. Exemplarily, the first joint probability can be represented by P(A). The at least one element attribute information includes n, namely: S1, S2, S3, ..., Sn. Then P(A) = P(F|S1)*P(F|S2)*P(F|S3)*...*P(F|Sn).

[0055] In other embodiments, the first joint probability is equal to the product of the first product and a preset probability, and the preset probability may be a priori probability corresponding to the first conditional probability. Exemplarily, the first joint probability may be represented by P(A). The priori probability may be represented by P(F). The at least one element attribute information includes n, namely: S1, S2, S3, ..., Sn. Then P(A) = P(F|S1)*P(F|S2)*P(F|S3)*...*P(F|Sn)*P(F).

[0056] Among them, according to the second conditional probabilities respectively corresponding to at least one element attribute information, determining the second joint probability that the page element of the non-target element type has at least one element attribute information includes: determining the second product of the second conditional probabilities respectively corresponding to at least one element attribute information, and determining the second product as the second joint probability that the page element of the non-target element type has at least one element attribute information.

[0057] In some embodiments, the second joint probability is equal to the second product. Exemplarily, the second joint probability can be represented by P(B). The at least one element attribute information includes n, namely: S1, S2, S3, ..., Sn. Then P(B) = P(F'|S1)*P(F'|S2)*P(F'|S3)*...*P(F'|Sn).

[0058] In other embodiments, the second joint probability is equal to the product of the second product and a preset probability, and the preset probability may be a priori probability corresponding to the second conditional probability. Exemplarily, the second joint probability may be represented by P(B). The priori probability may be represented by P(F'). At least one element attribute information includes n, namely: S1, S2, S3, ..., Sn. Then P(B) = P(F'|S1)*P(F'|S2)*P(F'|S3)*...*P(F'|Sn)*P(F').

[0059] It should be noted that in the embodiment of the present disclosure, the prior probability P(F) corresponding to the first conditional probability and the prior probability P(F') corresponding to the second conditional probability can be set equal to 0.5.

[0060] Step S2032: Identify the element type corresponding to the page element according to the first joint probability and the second joint probability.

[0061] Optionally, this step may include: determining the sum of the first joint probability and the second joint probability; determining a third ratio of the first joint probability to the sum of the first joint probability and the second joint probability; if the third ratio is greater than a preset classification threshold, determining that the element type corresponding to the page element is the target element type; if the third ratio is not greater than the preset classification threshold, determining that the element type corresponding to the page element is not the target element type.

[0062] Exemplarily, the sum of the first joint probability and the second joint probability is: P(A)+P(B). The third ratio can be represented by P. Where,

[0063]

[0064] Exemplarily, the target element type is "function point", and the preset classification threshold can be represented by T. In the embodiment of the present disclosure, the value of the preset classification threshold is not specifically limited and can be set and modified as needed. Accordingly, this step is: if P is greater than T, it is determined that the element type corresponding to the page element is "function point", and if P is not greater than T, it is determined that the element type corresponding to the page element is not "function point".

[0065] The disclosed embodiment provides a method for identifying page elements, including: obtaining at least one element attribute information corresponding to a page element to be identified in a target page; for each element attribute information, determining a first conditional probability and a second conditional probability corresponding to the element attribute information through an attribute statistical model, wherein the first conditional probability is used to represent the probability that a page element of a target element type has the element attribute information, and the second conditional probability is used to represent the probability that a page element of a non-target element type has the element attribute information; identifying the element type corresponding to the page element according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information. In the disclosed embodiment, since the element type corresponding to the page element is identified from the perspective of the conditional probability of the element attribute information corresponding to the page element through the attribute statistical model, and the accuracy of the attribute statistical model is high, the accuracy of the determined element type is improved.

[0066] One point that needs to be explained is that, in one embodiment of the present disclosure, after step S204, it may also include: S205, if it is determined that the element type corresponding to the page element is the target element type (for example, a function point), then the page element is associated with the target page, thereby obtaining the associated data between the target page and the function point; that is, it is achieved to filter out the function points from the multiple page elements included in the target page, and establish the associated data between the target page and the function point.

[0067] Figure 4 A flowchart of another method for identifying page elements provided by an embodiment of the present disclosure. In an embodiment of the present disclosure, a method for obtaining an attribute statistical model in step S202 is described in detail. Optionally, the attribute statistical model includes conditional probabilities corresponding to each of a plurality of sample attribute information. Accordingly, refer to Figure 4 , the method comprising:

[0068] S401: For each piece of sample attribute information, determine a first number of target page elements in a target corpus that include the sample attribute information.

[0069] Exemplarily, there are n pieces of sample attribute information, which are respectively: S1, S2, S3, ..., Sn. In the target corpus: the first quantity of multiple target page elements including sample attribute information S1 is A1, the first quantity of multiple target page elements including sample attribute information S2 is A2, ..., the first quantity of multiple target page elements including sample attribute information Sn is An.

[0070] S402. Statistically count the second quantity of the first target page elements belonging to the target element type among the multiple target page elements, and determine the first ratio of the second quantity to the first quantity as the first conditional probability corresponding to the sample attribute information. Also, statistically count the third quantity of the second target page elements belonging to the non-target element type among the multiple target page elements, and determine the second ratio of the third quantity to the first quantity as the second conditional probability corresponding to the sample attribute information.

[0071] Exemplarily, the target element type is: "function point". The second quantity of the first target page elements belonging to "function point" among the multiple target page elements is F. The first conditional probability corresponding to sample attribute information S1 is: F / A1, the first conditional probability corresponding to sample attribute information S2 is: F / A2, ..., the first conditional probability corresponding to sample attribute information Sn is: F / An. The third quantity of the second target page elements belonging to non-"function point" among the multiple target page elements is F'. The second conditional probability corresponding to sample attribute information S1 is: F' / A1, the second conditional probability corresponding to sample attribute information S2 is: F' / A2, ..., the second conditional probability corresponding to sample attribute information Sn is: F' / An.

[0072] S403. Obtain an attribute statistical model through the first conditional probability and the second conditional probability respectively corresponding to the multiple pieces of sample attribute information.

[0073] In an embodiment of the present disclosure, the trained attribute statistical model can also be evaluated. A variable T (0 < T < 1) can be defined as the judgment threshold of the attribute statistical model. Among them, the magnitude of the T value determines the accuracy rate and recall rate of the model, and can be adjusted as appropriate in different application scenarios.

[0074] Exemplarily, during the evaluation process, 500 test samples can be extracted, and the positive and negative sample ratio is 1:4 (function point is the positive sample, non-function point is the negative sample), and the recall rate and accuracy rate of the model are evaluated by fixing the T value:

[0075] T 0.7 0.75 0.8 0.85 0.9 0.95 Accuracy 0.818 0.926 0.926 0.931 0.942 0.942 Recall 0.975 0.944 0.942 0.941 0.942 0.941

[0076] Figure 5 It is a structural block diagram of a page element recognition device provided by an embodiment of the present disclosure. As Figure 5As shown, the page element identification device includes: an acquisition module 501, a determination module 502 and an identification module 503.

[0077] The acquisition module 501 is used to acquire at least one element attribute information corresponding to the page element to be identified in the target page;

[0078] A determination module 502 is used to determine, for each element attribute information, a first conditional probability and a second conditional probability corresponding to the element attribute information by using an attribute statistical model, wherein the first conditional probability is used to represent the probability that a page element of a target element type in a target corpus has the element attribute information, and the second conditional probability is used to represent the probability that a page element of a non-target element type in the target corpus has the element attribute information, wherein the target corpus includes a plurality of preset page elements of the target element type and a plurality of page elements of the non-target element type, wherein the non-target element type is different from the target element type;

[0079] The identification module 503 is used to identify the element type corresponding to the page element according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information.

[0080] According to one or more embodiments of the present disclosure, the attribute statistical model includes conditional probabilities corresponding to multiple sample attribute information respectively; accordingly, the determination module 502, for each element attribute information, determines the first conditional probability and the second conditional probability corresponding to the element attribute information through the attribute statistical model, including: according to the element attribute information, querying the target attribute information that is the same as the element attribute information from multiple sample attribute information; determining the first conditional probability and the second conditional probability corresponding to the target attribute information as the first conditional probability and the second conditional probability corresponding to the element attribute information; wherein the first conditional probability corresponding to the target attribute information is determined based on a first number of multiple target page elements in the target corpus that include the target attribute information and a second number of first target page elements that belong to the target element type among the multiple target page elements, and the second conditional probability corresponding to the target attribute information is determined based on the first number and a third number of second target page elements that belong to the non-target element type among the multiple target page elements.

[0081] According to one or more embodiments of the present disclosure, the device also includes: a training module, the training module is used to determine, for each sample attribute information, a first number of multiple target page elements in the target corpus that include the sample attribute information; count the second number of first target page elements that belong to the target element type among the multiple target page elements, and determine a first ratio of the second number to the first number as a first conditional probability corresponding to the sample attribute information; and count the third number of second target page elements that belong to the non-target element type among the multiple target page elements, and determine a second ratio of the third number to the first number as a second conditional probability corresponding to the sample attribute information; obtain the attribute statistical model through the first conditional probabilities and the second conditional probabilities corresponding to each of the multiple sample attribute information.

[0082] According to one or more embodiments of the present disclosure, the identification module 503 identifies the element type corresponding to the page element according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information, including: determining a first joint probability that the page element of the target element type has the at least one element attribute information according to the first conditional probability respectively corresponding to the at least one element attribute information, and determining a second joint probability that the page element of the non-target element type has the at least one element attribute information according to the second conditional probability respectively corresponding to the at least one element attribute information; identifying the element type corresponding to the page element according to the first joint probability and the second joint probability.

[0083] According to one or more embodiments of the present disclosure, the identification module 503 determines the first joint probability that the page element of the target element type has the at least one element attribute information based on the first conditional probabilities respectively corresponding to the at least one element attribute information, including: determining the first product of the first conditional probabilities respectively corresponding to the at least one element attribute information, and determining the first product as the first joint probability that the page element of the target element type has the at least one element attribute information.

[0084] According to one or more embodiments of the present disclosure, the identification module 503 determines the second joint probability that the page element of the non-target element type has the at least one element attribute information based on the second conditional probability respectively corresponding to the at least one element attribute information, including: determining the second product of the second conditional probabilities respectively corresponding to the at least one element attribute information, and determining the second product as the second joint probability that the page element of the non-target element type has the at least one element attribute information.

[0085] According to one or more embodiments of the present disclosure, the identification module 503 identifies the element type corresponding to the page element based on the first joint probability and the second joint probability, including: determining the sum of the first joint probability and the second joint probability; determining a third ratio of the first joint probability to the sum of the first joint probability and the second joint probability; if the third ratio is greater than a preset classification threshold, determining that the element type corresponding to the page element is the target element type; if the third ratio is not greater than the preset classification threshold, determining that the element type corresponding to the page element is not the target element type.

[0086] According to one or more embodiments of the present disclosure, the element attribute information includes one or more of text length, font size, font thickness, text line height, text color, interaction type, and label type.

[0087] The acquisition module 501, the determination module 502 and the identification module 503 are connected in sequence. The data processing device provided in this embodiment can execute the technical solution of the above method embodiment, and its implementation principle and technical effect are similar, which will not be described in detail in this embodiment.

[0088] In order to implement the above embodiment, the embodiment of the present disclosure also provides an electronic device.

[0089] Figure 6 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present disclosure. Figure 6 The electronic device 600 may be a terminal device or a server. The terminal device may include but is not limited to mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, personal digital assistants (PDAs), tablet computers (Portable Android Devices, PADs), portable multimedia players (PMPs), vehicle terminals (such as vehicle navigation terminals), etc., and fixed terminals such as digital TVs, desktop computers, etc. Figure 6 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0090] like Figure 6As shown, the electronic device 600 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 602 or a program loaded from a storage device 608 to a random access memory (RAM) 603. Various programs and data required for the operation of the electronic device 600 are also stored in the RAM 603. The processing device 601, the ROM 602, and the RAM 603 are connected to each other via a bus 604. An input / output (I / O) interface 605 is also connected to the bus 604.

[0091] Typically, the following devices may be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a magnetic tape, a hard disk, etc.; and communication devices 609. The communication device 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 6 The electronic device 600 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead.

[0092] In particular, according to an embodiment of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program contains program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device 609, or installed from a storage device 608, or installed from a ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the method of the embodiment of the present disclosure are executed.

[0093] It should be noted that the computer-readable medium disclosed above may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by or in combination with an instruction execution system, device or device. In the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0094] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0095] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device executes the method shown in the above embodiment.

[0096] Computer program code for performing the operations of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0097] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0098] The units involved in the embodiments described in the present disclosure may be implemented by software or hardware. The name of a unit does not limit the unit itself in some cases. For example, the first acquisition unit may also be described as a "unit for acquiring at least two Internet Protocol addresses".

[0099] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0100] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0101] In a first aspect, according to one or more embodiments of the present disclosure, a method for identifying page elements is provided, comprising:

[0102] Obtain at least one element attribute information corresponding to the page element to be identified in the target page;

[0103] For each element attribute information, a first conditional probability and a second conditional probability corresponding to the element attribute information are determined by an attribute statistical model, wherein the first conditional probability is used to represent the probability that a page element of a target element type in a target corpus has the element attribute information, and the second conditional probability is used to represent the probability that a page element of a non-target element type in the target corpus has the element attribute information, wherein the target corpus includes a plurality of preset page elements of target element types and a plurality of non-target element types, wherein the non-target element type is different from the target element type;

[0104] The element type corresponding to the page element is identified according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information.

[0105] According to one or more embodiments of the present disclosure, the attribute statistical model includes conditional probabilities corresponding to multiple sample attribute information respectively; accordingly, for each element attribute information, determining the first conditional probability and the second conditional probability corresponding to the element attribute information through the attribute statistical model includes: according to the element attribute information, querying the target attribute information that is the same as the element attribute information from multiple sample attribute information; determining the first conditional probability and the second conditional probability corresponding to the target attribute information as the first conditional probability and the second conditional probability corresponding to the element attribute information; wherein the first conditional probability corresponding to the target attribute information is determined based on a first number of multiple target page elements in the target corpus that include the target attribute information and a second number of first target page elements that belong to the target element type among the multiple target page elements, and the second conditional probability corresponding to the target attribute information is determined based on the first number and a third number of second target page elements that belong to the non-target element type among the multiple target page elements.

[0106] According to one or more embodiments of the present disclosure, before querying the target attribute information identical to the element attribute information from multiple sample attribute information based on the element attribute information, it also includes: determining, for each sample attribute information, a first number of multiple target page elements in the target corpus that include the sample attribute information; counting a second number of first target page elements of the target element type among the multiple target page elements, and determining a first ratio of the second number to the first number as a first conditional probability corresponding to the sample attribute information; and counting a third number of second target page elements of the non-target element type among the multiple target page elements, and determining a second ratio of the third number to the first number as a second conditional probability corresponding to the sample attribute information; and obtaining the attribute statistical model through the first conditional probabilities and the second conditional probabilities corresponding to each of the multiple sample attribute information.

[0107] According to one or more embodiments of the present disclosure, the identifying the element type corresponding to the page element according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information includes: determining a first joint probability that the page element of the target element type has the at least one element attribute information according to the first conditional probability respectively corresponding to the at least one element attribute information, and determining a second joint probability that the page element of the non-target element type has the at least one element attribute information according to the second conditional probability respectively corresponding to the at least one element attribute information; identifying the element type corresponding to the page element according to the first joint probability and the second joint probability.

[0108] According to one or more embodiments of the present disclosure, determining the first joint probability that the page element of the target element type has the at least one element attribute information based on the first conditional probabilities respectively corresponding to the at least one element attribute information includes: determining the first product of the first conditional probabilities respectively corresponding to the at least one element attribute information, and determining the first product as the first joint probability that the page element of the target element type has the at least one element attribute information.

[0109] According to one or more embodiments of the present disclosure, determining the second joint probability that the page element of the non-target element type has the at least one element attribute information based on the second conditional probability respectively corresponding to the at least one element attribute information includes: determining the second product of the second conditional probabilities respectively corresponding to the at least one element attribute information, and determining the second product as the second joint probability that the page element of the non-target element type has the at least one element attribute information.

[0110] According to one or more embodiments of the present disclosure, identifying the element type corresponding to the page element based on the first joint probability and the second joint probability includes: determining the sum of the first joint probability and the second joint probability; determining a third ratio of the first joint probability to the sum of the first joint probability and the second joint probability; if the third ratio is greater than a preset classification threshold, determining that the element type corresponding to the page element is a target element type; if the third ratio is not greater than the preset classification threshold, determining that the element type corresponding to the page element is not the target element type.

[0111] According to one or more embodiments of the present disclosure, the element attribute information includes one or more of text length, font size, font thickness, text line height, text color, interaction type, and label type.

[0112] In a second aspect, according to one or more embodiments of the present disclosure, a device for identifying page elements is provided, the device comprising:

[0113] An acquisition module, used to acquire at least one element attribute information corresponding to a page element to be identified in a target page;

[0114] A determination module, configured to determine, for each element attribute information, a first conditional probability and a second conditional probability corresponding to the element attribute information by using an attribute statistical model, wherein the first conditional probability is used to represent the probability that a page element of a target element type in a target corpus has the element attribute information, and the second conditional probability is used to represent the probability that a page element of a non-target element type in the target corpus has the element attribute information, wherein the target corpus includes a plurality of preset page elements of target element types and a plurality of non-target element types, wherein the non-target element type is different from the target element type;

[0115] The identification module is used to identify the element type corresponding to the page element according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information.

[0116] According to one or more embodiments of the present disclosure, the attribute statistical model includes conditional probabilities corresponding to multiple sample attribute information respectively; accordingly, the determination module determines, for each element attribute information, the first conditional probability and the second conditional probability corresponding to the element attribute information through the attribute statistical model, including: according to the element attribute information, querying the target attribute information that is the same as the element attribute information from multiple sample attribute information; determining the first conditional probability and the second conditional probability corresponding to the target attribute information as the first conditional probability and the second conditional probability corresponding to the element attribute information; wherein the first conditional probability corresponding to the target attribute information is determined based on a first number of multiple target page elements in the target corpus that include the target attribute information and a second number of first target page elements that belong to the target element type among the multiple target page elements, and the second conditional probability corresponding to the target attribute information is determined based on the first number and a third number of second target page elements that belong to the non-target element type among the multiple target page elements.

[0117] According to one or more embodiments of the present disclosure, the device also includes: a training module, the training module is used to determine, for each sample attribute information, a first number of multiple target page elements in the target corpus that include the sample attribute information; count the second number of first target page elements that belong to the target element type among the multiple target page elements, and determine a first ratio of the second number to the first number as a first conditional probability corresponding to the sample attribute information; and count the third number of second target page elements that belong to the non-target element type among the multiple target page elements, and determine a second ratio of the third number to the first number as a second conditional probability corresponding to the sample attribute information; obtain the attribute statistical model through the first conditional probabilities and the second conditional probabilities corresponding to each of the multiple sample attribute information.

[0118] According to one or more embodiments of the present disclosure, the identification module identifies the element type corresponding to the page element according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information, including: determining the first joint probability that the page element of the target element type has the at least one element attribute information according to the first conditional probability respectively corresponding to the at least one element attribute information, and determining the second joint probability that the page element of the non-target element type has the at least one element attribute information according to the second conditional probability respectively corresponding to the at least one element attribute information; identifying the element type corresponding to the page element according to the first joint probability and the second joint probability.

[0119] According to one or more embodiments of the present disclosure, the identification module determines the first joint probability that the page element of the target element type has the at least one element attribute information based on the first conditional probability respectively corresponding to the at least one element attribute information, including: determining the first product of the first conditional probabilities respectively corresponding to the at least one element attribute information, and determining the first product as the first joint probability that the page element of the target element type has the at least one element attribute information.

[0120] According to one or more embodiments of the present disclosure, the identification module determines the second joint probability that the page element of the non-target element type has the at least one element attribute information based on the second conditional probability respectively corresponding to the at least one element attribute information, including: determining the second product of the second conditional probabilities respectively corresponding to the at least one element attribute information, and determining the second product as the second joint probability that the page element of the non-target element type has the at least one element attribute information.

[0121] According to one or more embodiments of the present disclosure, the identification module identifies the element type corresponding to the page element based on the first joint probability and the second joint probability, including: determining the sum of the first joint probability and the second joint probability; determining a third ratio of the first joint probability to the sum of the first joint probability and the second joint probability; if the third ratio is greater than a preset classification threshold, determining that the element type corresponding to the page element is the target element type; if the third ratio is not greater than the preset classification threshold, determining that the element type corresponding to the page element is not the target element type.

[0122] According to one or more embodiments of the present disclosure, the element attribute information includes one or more of text length, font size, font thickness, text line height, text color, interaction type, and label type.

[0123] In a third aspect, according to one or more embodiments of the present disclosure, there is provided an electronic device, comprising: a processor, and a memory communicatively connected to the processor;

[0124] The memory stores computer-executable instructions;

[0125] The processor executes the computer-executable instructions stored in the memory to implement the data processing method described in the first aspect and various possible designs of the first aspect.

[0126] In a fourth aspect, according to one or more embodiments of the present disclosure, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer execution instructions. When a processor executes the computer execution instructions, the data processing method described in the first aspect and various possible designs of the first aspect is implemented.

[0127] In a fifth aspect, an embodiment of the present disclosure provides a computer program product, including a computer program, which, when executed by a processor, implements the data processing method described in the first aspect and various possible designs of the first aspect.

[0128] The above description is only a preferred embodiment of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of disclosure involved in the present disclosure is not limited to the technical solutions formed by a specific combination of the above technical features, but should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above disclosed concept. For example, the above features are replaced with the technical features with similar functions disclosed in the present disclosure (but not limited to) by each other to form a technical solution.

[0129] In addition, although each operation is described in a specific order, this should not be understood as requiring these operations to be performed in the specific order shown or in a sequential order. Under certain circumstances, multitasking and parallel processing may be advantageous. Similarly, although some specific implementation details are included in the above discussion, these should not be interpreted as limiting the scope of the present disclosure. Some features described in the context of a separate embodiment can also be implemented in a single embodiment in combination. On the contrary, the various features described in the context of a single embodiment can also be implemented in multiple embodiments individually or in any suitable sub-combination mode.

[0130] Although the subject matter has been described in language specific to structural features and / or methodological logical actions, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. On the contrary, the specific features and actions described above are merely example forms of implementing the claims.

Claims

1. A page element recognition method, characterized in that: include: Obtaining at least one element attribute information corresponding to the page element to be identified in the target page; For each element attribute information, a first conditional probability and a second conditional probability corresponding to the element attribute information are determined by an attribute statistical model, wherein the first conditional probability is used to represent the probability that a page element of a target element type in a target corpus has the element attribute information, and the second conditional probability is used to represent the probability that a page element of a non-target element type in the target corpus has the element attribute information, wherein the target corpus includes a plurality of preset page elements of target element types and a plurality of non-target element types, wherein the non-target element type is different from the target element type; The element type corresponding to the page element is identified according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information.

2. The method according to claim 1, characterized in that The attribute statistical model includes conditional probabilities corresponding to each of the plurality of sample attribute information; Accordingly, for each element attribute information, determining the first conditional probability and the second conditional probability corresponding to the element attribute information by using the attribute statistical model includes: According to the element attribute information, searching for target attribute information identical to the element attribute information from a plurality of sample attribute information; Determine the first conditional probability and the second conditional probability corresponding to the target attribute information as the first conditional probability and the second conditional probability corresponding to the element attribute information; Wherein, the first conditional probability corresponding to the target attribute information is determined based on a first number of target page elements in the target corpus that include the target attribute information and a second number of first target page elements that belong to the target element type among the multiple target page elements, and the second conditional probability corresponding to the target attribute information is determined based on the first number and a third number of second target page elements that belong to the non-target element type among the multiple target page elements.

3. The method according to claim 2, characterized in that Before searching for target attribute information identical to the element attribute information from a plurality of sample attribute information according to the element attribute information, the method further includes: For each sample attribute information, determining a first number of multiple target page elements in the target corpus that include the sample attribute information; counting a second number of first target page elements that are of a target element type among the multiple target page elements, and determining a first ratio of the second number to the first number as a first conditional probability corresponding to the sample attribute information; and counting a third number of second target page elements that are of a non-target element type among the multiple target page elements, and determining a second ratio of the third number to the first number as a second conditional probability corresponding to the sample attribute information; The attribute statistical model is obtained through the first conditional probabilities and the second conditional probabilities respectively corresponding to the plurality of sample attribute information.

4. The method according to claim 1, characterized in that The identifying the element type corresponding to the page element according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information includes: Determine, based on the first conditional probabilities respectively corresponding to the at least one element attribute information, a first joint probability that the page element of the target element type has the at least one element attribute information, and determine, based on the second conditional probabilities respectively corresponding to the at least one element attribute information, a second joint probability that the page element of the non-target element type has the at least one element attribute information; An element type corresponding to the page element is identified according to the first joint probability and the second joint probability.

5. The method according to claim 4, characterized in that The determining, according to the first conditional probabilities respectively corresponding to the at least one element attribute information, a first joint probability that the page element of the target element type has the at least one element attribute information includes: A first product of first conditional probabilities respectively corresponding to the at least one element attribute information is determined, and the first product is determined as a first joint probability that a page element of the target element type has the at least one element attribute information.

6. The method according to claim 4, characterized in that The determining, according to the second conditional probabilities respectively corresponding to the at least one element attribute information, the second joint probability of the page element of the non-target element type having the at least one element attribute information comprises: A second product of second conditional probabilities respectively corresponding to the at least one element attribute information is determined, and the second product is determined as a second joint probability that the page element of the non-target element type has the at least one element attribute information.

7. The method according to claim 4, characterized in that The identifying, according to the first joint probability and the second joint probability, the element type corresponding to the page element includes: determining a sum of the first joint probability and the second joint probability; determining a third ratio of the first joint probability to a sum of the first joint probability and the second joint probability; If the third ratio is greater than the preset classification threshold, it is determined that the element type corresponding to the page element is the target element type; if the third ratio is not greater than the preset classification threshold, it is determined that the element type corresponding to the page element is not the target element type.

8. The method according to any one of claims 1 to 7, characterized in that: The element attribute information includes one or more of text length, font size, font weight, text line height, text color, interaction type and label type.

9. A page element recognition device, characterized in that: include: An acquisition module, used to acquire at least one element attribute information corresponding to a page element to be identified in a target page; A determination module, configured to determine, for each element attribute information, a first conditional probability and a second conditional probability corresponding to the element attribute information by using an attribute statistical model, wherein the first conditional probability is used to represent the probability that a page element of a target element type in a target corpus has the element attribute information, and the second conditional probability is used to represent the probability that a page element of a non-target element type in the target corpus has the element attribute information, wherein the target corpus includes a plurality of preset page elements of target element types and a plurality of non-target element types, wherein the non-target element type is different from the target element type; The identification module is used to identify the element type corresponding to the page element according to the first conditional probability and the second conditional probability respectively corresponding to the at least one element attribute information.

10. An electronic device, characterized in that: include: Processor and memory; The memory stores computer-executable instructions; The processor executes the computer-executable instructions stored in the memory, so that the processor executes the page element identification method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer-executable instructions. When the processor executes the computer-executable instructions, the page element recognition method according to any one of claims 1 to 8 is implemented.

12. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method for identifying page elements according to any one of claims 1 to 8 is implemented.

Images