A webpage data collecting method and device

Abstract

The application discloses a webpage data collection method and device, and relates to the field of computer application. The method comprises the following steps: receiving an opening instruction input by a user, wherein the opening instruction comprises a webpage identifier; starting and displaying a webpage represented by the webpage identifier according to the webpage identifier, wherein the webpage comprises webpage elements; if an operation instruction input by the user on the webpage elements is received and the webpage elements have been configured with a reporting attribute, generating a webpage data table under the webpage; and when a reporting time arrives, sending a first request to a database, wherein the first request comprises the webpage data table. In the process of webpage data reporting, the application avoids code writing, ensures that the data tables reported on each page are different, guarantees correct reporting, simplifies the webpage data reporting process, and realizes data monitoring on the webpage generated by a low-code platform.

Description

Technical Field

[0001] This invention relates to the field of computer applications, and in particular to a method and apparatus for collecting web page data. Background Technology

[0002] Webpage reporting refers to the process of collecting user actions on webpage elements and reporting the collected data to a database. Developers can then identify and analyze the webpage data recorded in the database to implement risk monitoring and alerts, facilitating webpage optimization and upgrades.

[0003] The existing data reporting process is as follows: The user device packages the reporting processing logic and the common fields of the data table to be reported into a plugin. Then, the user device receives external code input by the developer, obtains the characteristic fields (or non-common fields) of the data table, and integrates the common fields and the characteristic fields of the data table to obtain the webpage data. Finally, the user device sends a GET request carrying the webpage data to the database, transmitting the webpage data to the database, enabling the developer to perform subsequent identification and statistical operations.

[0004] Low-code platforms are a product of the development of web front-end technology, providing developers with a platform for quickly and visually building web pages. Within a low-code platform, developers can create web pages using operations such as dragging, dropping, and pressing keys.

[0005] However, the above solution also has the following drawbacks: 1) The existing reporting logic is complex. If the common fields or characteristic fields in the data table need to be modified, the developers need to manually modify the reporting code snippets, resulting in a long modification and reporting cycle; 2) Since the web pages generated by the low-code platform are all automatically generated, if data reporting is to be achieved, the data reporting process should also be automated. The existing reporting solution cannot achieve automated web page data collection and cannot monitor such web pages. Summary of the Invention

[0006] In view of this, embodiments of the present invention provide a method and apparatus for collecting web page data to solve the problems of complex reporting logic and long modification and reporting cycles in existing web page reporting schemes. At the same time, it also enables data monitoring of web pages generated by low-code platforms.

[0007] In a first aspect, embodiments of the present invention provide a method for collecting web page data, the method comprising:

[0008] Receives a user-inputted command to open the webpage, the command including a webpage identifier;

[0009] Based on the webpage identifier, the webpage represented by the webpage identifier is launched and displayed, the webpage including webpage elements;

[0010] If, under the webpage, an operation instruction for the webpage element is received from the user and the webpage element has been configured with a reporting attribute, a webpage data table is generated.

[0011] When the reporting time is reached, a first request is sent to the database, the first request including the web page data table.

[0012] Optionally, generating the webpage data table specifically includes:

[0013] From the webpage, obtain the configured product number and the table number of the webpage data table, wherein the webpage data table belongs to the product indicated by the product number;

[0014] Send a table field request to the reporting platform, the table field request including the product number and the table number;

[0015] Receive the table field response sent by the reporting platform, the table field response including the common fields included in the web page data table, the common field column corresponding to the common fields, the characteristic fields, and the characteristic field column corresponding to the characteristic fields;

[0016] The common field column and the characteristic field column are integrated to obtain the web page data table;

[0017] The characteristic field is used to characterize the webpage element.

[0018] Optionally, the table field response may also include descriptive attribute information of the feature field;

[0019] The step of sending a first request to the database when the reporting time is reached specifically includes:

[0020] The reporting timing is obtained from the described attribute information;

[0021] When the reporting time is reached, the first request is sent to the database.

[0022] Optionally, before receiving the user's input to open the program, the method further includes:

[0023] Receive the first configuration command input;

[0024] According to the first configuration instruction, the original page attributes of the webpage are configured to obtain the configured page attributes;

[0025] Receive the second configuration command input;

[0026] According to the second configuration instruction, the original page elements included in the webpage are configured to obtain the configured page elements;

[0027] Generate a reporting file based on the configured page attributes and page elements;

[0028] The reported file is referenced in the source code of the webpage using script tags, thereby generating the webpage.

[0029] Optionally, configuring the original page attributes of the webpage to obtain the configured page attributes specifically includes:

[0030] Configure the product number and the table number in the page attributes, and configure the attribute values ​​of the product number and the table number as global variables;

[0031] The configuration of the original page elements included in the webpage to obtain the configured page elements specifically includes:

[0032] Configure the reporting attributes of the page element to obtain the configured page element.

[0033] Optionally, after generating the webpage, the method further includes:

[0034] The reporting file is stored in the reporting platform so that the reporting platform can generate the characteristic field, the characteristic field column, and the descriptive attribute information accordingly.

[0035] Secondly, embodiments of the present invention provide a web page data collection device, the device comprising:

[0036] A receiving unit is configured to receive an opening instruction input by a user, wherein the opening instruction includes a webpage identifier;

[0037] The display unit is configured to initiate and display the webpage represented by the webpage identifier according to the webpage identifier, wherein the webpage includes webpage elements;

[0038] The first generation unit is configured to generate a webpage data table if the receiving unit receives an operation instruction for the webpage element input by the user and the webpage element has been configured with a reporting attribute.

[0039] The sending unit is used to send a first request to the database when the reporting time is reached, the first request including the web page data table.

[0040] Optionally, the first generation unit is specifically used to obtain the configured product number and the table number of the web page data table from the web page, wherein the web page data table belongs to the product indicated by the product number;

[0041] Send a table field request to the reporting platform, the table field request including the product number and the table number;

[0042] Receive the table field response sent by the reporting platform, the table field response including the common fields included in the web page data table, the common field column corresponding to the common fields, the characteristic fields, and the characteristic field column corresponding to the characteristic fields;

[0043] The common field column and the characteristic field column are integrated to obtain the web page data table;

[0044] The characteristic field is used to characterize the webpage element.

[0045] Optionally, the table field response may also include descriptive attribute information of the feature field;

[0046] The sending unit is specifically used to obtain the reporting timing from the description attribute information;

[0047] When the reporting time is reached, the first request is sent to the database.

[0048] Optionally, the receiving unit is further configured to receive an input first configuration instruction;

[0049] The device further includes: a first configuration unit, configured to configure the original page attributes of the webpage according to the first configuration instruction, to obtain the configured page attributes;

[0050] The receiving unit is further configured to receive a second configuration instruction input.

[0051] The device further includes: a second configuration unit, configured to configure the original page elements included in the webpage according to the second configuration instruction, so as to obtain the configured page elements;

[0052] The second generation unit is used to generate the reporting file based on the configured page attributes and page elements;

[0053] The third generation unit is used to reference the reported file into the source code of the webpage using script tags, thereby generating the webpage.

[0054] Optionally, the first configuration unit is specifically used to configure the product number and the table number in the page attributes, and to configure the attribute values ​​of the product number and the table number as global variables;

[0055] The second configuration unit is specifically used to configure the reported attributes of the page element to obtain the configured page element.

[0056] Optionally, the device further includes:

[0057] The storage unit is used to store the reporting file in the reporting platform so that the reporting platform can generate the characteristic field, the characteristic field column and the descriptive attribute information accordingly.

[0058] Thirdly, embodiments of the present invention provide an electronic device, the electronic device comprising: a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is disposed within the space enclosed by the housing, and the processor and the memory are disposed on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the electronic device; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, for executing the web page data collection method described in the first aspect above.

[0059] Fourthly, embodiments of the present invention provide a computer-readable storage medium storing one or more programs, which can be executed by one or more processors to implement the web page data collection method described in the first aspect.

[0060] This invention provides a method and apparatus for collecting webpage data. An electronic device receives an open command input by a user, the open command including a webpage identifier. Based on the webpage identifier, the electronic device launches and displays the webpage represented by the webpage identifier, the webpage including webpage elements. Within the webpage, if an operation command for a webpage element is received from the user and the webpage element has been configured with a reporting attribute, the electronic device generates a webpage data table. When a reporting opportunity arises, the electronic device sends a first request to a database, the first request including the webpage data table.

[0061] The aforementioned solution can solve the problems of complex reporting logic and long modification and reporting cycles in existing web page reporting solutions. In the process of web page data reporting, this invention avoids writing code, ensures that the data table reported on each page is different, and also ensures correct reporting, simplifies the web page data reporting process, and also realizes data monitoring of web pages generated by low-code platforms. Attached Figure Description

[0062] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0063] Figure 1 A flowchart illustrating a method for collecting web page data provided in an embodiment of the present invention;

[0064] Figure 2 A schematic diagram of the structure of the web page data collection device provided in an embodiment of the present invention;

[0065] Figure 3 This is a schematic diagram of an electronic device structure provided in an embodiment of the present invention. Detailed Implementation

[0066] The embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0067] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0068] The following is in conjunction with the appendix Figure 1 The solutions provided in the embodiments of the present invention will be described in detail below. Figure 1 This is a flowchart illustrating a webpage data collection method provided in an embodiment of the present invention. The solution provided in this embodiment can be applied to low-code platforms or non-low-code platforms. In this embodiment, the implementing entity is an electronic device. This electronic device can be a terminal device, such as a personal computer or desktop computer; it can also be a server with a display device. Figure 1 As shown, the method in this embodiment specifically includes the following steps: Figure 1 As shown, the method in this embodiment specifically includes the following steps:

[0069] Step 110: Receive an opening instruction input by the user, wherein the opening instruction includes a webpage identifier.

[0070] In this step, the user or administrator wants to view a specific webpage. For example, they might want to view images or retrieve information from the webpage.

[0071] Users or administrators can input opening commands into electronic devices via command lines, and these opening commands include web page identifiers.

[0072] Electronic devices receive user input commands to open web pages and obtain web page identifiers from them.

[0073] Step 120: Based on the webpage identifier, start and display the webpage represented by the webpage identifier, the webpage including webpage elements.

[0074] In this step, after the electronic device obtains the webpage identifier, it launches the webpage corresponding to the webpage identifier and displays the webpage through the display device so that the user can see it.

[0075] It is understandable that the above webpage contains multiple webpage elements, including but not limited to images, buttons, text, etc.

[0076] Step 130: Under the webpage, if the user inputs an operation instruction on the webpage element and the webpage element has been configured with a reporting attribute, then a webpage data table is generated.

[0077] In this step, after the display device shows the webpage, the user can browse the webpage and interact with its elements. For example, the user can click on a button, download an image, etc.

[0078] On a webpage, the electronic device identifies whether it has received user input instructions to operate on webpage elements. If it identifies that it has received user input instructions to operate on webpage elements, the electronic device further identifies whether the webpage element operated by the user has a reporting attribute configured.

[0079] If the web page element that the user interacts with has been configured with a reporting attribute, the electronic device generates a web page data table.

[0080] It should be noted that the reporting attributes of web page elements are options that are configured and selected by administrators (also known as product managers, or personnel who need web page data tables) during the web page generation process. In the subsequent embodiments, the reporting attributes of web page elements will be explained when describing the web page generation process.

[0081] Optionally, the aforementioned electronic device generates a web page data table, the specific process of which is as follows:

[0082] From the webpage, the electronic device obtains the configured product number and the table number of the webpage data table, which belongs to the product indicated by the product number. Both the product number and the table number are global variables configured by administrators during the webpage generation process, and will be explained in subsequent embodiments.

[0083] In this embodiment of the invention, the product can specifically refer to products under different business lines or product lines. For example, search products, tool products, etc. Under each product, multiple web page data tables can be created, each with different content, to achieve reporting of different aspects of the product.

[0084] The electronic device generates and sends a table field request to the reporting platform. This request includes the product number and the table number. Upon receiving the request, the reporting platform retrieves the product number and table number. Using the product number and table number as keys, the platform searches locally for and retrieves the value that matches them. This value represents the common fields, their corresponding common field columns, characteristic fields, and their corresponding characteristic field columns within the webpage data table.

[0085] It should be noted that a webpage data table consists of table fields, and table field columns are data columns that include pairs of table field names and attributes. For example...

[0086] [{

[0087] os:string,

[0088] url:string,

[0089] id: number

[0090] }]

[0091] Understandably, since each webpage data table under a product reports different aspects of the product, each webpage data table includes different webpage elements, or each webpage data table includes unique webpage elements that match its own reported content. The aforementioned characteristic fields can be used to characterize the unique webpage elements of each webpage data table.

[0092] After the reporting platform retrieves the value matching the product number and table number from its local machine, it generates a table field response. This table field response includes the common fields of the web page data table, the common field columns corresponding to the common fields, the characteristic fields, and the characteristic field columns corresponding to the characteristic fields.

[0093] The reporting platform sends a table field response to the electronic device. The electronic device receives the table field response from the reporting platform and retrieves the common fields, the common field columns corresponding to the common fields, the characteristic fields, and the characteristic field columns corresponding to the characteristic fields.

[0094] Electronic devices integrate common field columns and characteristic field columns to obtain a web page data table.

[0095] Furthermore, each webpage data table may include the same common fields as well as characteristic fields representing different meanings.

[0096] In one example, the searched product is specifically browser product A, which includes three webpage data tables. Webpage data table 1 reports the number of impressions of browser product A, webpage data table 2 reports the number of views of browser product A, and webpage data table 3 reports the number of clicks of browser product A. These three webpage data tables include common fields. For example, UUID is used to uniquely identify the user, and br is used to identify the user's browser type, etc. Additionally, these three webpage data tables also include characteristic fields. For example, webpage data table 1 includes an impression time field, webpage data table 2 includes a view count field, and webpage data table 3 includes a button field.

[0097] In another example, the searched product is specifically browser product B, which includes two sets of webpage data tables. Webpage data table 1 reports the number of likes for browser product B, and webpage data table 2 reports the number of complaints about browser product B. Both webpage data tables include common fields. For example, `os` identifies the user's system, and `url` identifies the webpage link the user is currently visiting. Additionally, both webpage data tables also include characteristic fields. For example, webpage data table 1 includes a positive review statistics field, and webpage data table 2 includes a negative review statistics field.

[0098] Step 140: When the reporting time is reached, send a first request to the database, the first request including the web page data table.

[0099] In this step, after the electronic device obtains the web page data table, when the reporting time arrives, the electronic device generates and sends a first request to the database, which includes the web page data table.

[0100] Optionally, in this embodiment of the invention, the table field response further includes descriptive attribute information of the characteristic field. This descriptive attribute information includes the reporting timing and reporting value of the characteristic field. The reporting timing can be specifically a reporting point, or a trigger point for reporting. The reporting value can be specifically configured by the administrator; that is, the characteristic field is reported after it meets the value and the reporting timing is reached.

[0101] Optionally, when the reporting time is reached, the electronic device sends a first request to the database. The specific process is as follows: the electronic device obtains the reporting time from the description attribute information; when the reporting time is reached, the electronic device sends a first request to the database.

[0102] Optionally, in this embodiment of the invention, the first request is specifically a GET request.

[0103] The aforementioned solution can solve the problems of complex reporting logic and long modification and reporting cycles in existing web page reporting solutions. In the process of web page data reporting, this invention avoids writing code, ensures that the data table reported on each page is different, and also ensures correct reporting, simplifies the web page data reporting process, and also realizes data monitoring of web pages generated by low-code platforms.

[0104] Optionally, in this embodiment of the invention, the electronic device performs a webpage generation process before executing step 110. This process allows for the webpage to be characterized and reported, and the reported file to be referenced in the webpage.

[0105] Specifically, the administrator wants to configure the original page attributes of a webpage. The administrator can input a first configuration command into the electronic device via command line, which instructs the administrator to configure the original page attributes of the webpage.

[0106] The electronic device receives a first configuration command from the administrator. Based on the first configuration command, the electronic device configures the original page attributes of the webpage to obtain the configured page attributes.

[0107] After configuring the page properties, the administrator continues to configure the original page elements included in the webpage. The administrator then uses the command line to input a second configuration command into the electronic device, which instructs on the configuration of the original page elements included in the webpage.

[0108] The electronic device receives a second configuration instruction from the administrator. Based on this instruction, the electronic device configures the original page elements of the webpage to obtain the configured page elements.

[0109] Based on the configured page attributes and elements, the electronic device generates a reporting file. Using script tags, the electronic device references this reporting file in the webpage's source code, thus generating the webpage.

[0110] Optionally, the aforementioned electronic device configures the original page attributes of the webpage to obtain the configured page attributes. The specific process is as follows:

[0111] When a webpage is generated, the electronic device first configures the original page attributes at the webpage level. The first configuration instructions received by the electronic device include the product number and table number entered by the administrator. The electronic device configures the product number and table number in the original page attributes and sets the attribute values ​​of the product number and table number as global variables.

[0112] For example, original page attributes at the webpage level include, but are not limited to: the webpage title, keywords, explanatory text, the name in the browser address bar, etc., and all of these attributes apply to the entire webpage.

[0113] The above configuration process allows administrators to configure the product number and table number according to current product requirements. For example, product number 1 and table number 11; or product number 2 and table number 22, etc.

[0114] Optionally, the aforementioned electronic device configures the original page elements included in the webpage to obtain the configured page elements. The specific process is as follows:

[0115] Specifically, the electronic device configures the reported attributes of the page elements to obtain the configured page elements.

[0116] Furthermore, the reporting attributes of page elements include reporting anchors and reporting values. When configuring reporting attributes, administrators can determine whether to check the reporting anchor for a page element based on current product requirements. For example, an administrator needs to obtain the click count of a button in a browser product. In this case, the browser product includes a button as a page element. When configuring the reporting attributes of this button, the electronic device receives a second configuration instruction including a checkmark entered by the administrator. Through this checkmark, the electronic device marks the reporting anchor for the button. Simultaneously, the second configuration instruction also includes the reporting value for the button entered by the administrator. The electronic device marks this reporting value under the button, for example, a reporting value of 1; a reporting value of 2, etc.

[0117] Optionally, the electronic device integrates the configured page attributes and page elements to obtain the reporting file.

[0118] Optionally, the aforementioned script tag can specifically be a `<script>` tag. The electronic device references the reporting file through the `<script>` tag and writes the reporting file referenced by the `<script>` tag into the source code of the webpage to generate the webpage. For example, "<scripttype="text / javascript"src=" / js / infoc.js"> "Electronic devices use script tags to write the reported files into the source code of a webpage, thus creating the webpage."

[0119] In one example, a raw page for product A is created within a low-code platform. The electronic device configures the raw page properties. The electronic device configures the product number (A1) and table number (A11) in the raw page properties, and configures the product number (A1) and table number (A11) as global variables.

[0120] The electronic device continues to configure the original page elements. The original page includes buttons, and the electronic device marks the button's report anchor and marks the report value (report=1) below the button.

[0121] In another example, an initial page for product B is created within a low-code platform. The electronic device configures the initial page properties. The electronic device configures the product number (B2) and table number (B22) in the initial page properties, and configures the product number (B2) and table number (B22) as global variables.

[0122] The electronic device continues to configure the original page elements. The original page includes text, and if the text has external links, the electronic device marks the report anchor of the text and marks the report value (report=2) below the text.

[0123] In the two examples above, the electronic device generates a reporting file based on the configured page attributes and elements. Using script tags, the electronic device references the reporting file in the webpage's source code, generating the webpage. Thus, when the page receives a user-input button click or click text, the electronic device generates a webpage data table and, upon reaching the reporting threshold, sends a first request to the database.

[0124] Optionally, in this embodiment of the invention, after the electronic device generates a webpage, it also performs the step of storing the reporting file in the reporting platform so that the reporting platform generates characteristic fields, characteristic field columns and descriptive attribute information accordingly.

[0125] Optionally, in this embodiment of the invention, the administrator also configures common fields and corresponding common field columns for multiple web page data tables within the reporting platform; and configures the correspondence between the common fields and their corresponding common field columns and the product number, table number, and characteristic fields and their corresponding characteristic field columns, respectively. This allows administrators to configure and generate different web page data tables for different products as needed, simplifying the reporting process and enabling personalized reporting.

[0126] Figure 2 This is a schematic diagram of a web page data collection device provided in an embodiment of the present invention, as shown below. Figure 2 As shown, the web page data collection device in this embodiment may include: a receiving unit 210, a display unit 220, a first generating unit 230, and a sending unit 240.

[0127] The receiving unit 210 is used to receive an opening instruction input by the user, the opening instruction including a webpage identifier;

[0128] Display unit 220 is configured to initiate and display a webpage represented by the webpage identifier according to the webpage identifier, the webpage including webpage elements;

[0129] The first generation unit 230 is configured to generate a web page data table if the receiving unit receives an operation instruction for the web page element input by the user and the web page element has been configured with a reporting attribute.

[0130] The sending unit 240 is configured to send a first request to the database when the reporting time is reached, the first request including the web page data table.

[0131] Optionally, the first generation unit 230 is specifically used to obtain the configured product number and the table number of the web page data table from the web page, wherein the web page data table belongs to the product indicated by the product number;

[0132] Send a table field request to the reporting platform, the table field request including the product number and the table number;

[0133] Receive the table field response sent by the reporting platform, the table field response including the common fields included in the web page data table, the common field column corresponding to the common fields, the characteristic fields, and the characteristic field column corresponding to the characteristic fields;

[0134] The common field column and the characteristic field column are integrated to obtain the web page data table;

[0135] The characteristic field is used to characterize the webpage element.

[0136] Optionally, the table field response may also include descriptive attribute information of the feature field;

[0137] The sending unit 240 is specifically used to obtain the reporting timing from the description attribute information;

[0138] When the reporting time is reached, the first request is sent to the database.

[0139] Optionally, the receiving unit 210 is further configured to receive an input first configuration instruction;

[0140] The device further includes: a first configuration unit (not shown in the figure), configured to configure the original page attributes of the webpage according to the first configuration instruction to obtain the configured page attributes;

[0141] The receiving unit 210 is further configured to receive an input second configuration instruction;

[0142] The device further includes: a second configuration unit (not shown in the figure), configured to configure the original page elements included in the webpage according to the second configuration instruction to obtain the configured page elements;

[0143] The second generation unit (not shown in the figure) is used to generate a reporting file based on the configured page attributes and the configured page elements.

[0144] The third generation unit (not shown in the figure) is used to generate the webpage by referencing the reported file into the source code of the webpage through script tags.

[0145] Optionally, the first configuration unit (not shown in the figure) is specifically used to configure the product number and the table number in the page attributes, and to configure the attribute value of the product number and the attribute value of the table number as global variables;

[0146] The second configuration unit (not shown in the figure) is specifically used to configure the reported attributes of the page element to obtain the configured page element.

[0147] Optionally, the device further includes a storage unit (not shown in the figure) for storing the reporting file in the reporting platform, so that the reporting platform generates the characteristic field, the characteristic field column, and the descriptive attribute information accordingly.

[0148] The apparatus of this embodiment can be used to perform Figure 1 The technical solutions of the method embodiments shown are similar in principle and in effect, and will not be described again here.

[0149] Accordingly, the web page data collection device provided in this embodiment of the invention can also be implemented using another structure. Figure 3 This is a schematic diagram of an embodiment of an electronic device provided by the present invention, which can realize the present invention. Figure 1 The process of the illustrated embodiment is as follows: Figure 3 As shown, the above-mentioned electronic device may include: a housing 31, a processor 32, a memory 33, a circuit board 34, and a power supply circuit 35, wherein the circuit board 34 is disposed inside the space enclosed by the housing 31, and the processor 32 and the memory 33 are disposed on the circuit board 34; the power supply circuit 35 is used to supply power to the various circuits or devices of the above-mentioned electronic device; the memory 33 is used to store executable program code; the processor 32 runs a program corresponding to the executable program code by reading the executable program code stored in the memory 33, for executing the methods described in the foregoing embodiments.

[0150] For details on the specific execution process of the above steps by processor 32, and the steps further executed by processor 32 through running executable program code, please refer to the present invention. Figure 1 The description of the illustrated embodiments will not be repeated here.

[0151] The electronic device is a device that provides computing services. It consists of a processor, hard drive, memory, system bus, etc. The electronic device is similar to the general computer architecture, but because it needs to provide highly reliable services, it has higher requirements in terms of processing power, stability, reliability, security, scalability, and manageability.

[0152] It should be noted that, in this document, terms such as "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0153] The various embodiments in this specification are described in a related manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

[0154] In particular, the device embodiment is basically similar to the method embodiment, so the description is relatively simple. For relevant details, please refer to the description of the method embodiment.

[0155] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as a ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (such as a computer-based system, a processor-including system, or other system that can fetch and execute instructions from, an instruction execution system, apparatus, or device). For the purposes of this specification, "computer-readable medium" can be any means that can contain, store, communicate, propagate, or transmit programs for use by, or in conjunction with, an instruction execution system, apparatus, or device.

[0156] More specific examples of computer-readable media (a non-exhaustive list) include: electrical connections (electronic devices) having one or more wires, portable computer disk drives (magnetic devices), random access memory (RAM), read-only memory (ROM), erasable and editable read-only memory (EPROM or flash memory), fiber optic devices, and portable optical disc read-only memory (CDROM). Furthermore, computer-readable media can even be paper or other suitable media on which the program can be printed, since the program can be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, interpreting, or otherwise processing as necessary, and then stored in computer memory.

[0157] It should be understood that various parts of the present invention can be implemented in hardware, software, firmware, or a combination thereof.

[0158] In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in memory and executed by a suitable instruction execution system. For example, if implemented in hardware, as in another embodiment, it can be implemented using any one or a combination of the following techniques known in the art: discrete logic circuits having logic gates for implementing logical functions on data signals, application-specific integrated circuits (ASICs) having suitable combinational logic gates, programmable gate arrays (PGAs), field-programmable gate arrays (FPGAs), etc.

[0159] Those skilled in the art will understand that all or part of the steps of the methods in the above embodiments can be implemented by a program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, the program includes one or a combination of the steps of the method embodiments.

[0160] For ease of description, the above apparatus is described by dividing it into various functional units / modules. Of course, in implementing this invention, the functions of each unit / module can be implemented in one or more software and / or hardware.

[0161] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that the present invention can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of the present invention.

Claims

1. A method for collecting webpage data, characterized in that, The method includes: Receives a user-inputted command to open the webpage, the command including a webpage identifier; Based on the webpage identifier, the webpage represented by the webpage identifier is launched and displayed, the webpage including webpage elements; If, under the webpage, an operation instruction for the webpage element is received from the user and the webpage element has been configured with a reporting attribute, a webpage data table is generated. When the reporting time is reached, a first request is sent to the database, the first request including the web page data table; The generation of the webpage data table specifically includes: From the webpage, obtain the configured product number and the table number of the webpage data table, wherein the webpage data table belongs to the product indicated by the product number; Send a table field request to the reporting platform, the table field request including the product number and the table number; Receive the table field response sent by the reporting platform, the table field response including the common fields included in the web page data table, the common field column corresponding to the common fields, the characteristic fields, and the characteristic field column corresponding to the characteristic fields; The common field column and the characteristic field column are integrated to obtain the web page data table; The characteristic fields are used to characterize the web page elements; the product number and table number are both global variables. The table field response also includes descriptive attribute information for the characteristic fields; The step of sending a first request to the database when the reporting time is reached specifically includes: The reporting timing is obtained from the described attribute information; When the reporting opportunity is reached, the first request is sent to the database; Before receiving the user's input to open the command, the method further includes: Receive the first configuration command input; According to the first configuration instruction, the original page attributes of the webpage are configured to obtain the configured page attributes; Receive the second configuration command input; According to the second configuration instruction, the original page elements included in the webpage are configured to obtain the configured page elements; Generate a reporting file based on the configured page attributes and page elements; The reported file is referenced in the source code of the webpage using script tags, thereby generating the webpage; After generating the webpage, the method further includes: The reporting file is stored in the reporting platform so that the reporting platform generates the characteristic field, the characteristic field column, and the descriptive attribute information accordingly. The reporting platform is configured with common fields of multiple web page data tables, common field columns corresponding to the common fields, and correspondences between the common fields, the common field columns corresponding to the common fields, and the product number, table number, characteristic fields, and characteristic field columns corresponding to the characteristic fields.

2. The method according to claim 1, characterized in that, The process of configuring the original page attributes of the webpage to obtain the configured page attributes specifically includes: Configure the product number and the table number in the page attributes, and configure the attribute values ​​of the product number and the table number as global variables; The configuration of the original page elements included in the webpage to obtain the configured page elements specifically includes: Configure the reporting attributes of the page element to obtain the configured page element.

3. A device for collecting web page data, characterized in that, The device includes: A receiving unit is configured to receive an opening instruction input by a user, wherein the opening instruction includes a webpage identifier; The display unit is configured to initiate and display the webpage represented by the webpage identifier according to the webpage identifier, wherein the webpage includes webpage elements; The first generation unit is configured to generate a webpage data table if the receiving unit receives an operation instruction for the webpage element input by the user and the webpage element has been configured with a reporting attribute. The sending unit is used to send a first request to the database when the reporting time is reached, the first request including the web page data table; The first generation unit is specifically used to obtain the configured product number and the table number of the web page data table from the web page, wherein the web page data table belongs to the product indicated by the product number; Send a table field request to the reporting platform, the table field request including the product number and the table number; Receive the table field response sent by the reporting platform, the table field response including the common fields included in the web page data table, the common field column corresponding to the common fields, the characteristic fields, and the characteristic field column corresponding to the characteristic fields; The common field column and the characteristic field column are integrated to obtain the web page data table; The characteristic fields are used to characterize the web page elements; The table field response also includes descriptive attribute information for the characteristic fields; The sending unit is specifically used to obtain the reporting timing from the description attribute information; When the reporting opportunity is reached, the first request is sent to the database; The receiving unit is further configured to receive an input first configuration instruction; The device further includes: a first configuration unit, configured to configure the original page attributes of the webpage according to the first configuration instruction, to obtain the configured page attributes; The receiving unit is further configured to receive a second configuration instruction input. The device further includes: a second configuration unit, configured to configure the original page elements included in the webpage according to the second configuration instruction, so as to obtain the configured page elements; The second generation unit is used to generate the reporting file based on the configured page attributes and page elements; The third generation unit is used to generate the webpage by referencing the reported file into the source code of the webpage through script tags; The device further includes: A storage unit is used to store the reporting file in the reporting platform so that the reporting platform can generate the characteristic field, the characteristic field column, and the descriptive attribute information accordingly. The reporting platform is configured with common fields of multiple web page data tables, common field columns corresponding to the common fields, and correspondences between the common fields, the common field columns corresponding to the common fields, and the product number, table number, characteristic fields, and characteristic field columns corresponding to the characteristic fields.

4. The apparatus according to claim 3, characterized in that, The first configuration unit is specifically used to configure the product number and the table number in the page attributes, and to configure the attribute values ​​of the product number and the table number as global variables; The second configuration unit is specifically used to configure the reported attributes of the page element to obtain the configured page element.

5. An electronic device, characterized in that, The electronic device includes: a housing, a processor, a memory, a circuit board, and a power supply circuit, wherein the circuit board is disposed inside the space enclosed by the housing, and the processor and the memory are disposed on the circuit board; the power supply circuit is used to supply power to various circuits or devices of the electronic device; the memory is used to store executable program code; the processor runs a program corresponding to the executable program code by reading the executable program code stored in the memory, for executing the web page data collection method according to any one of claims 1-2.

6. A computer-readable storage medium storing one or more programs, which can be executed by one or more processors, for implementing the web page data collection method according to any one of claims 1-2.

Images