A data extraction method and system based on page engine

By obtaining user identity information and identification sequence, combining storage path tendency mode, dynamically adjusting the data storage location, the problem of inflexible data storage in the prior art is solved, and the efficiency of data storage in the desired location of the user is improved.

CN115714707BActive Publication Date: 2024-11-15FUJIAN HUIZHOU INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202211163953.0
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-09-23
Publication Date
2024-11-15
Estimated Expiration
2042-09-23

AI Technical Summary

Technical Problem

The existing data extraction method based on page engine cannot flexibly store data according to user storage preferences, resulting in data being stored in a default address rather than a storage device that the user expects, affecting data extraction efficiency.

Method used

By obtaining the user's identity information and identity identification sequence, the data storage module is determined, and the data storage location is dynamically adjusted according to the user's storage path tendency pattern and data type, and the mobile storage device and different types of data storage modules are supported to ensure that the data is stored in The location specified by the user.

Benefits of technology

It improves data extraction efficiency, meets users' storage needs, ensures that data is stored in the expected location, and enhances the user experience.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115714707B_ABST
    Figure CN115714707B_ABST
Patent Text Reader

Abstract

The present application relates to a data extraction method and system based on a page engine, and relates to the field of computer information technology. It solves the problem that the data of the page engine is generally stored in a default storage device, and this default address is not necessarily the storage device that the current user wants to store. It includes: obtaining the identity information of the person who logs in to the system; analyzing and determining the data storage module that stores the extracted data according to the corresponding relationship between the identity information of the person who logs in and the data storage module; performing data extraction on the page engine and storing it in the data storage module. The present application has the following effects: improving the efficiency of users in finding the required category data.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of computer information technology, and in particular to a data extraction method and system based on a page engine. Background Art

[0002] With the rise of Web2.0, AJAX (Asynchronous JavaScript and XML) technology is all the rage. The asynchronous interaction between the client and the server reduces the pressure on the server and brings a better user experience. However, the large number of dynamic web pages generated by this technology has created new problems for network data acquisition. Traditional web data acquisition tools such as web crawlers used to collect static web pages capture far less content than the content presented on the page. The inability to obtain useful information in a large number of dynamic web pages makes it impossible to carry out work that mainly processes network data, which seriously affects the development of network content monitoring, network data mining and other disciplines.

[0003] Therefore, how to improve the traditional Web data collection system to support dynamic page parsing has become a research hotspot in current information collection technology.

[0004] There are generally two main methods for dynamic page collection at present: one is to use open source browser interfaces (such as Firefox) to collect browser output results in the form of writing plug-ins; the other is to use existing script interpretation engines (such as SpiderMonkey, Rhino, etc.) to bind relevant DOM (DocumentObject Model) objects according to the needs of information collection and collect output results.

[0005] The existing data extraction method based on the page engine will store all the corresponding data into the same storage device after extracting the data.

[0006] With respect to the above-mentioned related technologies, the inventors have found the following defects: the data of the page engine is generally stored in a default storage device, and this default address is not necessarily the storage device that the current user wants to store. Summary of the invention

[0007] In order to improve the efficiency of users in finding the required category data, the present application provides a data extraction method and system based on a page engine.

[0008] In the first aspect, the present application provides a data extraction method based on a page engine, which adopts the following technical solution:

[0009] A data extraction method based on a page engine, comprising:

[0010] Obtain the identity information of the person logging into the system;

[0011] According to the corresponding relationship between the login personnel identity information and the data storage module, analyze and determine the data storage module for storing the extracted data this time;

[0012] Execute data extraction on the page engine and store it in the data storage module.

[0013] Optionally, obtaining the identity information of the system login personnel includes:

[0014] Get the last digit of today's date;

[0015] According to the correspondence between the odd or even number of the last digit of the date and the sequence of identity information recognition, the sequence of identity recognition is analyzed and determined. The sequence of identity recognition includes two recognition sequences: fingerprint recognition first and then iris recognition, and iris recognition first and then fingerprint recognition.

[0016] Obtain the user's identification information and identification sequence;

[0017] The identity information of the logged-in person is determined if and only if the user's identity information is correct and the identity identification sequence is consistent with the identity identification sequence determined by analysis.

[0018] Optionally, the method further includes a step of determining the identity information of the login person after obtaining the user's identity information and the identity identification sequence, and in parallel with determining the identity information of the login person if and only if the user's identity identification information is correct and the identity identification sequence is consistent with the analyzed and determined identity identification sequence, which is specifically as follows:

[0019] If the user's identity identification information is correct and the identity identification sequence is inconsistent with the identity identification sequence determined by analysis, the user is prompted to perform a second identity identification;

[0020] Get the current period;

[0021] According to the correspondence between the odd or even number of the last digit of the time period of the day and the sequence of identity information recognition, the sequence of identity recognition is analyzed and determined. The sequence of identity recognition includes two recognition sequences: fingerprint recognition first and then iris recognition, and iris recognition first and then fingerprint recognition.

[0022] Obtain the user's identification information and identification sequence;

[0023] The identity information of the logged-in person is determined if and only if the user's identity information is correct and the identity identification sequence is consistent with the identity identification sequence determined by analysis.

[0024] Optionally, the method further includes a step of determining the identity information of the login person after obtaining the user's identity information and the identity identification sequence, and in parallel with determining the identity information of the login person if and only if the user's identity identification information is correct and the identity identification sequence is consistent with the analyzed and determined identity identification sequence, which is specifically as follows:

[0025] If the user's identity identification information is inconsistent and the identity identification sequence is consistent with the identity identification sequence determined by analysis, the user is prompted to perform a second identity identification;

[0026] Obtain user identification information;

[0027] The identity information of the logged-in person is determined if and only if the user's identity information is identified through both fingerprint recognition and iris recognition.

[0028] Optionally, according to the correspondence between the login personnel identity information and the data storage module, the data storage module for storing the extracted data is analyzed and determined to include:

[0029] Obtaining whether the terminal loaded with the data extraction level of the page engine is plugged into a mobile storage device;

[0030] If yes, then the corresponding mobile storage device is used as the overall data storage module for the data extracted this time;

[0031] If not, the data storage modules for different types of data are analyzed and determined according to the corresponding relationship between the login personnel identity information and the data storage modules for storing different types of data.

[0032] Optionally, the method further includes steps after extracting data from the page engine and storing the data in the data storage module, which are as follows:

[0033] Obtain the user's preference pattern for data storage paths;

[0034] The data storage modules where different data are stored are displayed according to the user's preference for data storage paths.

[0035] Optionally, obtaining the user's preference pattern for data storage path display includes:

[0036] Analyze and determine the display objects according to the corresponding relationship between the acquired data type combination and the display objects, and the display objects include the user himself and the public;

[0037] If the display object is the user himself, then according to the user's tendency pattern of displaying the data storage path at different time periods, the user's tendency pattern of displaying the data storage path is analyzed and determined;

[0038] If the display object is the public, the public's preference pattern for displaying the data storage path is used as the user's preference pattern for displaying the data storage path.

[0039] Optionally, the method further includes a step after displaying the data storage module in which the different data are stored this time according to the user's preference mode for displaying the data storage path, which is specifically as follows:

[0040] If the display object is the user himself, analyze whether the user has modified the storage path within the preset time;

[0041] If yes, the data storage is adjusted according to the storage path modified by the user, and the adjusted storage situation is sent to the terminal held by the user;

[0042] If not, the storage status will be sent to the terminal held by the user.

[0043] In the second aspect, the present application provides a data extraction system based on a page engine, which adopts the following technical solution:

[0044] A data extraction system based on a page engine comprises a memory, a processor and a program stored in the memory and executable on the processor, wherein the program can implement the data extraction method based on the page engine as described in the first aspect when loaded and executed by the processor. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0046] Figure 1 It is a flow chart of a data extraction method based on a page engine according to an embodiment of the present application.

[0047] Figure 2 It is a schematic diagram of the process of obtaining the identity information of the person logging in to the system according to another embodiment of the present application.

[0048] Figure 3 Another embodiment of the present application is a flowchart diagram that is located after obtaining the user's identity identification information and identity identification sequence, and in parallel with determining the identity information of the login person if and only if the user's identity identification information is correct and the identity identification sequence is consistent with the analyzed and determined identity identification sequence.

[0049] Figure 4Another embodiment of the present application is a flowchart diagram that is located after obtaining the user's identity identification information and identity identification sequence, and in parallel with determining the identity information of the login person if and only if the user's identity identification information is correct and the identity identification sequence is consistent with the analyzed and determined identity identification sequence.

[0050] Figure 5 This is a flowchart of another embodiment of the present application, which analyzes and determines the data storage module that stores the data extracted this time according to the correspondence between the login person's identity information and the data storage module.

[0051] Figure 6 It is a flowchart of another embodiment of the present application after data extraction from a page engine is performed and stored in a data storage module.

[0052] Figure 7 This is a schematic diagram of the process of obtaining the preference pattern displayed by users on data storage paths in another embodiment of the present application.

[0053] Figure 8 It is a flowchart diagram of another embodiment of the present application after displaying the data storage module in which different data are stored according to the user's preference mode for displaying the data storage path. DETAILED DESCRIPTION

[0054] The present application is further described in detail below in conjunction with the accompanying drawings.

[0055] Reference Figure 1 , is a data extraction method based on a page engine disclosed in this application, comprising:

[0056] Step S100, obtaining the identity information of the person logging into the system.

[0057] Step S200: Analyze and determine the data storage module for storing the extracted data according to the corresponding relationship between the login person's identity information and the data storage module.

[0058] Step S300, extracting data from the page engine and storing the data in a data storage module.

[0059] exist Figure 1 In step S100, further considering the situation where the identity information of the system login personnel is wrong, it is necessary to further analyze and determine the identity information of the system login personnel, specifically refer to Figure 2 The illustrated embodiment is described in detail.

[0060] Reference Figure 2 , the acquisition of the identity information of the system login personnel includes:

[0061] Step S110, obtaining the last digit of the current date.

[0062] Step S120, analyzing and determining the identity recognition sequence according to the correspondence between the odd or even number of the last digit of the current date and the identity information recognition sequence, the identity recognition sequence includes two recognition sequences: fingerprint recognition first and then iris recognition, and iris recognition first and then fingerprint recognition.

[0063] Among them, fingerprint recognition can be obtained by a fingerprint recognition device, and iris recognition can be obtained by an iris recognition device; the correspondence between the parity of the last digit of the current date and the identity information recognition order can be queried and obtained from a preset database that stores the correspondence between the parity of the last digit of the current date and the identity information recognition order.

[0064] Step S130, obtaining the user's identity identification information and identity identification sequence.

[0065] Step S140: Determine the login person's identity information if and only if the user's identity identification information is correct and the identity identification sequence is consistent with the analyzed and determined identity identification sequence.

[0066] exist Figure 2 Between step S130 and step S140, further considering the situation that the user's identity identification information is correct and the identity identification sequence is inconsistent with the identity identification sequence determined by analysis, specifically refer to Figure 3 The illustrated embodiment is described in detail.

[0067] Reference Figure 3 A data extraction method based on a page engine also includes a step of determining the identity information of a login person after obtaining the user's identity information and the identity identification sequence, and in parallel with the step of determining the identity information of a login person if and only if the user's identity information is correct and the identity identification sequence is consistent with the analyzed and determined identity identification sequence, specifically as follows:

[0068] Step S1A0: If the user's identity identification information is correct and the identity identification sequence is inconsistent with the identity identification sequence determined by analysis, the user is prompted to perform a second identity identification.

[0069] Step S1B0, obtaining the current time period.

[0070] Step S1C0, analyzing and determining the identity recognition sequence according to the correspondence between the odd or even number of the day's time period and the identity information recognition sequence, the identity recognition sequence includes two recognition sequences: fingerprint recognition first and then iris recognition, and iris recognition first and then fingerprint recognition.

[0071] The correspondence between the parity of the mantissa of the time period and the sequence of identity information recognition can be obtained by querying from a preset database storing the correspondence between the parity of the mantissa of the time period and the sequence of identity information recognition.

[0072] Step S1D0, obtaining the user's identity identification information and identity identification sequence.

[0073] Step S1E0, if and only if the user's identity identification information is correct and the identity identification sequence is consistent with the identity identification sequence determined by analysis, determine the login person's identity information.

[0074] exist Figure 2 Between step S130 and step S140, the user's identity identification information is further considered to be inconsistent and the identity identification sequence is consistent with the identity identification sequence determined by analysis. Figure 4 The illustrated embodiment is described in detail.

[0075] Reference Figure 4 A data extraction method based on a page engine also includes a step of determining the identity information of a login person after obtaining the user's identity information and the identity identification sequence, and in parallel with the step of determining the identity information of a login person if and only if the user's identity information is correct and the identity identification sequence is consistent with the analyzed and determined identity identification sequence, specifically as follows:

[0076] Step S1a0: If the user's identity identification information is inconsistent and the identity identification sequence is consistent with the analyzed and determined identity identification sequence, the user is prompted to perform a second identity identification.

[0077] Step S1b0, obtaining the user's identity information.

[0078] Step S1c0, if and only if the user's identity information is identified through both fingerprint recognition and iris recognition, the identity information of the logged-in person is determined.

[0079] exist Figure 1 In step S200, whether the user uses the mobile storage device is further considered to better judge the user's storage intention. Figure 5 The illustrated embodiment is described in detail.

[0080] Reference Figure 5 According to the corresponding relationship between the login personnel identity information and the data storage module, the data storage module for storing the extracted data is analyzed and determined to include:

[0081] Step S210, obtaining whether the terminal loaded with the data extraction program of the page engine is plugged into a mobile storage device. If yes, executing step S220. If no, executing step S230.

[0082] The mobile storage device may be a mobile storage device such as a mobile hard disk or a USB flash drive.

[0083] Step S220: Using the corresponding mobile storage device as the overall data storage module for the data extracted this time.

[0084] Step S230 , analyzing and determining the data storage modules for different types of data according to the correspondence between the login person's identity information and the data storage modules for storing different types of data.

[0085] The data types may be divided in many ways, for example, into image data, video data, text data, etc.

[0086] exist Figure 1 After step S300, it is further considered that the user can subsequently understand the storage path of the data so that he can query it when needed. Figure 6 The illustrated embodiment is described in detail.

[0087] Reference Figure 6 , a data extraction method based on a page engine, further comprising steps after extracting data from the page engine and storing the data in a data storage module, specifically as follows:

[0088] Step S400, obtaining the user's preference pattern for data storage paths.

[0089] The tendency mode may be a text display mode or a chart display mode.

[0090] Step S500: display the data storage modules in which the different data are stored this time according to the user's preference mode for displaying the data storage path.

[0091] The data storage module may be a C drive, a D drive or other storage units.

[0092] exist Figure 6 In step S400, it is further considered that due to the difference in data types, the objects that may be displayed are different, so the storage path will also change. Therefore, it is necessary to further analyze the acquisition of the user's tendency pattern for the data storage path display. For details, refer to Figure 7 The illustrated embodiment is described in detail.

[0093] Reference Figure 7 , the acquisition of the user's tendency pattern for data storage path display includes:

[0094] Step S410: analyzing and determining the display object according to the correspondence between the acquired data type combination and the display object, where the display object includes the user himself and the public.

[0095] Step S420: If the display object is the user himself, the user's display tendency pattern for the data storage path is analyzed and determined according to the user's display tendency pattern for the data storage path in different time periods.

[0096] Step S430: If the display object is the public, the public's preferred mode of displaying the data storage path is used as the user's preferred mode of displaying the data storage path.

[0097] exist Figure 6 In step S500, it is further considered that the user may make adjustments after the storage path is displayed. Figure 7 The illustrated embodiment is described in detail.

[0098] Reference Figure 8 A data extraction method based on a page engine also includes a step after displaying the data storage module in which the different data are stored according to the user's preference pattern for displaying the data storage path, which is specifically as follows:

[0099] Step S600: If the display object is the user himself, analyze whether the user has modified the storage path within the preset time. If yes, execute step S700; if no, execute step S800.

[0100] Step S700: adjust the data storage according to the storage path modified by the user, and send the adjusted storage status to the terminal held by the user.

[0101] Step S800 sends the storage status to the terminal held by the user.

[0102] The terminal held by the user may be a mobile phone, a computer or other communication terminal devices.

[0103] Based on the same inventive concept, an embodiment of the present invention provides a data extraction system based on a page engine, including a memory and a processor, wherein the memory stores a data that can be executed on the processor to implement the following Figures 1 to 8 Procedure for either method.

[0104] The embodiments of this specific implementation method are all preferred embodiments of the present application, and are not intended to limit the protection scope of the present application. Therefore, all equivalent changes made based on the structure, shape, and principle of the present application should be included in the protection scope of the present application.

Claims

1. A data extraction method based on a page engine, characterized in that: include: Obtain the identity information of the person logging into the system; According to the corresponding relationship between the login personnel identity information and the data storage module, analyze and determine the data storage module for storing the extracted data this time; Execute data extraction from the page engine and store it in the data storage module; The acquisition of the identity information of the system login personnel includes: Get the last digit of today's date; According to the correspondence between the odd or even number of the last digit of the date and the sequence of identity information recognition, the sequence of identity recognition is analyzed and determined. The sequence of identity recognition includes two recognition sequences: fingerprint recognition first and then iris recognition, and iris recognition first and then fingerprint recognition. Obtain the user's identification information and identification sequence; If and only if the user's identity information is correct and the identity identification sequence is consistent with the identity identification sequence determined by analysis, the identity information of the logged-in person is determined; A data extraction method based on a page engine also includes a step of determining the identity information of a login person after obtaining the user's identity information and the identity identification sequence, and in parallel with the step of determining the identity information of a login person if and only if the user's identity information is correct and the identity identification sequence is consistent with the analyzed and determined identity identification sequence, specifically as follows: If the user's identity identification information is correct and the identity identification sequence is inconsistent with the identity identification sequence determined by analysis, the user is prompted to perform a second identity identification; Secondary identification includes: Get the current period; According to the correspondence between the odd or even number of the last digit of the time period of the day and the sequence of identity information recognition, the sequence of identity recognition is analyzed and determined. The sequence of identity recognition includes two recognition sequences: fingerprint recognition first and then iris recognition, and iris recognition first and then fingerprint recognition. Obtain the user's identification information and identification sequence; The identity information of the logged-in person is determined if and only if the user's identity information is correct and the identity identification sequence is consistent with the identity identification sequence determined by analysis.

2. A data extraction method based on a page engine according to claim 1, characterized in that: It also includes a step of determining the identity information of the login person after obtaining the user's identity information and the identity identification sequence, and in parallel with determining the identity information of the login person if and only if the user's identity identification information is correct and the identity identification sequence is consistent with the analyzed and determined identity identification sequence, which is specifically as follows: If the user's identity identification information is inconsistent and the identity identification sequence is consistent with the identity identification sequence determined by analysis, the user is prompted to perform a second identity identification; Obtain user identification information; The identity information of the logged-in person is determined if and only if the user's identity information is identified through both fingerprint recognition and iris recognition.

3. A data extraction method based on a page engine according to any one of claims 1 to 2, characterized in that: According to the corresponding relationship between the login personnel identity information and the data storage module, the data storage module for storing the extracted data is analyzed and determined to include: Obtaining whether a terminal loaded with a data extraction program based on a page engine is plugged into a mobile storage device; If yes, then the corresponding mobile storage device is used as the overall data storage module for the data extracted this time; If not, the data storage modules for different types of data are analyzed and determined according to the corresponding relationship between the login personnel identity information and the data storage modules for storing different types of data.

4. A data extraction method based on a page engine according to claim 3, characterized in that: The method also includes the following steps after extracting data from the page engine and storing it in the data storage module: Obtain the user's preference pattern for data storage paths; The data storage modules where different data are stored are displayed according to the user's preference for data storage paths.

5. A data extraction method based on a page engine according to claim 4, characterized in that: The acquisition of the user's preference pattern for data storage paths includes: Analyze and determine the display objects according to the corresponding relationship between the acquired data type combination and the display objects, and the display objects include the user himself and the public; If the display object is the user himself, then according to the user's tendency pattern of displaying the data storage path at different time periods, the user's tendency pattern of displaying the data storage path is analyzed and determined; If the display object is the public, the public's preference pattern for displaying the data storage path is used as the user's preference pattern for displaying the data storage path.

6. A data extraction method based on a page engine according to claim 5, characterized in that: The step further includes the following steps after displaying the data storage module in which the different data are stored according to the user's preference mode for displaying the data storage path: If the display object is the user himself, analyze whether the user has modified the storage path within the preset time; If yes, the data storage is adjusted according to the storage path modified by the user, and the adjusted storage situation is sent to the terminal held by the user; If not, the storage status will be sent to the terminal held by the user.

7. A data extraction system based on a page engine, characterized in that: The invention comprises a memory, a processor and a program stored in the memory and executable on the processor, wherein the program can be loaded and executed by the processor to implement a data extraction method based on a page engine as described in any one of claims 1 to 6.