Internet page restoration method, system, access method, system, medium and terminal

Abstract

The present invention provides a method, system, access method, system, medium and terminal for restoring an online page. The method for restoring an online page of the present invention comprises the following steps: after receiving the restoration request, reading the mirror traffic through the restoration program to obtain the resource files in the online page; storing the resource files in the root directory of the customized server; traversing all the resource files of each online page, and generating an access index in the root directory to access the online page. Under the condition of the mirror network, the present invention does not need to operate the entity file content, and the browser will send a reference request according to the HTML file processing; and the online page restoration process and access process have high performance, can realize real-time restoration, and do not need to consider the massive data processing situation; it also supports dynamic resource references, and has coverage for complex resource dependencies, such as special resource requests, js, css file resource requests, etc., which can meet different needs.

Description

Technical Field

[0001] The present invention belongs to the field of Internet technology, and in particular relates to an Internet page restoration method, system, access method, system, medium and terminal. Background Art

[0002] When users surf the Internet, they browse a variety of web pages. By analyzing the pages they browse, we can analyze and summarize their behaviors and generate user behavior portraits. This technology is helpful for preventing and filtering illegal websites and locating illegal personnel.

[0003] After some analysis of the characteristics of the http protocol, some characteristics can be associated with resource files. When building an online page, the browser will use the HTML file as the basis and traverse the resource references in the HTML file. These references are in the form of network requests (for example: loading an image url="http: / / www.baidu.com / img / a.jpg") or path references (for example: url="img / a.jpg"). Path references can directly obtain resources from the web server of the online page, and resources that are not on the current web server need to be obtained through network requests.

[0004] Considering the path reference feature, if all resources are classified by host and all references in HTML are replaced by path reference, the loading of resource files of online pages can be realized. That is, in the mirror traffic, the restored resource files are saved in directories according to the URL and host, the HTML files are traversed and analyzed, the resource reference list in the HTML files is obtained, the resource references in the HTML files are traversed, and the network request references and absolute path references are changed to relative references (for example, the host is www.baidu.com, and there is an index.html file in the root directory that references url="http:: / / image.baidu.com / a.img", which is changed to url=".. / image.baidu.com / a.jpg. Or, the reference is url=" / img / b.jpg" which is changed to url=". / img / b.jpg"), so that the purpose of loading resources can be achieved. However, under massive traffic, this method requires continuous opening of operation files and the replacement of various complex references, which has low performance and incomplete coverage.

[0005] The current methods for restoring and accessing Internet pages mainly have the following problems:

[0006] 1. Rely on the Internet environment to revisit the user's web page and send a request. The existing technology relies on the Internet environment and cannot be executed under mirror traffic.

[0007] 2. It is necessary to operate the file content, analyze and convert the URL. In the process of analyzing HTML files, it is necessary to frequently open HTML files and request resources. Under massive mirror traffic, there will be performance problems.

[0008] 3. Incomplete coverage. There are some special references in HTML references (such as references generated based on real time, some non-standard usage), which may not be covered.

[0009] 4. It does not support dynamic and complex page references. When the web page references some js files and other dynamic references, if the corresponding web server's processing logic is not clear, there is no way to correctly obtain the resources. Summary of the invention

[0010] In view of the above-mentioned shortcomings of the prior art, the purpose of the present invention is to provide a web page restoration method, system, access method, system, medium and terminal, which are used to solve the problems of low performance and inability to restore in real time during web page restoration in the prior art.

[0011] To achieve the above-mentioned purpose and other related purposes, the present invention provides a method for restoring an online page, comprising the following steps:

[0012] After receiving the restore request, the mirror traffic is read through the restore program to obtain the resource files in the Internet page;

[0013] Storing the resource file in the root directory of the custom server;

[0014] All resource files of each of the online pages are traversed, and an access index is generated in the root directory to access the online page.

[0015] In one embodiment of the present invention, the resource file is stored in the root directory of the custom server, including:

[0016] The address host mapping file information host is used as the first-level directory, and the resource address url of the resource file is used as the remaining hierarchical directory, so as to store the resource file in layers in the customized server.

[0017] In one embodiment of the present invention, the customization server generates the access index corresponding to each resource file corresponding to the online page.

[0018] The present invention provides a method for restoring access to an Internet page, comprising the following steps:

[0019] The access host accesses the access index in the customized server through a browser, traverses all the resource files associated with the online page, and generates an http access request for the associated resource files;

[0020] Analyzing the http access request through the customized server to obtain the basic path information referer and the network request information url in the resource access request, and localizing the network request information url according to the basic path information referer to obtain the localized path url;

[0021] The customization server adds the localization path URL to the redirection address information location in the response information, and sends the response information to the access host;

[0022] After the access host obtains the response information, it obtains the redirection address information location, analyzes the redirection address information location to generate redirection path information url, and sends the redirection path information url to the customization server;

[0023] The customized server obtains the target resource file in the root directory according to the redirection path information URL, and sends the target resource file to the access host to complete the access process.

[0024] In one embodiment of the present invention, the localizing the network request information URL according to the basic path information referer to obtain the localized path URL includes:

[0025] After determining that the network request information url is a path request, obtaining the path information of the basic path information referer and the IP address of the customized server;

[0026] The localized path URL is regenerated according to the path information of the basic path information referer and the IP address of the customized server.

[0027] In one embodiment of the present invention, the method for restoring the online page further includes:

[0028] After determining that the network request information url is an Internet request, the custom server analyzes the path address in the network request information url and obtains the IP address of the custom server;

[0029] The real path information of the corresponding resource file in the customized server is obtained according to the path address in the network request information url and the IP address of the customized server, and the corresponding resource file is sent to the access host according to the real path information.

[0030] The present invention discloses a web page restoration system, comprising:

[0031] A reading module, configured to read the mirrored traffic through a restoration program after receiving the restoration request to obtain the resource file in the Internet page;

[0032] A storage module, used for storing the resource file in the root directory of the custom server;

[0033] The generating module is used for traversing all resource files of each of the online pages and generating an access index in the root directory to access the online pages.

[0034] The present invention also provides a system for accessing an Internet restoration page, comprising:

[0035] A request module, used to access the access index in the customized server through a browser, traverse all the resource files associated with the online page, and generate an http access request for the associated resource files;

[0036] A localization module, used for analyzing the http access request through the custom server to obtain the basic path information referer and the network request information url in the resource access request, and localizing the network request information url according to the basic path information referer to obtain a localized path url;

[0037] A response module, used for adding the localized path URL to the redirection address information location in the response information through the custom server, and sending the response information to the access host;

[0038] A redirection sending module, used for obtaining the redirection address information location after the access host obtains the response information, analyzing the redirection address information location to generate redirection path information url, and sending the redirection path information url to the custom server;

[0039] An access module is used to obtain the target resource file in the root directory according to the redirection path information URL through the customized server, and send the target resource file to the access host to complete the access process;

[0040] The present invention provides a storage medium on which a computer program is stored. When the program is executed by a processor, the program implements the above-mentioned method for restoring an online page or the above-mentioned method for accessing a restored online page.

[0041] The present invention provides a terminal, characterized by comprising: a processor and a memory;

[0042] The memory is used to store computer programs;

[0043] The processor is used to execute the computer program stored in the memory, so that the terminal executes the above-mentioned method for restoring the online page or the above-mentioned method for accessing the restored online page.

[0044] As described above, the Internet page restoration method, system, access method, system, medium and terminal of the present invention have the following beneficial effects:

[0045] Under the condition of a mirror network, there is no need to operate the content of the physical file. The browser will send a reference request based on the HTML file processing. Moreover, the Internet page restoration process and access process have high performance and can achieve real-time restoration. There is no need to consider the processing of massive data. Only when accessing the restored Internet page will real-time processing be performed, and there is no performance problem. It also supports dynamic resource references and covers complex resource dependencies, such as special resource requests, js, css file resource requests, etc., to meet different needs. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 Shown is a flow chart of the web page restoration method of the present invention.

[0047] Figure 2 Shown is a schematic diagram of the restoration process of the Internet page restoration method of the present invention.

[0048] Figure 3 Shown is a flow chart of the method for restoring access to Internet pages of the present invention.

[0049] Figure 4 Shown is a flow chart of the access process of the method for restoring access to a web page of the present invention.

[0050] Figure 5 Shown is a schematic diagram of the process of restoring and accessing the Internet page of the present invention.

[0051] Figure 6 Shown is a structural block diagram of the web page restoration system of the present invention.

[0052] Figure 7 Shown is a structural block diagram of the system for restoring access to Internet pages of the present invention. DETAILED DESCRIPTION

[0053] The following describes the embodiments of the present invention through specific embodiments, and those skilled in the art can easily understand other advantages and effects of the present invention from the contents disclosed in this specification. The present invention can also be implemented or applied through other different specific embodiments, and the details in this specification can also be modified or changed in various ways based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the following embodiments and features in the embodiments can be combined with each other without conflict.

[0054] It should be noted that the illustrations provided in the following embodiments are only schematic illustrations of the basic concept of the present invention. The illustrations only show components related to the present invention rather than being drawn according to the number, shape and size of components in actual implementation. In actual implementation, the type, quantity and proportion of each component may be changed arbitrarily, and the component layout may also be more complicated.

[0055] The web page restoration method, system, access method, system, medium and terminal of the present invention do not need to operate the entity file content under the condition of a mirror network, and the browser will send a reference request according to the HTML file processing; the web page restoration process and access process have high performance and can achieve real-time restoration without considering the processing of massive data. Only when the web page after access is restored will real-time processing be performed, and there is no performance problem; dynamic resource reference is also supported, and complex resource dependency relationships are covered, such as special resource requests, js, css file resource requests, etc., which can meet different needs.

[0056] like Figure 1 As shown, in one embodiment, a method for restoring a web page of the present invention, referring to Figure 1 , including the following steps:

[0057] S101, after receiving the restoration request, reading the mirror traffic through the restoration program to obtain the resource file in the Internet page;

[0058] S102, storing the resource file in the root directory of the customization server;

[0059] S103, traversing all resource files of each of the online pages, and generating an access index in the root directory to access the online pages.

[0060] In one embodiment of the present invention, the resource file is stored in the root directory of the custom server, including:

[0061] The address host mapping file information host is used as the first-level directory, and the resource address url of the resource file is used as the remaining hierarchical directory, so as to store the resource file in layers in the customized server.

[0062] In an embodiment of the present invention, the customization server generates the access index corresponding to each resource file corresponding to the online page.

[0063] Specifically, refer to Figure 2 During the restoration process, the restoration program will first read the mirror traffic and restore it to a physical resource file. The physical resource file after restoration is stored in the root directory of the custom server. The storage method is to use the host as the first-level directory and the url as the remaining levels for hierarchical storage in the custom server. At the same time, the resource files will be traversed and an index will be generated in the root directory to access all HTML Internet pages.

[0064] The present invention also provides a method for restoring access to an online page, referring to Figure 3 , including the following steps:

[0065] S301, the access host accesses the access index in the customized server through a browser, traverses all the resource files associated with the online page, and generates an http access request for the associated resource files;

[0066] S302, analyzing the http access request through the customized server to obtain the basic path information referer and the network request information url in the resource access request, and localizing the network request information url according to the basic path information referer to obtain the localized path url;

[0067] S303, the customization server adds the localized path URL to the redirection address information location in the response information, and sends the response information to the access host;

[0068] S304, after obtaining the response information, the access host obtains the redirection address information location, analyzes the redirection address information location to generate redirection path information url, and sends the redirection path information url to the customization server;

[0069] S305: The customization server obtains the target resource file in the root directory according to the redirection path information URL, and sends the target resource file to the access host to complete the access process.

[0070] Referer is a part of the http header. When a browser sends a request to a server, it usually includes a referer to tell the server which page the webpage is linked from, so that the server can get some information for processing. When a resource in an html file is requested, the header will include referer information, which contains relevant information about the html file.

[0071] Location indicates redirection, which usually appears in the http response header and identifies the URL to which the response redirects. When the response header received by the browser contains location, it will obtain the redirection address in location and redirect.

[0072] Specifically, refer to Figure 4 After the access host accesses one of the indexes in the index page of the customized server through the browser, it will request the HTML file of the online page. According to the characteristics of the browser, it will obtain all the related resources of the online page and request them one by one. Then the access host traverses the resource list and sends an http request to the customized server. The customized server analyzes the http request, obtains the basic path information referer and the network request information url in the request, localizes the network request information url according to the basic path information referer to obtain the localized path url, fills the localized localized path url into the redirection address information location in the response information, and sends the response information to the access host at the same time. After the access host obtains the response information, it obtains the redirection address information location in the response information, analyzes the redirection address information location to generate the redirection path information url, and then redirects the path information url to the customized server. The customized server sends the target resource file to the access host, so that the access host obtains the resource file and completes the access process.

[0073] In one embodiment of the present invention, the localizing the network request information URL according to the basic path information referer to obtain the localized path URL includes:

[0074] After determining that the network request information url is a path request, obtaining the path information of the basic path information referer and the IP address of the customized server;

[0075] The localized path URL is regenerated according to the path information of the basic path information referer and the IP address of the customized server.

[0076] Specifically, after determining that the network request information url is a path request, the customized server obtains the network request information url, analyzes the network request information url and obtains the basic path information referer, such as http: / / localhost / www.baidu.com / index.html, and regenerates the localized path url according to the basic path information referer and the IP address of the customized server, such as http: / / localhost / www.baidu.com / a / b.png, and fills the localized path url into the redirection address information location of the customized server response, and sends the response information to the access host. After the access host obtains the response information, it reads the localized path url in the redirection address information location, and resends the redirection path information url after redirection to the customized server. Then, the customized server obtains the target resource file according to the redirection path information url, and sends the target resource file to the access host, and the access process.

[0077] In one embodiment of the present invention, the method for restoring the online page further includes:

[0078] After determining that the network request information url is an Internet request, the custom server analyzes the path address in the network request information url and obtains the IP address of the custom server;

[0079] The real path information of the corresponding resource file in the customized server is obtained according to the path address in the network request information url and the IP address of the customized server, and the corresponding resource file is sent to the access host according to the real path information.

[0080] Specifically, for example, when the beginning of the network request information URL is any one of http, https and " / / ", it is determined that the network request information URL is an Internet request. Since the Internet request is unreachable in the mirror traffic, it needs to be redirected to the real resource path on the custom server. The custom server obtains the network request information URL, analyzes the information in the network request information URL and the IP address of the custom server, and obtains the real location of the resource on the custom server according to the information in the network request information URL and the IP address of the custom server, such as http: / / localhost / www.baiducom / a.jgp, so as to directly send the target resource file to the access host and complete the access process.

[0081] In some embodiments, the browser accessing the host will parse all resource file references of the Internet page and send a request to the customized server. The customized server will perform the above processing according to different types of resources, and finally restore the complete Internet page in the mirror network environment, completing the entire Internet page access process.

[0082] Since the http request header of resource references contains referer information, in a mirror environment, the html file and the physical resource file cannot be associated. However, this solution analyzes the referer attribute through the referer resolution algorithm to associate the html file with the resource file; and localizes the resource reference by localizing the URL, so that the restored access to the Internet page can be achieved in a mirror network environment.

[0083] The web page restoration and access process of this solution is as follows: Figure 5 As shown, since the specific process has been described in the previous content, it will not be repeated here.

[0084] The present invention provides a web page restoration system, referring to Figure 6 ,include:

[0085] The reading module 601 is used to read the mirror traffic through the restoration program to obtain the resource file in the Internet page after receiving the restoration request;

[0086] Storage module 602, used to store the resource file in the root directory of the customization server;

[0087] The generation module 603 is used to traverse all resource files of each of the online pages and generate an access index in the root directory to access the online page.

[0088] The present invention also provides a system for accessing a web page restoration, referring to Figure 7 ,include:

[0089] The request module 701 is used to access the access index in the customized server through a browser, traverse all the resource files associated with the online page, and generate an http access request for the associated resource files;

[0090] The localization module 702 is used to analyze the http access request through the customized server to obtain the basic path information referer and the network request information url in the resource access request, and localize the network request information url according to the basic path information referer to obtain the localized path url;

[0091] A response module 703 is used to add the localized path URL to the redirection address information location in the response information through the custom server, and send the response information to the access host;

[0092] The redirection sending module 704 is used to obtain the redirection address information location after the access host obtains the response information, analyze the redirection address information location to generate redirection path information url, and send the redirection path information url to the custom server;

[0093] Access module 705, used to obtain the target resource file in the root directory according to the redirection path information URL through the customization server, and send the target resource file to the access host to complete the access process;

[0094] It should be noted that the protection scope of the web page restoration method described in the present invention is not limited to the execution order of the steps listed in this embodiment. All solutions implemented by adding, reducing or replacing steps in the prior art based on the principles of the present invention are included in the protection scope of the present invention.

[0095] It should be noted that it should be understood that the division of the various modules of the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software called by processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software called by processing elements, and some modules can be implemented in the form of hardware. For example, the x module can be a separately established processing element, or it can be integrated in a chip of the above device. In addition, it can also be stored in the memory of the above device in the form of program code, and called and executed by a processing element of the above device. The implementation of other modules is similar. In addition, these modules can be fully or partially integrated together, or they can be implemented independently. The processing element described here can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above-mentioned Internet page restoration method or each of the above modules can be completed by the hardware integrated logic circuit in the processor element or the instructions in the form of software.

[0096] For example, the above modules may be one or more integrated circuits configured to implement the above web page restoration method, such as one or more application specific integrated circuits (ASIC), or one or more digital signal processors (DSP), or one or more field programmable gate arrays (FPGA). For another example, when a certain module above is implemented in the form of a processing element scheduling program code, the processing element may be a general-purpose processor, such as a central processing unit (CPU) or other processors that can call program code. For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0097] The storage medium of the present invention stores a computer program, which, when executed by a processor, implements the above-mentioned method for restoring an online page or the above-mentioned method for accessing a restored online page. The storage medium includes: ROM, RAM, disk, U disk, memory card or optical disk, etc., various media that can store program codes.

[0098] like Figure 3 As shown, the terminal of the present invention includes a processor and a memory.

[0099] The memory is used to store computer programs. Preferably, the memory includes: ROM, RAM, disk, USB flash drive, memory card or CD, etc., which can store program codes.

[0100] The processor is connected to the memory and is used to execute the computer program stored in the memory so as to enable the terminal to execute the above-mentioned method for restoring the online page or the above-mentioned method for accessing the restored online page.

[0101] Preferably, the processor 3 can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components.

[0102] It should be noted that the Internet page restoration system of the present invention can implement the Internet page restoration method of the present invention, and the Internet page restoration page access system of the present invention can implement the Internet page restoration access method of the present invention, but the implementation device of the Internet page restoration method of the present invention includes but is not limited to the structure of the Internet page restoration system listed in this embodiment, and the implementation device of the Internet page access method of the present invention also includes but is not limited to the structure of the Internet page access system listed in this embodiment. All structural deformations and replacements of the prior art made according to the principles of the present invention are included in the protection scope of the present invention.

[0103] In summary, the web page restoration method, system, access method, system, medium and terminal described in the present invention do not need to operate the physical file content under the condition of a mirror network, and the browser will send a reference request according to the HTML file processing; and the web page restoration process and access process have high performance, and can achieve real-time restoration, without considering the massive data processing situation, and only when accessing the restored web page will real-time processing be performed, and there is no performance problem; and it also supports dynamic resource reference, and has coverage for complex resource dependencies, such as special resource requests, js, css file resource requests, etc., and can meet different needs; therefore, the present invention effectively overcomes various shortcomings in the prior art and has high industrial utilization value.

[0104] The above embodiments are merely illustrative of the principles and effects of the present invention, and are not intended to limit the present invention. Anyone familiar with the art may modify or alter the above embodiments without departing from the spirit and scope of the present invention. Therefore, all equivalent modifications or alterations made by a person of ordinary skill in the art without departing from the spirit and technical concept disclosed by the present invention shall still be covered by the claims of the present invention.

Claims

1. A method for restoring a web page, characterized in that: The following steps are involved: After receiving the restore request, the mirror traffic is read through the restore program to obtain the resource files in the Internet page; Storing the resource file in the root directory of the custom server; The storing of the resource file in the root directory of the customized server includes taking the address host mapping file information host as the first-level directory and taking the resource address url of the resource file as the remaining hierarchical directories, so as to store the resource file in layers in the customized server; Traversing all resource files of each of the online pages, and generating an access index in the root directory to access the online pages; The customization server generates the access index corresponding to each resource file corresponding to the online page.

2. A method for restoring access to an Internet page, wherein the Internet page is restored by the restoration method according to claim 1, characterized in that: The steps include: The access host accesses the access index in the customized server through a browser, traverses all the resource files associated with the online page, and generates an http access request for the associated resource files; Analyzing the http access request through the customized server to obtain the basic path information referer and the network request information url in the resource access request, and localizing the network request information url according to the basic path information referer to obtain the localized path url; The customization server adds the localization path URL to the redirection address information location in the response information, and sends the response information to the access host; After the access host obtains the response information, it obtains the redirection address information location, analyzes the redirection address information location to generate redirection path information url, and sends the redirection path information url to the customization server; The customized server obtains the target resource file in the root directory according to the redirection path information URL, and sends the target resource file to the access host to complete the access process.

3. The method for restoring access to an Internet page according to claim 2, characterized in that: The method of performing localization processing on the network request information URL according to the basic path information referer to obtain a localized path URL includes: After determining that the network request information url is a path request, obtaining the path information of the basic path information referer and the IP address of the customized server; The localized path URL is regenerated according to the path information of the basic path information referer and the IP address of the customized server.

4. The method for restoring access to an Internet page according to claim 2, characterized in that: The Internet page restoration method further includes: After determining that the network request information url is an Internet request, the custom server analyzes the path address in the network request information url and obtains the IP address of the custom server; The real path information of the corresponding resource file in the customized server is obtained according to the path address in the network request information url and the IP address of the customized server, and the corresponding resource file is sent to the access host according to the real path information.

5. A web page restoration system, characterized in that: include: A reading module, configured to read the mirrored traffic through a restoration program after receiving the restoration request to obtain the resource file in the Internet page; A storage module, used for storing the resource file in the root directory of the custom server; The storing of the resource file in the root directory of the customized server includes taking the address host mapping file information host as the first-level directory and taking the resource address url of the resource file as the remaining hierarchical directories, so as to store the resource file in layers in the customized server; A generation module, used for traversing all resource files of each of the online pages, and generating an access index in the root directory to access the online page; The customization server generates the access index corresponding to each resource file corresponding to the online page.

6. A system for accessing a restored web page, wherein the web page is restored by the system of claim 5, characterized in that: include: A request module, used to access the access index in the customized server through a browser, traverse all the resource files associated with the online page, and generate an http access request for the associated resource files; A localization module, used for analyzing the http access request through the customized server to obtain the basic path information referer and the network request information url in the resource access request, and localizing the network request information url according to the basic path information referer to obtain the localized path url; A response module, used for adding the localized path URL to the redirection address information location in the response information through the custom server, and sending the response information to the access host; A redirection sending module, used for obtaining the redirection address information location after the access host obtains the response information, analyzing the redirection address information location to generate redirection path information url, and sending the redirection path information url to the custom server; The access module is used to obtain the target resource file in the root directory according to the redirection path information URL through the customized server, and send the target resource file to the access host to complete the access process.

7. A storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method for restoring an online page according to claim 1 or the method for restoring access to an online page according to any one of claims 2 to 4 is implemented.

8. A terminal, characterized in that: include: Processor and memory; The memory is used to store computer programs; The processor is used to execute the computer program stored in the memory, so that the terminal executes the method for restoring the online page according to claim 1 or the method for restoring the online page according to any one of claims 2 to 4.

Images