Network link processing method and apparatus, electronic device, and storage medium

Abstract

Embodiments of the present application provide a network link processing method and device, electronic equipment and storage medium, and relate to the technical field of Internet. The method comprises: obtaining a to-be-accessed link and a keyword field; filtering a preset information source database according to the to-be-accessed link and the keyword field to obtain a selected information source; filtering a preset analysis template database according to the selected information source to obtain a selected analysis template; analyzing the to-be-accessed link according to the selected analysis template to obtain a target data request; and sending the target data request to the to-be-accessed link to obtain target link data. In the embodiments of the present application, the obtained target data request is paired with the information source, thereby improving the completeness of the obtained link data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet, and particularly relates to a network link processing method and device, electronic equipment and storage medium. BACKGROUND

[0002] Generally, in the case of obtaining a network link of an article, the link data of the article can be obtained only by accessing the network link. The link data includes the title of the article, the publishing time of the article, the author of the article, the body of the article and the like. However, due to the limitation of the network link, the relatively complete link data cannot be obtained by directly accessing the network link. Therefore, how to provide a network link processing method capable of improving the completeness of the obtained link data has become a technical problem to be solved. SUMMARY

[0003] The main purpose of the embodiments of the present application is to provide a network link processing method and device, electronic equipment and storage medium, which can improve the completeness of the obtained link data.

[0004] To achieve the above purpose, a first aspect of the embodiments of the present application provides a network link processing method, which comprises the following steps:

[0005] obtaining a to-be-accessed link and a keyword field;

[0006] screening a preset information source database according to the to-be-accessed link and the keyword field to obtain a selected information source;

[0007] screening a preset analysis template database according to the selected information source to obtain a selected analysis template;

[0008] analyzing the to-be-accessed link according to the selected analysis template to obtain a target data request;

[0009] sending the target data request to the to-be-accessed link to obtain target link data.

[0010] In some embodiments, the step of screening the preset information source database according to the to-be-accessed link and the keyword field to obtain the selected information source comprises:

[0011] if the keyword field is a null field, performing domain name resolution on the to-be-accessed link to obtain a link domain name;

[0012] screening the information source database according to the link domain name to obtain the selected information source; wherein the information source database comprises matching information of the link domain name and the selected information source.

[0013] In some embodiments, the preset information source database is filtered according to the to-be-accessed link and the keyword field to obtain a selected information source, including:

[0014] If the keyword field is a non-empty field, a keyword is extracted from the keyword field;

[0015] Similarity calculation is performed according to the keyword and a name of each information source of the information source database to obtain a similarity;

[0016] The selected information source is obtained according to the information source whose similarity meets a preset similarity condition.

[0017] In some embodiments, the to-be-accessed link is parsed according to the selected parsing template to obtain a target data request, including:

[0018] A request parameter of the to-be-accessed link is identified to obtain an information identification code; wherein the information identification code is used to represent an identification of target link data corresponding to the to-be-accessed link;

[0019] The preset data request database is filtered according to the information identification code to obtain the target data request; wherein the data request database includes matching information of the information identification code and the target data request, and the target data request is used to request to obtain the to-be-accessed link data.

[0020] In some embodiments, before the preset data request database is filtered according to the information identification code to obtain the target data request, the method further includes:

[0021] The data request database is created, specifically including:

[0022] At least one initial data request of a preset information source is obtained;

[0023] The initial data request is parameter modified to obtain at least one intermediate data request; the parameter modification includes deleting a parameter or modifying a value of the parameter;

[0024] The intermediate data request is sent to the to-be-accessed link to obtain intermediate link data;

[0025] The intermediate data request whose completeness meets a preset completeness condition is filtered from the intermediate link data to obtain the target data request;

[0026] The data request database is obtained according to the target data request.

[0027] In some embodiments, before the screening the intermediate data request whose completeness of the intermediate link data meets the preset completeness condition to obtain the target data request, the method further comprises:

[0028] The filtering the intermediate data request specifically comprises:

[0029] Obtaining any two intermediate link data to obtain first intermediate link data and second intermediate link data;

[0030] Comparing the data amount of the first intermediate link data with the data amount of the second intermediate link data to obtain a comparison result;

[0031] According to the comparison result, deleting the intermediate data request corresponding to the first intermediate link data or deleting the intermediate data request corresponding to the second intermediate link data.

[0032] In some embodiments, the deleting the intermediate data request corresponding to the first intermediate link data or deleting the intermediate data request corresponding to the second intermediate link data according to the comparison result comprises:

[0033] If the comparison result is that the data amount of the first intermediate link data is greater than or equal to the data amount of the second intermediate link data, the intermediate data request corresponding to the first intermediate link data is deleted;

[0034] If the comparison result is that the data amount of the first intermediate link data is less than the data amount of the second intermediate link data, the intermediate data request corresponding to the second intermediate link data is deleted.

[0035] To achieve the above object, a second aspect of the embodiment of the present application proposes a network link processing device, the device comprises:

[0036] An obtaining module is configured to obtain a to-be-accessed link and a keyword field;

[0037] An information source screening module is configured to screen a preset information source database according to the to-be-accessed link and the keyword field to obtain a selected information source;

[0038] A target screening module is configured to screen a preset analysis template database according to the selected information source to obtain a selected analysis template;

[0039] An analysis module is configured to analyze the to-be-accessed link according to the selected analysis template to obtain a target data request;

[0040] A sending module is configured to send the target data request to the to-be-accessed link to obtain target link data.

[0041] To achieve the above object, a third aspect of the embodiments of the present application provides an electronic device, comprising a memory and a processor, the memory stores a computer program, and the processor implements the method of the first aspect when executing the computer program.

[0042] To achieve the above object, a fourth aspect of the embodiments of the present application provides a storage medium, which is a computer readable storage medium, and the storage medium stores a computer program, and the computer program is executed by a processor to implement the method of the first aspect.

[0043] The network link processing method and device, electronic device and storage medium provided by the present application determine a matched information source according to the to-be-accessed link and the keyword field, select a parsing template according to the information source, parse the to-be-accessed link according to the parsing template to obtain a target data request, then send the target data request to the to-be-accessed link, so that the server to which the to-be-accessed link belongs returns link data according to the target data request, and finally obtain the target link data. In the embodiments of the present application, the target data request is paired with the information source, so that the integrity of the obtained link data can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0044] Figure 1 is a flowchart of the network link processing method provided by the embodiments of the present application;

[0045] Figure 2 is a schematic diagram of one embodiment of step S101 in Figure 1

[0046] Figure 3 is a flowchart of step S102 in Figure 1

[0047] Figure 4 is a flowchart of step S102 in Figure 1

[0048] Figure 5 is a flowchart of step S104 in Figure 1

[0049] Figure 6 is a flowchart of the network link processing method provided by another embodiment of the present application;

[0050] Figure 7 is a flowchart of the network link processing method provided by another embodiment of the present application;

[0051] Figure 8 is a module structure block diagram of the network link processing device provided by the embodiments of the present application;

[0052] ​​​​Figure 9 Fig. 1 is a schematic diagram of a hardware structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0053] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below with reference to the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and should not be used to limit the present application.

[0054] It should be noted that although the functional modules are divided in the device schematic diagram, and the logical order is shown in the flowchart, in some cases, the steps shown or described can be performed in a manner different from the module division in the device or the order in the flowchart. The terms "first", "second", and the like in the specification and claims and the above-described drawings are used to distinguish similar objects, and do not necessarily describe a specific order or sequence.

[0055] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terms used herein are only for the purpose of describing the embodiments of the present application and are not intended to limit the present application.

[0056] First, the terms involved in the present application are analyzed:

[0057] Uniform Resource Locator (URL): Each web page on the Internet has a unique name identifier, commonly known as URL. It is the uniform resource locator of the web page. Simply put, URL is the web address, commonly known as "website address". The network link referred to in the embodiments of the present application is the URL, also known as the link address or web address.

[0058] URL usually starts with http: / / or https: / / . A basic URL may include, for example, a protocol type, a domain name, a path, and parameters. The protocol type, for example, http or https, where http is the HyperText Transfer Protocol, https is the Hyper Text Transfer Protocol over Secure Socket Layer, and https is the HyperText Transfer Protocol transmitted by the Secure Socket Layer. The httpHeader parameter includes the operation parameters of the http request and response in the httpHeader. The domain name, path, and parameters, etc. The parameters include the parameters of the requested data.

[0059] Generally, in the case of obtaining a network link of an article, if the link data of the article, such as the title of the article, the publishing time of the article, the author of the article, and the body of the article, is to be obtained, the network link needs to be checked in a browser to determine whether the network link is valid, and the link data of the article is manually searched. However, the body of the article is missing in some link data. Therefore, how to provide a network link processing method to improve the completeness of the obtained link data has become a technical problem to be solved.

[0060] In addition, when the network link is obtained, the article data information is determined after the link is manually verified to be valid and accessible. When different information sources (websites and APPs) are checked, a large amount of manpower is consumed and the checking efficiency is low. Therefore, in addition to improving the completeness of the obtained link data, the efficiency of verifying the validity of the network link can also be improved.

[0061] The technical scheme of the embodiment of the application is mainly applicable to a master-slave architecture, that is, a communication network of a client and a server architecture. The client can send a request to the server and receive data returned by the server. The client can be an APP installed on a user device. The user device can be various smart phones, tablet computers, and the like. The server can be a computer server.

[0062] The network link processing method provided by the embodiment of the application can be used in many general or special computer system environments or configurations. For example, a server computer, a multi-processor system, a microprocessor-based system, a set-top box, a programmable consumer electronic device, a network PC, a distributed computing environment including any of the above systems or devices, and the like. The application can be described in the general context of computer-executable instructions executed by a computer, such as a program module. Generally, program modules include routines, programs, objects, components, data structures, and the like that perform specific tasks or implement specific abstract data types. The application can also be practiced in a distributed computing environment in which tasks are performed by remote processing devices connected by a communication network. In a distributed computing environment, program modules can be located in local and remote computer storage media, including storage devices.

[0063] The network link processing method and the network link processing device, the electronic device, and the storage medium provided by the embodiment of the application are described in detail.

[0064] According to an embodiment of the application, a network link processing method is provided.

[0065] The network link is specifically a URL, and the corresponding link data can be obtained by accessing the network link. The process of accessing the network link includes assembling a plurality of data requests according to the network link, sending the plurality of data requests to a server corresponding to the network link, and obtaining link data returned by the server based on the data requests, thereby completing the process of accessing the network link.

[0066] Figure 1 The optional flowchart of the network link processing method provided in the embodiment of the present application can include, but is not limited to, steps S101 to S105.

[0067] In step S101, a to-be-accessed link and a keyword field are obtained.

[0068] In step S102, a preset information source database is filtered according to the to-be-accessed link and the keyword field, and a selected information source is obtained.

[0069] In step S103, a preset parsing template database is filtered according to the selected information source, and a selected parsing template is obtained.

[0070] In step S104, the to-be-accessed link is parsed according to the selected parsing template, and a target data request is obtained.

[0071] In step S105, the target data request is sent to the to-be-accessed link, so as to obtain target link data.

[0072] The steps S101 to S105 shown in the embodiment of the present application first determine the matched information source according to the to-be-accessed link and the keyword field, then select the parsing template according to the information source, and then parse the to-be-accessed link according to the parsing template to obtain the target data request. Then, the target data request is sent to the to-be-accessed link, so that the server to which the to-be-accessed link belongs returns the link data according to the target data request, and finally the target link data is obtained. In the embodiment of the present application, the method of determining the parsing template through the information source can improve the parsing efficiency of the link, and the target data request obtained by pairing with the information source can improve the completeness of obtaining the link data.

[0073] The steps S101 to S105 are described in detail below.

[0074] In step S101 of some embodiments, the to-be-accessed link is specifically a URL, and can also be referred to as a network link, a network address, a web address, etc. The keyword field is an optional field, which can be a null field or a non-null field. If the keyword field is a null field, it means that there is no keyword. If the keyword field is a non-null field, it means that the keyword can be extracted from the keyword field.

[0075] The steps S101 to S105 are described in detail below. Figure 2Different situations of obtaining the to-be-accessed link and the keyword field are described.

[0076] In one embodiment, the user inputs the URL into the input box of the verification system, but does not input the keyword, at this time, the verification system obtains the to-be-accessed network link and the keyword field, but the keyword field is empty.

[0077] In another embodiment, the user inputs the URL and the keyword into the input box of the verification system, at this time, the verification system obtains the to-be-accessed network link and the keyword field, and the keyword field is a non-empty field.

[0078] In the case that the user inputs the keyword into the verification system, when the cursor moves to the input box corresponding to the keyword, a plurality of candidate keywords preset by the verification system are displayed. The information source referred to in the embodiment of the application includes a website and an APP, and one of the plurality of candidate keywords can be selected as the input keyword field in response to the touch or click of the user.

[0079] In step S102 of some embodiments, the preset information source database is filtered according to the to-be-accessed link and the keyword field, and the selected information source is obtained.

[0080] Specifically, the information source to which the link belongs can be known from the domain name of the URL, but the URL needs to be domain name resolved to obtain the link domain name, and then the link domain name is subjected to information source filtering to obtain the selected information source. For details, please refer to Figure 3 Step S102 can include but is not limited to steps S201 to S202:

[0081] Step S201, if the keyword field is an empty field, the to-be-accessed link is subjected to domain name resolution to obtain the link domain name;

[0082] Step S202, the information source database is filtered according to the link domain name, and the selected information source is obtained; wherein the information source database includes matching information of the link domain name and the selected information source.

[0083] The steps S201 to S202 shown in the embodiment of the application show that the URL includes the domain name, so that the to-be-accessed link can be subjected to domain name resolution, and then the link domain name obtained by the resolution is subjected to information source matching, so as to obtain the selected information source.

[0084] In another embodiment, considering that the domain name resolution and the information source matching consume time and affect the processing efficiency, the keyword field is introduced, if the keyword field is a non-empty field, the keyword is extracted from the keyword field, and the information source corresponding to the keyword is used as the selected information source.

[0085] It should be noted that if the keyword field is a candidate keyword selected by the user through touch or click, no information source matching is needed, and the selected keyword is taken as the selected information source. Alternatively, please refer to Figure 4 Step S102 further includes but is not limited to steps S301 to S303:

[0086] Step S301, if the keyword field is a non-empty field, the keyword is extracted from the keyword field;

[0087] Step S302, similarity calculation is performed according to the keyword and the name of each information source in the information source database, to obtain the similarity;

[0088] Step S303, the selected information source is obtained according to the information source whose similarity meets the preset similarity condition.

[0089] The steps S301 to S303 shown in the embodiments of the present application determine the selected information source by the similarity between the keyword and the information source, without the need for domain name resolution, which greatly improves the processing efficiency.

[0090] In step S302 of some embodiments, the keyword is vectorized to obtain a first vector; the name of the information source is vectorized to obtain a second vector; and the cosine similarity or the Euclidean distance is calculated according to the first vector and the second vector to obtain the similarity.

[0091] In step S303 of some embodiments, the selected information source is obtained according to the information source with the maximum similarity.

[0092] In step S103 of some embodiments, the selected information source is used to screen the preset resolution template database to obtain a selected resolution template. Specifically, the resolution template database includes a plurality of candidate resolution templates and matching information of each candidate resolution template and the information source. After the selected information source is obtained in step S102, the resolution template database is screened according to the selected information source, which is actually to find the matching information, find the candidate resolution template matched with the selected information source, and obtain the selected resolution template.

[0093] In step S104 of some embodiments, the selected resolution template is used to resolve the to-be-accessed link to obtain a target data request.

[0094] Please refer to Figure 5 Step S104 includes but is not limited to steps S401 to S402:

[0095] Step S401, the request parameter of the to-be-accessed link is identified to obtain an information identification code; wherein the information identification code is used to represent the identification of the target link data of the to-be-accessed link data;

[0096] Step S402, screening the preset data request database according to the information identification code to obtain a target data request; wherein, the data request database comprises matching information of the information identification code and the target data request, and the target data request is used to request to obtain target link data.

[0097] In the embodiment of the application, each to-be-accessed link URL has a corresponding request parameter, which is used to represent the information identification code to be obtained, and the information identification code can represent which target link data is to be obtained. For example, the URL unique identifier can be obtained from the to-be-accessed URL link, that is, the information identification code in the embodiment of the application, so that it can be known which article is to be obtained or which video is to be obtained. The data request database comprises a plurality of candidate data requests and matching information of the information identification code and the candidate data requests. The candidate data requests of the data request database are screened according to the information identification code to obtain the target data request. Then, the target link data can be obtained by sending the target data request.

[0098] Please refer to Figure 6 Before step S402, the network link processing method provided by the embodiment of the application further comprises:

[0099] Creating a data request database, specifically comprising steps S501 to S505:

[0100] Step S501, obtaining at least one initial data request of a preset information source;

[0101] Step S502, modifying the parameters of the initial data request to obtain at least one intermediate data request: the parameter modification comprises deleting parameters or modifying the values of the parameters;

[0102] Step S503, sending the intermediate data request to the to-be-accessed link to obtain intermediate link data;

[0103] Step S504, screening the intermediate data request whose intermediate link data meets the preset completeness condition to obtain a target data request;

[0104] Step S505, obtaining a data request database according to the target data request.

[0105] The steps S501 to S505 shown in the embodiments of the present application obtain at least one initial data request of a preset information source through a packet capturing tool, access a specified link according to the initial data request, and receive corresponding link data. However, considering that there are redundant parameters in the initial data request or the values of the parameters are dynamically changed, the initial data request can be processed to delete parameters and modify the values of the parameters to obtain intermediate data requests. The obtained intermediate data requests are sent to the link to be accessed to receive returned intermediate link data. The target data request is determined according to the completeness of the intermediate link data. It can be understood that the higher the completeness of the intermediate link data, the more likely the corresponding intermediate link data is selected as the target data request. Finally, the target data request and the information type are paired and stored in a database to finally obtain a data request database.

[0106] It should be noted that in step S504, if the completeness of the intermediate link data meets the preset completeness condition, the corresponding intermediate data request can be added to the target data request. In this way, there can be two same intermediate data requests that can obtain consistent intermediate link data, resulting in redundant data requests in the target data request.

[0107] Therefore, referring to Figure 7 Before step S504, the network link processing method of the embodiments of the present application further includes:

[0108] The intermediate data requests are filtered, specifically including:

[0109] In step S601, any two intermediate link data are obtained to obtain first intermediate link data and second intermediate link data.

[0110] In step S602, the first intermediate link data and the second intermediate link data are compared to obtain a comparison result.

[0111] In step S603, the intermediate data request corresponding to the first intermediate link data or the intermediate data request corresponding to the second intermediate link data is deleted according to the comparison result.

[0112] The steps S601 to S603 shown in the embodiments of the present application consider that the same intermediate data request can obtain the same intermediate link data, so the intermediate data requests are filtered to reduce the number of times of sending requests to the link to be accessed and improve the communication efficiency.

[0113] If the comparison result is that the data amount of the first intermediate link data is greater than or equal to the data amount of the second intermediate link data, the intermediate data request corresponding to the first intermediate link data is deleted.

[0114] If the comparison result shows that the amount of data in the first intermediate link is less than the amount of data in the second intermediate link, then delete the intermediate data request corresponding to the second intermediate link.

[0115] Specifically, if the comparison result shows that the amount of data in the first intermediate link data is equal to the amount of data in the second intermediate link data, then the intermediate data request corresponding to the first intermediate link data is deleted; or, if the comparison result shows that the amount of data in the first intermediate link data is equal to the amount of data in the second intermediate link data, then the intermediate data request corresponding to the second intermediate link data is deleted.

[0116] In some embodiments, considering the number of parameters, deleting intermediate data requests with a large number of parameters may include: obtaining the intermediate data request corresponding to the first intermediate link data to obtain a first intermediate data request; obtaining the intermediate data request corresponding to the second intermediate link data to obtain a second intermediate data request; comparing the number of parameters in the first and second intermediate data requests to obtain a comparison result; if the number of parameters in the first intermediate data request is greater than or equal to the number of parameters in the second intermediate data request, then the intermediate data request corresponding to the first intermediate link data is deleted; if the number of parameters in the first intermediate data request is less than the number of parameters in the second intermediate data request, then the intermediate data request corresponding to the second intermediate link data is deleted. By deleting intermediate data requests with fewer parameters through the above steps, the number of accesses is reduced, parameter parsing is reduced, parameter parsing speed is improved, and the speed of obtaining connection data is further improved.

[0117] In step S105 of some embodiments, a target data request is sent to the link to be accessed in order to obtain the target link data.

[0118] It should be noted that the target data request may include one or multiple data requests to be sent. Each data request to be sent may include one or more request headers. Request headers may include User-Agent, cookie, token, host, etc.

[0119] 1. Obtain all data information in the link by sending a data request to be sent, thus obtaining the target link data, specifically including:

[0120] 1) You need to add a request header. Adding the most basic User-Agent request header will allow you to access the link and obtain all the necessary data fields to get the target link data.

[0121] 2) Multiple request headers need to be added. Access to some request header parameters is time-sensitive. The data to be accessed is obtained by dynamically changing real-time data requests.

[0122] Two: obtain the target link data by sending multiple data requests to obtain the link data. Considering that a single access to the link cannot obtain the most complete link data, multiple accesses to the link are required, which is equivalent to sending multiple data requests to the link to obtain the most complete data information, and obtaining the target link data. The multiple accesses are the same as the single access in the above embodiment, and will not be described again.

[0123] In an embodiment, after step S105, the network link processing method further comprises:

[0124] determining a data format parsing method according to the selected information source;

[0125] parsing the target link data according to the data format parsing method to obtain target parsed data;

[0126] returning the target parsed data to the front end.

[0127] It should be noted that the link data obtained by sending the request according to the above embodiment is different in data structure according to different parsing methods: the final data returned by each information source (website, APP), some of which are html, json format, XML, and some of which are encrypted and need to be cracked to obtain the final data. And return the data to the front end.

[0128] Please refer to Figure 8 The embodiments of the present application also provide a network link processing device, which can implement the above network link processing method, Figure 8 The module structure block diagram of the network link processing device provided by the embodiments of the present application, the device comprises: an acquisition module 701, an information source screening module 702, a target screening module 703, an analysis module 704, and a sending module 705. The acquisition module 701 is configured to acquire a link to be accessed and a keyword field; the information source screening module 702 is configured to screen a preset information source database according to the link to be accessed and the keyword field to obtain a selected information source; the target screening module 703 is configured to screen a preset analysis template database according to the selected information source to obtain a selected analysis template; the analysis module 704 is configured to analyze the link to be accessed according to the selected analysis template to obtain a target data request; and the sending module 705 is configured to send the target data request to the link to be accessed to obtain target link data.

[0129] It should be noted that the specific embodiments of the network link processing device are basically the same as the specific embodiments of the above network link processing method, and will not be described again here.

[0130] The electronic device provided by the embodiment of the present application comprises a memory, a processor, a program stored in the memory and executable on the processor, and a data bus for realizing connection communication between the processor and the memory, and the program is executed by the processor to realize the cross-domain session data processing method. The electronic device can be any smart terminal including a tablet computer, a vehicle-mounted computer, etc.

[0131] Please refer to Figure 9 , Figure 9 The hardware structure of the electronic device of another embodiment is illustrated, and the electronic device comprises:

[0132] The processor 801 can be implemented in the form of a general CPU (Central Processing Unit), a microprocessor, an ASIC (Application Specific Integrated Circuit), or one or more integrated circuits, etc., for executing related programs to realize the technical solutions provided by the embodiments of the present application.

[0133] The memory 802 can be implemented in the form of a ROM (Read Only Memory), a static storage device, a dynamic storage device, or a RAM (Random Access Memory), etc. The memory 802 can store an operating system and other application programs, and when the technical solutions provided by the embodiments of the present application are implemented by software or firmware, the related program codes are stored in the memory 802 and called and executed by the processor 801 to realize the cross-domain session data processing method of the embodiments of the present application.

[0134] The input / output interface 803 is used to realize information input and output.

[0135] The communication interface 804 is used to realize the communication interaction between the device and other devices, which can realize communication through a wired manner (for example, a USB, a network cable, etc.) or a wireless manner (for example, a mobile network, WIFI, Bluetooth, etc.).

[0136] The bus 805 is used to transmit information between various components (for example, the processor 801, the memory 802, the input / output interface 803, and the communication interface 804) of the device.

[0137] The processor 801, the memory 802, the input / output interface 803, and the communication interface 804 realize the communication connection between each other in the device through the bus 805.

[0138] This application embodiment also provides a storage medium, which is a computer-readable storage medium for computer-readable storage. The storage medium stores one or more programs, which can be executed by one or more processors to implement the above-described cross-domain session data processing method.

[0139] Memory, as a non-transitory computer-readable storage medium, can be used to store non-transitory software programs and non-transitory computer-executable programs. Furthermore, memory may include high-speed random access memory, and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some embodiments, memory may optionally include memory remotely located relative to the processor, and these remote memories can be connected to the processor via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0140] The network link processing method, apparatus, electronic device, and storage medium provided in this application first determine the matching information source based on the link to be accessed and the keyword field. Then, a parsing template is selected based on the information source. Next, the link to be accessed is parsed according to the parsing template to obtain the target data request. Finally, the target data request is sent to the link to be accessed, causing the server to return link data based on the target data request, thus obtaining the target link data. In this application embodiment, determining the parsing template through the information source improves the efficiency of link parsing, and the matching of the obtained target data request with the information source enhances the integrity of the obtained link data.

[0141] The embodiments described in this application are for the purpose of more clearly illustrating the technical solutions of the embodiments of this application, and do not constitute a limitation on the technical solutions provided by the embodiments of this application. As those skilled in the art will know, with the evolution of technology and the emergence of new application scenarios, the technical solutions provided by the embodiments of this application are also applicable to similar technical problems.

[0142] It will be understood by those skilled in the art that Figure 1 , 4 The technical solutions shown in Figure 8 do not constitute a limitation on the embodiments of this application. They may include more or fewer steps than shown in the figure, or combine certain steps, or different steps.

[0143] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs.

[0144] Those skilled in the art can understand that all or some of the steps in the method disclosed above, the function modules / units in the system and the device can be implemented as software, firmware, hardware or appropriate combination thereof.

[0145] The terms "first", "second", "third", "fourth" and the like in the description of the application and in the claims, if any, are used for distinguishing between similar elements and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of these terms herein is to be construed to cover a general order and / or structure unless otherwise indicated. Furthermore, the terms "comprise", "comprising", "has", "having", "includes", "including", "contain", "containing" or any other similar forms are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises, has, includes, contains items or components does not include items or components not explicitly recited. The terms "a" or "an", as used herein in the detailed description and in the claims, mean "one or more" or "at least one", unless otherwise indicated.

[0146] It should be understood that, in the present application, "at least one" means one or more, and "multiple" means two or more. "And / or" is used to describe the relationship between associated objects, which means that there can be three relationships, for example, "A and / or B" can mean that there are three cases: only A, only B, and A and B at the same time, where A and B can be singular or plural. The character " / " generally represents an "or" relationship between the associated objects. "At least one of the following" or similar expressions means any combination of these items, including any combination of single or multiple items. For example, at least one of a, b or c can mean a, b, c, "a and b", "a and c", "b and c", or "a and b and c", where a, b, and c can be singular or plural.

[0147] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, and the division of the units is only a logical function division, and actual implementation can have another division manner, for example, multiple units or components can be combined or integrated into another system, or some features can be omitted or not executed. In addition, the coupling or direct coupling or communication connection between the displayed or discussed objects can be indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0148] The units described as separate components may or may not be physically separate, and the components displayed as units may or may not be physical units, that is, may be located in one place, or may be distributed on multiple network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0149] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can be physically present alone, or two or more units can be integrated in one unit. The integrated unit can be realized in the form of hardware or in the form of a software functional unit.

[0150] The integrated unit, if realized in the form of a software functional unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the part of the prior art that contributes to the technical solutions or the whole or part of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium, including multiple instructions for making an electronic device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the method described in each embodiment of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various program storage media.

[0151] The preferred embodiments of the embodiments of the present application are described above with reference to the accompanying drawings, and are not limited to the scope of the embodiments of the present application. Any modifications, equivalent replacements and improvements made by those skilled in the art without departing from the scope and essence of the embodiments of the present application shall be within the scope of the embodiments of the present application.

Claims

1. A network link processing method characterized by, The method comprises: acquiring a to-be-accessed link and a keyword field; screening a preset information source database according to the to-be-accessed link and the keyword field to obtain a selected information source; screening a preset analysis template database according to the selected information source to obtain a selected analysis template; analyzing the to-be-accessed link according to the selected analysis template to obtain a target data request; sending the target data request to the to-be-accessed link to obtain target link data.

2. The method of claim 1, wherein, The screening of the preset information source database according to the to-be-accessed link and the keyword field comprises: if the keyword field is an empty field, performing domain name analysis on the to-be-accessed link to obtain a link domain name; screening the information source database according to the link domain name to obtain the selected information source; wherein the information source database comprises matching information of a link domain name and a selected information source.

3. The method of claim 2, wherein, The screening of the preset information source database according to the to-be-accessed link and the keyword field comprises: if the keyword field is a non-empty field, extracting a keyword from the keyword field; performing similarity calculation according to the keyword and the name of each information source of the information source database to obtain a similarity; obtaining the selected information source according to the information source whose similarity meets a preset similarity condition.

4. The method of claim 1, wherein, The analysis of the to-be-accessed link according to the selected analysis template to obtain a target data request comprises: identifying a request parameter of the to-be-accessed link to obtain an information identification code; wherein the information identification code is used to represent the identification of target link data corresponding to the to-be-accessed link; screening a preset data request database according to the information identification code to obtain the target data request; wherein the data request database comprises matching information of an information identification code and the target data request, and the target data request is used to request to acquire the target link data.

5. The method of claim 4, wherein, Before the screening of the preset data request database according to the information identification code to obtain the target data request, the method further comprises: creating the data request database, specifically comprising: acquiring at least one initial data request of a preset information source; performing parameter modification on the initial data request to obtain at least one intermediate data request; the parameter modification comprises deleting a parameter or modifying the value of a parameter; sending the intermediate data request to the to-be-accessed link to obtain intermediate link data; screening the intermediate data request whose completeness meets a preset completeness condition to obtain the target data request; obtaining the data request database according to the target data request.

6. The method of claim 5, wherein, Before the screening of the intermediate data request whose completeness meets a preset completeness condition to obtain the target data request, the method further comprises: filtering the intermediate data request, specifically comprising: acquiring any two intermediate link data to obtain first intermediate link data and second intermediate link data; The first intermediate link data and the second intermediate link data are compared to obtain a comparison result; According to the comparison result, the intermediate data request corresponding to the first intermediate link data is deleted, or the intermediate data request corresponding to the second intermediate link data is deleted.

7. The method of claim 6, wherein, The method according to the comparison result, the intermediate data request corresponding to the first intermediate link data is deleted, or the intermediate data request corresponding to the second intermediate link data is deleted, comprising: If the comparison result is that the data amount of the first intermediate link data is equal to the data amount of the second intermediate link data, the intermediate data request corresponding to the first intermediate link data is deleted; Or, if the comparison result is that the data amount of the first intermediate link data is equal to the data amount of the second intermediate link data, the intermediate data request corresponding to the second intermediate link data is deleted.

8. A network link processing apparatus, characterized by comprising: The device comprises: An acquisition module configured to acquire a to-be-accessed link and a keyword field; An information source screening module configured to screen a preset information source database according to the to-be-accessed link and the keyword field to obtain selected information sources; A target screening module configured to screen a preset analysis template database according to the selected information sources to obtain selected analysis templates; An analysis module configured to analyze the to-be-accessed link according to the selected analysis templates to obtain a target data request; A sending module configured to send the target data request to the to-be-accessed link to obtain target link data.

9. An electronic device, comprising: The electronic device comprises a memory and a processor, the memory stores a computer program, and the processor implements the method of any one of claims 1 to 7 when executing the computer program.

10. A computer readable storage medium, the storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the method of any one of claims 1 to 7.

Images