A method, device, and medium for expanding homologous websites

Abstract

The present application relates to the field of the Internet, and discloses a method, device and medium for expanding homologous websites, including: obtaining the URL of the website to be expanded and task-related parameters, initiating an HTTP or HTTPS request for the URL of the website to be expanded to obtain response information, extracting website features from the response information to obtain a website feature set, selecting target website features from the website feature set, and converting them into query strings recognizable by a cyberspace search engine. The cyberspace search engine verifies the distinctiveness of the target website features according to the task-related parameters to determine the homologous websites corresponding to the website to be expanded. Thus, based on the cyberspace search engine, the website features of the website to be expanded are automatically extracted, automatically converted into query languages recognizable by the cyberspace search engine, and automatically verified to determine the homologous websites corresponding to the website to be expanded, avoiding the low efficiency of manually extracting, converting and verifying website features.

Description

Technical Field

[0001] The present application relates to the field of the Internet, and in particular to a method, device, and medium for expanding homologous websites. Background Art

[0002] A cyberspace search engine is a cyberspace asset security management system that integrates network asset collection, asset information, vulnerability impact assessment, and security management. When expanding services on a homologous website, a cyberspace search engine is required. Homologous website expansion involves finding other websites on the internet that share the same architecture and template as the one being expanded.

[0003] When expanding a same-origin website, you first need to use a browser to access the website to be expanded to obtain the response information generated during the access process, and then manually extract website features from the obtained response information based on past experience. When accessing a website, a complete HTTP or HTTPS request contains a request header, a response header, a request body, and a response body. Website features refer to the signature features that are different from other websites extracted from the response header and response body. Each time a website feature is extracted, it needs to be manually converted into a specific query syntax that can be recognized by the cyberspace search engine, and the cyberspace search engine will query the website feature. After obtaining the query results, the query results are manually evaluated and verified. During the entire query process, if the website feature extraction is not appropriate, the query result will be a false positive. If the number of website features extracted is not appropriate, the query result will be a missed negative. The extracted website features corresponding to false positives and missed negatives are not iconic. The above query steps need to be repeated until a website feature with an iconic feature is obtained.

[0004] It can be seen that the efficiency of extracting website features based on previous manual experience is low when conducting homologous website searches. In addition, manual conversion of website features into the search engine query language of cyberspace and manual repeated verification of query results will reduce the efficiency of homologous website expansion.

[0005] It can be seen that how to improve the efficiency of same-origin website expansion is an urgent problem to be solved by those skilled in the art. Summary of the Invention

[0006] The purpose of this application is to provide a method, device and medium for expanding homologous websites, which automatically extracts website features corresponding to the website to be expanded based on a cyberspace search engine, automatically converts the website features into a query string that can be recognized by the cyberspace search engine, and automatically verifies the landmarks of the website features to determine the homologous website corresponding to the website to be expanded, avoiding the inefficiency of manual extraction, conversion and verification of website features, and improving the expansion efficiency of homologous websites.

[0007] To solve the above technical problems, the present application provides a method for expanding homologous websites, including:

[0008] Obtain the URL of the website to be expanded and task-related parameters;

[0009] Initiate an HTTP or HTTPS request for the URL of the website to be expanded to obtain response information;

[0010] Extract the website features corresponding to the website to be expanded from the response information to obtain a website feature set;

[0011] Select target website features from the website feature set, and convert the target website features into query strings recognizable by a cyber space search engine;

[0012] Invoke the cyber space search engine so that the cyber space search engine verifies the iconicity of the target website features according to the task-related parameters to determine the homologous websites corresponding to the website to be expanded.

[0013] Preferably, the extracting the website features corresponding to the website to be expanded from the response information to obtain a website feature set includes:

[0014] Parse the response information to obtain a response body;

[0015] Extract the website features in the specified tags from the response body; wherein, there are multiple specified tags;

[0016] Filter all the extracted website features to obtain the website feature set.

[0017] Preferably, selecting target website features from the website feature set includes:

[0018] Sort the website features in the website feature set according to the values of the website features corresponding to the specified tags to obtain a sorting result;

[0019] According to the sorting result, sequentially select the website features with decreasing values as the target website features.

[0020] Preferably, the task-related parameters include the number of combinations of target website features and the number of verification times, wherein the initial value of the number of verification times is zero, and the initial value of the number of combinations of target website features is greater than zero.

[0021] Preferably, the cyber space search engine verifying the iconicity of the target website features according to the task-related parameters includes:

[0022] Sequentially add the target website features to a query group and combine them with other target website features to obtain a latest query group;

[0023] Each time a latest query group is obtained, the network space search engine performs a query on the query string corresponding to the target website feature combination in the latest query group to obtain a query result, and the number of target website feature combinations is reduced by one, and the number of verification times is increased by one;

[0024] Determining whether the number of target website features in the latest query group is greater than or equal to a preset value;

[0025] If it is greater than or equal to the preset value, verify whether the query results of the latest preset number of times are the same. If they are the same, end the query and determine whether the target website features of the preset number of times are iconic;

[0026] If it is less than the preset value, the step of sequentially adding the target website feature to the query group and combining it with other target website features to obtain the latest query group is entered.

[0027] Preferably, if the query results of the latest preset number of times are different, the method further includes:

[0028] Removing the target website feature with the highest value in the latest query group, and adding the next target website feature in the website feature set to combine to obtain the current latest query group;

[0029] The step of determining whether the number of target website features in the latest query group is greater than or equal to a preset value is entered.

[0030] Preferably, after determining that the number of target website features in the latest query group is greater than or equal to a preset value, the method further includes:

[0031] Determining whether the number of target website feature combinations is equal to zero;

[0032] If it is not equal to zero, then entering the step of verifying whether the query results of the latest preset number of times are the same;

[0033] If it is equal to zero, then the process ends and determines that the expansion of the website to be expanded has failed.

[0034] In order to solve the above technical problems, the present application also provides a device for expanding homologous websites, including:

[0035] The acquisition module is used to obtain the URL of the website to be expanded and task-related parameters;

[0036] A request module is used to initiate an HTTP or HTTPS request to the URL of the website to be expanded to obtain response information;

[0037] An extraction module, configured to extract website features corresponding to the website to be expanded from the response information to obtain a website feature set;

[0038] A selection module, configured to select target website features from the website feature set;

[0039] A conversion module, configured to convert the target website features into query strings recognizable by a cyberspace search engine;

[0040] An invocation module, configured to invoke the cyberspace search engine, so that the cyberspace search engine verifies the iconicity of the target website features according to the task-related parameters to determine the homologous websites corresponding to the website to be expanded.

[0041] To solve the above technical problems, the present application further provides a device for expanding homologous websites, including a memory for storing a computer program;

[0042] A processor, configured to implement the steps of the method for expanding homologous websites as described when executing the computer program.

[0043] To solve the above technical problems, the present application further provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the steps of the method for expanding homologous websites as described are implemented.

[0044] A method for expanding homologous websites provided by the present invention includes: obtaining the URL of the website to be expanded and task-related parameters, initiating an HTTP or HTTPS request for the URL of the website to be expanded to obtain response information, extracting website features corresponding to the website to be expanded from the response information to obtain a website feature set, then selecting target website features from the website feature set, converting the target website features into query strings recognizable by a cyberspace search engine, invoking the cyberspace search engine to verify the iconicity of the target website features according to the task-related parameters, and further determining the homologous websites corresponding to the website to be expanded. It can be seen that the technical solution provided by the present application realizes automatic extraction of the website features of the website to be expanded based on the cyberspace search engine, automatically converts the website features into query languages recognizable by the cyberspace search engine, and automatically verifies the website features to determine the homologous websites corresponding to the website to be expanded, avoiding the low efficiency of manually extracting website features, converting website feature query languages, and verifying website features, and thus improving the efficiency of expanding homologous websites.

[0045] In addition, the present application further provides a device and a medium for expanding homologous websites, corresponding to the method for expanding homologous websites as described, and having the same effect. Description of the Drawings

[0046] To more clearly illustrate the embodiments of the present application, the following will briefly introduce the drawings required in the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, without creative efforts, other drawings can be obtained based on these drawings.

[0047] Figure 1 It is a flowchart of a method for expanding homologous websites provided by an embodiment of the present application;

[0048] Figure 2 It is a flowchart of a method for expanding homologous websites provided by another embodiment of the present application;

[0049] Figure 3 It is a structural diagram of a device for expanding homologous websites provided by an embodiment of the present application;

[0050] Figure 4 It is a structural diagram of a device for expanding homologous websites provided by another embodiment of the present application. Specific embodiments

[0051] The following will clearly and completely describe the technical solutions in the embodiments of the present application in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, rather than all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the protection scope of the present application.

[0052] The core of the present application is to provide a method, device and medium for expanding homologous websites, which realizes automatically extracting the website features of the website to be expanded, and automatically converting the website features into query strings recognizable by the network space search engine. The network space search engine automatically queries the query strings to verify the signature of the website features, and further realizes the purpose of determining the homologous websites of the website to be expanded, avoiding the low efficiency of manual extraction, conversion and verification of website features, and improving the efficiency of expanding homologous websites.

[0053] To enable those skilled in the art to better understand the solution of the present application, the following will further elaborate on the present application in conjunction with the drawings and specific embodiments.

[0054] Cyberspace search engines are integrated cyberspace asset security management systems that collect network assets, provide asset information, assess vulnerability impacts, and conduct security management. They enable the mapping of internet space and employ specialized query syntax for retrieving relevant assets. Consequently, cyberspace search engines are often used in conjunction with the term "same-origin website expansion," which refers to finding other websites on the internet that share the same architecture and template as the website being expanded. For example, if website A needs to be expanded, a cyberspace search engine would be used to search the internet for websites with the same architecture and template as website A.

[0055] Currently, when expanding a same-origin website, one first needs to use a browser to access the website to be expanded to obtain the response information generated during the access process, and then manually extract website features from the obtained response information based on past experience. When accessing a website, a complete HTTP or HTTPS request includes a request header, a response header, a request body, and a response body. Website features refer to the signature features that are extracted from the response header and the response body that distinguish them from other websites. Each time a website feature is extracted, it needs to be manually converted into a specific query syntax that can be recognized by the cyberspace search engine, and the cyberspace search engine will query the website feature. After obtaining the query results, the query results are manually evaluated and verified. During the entire query process, if the website feature extraction is not appropriate, the query result will be a false positive. If the number of website features extracted is not appropriate, the query result will be a missed negative. The extracted website features corresponding to false positives and missed negatives are not signature. The above query steps need to be repeated until a signature website feature is obtained.

[0056] It can be seen that the efficiency of extracting website features based on previous manual experience is low when conducting homologous website searches. In addition, manual conversion of website features into the search engine query language of cyberspace and manual repeated verification of query results will reduce the efficiency of homologous website expansion.

[0057] In order to improve the efficiency of homologous website expansion, this application provides a homologous website expansion method, which realizes the purpose of automatically extracting, converting and verifying website features based on a website space search engine, and then determines the homologous website of the website to be expanded.

[0058] Figure 1 A flowchart of a method for expanding a same-origin website provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method includes:

[0059] S10: Obtain the URL of the website to be expanded and task-related parameters.

[0060] S11: Initiate an HTTP or HTTPS request to the extension website URL and obtain response information.

[0061] S12: Extract the website features corresponding to the website to be expanded from the response information to obtain a website feature set.

[0062] In a specific implementation, it is necessary to obtain the Uniform Resource Locator (URL) of the website to be expanded, and initiate an HTTP or HTTPS request for the URL of the website to be expanded to obtain response information. It can be understood that a complete HTTP or HTTPS request includes a request header, a response header, a request body, and a response body, and the website features of the website to be expanded exist in the response body. Therefore, it is necessary to parse the response information to obtain the response body, and extract the website features corresponding to the website to be expanded from the response body to obtain a website feature set.

[0063] It should be noted that when extracting website features from the response body, extract the website features in the specified tags, and filter all the extracted website features to obtain a website feature set. It can be understood that there are multiple specified tags, and this application does not limit the number and types of specified tags.

[0064] S13: Select target website features from the website feature set, and convert the target website features into query strings recognizable by a cyber space search engine.

[0065] After obtaining the website feature set, through step S13, select target website features from the website feature set, and convert the target website features into query strings recognizable by a cyber space search engine. It should be noted that when selecting target website features, first sort each website feature in the website feature set according to its value, and select the website features as target website features in order from high to low according to the value.

[0066] S14: Invoke a cyber space search engine so that the cyber space search engine verifies the signature of the target website features according to the task-related parameters to determine the homologous websites corresponding to the website to be expanded.

[0067] In step S14, invoke a cyber space search engine, and verify the signature of the target website features through the cyber space search engine according to the task-related parameters, and then determine the homologous websites corresponding to the website to be expanded. Among them, the task-related parameters include the number of combinations of target website features and the number of verification times. The initial value of the number of verification times is zero, and the initial value of the number of combinations of target website features is greater than zero. It should be noted that the number of combinations of target website features in the task-related parameters is used to avoid falling into infinite queries, and the number of verification times is used to accumulate the number of times the cyber space search engine verifies the signature of the target website features.

[0068] When verifying the iconicity of the target website features, the target website features selected from the website feature set are successively added to the query group to be combined with other target website features to obtain the latest query group. For each obtained latest query group, the cyber space search engine performs a query on the query string composed of each target website feature combination in the current latest query group to obtain a query result, and subtracts one from the number of target website feature combinations and adds one to the verification times. For each obtained query result, it is judged whether the number of target website features in the current latest query group is greater than or equal to the preset value. If it is greater than or equal to the preset value, it is verified whether the query results in the latest preset value times are the same. If they are the same, it can be determined that the target website features in the current latest query group have iconicity, and all the homologous websites corresponding to the website to be extended are the websites queried through this latest query group. When the number of target website features in the latest query group is less than the preset value, the next target website feature is continuously selected from the website feature set and added to the query group to be combined with other target website features to obtain the latest query group.

[0069] When the query results in the latest preset value times are different, the target website feature with the highest value in the current latest query group is removed, and the next target website feature in the website feature set is added for combination to obtain a new latest query group, and then it enters the step of judging whether the number of target website features in the current latest query group is greater than or equal to the preset value. It should be noted that the present application does not limit the setting of the preset value.

[0070] In a specific embodiment, after determining that the number of target website features in the latest query group is greater than or equal to the preset value, it is judged whether the number of target website feature combinations is equal to zero. If it is not equal to zero, it enters the step of verifying whether the query results in the latest preset value times are the same. If it is equal to zero, it ends and determines that the extension of the website to be extended fails.

[0071] The method for expanding homologous websites provided by the embodiments of the present application includes: obtaining the URL of the website to be expanded and task-related parameters, initiating an HTTP or HTTPS request for the URL of the website to be expanded to obtain response information, extracting the website features corresponding to the website to be expanded from the response information to obtain a website feature set, then selecting target website features from the website feature set, converting the target website features into query strings recognizable by a cyberspace search engine, and calling the cyberspace search engine to verify the iconicity of the target website features according to the task-related parameters, so as to determine the homologous websites corresponding to the website to be expanded. It can be seen that the technical solution provided by the present application realizes the automatic extraction of the website features of the website to be expanded based on the cyberspace search engine, automatically converts the website features into query languages recognizable by the cyberspace search engine, and automatically verifies the website features to determine the homologous websites corresponding to the website to be expanded, avoiding the low efficiency of manually extracting website features, converting website feature query languages, and verifying website features, thereby improving the efficiency of expanding homologous websites.

[0072] In a specific embodiment, the response information obtained by initiating a complete HTTP or HTTPS request for the URL of the website to be expanded includes a request header, a response header, a request body, and a response body. After obtaining the response information, it is parsed to obtain the response body, and the website features in the specified tags are extracted from the response body to obtain a website feature set. For the convenience of understanding, an example will be given below.

[0073] For example, the URL of the website to be expanded is http: / / www.abcxxxx.com. According to the values of the website features in each tag of the response body, the a tag, img tag, link tag, and script tag are selected as the specified tags. After initiating an HTTP or HTTPS request for the URL of the website to be expanded, the response information is parsed to obtain the response body, and the website features in the a tag, img tag, link tag, and script tag are respectively obtained from the response body.

[0074] It should be noted that in the response body, the form of a complete tag is: <tag name>KDF CAPITAL< / tag name>, where "KDF CAPITAL" is the content filled in. For example, the a tag is <a class＝"mui-tab-itemcur"href＝" index.php？m＝index&a＝index"> <span class＝"mui-icon cur"> <span class＝"mui-tab-label">xxxx , and the website feature in this a tag is " / index.php?m=Index&a=index". A certain script tag is: <script src＝" public home js2 jquery.dependclass.js">< / script> , and the website feature in this script tag is " / Public / home / js2 / jquery.dependClass.js".

[0075] It should be noted that there are multiple specified tags. According to the value of the website features in the tags, the tags with high value are preferentially used as the specified tags. The number of specified tags selected in this application is not limited. It is worth noting that after extracting the website features from each specified tag, all the website features are de-duplicated, classified, and filtered to filter out website features with shorter feature lengths or empty data.

[0076] In the method for expanding homologous websites provided by the embodiments of this application, after obtaining the response information of the website URL to be expanded, the response information is parsed to obtain the response body, so as to extract the website features in the specified tags from the response body to obtain a website feature set. Among them, there are multiple specified tags, and the tags with high website feature value are preferably used as the specified tags. Furthermore, on the premise of ensuring the accuracy of extracting website tags, the purpose of automatically extracting website features is achieved, avoiding the low efficiency in manual extraction of website features.

[0077] On the basis of the above embodiments, after extracting the website features from the specified tags to obtain a website feature set, the website features in the website feature set are sorted according to the website feature values in each specified tag to obtain a sorting result, and the website features with decreasing value are sequentially selected as the target website features according to the sorting result.

[0078] For example, select the a tag, img tag, link tag, and script tag from the response body as the specified tags. Usually, the website feature value in the a tag is the same as that in the img tag, and is greater than the website feature value in the link tag, while the website feature value in the link tag is greater than the website feature value in the script tag, that is, a tag = img tag > link tag > script tag.

[0079] It can be understood that the website feature value refers to the number of homologous websites that can be queried when the website feature is converted into a query string recognizable by a network space search engine and the network space search engine is used for query. For example, the result of the homologous websites queried by converting "banana" into a query string recognizable by a network space search engine is less than the result of the homologous websites queried by converting "fruit" into a query string recognizable by a network space search engine. Therefore, the value of the website feature "banana" is less than the value of the website feature "fruit".

[0080] After obtaining the website feature set by selecting website features from the a tag, img tag, link tag, and script tag, sort the website features in the set according to the value of the website features in each specified tag to obtain a sorting result, and sequentially use the website features with the highest to lowest value as the target website features according to the sorting result. It should be noted that the website feature values in the a tag and img tag are the same. Therefore, when selecting the target website features, the website features in the a tag and img tag can be randomly selected as the target website features. When the website features in the a tag and img tag are selected, then select the website features in the link tag and script tag according to the rule of highest to lowest value. The present application does not limit the method of selecting website features with the same value in the selected tags as target features. In addition, the present application also does not limit the types and quantities of the selected specified tags.

[0081] The method for expanding homologous websites provided by the embodiments of the present application, after obtaining the website feature set, sorts each website feature in the website feature set according to the value corresponding to the website feature in each specified tag to obtain a sorting result, and sequentially selects the website features with the highest to lowest value as the target website features according to the sorting result, thereby ensuring the accuracy of the selected website features.

[0082] Figure 2 is a flowchart of a method for expanding homologous websites provided by another embodiment of the present application, as Figure 2 shown, the method includes:

[0083] S20: Sequentially add the target website features to the query group and combine them with other target website features to obtain the latest query group.

[0084] S21: For each obtained latest query group, the network space search engine performs a query on the query string corresponding to the combination of target website features in the latest query group to obtain a query result, and the number of combinations of target website features is reduced by one, and the number of verification times is increased by one.

[0085] S22: Determine whether the number of target website features in the latest query group is greater than or equal to a preset value. If it is greater than or equal to the preset value, go to step S23. If it is less than the preset value, go to step S20.

[0086] S23: Determine whether the number of combinations of target website features is equal to zero. If it is not equal to zero, go to step S24. If it is equal to zero, go to step S25.

[0087] S24: Verify whether the query results of the latest preset number of times are the same. If they are the same, go to step S26. If they are not the same, go to step S27.

[0088] S25: End and determine that the expansion of the website to be expanded fails.

[0089] S26: End the query and determine that each target website feature in the preset value times is iconic.

[0090] S27: Remove the target website feature with the highest value in the latest query group, and add the next target website feature in the website feature set for combination to obtain the current latest query group, and return to step S22.

[0091] In the above embodiment, each website feature in the website feature set is sorted according to the value of the website feature in each specified tag, and the website features with the highest value to the lowest value are sequentially used as target website features according to the sorting. Then, in step S20, the target website features are sequentially added to the query group for combination with other target website features to obtain the latest query group. In fact, step S20 is to convert the first target website feature into a query string recognizable by the cyber space search engine for query to obtain a query result, then select the second website feature from the website feature set to combine with the first website feature, and convert it into a query string for query to obtain a query result, and so on for combined query.

[0092] It should be noted that step S21 needs to be executed once every time step S20 is executed. That is, for each obtained latest query group, the cyber space search engine queries the query string corresponding to the target website feature combination in the latest query group once to obtain a query result, and the number of target website feature combinations is reduced by one, and the verification times is increased by one. Of course, step S22 needs to be executed once every time step S21 is executed. That is, it is judged whether the number of target website features in the latest query group is greater than or equal to the preset value. If it is not greater than or equal to the preset value, return to step S20 to continue combined query to obtain a query result.

[0093] When the number of target website features in the latest query group is greater than or equal to the preset value, first judge whether the number of target website feature combinations is equal to zero. If it is equal to zero, it is determined that the website features corresponding to the to-be-expanded website do not have iconicity, and the expansion of the to-be-expanded website fails, and the expansion of the to-be-expanded website ends. If it is not equal to zero, verify whether the query results of each of the latest preset value times are the same. If they are the same, end the query and determine that each target website feature in the preset value times is iconic. If they are not the same, remove the target website feature with the highest value in the latest query group, and add the next target website feature in the website feature set for combination to obtain the current latest query group, and then return to step S22 to continue judging whether the number of target website features in the latest query group is greater than or equal to the preset value.

[0094] It should be noted that the task-related parameters include the number of combinations of target website features and the number of verification times. The initial value of the number of verification times is zero, and the initial value of the number of combinations of target website features is greater than zero. For the sake of easy understanding, an example will be given below.

[0095] For example, name the number of combinations of target website features as Max_num, and the initial value of Max_num is 5. Name the number of verification times as try_num, and try_num is 0. Set the preset value to 3. Add the first target website feature to the query group according to the value from the website feature set, and automatically convert it into a string recognizable by the cyber space search engine. Then call the query interface of the cyber space search engine to query the query string to obtain the query result search_text_base. Based on this, try_num is incremented by 1, and Max_num is decremented by 1. At this time, try_num is 1 and Max_num is 4. It can be understood that there is only one target website feature in the first latest query group obtained.

[0096] At this time, it is determined that try_num1 is less than the preset value 3. Continue to add the second target website feature to the query group for combination according to the value from the website feature set, and automatically convert it into a string. Then query through the query interface of the cyber space search engine to obtain the query result search_text_one. Similarly, try_num is incremented by 1, and Max_num is decremented by 1. At this time, try_num is 2 and Max_num is 3.

[0097] try_num2 is less than the preset value 3. Continue to add the third target website feature to the query group for combination according to the value from the website feature set, and automatically convert it into a string. Then query through the query interface of the cyber space search engine to obtain the query result search_text_sec. try_num is incremented by 1, and Max_num is decremented by 1. At this time, try_num is 3 which is equal to the preset value, and Max_num is 2 which is greater than zero, that is, try_num >= 3 and Max_num > 0. Then judge whether the results of the three queries are the same. If search_text_base = search_text_one = search_text_sec = 400, it is determined that the results of the three queries are the same, and the number of homologous websites queried for the website to be expanded is 400. Then it can be determined that the three target website features output are iconic.

[0098] If the results of the three queries are different, then remove the target website feature with the highest value in the latest query group, and add the fourth target website feature in the website feature set for combination to obtain the current latest query group. Call the query interface to perform a query to obtain the query result. After that, try_num continuously increases and will always be greater than the preset value of 3. Then, every time the target website feature with the highest value in the latest query group is removed and the next target website feature in the website feature set is added for combination to obtain the current latest query group, it is necessary to verify whether the query results of the latest preset number of times are equal, that is, to verify whether the latest 3 query results are equal, until the target website feature with equal verification results is found.

[0099] During this process, Max_num continuously decreases. When Max_num = 0, it is determined that the expansion of the website to be expanded fails, that is, none of the website features in the website to be expanded are iconic. It can be understood that the setting of the number of combinations of target website features is to avoid infinite verification of query results.

[0100] The method for expanding homologous websites provided by the embodiments of the present application realizes the automatic extraction of the website features of the website to be expanded through a cyber space search engine, automatically converts the website features into a query language recognizable by the cyber space search engine, and automatically verifies the website features to determine the homologous websites corresponding to the website to be expanded, avoiding the low efficiency of manual extraction, conversion, and verification of website features, and thus improving the efficiency of expanding homologous websites.

[0101] In the above embodiments, the method for expanding homologous websites is described in detail. The present application also provides corresponding embodiments of the device for expanding homologous websites. It should be noted that the present application describes the embodiments of the device part from two perspectives, one is from the perspective of functional modules, and the other is from the perspective of hardware structure.

[0102] Figure 3 For the structure diagram of a device for expanding homologous websites provided by the embodiments of the present application, as Figure 3 shown, the device includes:

[0103] An acquisition module 10, configured to acquire the URL of the website to be expanded and task-related parameters;

[0104] A request module 11, configured to initiate an HTTP or HTTPS request for the URL of the website to be expanded to obtain response information;

[0105] An extraction module 12, configured to extract the website features corresponding to the website to be expanded from the response information to obtain a website feature set;

[0106] A selection module 13, configured to select target website features from the website feature set;

[0107] A conversion module 14, configured to convert the target website features into query strings recognizable by a cyberspace search engine;

[0108] An invocation module 15, configured to invoke a cyberspace search engine, so that the cyberspace search engine validates the signature of the target website features according to task-related parameters to determine the homologous websites corresponding to the website to be extended.

[0109] Since the embodiments in the apparatus part correspond to those in the method part, please refer to the descriptions of the embodiments in the method part for the embodiments in the apparatus part, which will not be elaborated here.

[0110] The apparatus for homologous website extension provided by the embodiments of the present application includes: obtaining the URL of the website to be extended and task-related parameters, initiating an HTTP or HTTPS request for the URL of the website to be extended to obtain response information, extracting the website features corresponding to the website to be extended from the response information to obtain a website feature set, then selecting target website features from the website feature set, converting the target website features into query strings recognizable by a cyberspace search engine, invoking the cyberspace search engine to validate the signature of the target website features according to task-related parameters, and further determining the homologous websites corresponding to the website to be extended. It can be seen that the technical solution provided by the present application realizes the automatic extraction of the website features of the website to be extended, automatically converts the website features into query languages recognizable by a cyberspace search engine, and automatically validates the website features to determine the homologous websites corresponding to the website to be extended based on the cyberspace search engine, avoiding the low efficiency of manually extracting website features, converting website feature query languages, and validating website features, and thus improving the efficiency of homologous website extension.

[0111] Figure 4 The structural diagram of the apparatus for homologous website extension provided by another embodiment of the present application is as Figure 4 shown. The apparatus for homologous website extension includes: a memory 20, configured to store computer programs;

[0112] A processor 21, configured to implement the steps of the method for homologous website extension as mentioned in the above embodiments when executing the computer programs.

[0113] The apparatus for homologous website extension provided by this embodiment may include but is not limited to smart phones, tablet computers, laptop computers, desktop computers, etc.

[0114] Among them, the processor 21 may include one or more processing cores, such as a 4-core processor, an 8-core processor, etc. The processor 21 may be implemented in at least one hardware form of a digital signal processor (DSP for short), a field-programmable gate array (FPGA for short), or a programmable logic array (PLA for short). The processor 21 may also include a main processor and a coprocessor. The main processor is a processor for processing data in the wake state, also known as the central processing unit (CPU for short); the coprocessor is a low-power processor for processing data in the standby state. In some embodiments, the processor 21 may be integrated with a graphics processing unit (GPU for short), and the GPU is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 21 may also include an artificial intelligence (AI for short) processor, and the AI processor is used to process computational operations related to machine learning.

[0115] The memory device 20 may include one or more computer-readable storage media, and the computer-readable storage media may be non-transitory. The memory device 20 may also include high-speed random access memory and non-volatile memory, such as one or more disk storage devices and flash storage devices. In this embodiment, the memory device 20 is at least used to store the following computer program 201. After the computer program is loaded and executed by the processor 21, it can implement the relevant steps of the method for expanding homologous websites disclosed in any of the foregoing embodiments. In addition, the resources stored in the memory device 20 may also include an operating system 202 and data 203, etc., and the storage method may be temporary storage or permanent storage. Among them, the operating system 202 may include Windows, Unix, Linux, etc. The data 203 may include, but is not limited to, the relevant data involved in the method for expanding homologous websites.

[0116] In some embodiments, the device for expanding homologous websites may further include a display screen 22, an input / output interface 23, a communication interface 24, a power supply 25, and a communication bus 26.

[0117] Those skilled in the art can understand that Figure 4 the structure shown in

[0118] The apparatus for expanding homologous websites provided by an embodiment of the present application includes a memory and a processor. When the processor executes the program stored in the memory, the following method can be implemented: the method for expanding homologous websites.

[0119] The apparatus for expanding homologous websites provided by an embodiment of the present application realizes automatic extraction of website features of a website to be expanded based on a cyberspace search engine, automatically converts the website features into a query language recognizable by the cyberspace search engine, and automatically verifies the website features to determine the homologous websites corresponding to the website to be expanded, avoiding the low efficiency of manual extraction of website features, conversion of website feature query languages, and verification of website features, thereby improving the efficiency of expanding homologous websites.

[0120] Finally, the present application also provides an embodiment corresponding to a computer-readable storage medium. A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps recorded in the above method embodiment are implemented.

[0121] It can be understood that if the method in the above embodiment is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and executes all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical discs that can store program codes.

[0122] The above has introduced in detail a method, apparatus, and medium for expanding homologous websites provided by the present application. The embodiments in the specification are described in a progressive manner, and the key point of each embodiment is the difference from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the apparatus disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the method part. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the protection scope of the claims of the present application.

[0123] It should also be noted that in this specification, relational terms such as first and second are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any actual relationship or order between these entities or operations. Moreover, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or device comprising a series of elements not only includes those elements but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article or device. Without further limitation, an element defined by the phrase "comprising an..." does not exclude the presence of additional identical elements in the process, method, article or device comprising said element.

Claims

1. A method for expanding a homologous website, characterized in that: include: Obtaining the URL of the website to be expanded and task-related parameters; wherein the task-related parameters include the number of target website feature combinations and the number of verifications, wherein the initial value of the number of verifications is zero, and the initial value of the number of target website feature combinations is greater than zero; Initiate an HTTP or HTTPS request to the URL of the website to be expanded and obtain response information; Extracting website features corresponding to the website to be expanded from the response information to obtain a website feature set; Selecting target website features from the website feature set, and converting the target website features into a query string recognizable by a cyberspace search engine; Invoking the network space search engine so that the network space search engine verifies the landmark of the target website feature according to the task-related parameters to determine the homologous website corresponding to the website to be expanded; Wherein, the cyberspace search engine verifies the characteristic of the target website feature according to the task-related parameters, including: sequentially adding the target website feature to the query group and combining it with other target website features to obtain the latest query group; each time the latest query group is obtained, the cyberspace search engine performs a query on the query string corresponding to the target website feature combination in the latest query group to obtain a query result, and the number of target website feature combinations is reduced by one, and the number of verifications is increased by one; judging whether the number of target website features in the latest query group is greater than or equal to a preset value; if it is greater than or equal to the preset value, verifying whether the query results of the latest preset number of times are the same, if so, ending the query and determining that the target website features of the preset number of times are characteristic; if it is less than the preset value, entering the step of sequentially adding the target website feature to the query group and combining it with other target website features to obtain the latest query group; If the query results of the latest preset number of times are different, it also includes: Removing the target website feature with the highest value in the latest query group, and adding the next target website feature in the website feature set to combine to obtain the current latest query group; Entering the step of determining whether the number of target website features in the latest query group is greater than or equal to a preset value; After determining that the number of target website features in the latest query group is greater than or equal to a preset value, the method further includes: Determining whether the number of target website feature combinations is equal to zero; If it is not equal to zero, then entering the step of verifying whether the query results of the latest preset number of times are the same; If it is equal to zero, then the process ends and determines that the expansion of the website to be expanded has failed.

2. The method for expanding homologous websites according to claim 1, characterized in that: Extracting the website features corresponding to the website to be expanded from the response information to obtain a website feature set includes: Parsing the response information to obtain a response body; Extracting the website features in a specified tag from the response body; wherein the specified tags are multiple; All the extracted website features are filtered to obtain the website feature set.

3. The method for expanding homologous websites according to claim 2, characterized in that: Selecting target website features from the website feature set includes: Sorting the website features in the website feature set according to the values corresponding to the website features in the designated tags to obtain a ranking result; According to the ranking result, the website features with the highest value to the lowest value are selected in sequence as the target website features.

4. A device for expanding homologous websites, characterized in that: include: An acquisition module, configured to acquire the URL of the website to be expanded and task-related parameters; wherein the task-related parameters include the number of target website feature combinations and the number of verifications, wherein the initial value of the number of verifications is zero, and the initial value of the number of target website feature combinations is greater than zero; A request module is used to initiate an HTTP or HTTPS request to the URL of the website to be expanded to obtain response information; An extraction module is used to extract website features corresponding to the website to be expanded from the response information to obtain a website feature set; A selection module, configured to select target website features from the website feature set; A conversion module, configured to convert the target website features into a query string recognizable by a cyberspace search engine; A calling module, configured to call the network space search engine so that the network space search engine verifies the landmark of the target website feature according to the task-related parameters to determine the homologous website corresponding to the website to be expanded; Wherein, the cyberspace search engine verifies the characteristic of the target website feature according to the task-related parameters, including: sequentially adding the target website feature to the query group and combining it with other target website features to obtain the latest query group; each time the latest query group is obtained, the cyberspace search engine performs a query on the query string corresponding to the target website feature combination in the latest query group to obtain a query result, and the number of target website feature combinations is reduced by one, and the number of verifications is increased by one; judging whether the number of target website features in the latest query group is greater than or equal to a preset value; if it is greater than or equal to the preset value, verifying whether the query results of the latest preset number of times are the same, if so, ending the query and determining that the target website features of the preset number of times are characteristic; if it is less than the preset value, entering the step of sequentially adding the target website feature to the query group and combining it with other target website features to obtain the latest query group; If the query results of the latest preset number of times are different, it also includes: Removing the target website feature with the highest value in the latest query group, and adding the next target website feature in the website feature set to combine to obtain the current latest query group; Entering the step of determining whether the number of target website features in the latest query group is greater than or equal to a preset value; After determining that the number of target website features in the latest query group is greater than or equal to a preset value, the method further includes: Determining whether the number of target website feature combinations is equal to zero; If it is not equal to zero, then entering the step of verifying whether the query results of the latest preset number of times are the same; If it is equal to zero, then the process ends and determines that the expansion of the website to be expanded has failed.

5. A device for expanding homologous websites, characterized in that: including a memory for storing a computer program; A processor is configured to implement the steps of the method for extending same-origin websites as claimed in any one of claims 1 to 3 when executing the computer program.

6. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the steps of the method for extending homologous websites according to any one of claims 1 to 3.

Images