Website retrieval method and device based on dynamic keyword library

Through the round search and update mechanism of dynamic keyword databases, the problems of low search accuracy and poor timeliness caused by static keyword databases are solved, and more accurate and timely understanding and retrieval of public service website content is achieved.

CN119003889AInactive Publication Date: 2024-11-22HARBIN INST OF TECH AT WEIHAI +1

Patent Information

Application Number
CN202411473151.9
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-10-22
Publication Date
2024-11-22
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

In the prior art, public service website search methods rely on static keyword databases, resulting in low search accuracy and poor timeliness of keyword databases, inability to update in time, and inability to deeply understand the professional language environment of a specific website, resulting in generalization of search results.

Method used

Using a website search method based on dynamic keyword database, we can select different types of target keywords to build keyword groups in each round of search, obtain the second keyword of the newly added effective website, and update the dynamic keyword database according to the word frequency to form Multi-element composite query to improve retrieval accuracy and keyword database timeliness.

Benefits of technology

It improves the search accuracy and timeliness of keyword databases, avoids the problem that static keyword databases cannot be updated in time, ensures the pertinence and freshness of search results, and enhances the understanding of the content of public service websites.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119003889A_ABST
    Figure CN119003889A_ABST
Patent Text Reader

Abstract

The invention provides a website retrieval method and device based on a dynamic keyword library, and relates to the technical field of Internet, the method comprises the following steps: in the retrieval process of this round, selecting different types of target first keywords from the dynamic keyword library by using retrieval keywords, and constructing a keyword group according to the target first keywords; performing website retrieval by utilizing the keyword group, obtaining a newly-added effective website corresponding to the retrieval keyword, and obtaining a second keyword from the newly-added effective website; updating the dynamic keyword library by using the second keyword, and performing the next round of retrieval process by using the updated dynamic keyword library; and taking the newly-added effective websites retrieved in all rounds as final retrieval results when the retrieval end condition is met. By the adoption of the website retrieval method and device based on the dynamic keyword library, the problems that in the website retrieval process, the retrieval accuracy is low, and the timeliness of the keyword library is poor are solved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of Internet technology, and in particular to a website search method and device based on a dynamic keyword library. Background Art

[0002] With the rapid development of Internet technology, various public service websites emerge in an endless stream on the Internet, such as online video websites, novel websites, information websites, etc., which greatly enrich the content of the Internet while also bringing security risks. Therefore, there is an urgent need for a method to efficiently and quickly obtain a large number of public service websites to collect whether the retrieved public service websites contain bad information, spread illegal advertisements, etc. In the prior art, public service websites are mainly obtained by inputting a single keyword and interacting with the search engine based on a static keyword library.

[0003] However, the above public service website search method is to use a static keyword library to search for a single keyword. Since a single keyword lacks in-depth understanding and targeted screening of the professional language environment of a specific website, the search results tend to be generalized and the search accuracy is low. At the same time, since the static keyword library requires manual intervention and management, it cannot be updated in a timely manner, resulting in the problem of delayed timeliness of the static keyword library. Summary of the invention

[0004] In view of this, the purpose of the present application is to provide a website search method and device based on a dynamic keyword library to solve the problems of low search accuracy and poor timeliness of the keyword library during the website search process.

[0005] In a first aspect, an embodiment of the present application provides a website search method based on a dynamic keyword library, comprising: In this round of search, different types of target first keywords are selected from the dynamic keyword library using the search keywords, and keyword groups are constructed based on the target first keywords; Using the keyword group to search for websites, obtaining newly added valid websites corresponding to the search keywords, and obtaining the second keyword from the newly added valid websites; Using the second keyword to update the dynamic keyword library, so as to use the updated dynamic keyword library to perform the next round of search process; When the search end conditions are met, all newly added valid websites retrieved in all rounds will be regarded as the final search results.

[0006] Optionally, the second keyword includes at least one, and using the second keyword to update the dynamic keyword library includes: determining the frequency of each second keyword in the newly added valid website; and updating the dynamic keyword library according to the frequency of each second keyword.

[0007] Optionally, the dynamic keyword library is updated according to the word frequency of each second keyword, including: selecting the second keywords to be updated that are already in the dynamic keyword library from the second keywords; updating the word frequency of the first keyword to be updated according to the word frequency of the second keyword to be updated, the first keyword to be updated being the first keyword corresponding to the second keyword to be updated; sorting all other second keywords according to their frequency, and selecting a preset number of other second keywords with high rankings to add to the dynamic keyword library, the other second keywords being the keywords in the second keywords except the second keyword to be updated.

[0008] Optionally, the method also includes: determining whether the number of first keywords in the updated dynamic keyword library meets the keyword number condition; if the keyword number condition is not met, deleting at least one first keyword with the smallest frequency from the dynamic keyword library so that the number of first keywords in the dynamic keyword library meets the keyword number condition.

[0009] Optionally, different types of target first keywords include target first general keywords, target first popular keywords and target first classification keywords, and different types of target first keywords are selected from the dynamic keyword library using the search keywords, including: determining the selection probability of the first keyword based on the word frequency of the first keyword in the dynamic keyword library; and randomly selecting the target first general keyword, the target first popular keyword and the first target classification keyword corresponding to the search keyword from the dynamic keyword library based on the selection probability.

[0010] Optionally, using a keyword group to perform a website search to obtain a newly added valid website corresponding to the search keyword includes: using multiple search engines to search the websites corresponding to the keyword group respectively to obtain at least one candidate website; determining whether each candidate website meets preset conditions, so as to use the candidate website that meets the preset conditions as a newly added valid website.

[0011] Optionally, determining whether each candidate website meets preset conditions includes: for each candidate website, determining whether the candidate website is an accessible website, whether the keywords in the candidate website meet the keyword proportion conditions, and whether the candidate website is an official website; if the candidate website is an accessible website, the keywords in the candidate website meet the keyword proportion conditions, and the candidate website is an unofficial website, then determining that the candidate website meets the preset conditions.

[0012] Optionally, before using the updated dynamic keyword library for the next round of search process, it also includes: determining whether the number of newly added valid websites meets a set number condition; if not, performing the next round of search process; if meeting the set number condition, the entire search process ends.

[0013] Optionally, before selecting different types of target keywords from the dynamic keyword library using the search keywords, the method further includes: performing a preliminary search based on the initial search keywords under each set website type, constructing an initial dynamic keyword library, and performing website search based on the initial dynamic keyword library.

[0014] In a second aspect, the embodiment of the present application further provides a website search device based on a dynamic keyword library, the device comprising: A keyword group construction module is used to select different types of target first keywords from the dynamic keyword library using the search keywords during this round of search, and to construct a keyword group based on the target first keywords; A new keyword acquisition module is added to search websites using keyword groups, obtain newly added valid websites corresponding to the search keywords, and obtain the second keyword from the newly added valid websites; A keyword library updating module, used to update the dynamic keyword library using the second keyword, so as to use the updated dynamic keyword library for the next round of search process; The search result determination module is used to take the newly added valid websites retrieved in all rounds as the final search results when the search end condition is met.

[0015] The embodiments of the present application bring the following beneficial effects: The embodiment of the present application provides a website search method and device based on a dynamic keyword library, which can flexibly combine the keywords in the dynamic keyword library to form a multi-element composite query, thereby improving the search accuracy. At the same time, the second keyword can be obtained according to the search results of each round, and the second keyword can be used to reflect which keywords are more likely to appear at present, and then the second keyword is used to dynamically update the keyword library, thereby improving the timeliness of the keyword library and avoiding the problem that the static keyword library cannot be updated in time. Compared with the website search method in the prior art, it solves the problems of low search accuracy and poor timeliness of the keyword library during the website search process.

[0016] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments will be briefly introduced below. It should be understood that the following drawings only show certain embodiments of the present application and therefore should not be regarded as limiting the scope. For ordinary technicians in this field, other related drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1A flow chart of a website search method based on a dynamic keyword library provided in an embodiment of the present application is shown; Figure 2 A flowchart showing the entire website search process provided by an embodiment of the present application is shown; Figure 3 A schematic diagram showing the structure of a website search device based on a dynamic keyword library provided in an embodiment of the present application is shown; Figure 4 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0019] To make the purpose, technical scheme and advantages of the embodiments of the present application clearer, the technical scheme in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all of the embodiments. The components of the embodiments of the present application usually described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the application claimed for protection, but merely represents the selected embodiments of the present application. Based on the embodiments of the present application, each other embodiment obtained by those skilled in the art without making creative work belongs to the scope of protection of the present application.

[0020] It is worth noting that before this application was filed, with the rapid development of Internet technology, various public service websites emerged in an endless stream on the Internet, such as online video websites, novel websites, information websites, etc., which greatly enriched the content of the Internet while also bringing security risks. Therefore, there is an urgent need for a method of efficiently and quickly obtaining a large number of public service websites. In the prior art, public service websites are mainly obtained by inputting a single keyword and interacting with a search engine based on a static keyword library.

[0021] However, the retrieval method of the above public service websites uses a static keyword library as the basic resource for the search. The static keyword library is a pre-set, relatively fixed vocabulary set that contains subject terms closely related to the target public service. However, the content of public service websites is not static. They will continue to change with actual conditions such as policy adjustments, service upgrades, and information updates. When facing the dynamic evolution of the content of public service websites, traditional acquisition methods may have obvious timeliness lags in search technology, that is, search results returned based on outdated keywords cannot accurately reflect the actual content and service status of the current website.

[0022] Secondly, the existing public service website retrieval method adopts a single keyword search strategy, which means that in actual operation, the user or system only needs to select a word from the keyword library and submit it directly to the search engine for retrieval. The search engine returns public service website links that directly match or are highly relevant to the keyword based on its indexing rules and algorithms. At this time, a large number of web pages containing the word may be returned, and the content of these web pages may only have a shallow connection with the keyword. Due to the lack of in-depth understanding and targeted screening of the professional language environment of a specific website, the resulting search results often show a greater tendency to generalize. Based on this, an embodiment of the present application provides a website search method based on a dynamic keyword library to improve the search accuracy and timeliness of the keyword library.

[0023] See also Figure 1 , Figure 1 This is a flow chart of a website search method based on a dynamic keyword library provided in an embodiment of the present application. Figure 1 As shown, the website search method based on the dynamic keyword library provided in the embodiment of the present application includes: Step S101, in this round of search, using the search keyword to select different types of target first keywords from the dynamic keyword library, and constructing a keyword group based on the target first keyword; Step S102, using the keyword group to search for websites, obtaining newly added valid websites corresponding to the search keywords, and obtaining a second keyword from the newly added valid websites; Step S103, using the second keyword to update the dynamic keyword library, so as to use the updated dynamic keyword library to perform the next round of search process; Step S104, when the search end condition is met, all newly added valid websites retrieved in all rounds are taken as the final search results.

[0024] This application can flexibly combine the keywords in the dynamic keyword library to form a multi-element composite query, thereby improving the accuracy of the search. At the same time, it can obtain the second keyword based on the search results of each round, and reflect which keywords are more likely to appear at present through the second keyword, and then use the second keyword to dynamically update the keyword library, thereby improving the timeliness of the keyword library and avoiding the problem that the static keyword library cannot be updated in time. Compared with the website search method in the prior art, it solves the problems of low search accuracy and poor timeliness of the keyword library during the website search process.

[0025] It should be noted that the above-mentioned website retrieval method based on a dynamic keyword library can be applied to the server side or the client side. The entire public service website retrieval process includes multiple rounds of retrieval processes. Different target first keywords need to be selected between different rounds, and the selected target first keywords are used to construct a keyword group, and then the keyword group is used for the current round of retrieval.

[0026] Refer to the following Figure 2 To introduce the entire website retrieval process.

[0027] Figure 2 The flowchart of the entire website search process provided by the embodiment of the present application is shown as follows: Figure 2 As shown, before performing multiple rounds of retrieval processes, step S201 is first performed.

[0028] Step S201, determine the type of the public service website. Here, the type of the public service website may refer to the type of the set public service website, and the types of public service websites include online videos, novels, information, etc. The developer may determine the specific type of the public service website according to the actual situation, and this application does not limit it here. After determining the type of the public service website, execute step S202.

[0029] Step S202, construct a dynamic keyword library. Here, an initial search keyword can be determined for each type of public service website, for example, for online video websites, the initial search keyword that can be set is "video"; for information websites, the initial search keyword that can be set is "news".

[0030] Then, a preliminary search is performed based on the initial search keywords under each type of public service website, that is, the initial search keywords under each type of public service website are input into the search engine for preliminary search, and different types of keywords are extracted from the retrieved public service websites. These different types of keywords include general keywords, popular keywords and classified keywords, and each type of keyword includes multiple. An initial dynamic keyword library is constructed based on the extracted multiple general keywords, popular keywords and classified keywords. Among them, general keywords refer to keywords that frequently appear in public service websites of all types, popular keywords refer to keywords in recent hot online events, and classified keywords refer to keywords that frequently appear in a type of public service website. For example: for online video public service websites, the classification keyword can be "short video"; for information public service websites, the classification keyword can be "news".

[0031] After the initial dynamic keyword library is established, the initial dynamic keyword library can be used to perform multiple rounds of website searches, ie, execute step S203.

[0032] Step S203, searching websites. Here, taking the website search method based on the dynamic keyword library of the present application applied to a search application as an example, when searching for online video websites, you can enter "online video website" in the search application corresponding to the present application or select the "online video website" tab to determine the type of public service website to be searched, so as to start a multi-round website search process.

[0033] The single-round retrieval process is described in detail below.

[0034] In this round of search, different types of target first keywords are selected from the dynamic keyword library using the search keywords, and keyword groups are constructed based on the target first keywords.

[0035] The search keyword is used to characterize the type of public service website that is desired to be searched. Exemplarily, the search keyword may be the “online video website” input or selected in the above example.

[0036] The dynamic keyword library may be an initial dynamic keyword library used in the first search, or may be a dynamic keyword library that has been updated after multiple searches.

[0037] The first keyword may refer to a keyword stored in a dynamic keyword library, which stores not only the first keyword but also the frequency of each first keyword. The frequency represents the frequency of a keyword appearing on a website. The higher the frequency, the more often the keyword appears on the website; the lower the frequency, the less likely the keyword appears on the website.

[0038] Different types refer to the types of keywords in the dynamic keyword library, including general type, popular type and classified type. Different types of first keywords include first general keyword, first popular keyword and first classified keyword. Different types of target first keywords include target first general keyword, target first popular keyword and target first classified keyword.

[0039] A keyword group may refer to a combination of keywords, including multiple first keywords of different types, and the keyword group must include at least one first keyword under each type. Exemplarily, the keyword group includes a first general keyword, a first hot keyword, and two first classification keywords, or the keyword group includes two first general keywords, two first hot keywords, and three first classification keywords.

[0040] In an embodiment of the present application, when selecting the target first keyword, the selection probability of the first keyword can be determined based on the word frequency of the first keyword in the dynamic keyword library. Then, based on the selection probability, the target first general keyword, the target first hot keyword and the first target classification keyword corresponding to the search keyword are randomly selected from the dynamic keyword library. For example: the dynamic keyword library includes the first keyword A, whose word frequency is 0.3; the first keyword B, whose word frequency is 0.15; the first keyword C, whose word frequency is 0.2; the first keyword D, whose word frequency is 0.15; the first keyword E, whose word frequency is 0.16, and the first keyword F, whose word frequency is 0.24. The first keyword A and the first keyword B are the first general keywords, the first keyword C and the first keyword D are the first hot keywords, and the first keyword E and the first keyword F are the first classification keywords. Then, the word frequency is used as the selection probability, and the number of first keywords under each type is randomly determined. It is assumed that the number of selected first general keywords and first popular keywords is one, the number of selected first classification keywords is two, and the first keywords randomly selected from the dynamic keyword library are first keyword A, first keyword C, first keyword E and first keyword F, respectively. Then, the first keyword A is the target first general keyword, the first keyword C is the target first popular keyword, and the first keyword E and the first keyword F are the first target classification keywords corresponding to the search keyword.

[0041] After selecting the target first keywords, the target first keywords are combined to obtain a keyword group. Taking the above example, the keyword group is obtained by combining the first keyword A, the first keyword C, the first keyword E and the first keyword F.

[0042] The keyword group is used to search for websites, newly added valid websites corresponding to the search keywords are obtained, and the second keyword is obtained from the newly added valid websites.

[0043] The second keyword may refer to a keyword extracted from a newly added valid website. The second keyword may be the same keyword as the first keyword. For the sake of distinction, the keyword extracted from the newly added valid website is referred to as the second keyword, and the keyword stored in the dynamic keyword library is referred to as the first keyword.

[0044] Specifically, multiple search engines can be used to search for websites corresponding to the keyword group to obtain at least one candidate website, wherein the multiple search engines can be search engines such as Google and Baidu. Here, the keyword group can be used as a request parameter and the URL of these search engines to string together to form a GET request, and a crawler technology is used to send a GET request to the URL to obtain the retrieved HTML document. The retrieved website addresses are listed in the retrieved HTML document, and the searched website addresses (page structure) are analyzed to obtain the structural commonality of the websites listed in the HTML document, and the websites with the same structural commonality are retained, and the remaining websites are deleted. Exemplarily, the website addresses with the same class can be retained, and the remaining website addresses can be deleted. These retained websites are candidate websites, and then, step S204 is performed, and the retained candidate websites need to be further screened to screen out the newly added valid websites that meet the preset conditions.

[0045] Step S204, determining whether each candidate website meets the preset conditions. When determining whether each candidate website meets the preset conditions, for each candidate website, it can be determined whether the candidate website is an accessible website, whether the keywords in the candidate website meet the keyword ratio conditions, and whether the candidate website is an official website.

[0046] Specifically, when determining whether a candidate website is accessible, a GET request may be sent to the candidate website to detect whether the candidate website is accessible. If the candidate website can be loaded normally after receiving the GET request, it indicates that it is accessible. If the candidate website cannot be opened after receiving the GET request or prompts "404 Not Found" that the web page is not found, it indicates that it is inaccessible.

[0047] When determining whether the keywords in a candidate website satisfy the keyword proportion condition, the keywords in the candidate website may be extracted to determine the proportion of the keywords in the candidate website to the keywords in the dynamic keyword library. This proportion is referred to as the keyword effective proportion. If the keyword effective proportion is greater than or equal to the set proportion threshold, it is determined that the keyword proportion condition is satisfied; if the keyword effective proportion is less than the set proportion threshold, it is determined that the keyword proportion condition is not satisfied. For example: there are 100 keywords in the candidate website, and there are 1000 first keywords in the dynamic keyword library. If 10 keywords in the candidate website appear in the dynamic keyword library, the ratio of 10 / 1000 is determined as the effective keyword ratio. Assume that the preset ratio threshold is 0.15. Since the effective keyword ratio of 0.1 is less than the set ratio threshold of 0.15, it is determined that the candidate website does not meet the keyword ratio condition, and therefore the candidate website cannot be used as a new valid website; if 20 keywords in the candidate website appear in the dynamic keyword library, the ratio of 20 / 1000 is determined as the effective keyword ratio. Since the effective keyword ratio of 0.2 is greater than the set ratio threshold of 0.15 at this time, it is determined that the candidate website meets the keyword ratio condition, and therefore the candidate website is used as a new valid website.

[0048] When determining whether a candidate website is an official website, it can be checked whether the candidate website has a registration number. If the candidate website has a registration number, the website is determined to be an official website; if the candidate website does not have a registration number, the website is determined not to be an official website.

[0049] If the candidate website is an accessible website, the keywords in the candidate website meet the keyword ratio condition, and the candidate website is not an official website, it is determined that the candidate website meets the preset conditions. That is, if the candidate website does not meet any of the above conditions, it is determined that the candidate website is not a newly added valid website, and step S205 is executed. If the candidate website meets the above three conditions at the same time, it is determined that the candidate website is a newly added valid website, and step S206 is executed.

[0050] Step S205, as a non-newly added valid website. At this time, if the candidate website is not a newly added valid website, the candidate website is deleted.

[0051] Step S206, as a newly added valid website. At this time, the candidate website is added to the newly added valid websites. The above judgment is performed for each candidate website, so that which candidate websites are newly added valid websites can be determined, and the number of newly added valid websites can be calculated.

[0052] For the newly added valid website, execute step S207.

[0053] Step S207, determine whether the search end condition is met. The search end condition can be a condition on the number of newly added valid websites, and / or a condition on the number of rounds. If the search end condition is a condition on the number of newly added valid websites, determine whether the number of newly added valid websites meets the set number condition. The set number condition refers to a condition that is less than a set number threshold. If the set number condition is met, for example: the set number threshold is 3, when the number of newly retrieved valid websites is less than 3, it means that it is difficult to retrieve more newly added websites in the future. Therefore, it can be determined that the public service websites that can be retrieved by the search engine are basically completed, and the entire search process can be stopped, and step S209 is executed. If the set number condition is not met, that is, the number of newly retrieved valid websites is greater than or equal to 3, it means that more newly added websites can continue to be retrieved in the future, and the next round of search process can be performed, and step S208 is executed.

[0054] Step S208, updating the dynamic keyword library. Here, the dynamic keyword library can be updated using the second keyword, so as to use the updated dynamic keyword library for the next round of search process. The second keyword includes at least one keyword.

[0055] In an embodiment of the present application, after obtaining the second keyword, the second keyword can be used to update the existing dynamic keyword library. Here, the frequency of each second keyword in the newly added valid website can be determined first, and then the dynamic keyword library can be updated according to the frequency of each second keyword. Among them, the frequency of the second keyword in the newly added valid website can refer to the frequency of the second keyword appearing in all the newly added valid websites. For example: after this round of retrieval, a total of 10 newly added valid websites are obtained, and the second keyword "movie" appears in 6 of the newly added valid websites, then the frequency of the second keyword "movie" is , the dynamic keyword library can be updated according to the word frequency 0.6.

[0056] When updating the dynamic keyword library, one method is to update the frequency of existing keywords in the current dynamic keyword library, and the other method is to add new keywords. To this end, the second keywords to be updated in the dynamic keyword library can be selected from the second keywords. For example, 12 second keywords are extracted from the newly added valid websites, and three of them are keywords in the current dynamic keyword library. These three second keywords are used as the second keywords to be updated.

[0057] When the word frequency of existing keywords in the current dynamic keyword library is updated, the word frequency of the first keyword to be updated in the current dynamic keyword library can be updated according to the word frequency of the second keyword to be updated, for example: the target statistical value of the word frequency of the first keyword to be updated and the second keyword to be updated is used as the latest word frequency of the first keyword after the update. Among them, the first keyword to be updated is the first keyword corresponding to the second keyword to be updated, and the target statistical value can be a mean value, or a preset percentile, etc. For example: the first keyword to be updated is "movie", the word frequency of the first keyword to be updated is 0.7, and the word frequency of the second keyword to be updated is 0.6. Then, when the target statistical value is the mean value, the mean of 0.6 and 0.7, 0.65, is used as the latest word frequency of the updated first keyword "movie". When the target statistical value is the 70th percentile, 70% of the numbers between 0.6 and 0.7 are used as the latest word frequency, that is, 0.67 is used as the latest word frequency of "movie".

[0058] When adding new keywords, all other second keywords can be sorted according to the frequency of words, and a preset number of other second keywords with high rankings can be selected and added to the dynamic keyword library. The other second keywords are the keywords in the second keywords except the second keywords to be updated. For example: there are ten second keywords in total, seven of which are second keywords to be updated, and three are other second keywords. These three second keywords are sorted in order of frequency from high to low, and one or more other second keywords with high rankings are added to the dynamic keyword library according to their frequency in the newly added valid websites. Taking the above example, if the second keyword "movie" does not appear in the current dynamic keyword library, 0.6 is used as the frequency of the keyword in the dynamic keyword library. Among them, those skilled in the art can select the specific value of the preset number according to the actual situation, and this application is not limited here.

[0059] After the dynamic keyword library is updated, in order to ensure that the number of first keywords in the dynamic keyword library does not expand indefinitely, it is also necessary to determine whether the number of first keywords in the updated dynamic keyword library meets the keyword quantity condition after adding new keywords. Among them, the keyword quantity condition refers to the condition that the number of first keywords in the updated dynamic keyword library is less than or equal to the set threshold. Those skilled in the art can select the specific value of the set threshold according to actual conditions, and this application does not limit it. If the number of first keywords in the dynamic keyword library is greater than the set threshold after a preset number of keywords are newly added, the word frequencies of the first keywords in the updated dynamic keyword library are sorted, and at least one first keyword with the smallest word frequency is deleted from the dynamic keyword library, so that the number of first keywords in the dynamic keyword library meets the keyword quantity condition. For example: if three new keywords are added, so that the number of first keywords in the dynamic keyword library exceeds two, the two first keywords with the smallest word frequency are deleted from the dynamic keyword library.

[0060] In this way, the current search process ends and the next search process begins. In the next search process, the search keywords can be used to reselect new different types of target first keywords from the updated dynamic keyword library to construct the next round of keyword groups based on the new different types of target first keywords.

[0061] If the search end condition is met, step S209 is executed to end the entire search process. The search end condition may refer to the number of newly added valid websites in a single round of search meeting a set number condition, or the number of search rounds reaching a set round limit.

[0062] Step S209, obtaining the final search result. When the search end condition is met, all newly added valid websites retrieved in all rounds are taken as the final search result, and at this point, the entire search process is completely finished.

[0063] At the end of the entire search process, all newly retrieved valid websites are saved, and these newly retrieved valid websites are the final search results. At the same time, the dynamic keyword library is also saved as the updated dynamic keyword library at the end of the last round of search process.

[0064] This application has the following beneficial effects: 1. This application can dynamically build a keyword library, analyze text on a large number of specific websites, count the frequency of occurrence of each keyword, and extract the most frequently appearing keyword set on the target website. This process abandons the preset limitations of the static keyword library, ensures that the keyword library is closely aligned with the actual website content, keeps pace with the times, and effectively solves the problem of the existing technology that the keyword library content is solidified, resulting in delayed and inaccurate search results, and improves the consistency and freshness of the search results with the current public service website content.

[0065] Second, this application introduces a keyword search strategy. It can flexibly combine the words in the keyword library to form a multi-element compound query. It helps to capture specific professional expressions and terms in public service websites, improve the accuracy of searches, and make search results more targeted. Keyword groups can not only reveal the semantic associations and contextual relationships between words, but also reduce the interference of irrelevant information, significantly improving search efficiency.

[0066] 3. This application also uses a random combination method to construct keyword groups to enhance the comprehensiveness and exploratory nature of the search. By introducing a random combination strategy, this application can break the routine, explore potential keyword combinations, further broaden the search coverage, and increase the possibility of discovering unknown or rare public service information, thereby avoiding the information blind spots that may be caused by a single keyword search.

[0067] Based on the same inventive concept, the embodiment of the present application also provides a website retrieval device based on a dynamic keyword library corresponding to the website retrieval method based on a dynamic keyword library. Since the principle of solving the problem by the device in the embodiment of the present application is similar to the above-mentioned website retrieval method based on a dynamic keyword library in the embodiment of the present application, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0068] See also Figure 3 , Figure 3 The schematic diagram of the structure of a website search device based on a dynamic keyword library provided in an embodiment of the present application. Figure 3 As shown in , the website search device 300 based on the dynamic keyword library includes: The keyword group construction module 301 is used to select different types of target first keywords from the dynamic keyword library using the search keywords during the current search process, and to construct a keyword group based on the target first keywords; A newly added keyword acquisition module 302 uses the keyword group to perform website search, obtains newly added valid websites corresponding to the search keyword, and obtains a second keyword from the newly added valid websites; The keyword library updating module 303 is used to update the dynamic keyword library using the second keyword, so as to use the updated dynamic keyword library to perform the next round of search process; The search result determination module 304 is used to take the newly added valid websites retrieved in all rounds when the search end condition is met as the final search result.

[0069] See also Figure 4 , Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. Figure 4As shown in , the electronic device 400 includes a processor 410 , a memory 420 and a bus 430 .

[0070] The memory 420 stores machine-readable instructions executable by the processor 410. When the electronic device 400 is running, the processor 410 communicates with the memory 420 via the bus 430. When the machine-readable instructions are executed by the processor 410, the above-mentioned Figure 1 The steps of the website search method based on the dynamic keyword library in the method embodiment shown in the figure can be specifically implemented by referring to the method embodiment, which will not be described in detail here.

[0071] The present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the computer program can execute the above-mentioned Figure 1 The steps of the website search method based on the dynamic keyword library in the method embodiment shown in the figure can be specifically implemented by referring to the method embodiment, which will not be described in detail here.

[0072] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0073] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of devices or units can be electrical, mechanical or other forms.

[0074] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0075] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0076] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk, and other media that can store program codes.

[0077] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present application, which are used to illustrate the technical solutions of the present application, rather than to limit them. The protection scope of the present application is not limited thereto. Although the present application is described in detail with reference to the above-mentioned embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-mentioned embodiments within the technical scope disclosed in the present application, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application, and should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A website search method based on a dynamic keyword library, characterized in that: include: In this round of search, different types of target first keywords are selected from the dynamic keyword library using the search keywords, and keyword groups are constructed based on the target first keywords; Using the keyword group to search for websites, obtaining newly added valid websites corresponding to the search keyword, and obtaining a second keyword from the newly added valid websites; Using the second keyword to update the dynamic keyword library, so as to use the updated dynamic keyword library to perform the next round of search process; When the search end conditions are met, all newly added valid websites retrieved in all rounds will be regarded as the final search results.

2. The method according to claim 1, characterized in that The second keyword includes at least one, and the updating of the dynamic keyword library by using the second keyword includes: Determine the frequency of each second keyword in the newly added valid website; The dynamic keyword library is updated according to the word frequency of each second keyword.

3. The method according to claim 2, characterized in that The updating of the dynamic keyword library according to the word frequency of each second keyword includes: Selecting from the second keywords the second keywords to be updated that are already in the dynamic keyword library; According to the word frequency of the second keyword to be updated, the word frequency of the first keyword to be updated is updated, and the first keyword to be updated is the first keyword corresponding to the second keyword to be updated; All other second keywords are sorted according to their frequency, and a preset number of other second keywords with high rankings are selected to be added to the dynamic keyword library, wherein the other second keywords are keywords in the second keywords except the second keyword to be updated.

4. The method according to claim 1, characterized in that: The method further comprises: Determining whether the number of the first keyword in the updated dynamic keyword library meets the keyword quantity condition; If the keyword quantity condition is not met, at least one first keyword with the smallest word frequency is deleted from the dynamic keyword library, so that the number of first keywords in the dynamic keyword library meets the keyword quantity condition.

5. The method according to claim 1, characterized in that The different types of target first keywords include target first general keywords, target first hot keywords and target first classification keywords. The method of selecting different types of target first keywords from the dynamic keyword library using search keywords includes: Determining the selection probability of the first keyword according to the word frequency of the first keyword in the dynamic keyword library; According to the selection probability, a target first general keyword, a target first hot keyword, and a first target classification keyword corresponding to the search keyword are randomly selected from the dynamic keyword library.

6. The method according to claim 1, characterized in that The step of using the keyword group to search for websites and obtain newly added valid websites corresponding to the search keywords includes: Using multiple search engines to search for websites corresponding to the keyword group to obtain at least one candidate website; Determine whether each candidate website meets the preset conditions, and take the candidate websites that meet the preset conditions as newly added valid websites.

7. The method according to claim 6, characterized in that Determining whether each candidate website meets a preset condition includes: For each candidate website, determine whether the candidate website is an accessible website, whether the keywords in the candidate website meet the keyword ratio condition, and whether the candidate website is an official website; If the candidate website is an accessible website, the keywords in the candidate website meet the keyword ratio condition, and the candidate website is not an official website, it is determined that the candidate website meets the preset condition.

8. The method according to claim 1, characterized in that Before the next round of search process using the updated dynamic keyword library, it also includes: Determine whether the number of the newly added valid websites meets a set number condition; If the set quantity condition is not met, the next round of search process will be carried out; If the set quantity conditions are met, the entire retrieval process is completed.

9. The method according to claim 1, characterized in that: Before selecting different types of target keywords from the dynamic keyword library using the search keywords, the method further includes: A preliminary search is performed according to the initial search keywords under each set website type, and an initial dynamic keyword library is constructed to perform website search based on the initial dynamic keyword library.

10. A website search device based on a dynamic keyword library, characterized in that: include: A keyword group construction module is used to select different types of target first keywords from the dynamic keyword library using the search keywords during this round of search, and to construct a keyword group based on the target first keywords; A newly added keyword acquisition module uses the keyword group to perform website search, obtains newly added valid websites corresponding to the search keyword, and obtains a second keyword from the newly added valid website; A keyword library updating module, used to update the dynamic keyword library using the second keyword, so as to use the updated dynamic keyword library for the next round of search process; The search result determination module is used to take the newly added valid websites retrieved in all rounds as the final search results when the search end condition is met.

Citation Information

Patent Citations

  • Website statistical information processing method and website statistical information processing system

    CN103064852A

  • Keyword determination method and device based on search platform, equipment and storage medium

    CN112070550A

  • Generating and implementing keyword clusters

    US20210191995A1

Cited By

  • News keyword library construction method and device and storage medium

    CN120653848A