Webpage searching method and device, and storage medium

Abstract

The application discloses a webpage search method and device and a storage medium. The method comprises the following steps: obtaining a semantic vector of a query statement; determining the similarity between the query statement and each webpage according to the semantic vector of the query statement and the characteristic information of each webpage, wherein the characteristic information of each webpage is used for representing the first semantic aggregation information and at least one second semantic aggregation information of each webpage, the first semantic aggregation information is obtained by performing semantic aggregation on the semantic information of multiple webpages, the at least one second semantic aggregation information is obtained by performing semantic aggregation on the semantic information of the webpages with the same theme as each webpage in the multiple webpages, and the weight of each webpage is greater than the weight of other webpages in the semantic aggregation process; and obtaining the query result of the query statement according to the similarity between the query statement and each webpage. The embodiment of the application is beneficial to improving the webpage search precision.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence technology, specifically to a web search method, apparatus, and storage medium. Background Technology

[0002] Search is one of the key technologies in the internet field, directly impacting the efficiency with which users obtain information. On the other hand, search is also a crucial application in the ecosystems of internet giants like Google and Baidu. For example, Google's total revenue in 2019 was $160.743 billion, of which advertising revenue from Google Search reached $98.115 billion, accounting for a staggering 61.0%.

[0003] Web search mainly involves several steps: analyzing web pages in a web page database and indexing them into a specific space; analyzing user input online and projecting it into the same space as the web page database; matching user input with web pages within that space; and sorting the results by matching degree and returning the search results to the user.

[0004] Traditional web search technologies rely on keyword analysis and similarity calculations based on candidate web pages and user-inputted text characters. This approach is slow and has low accuracy. To continuously improve the web search experience and product competitiveness, web search technology is constantly evolving and improving, gradually shifting from symbolic search based on text matching to deep semantic search based on semantic matching. In deep semantic search, deep representation models (e.g., the BERT model) are used to represent candidate web pages and user input. These models represent explicit information, such as text characters, as implicit semantic vectors, and the matching degree between these semantic vectors is calculated in the semantic space to complete the search ranking process.

[0005] Although deep semantic search can solve some search problems in complex semantic scenarios, it determines the semantic vector of each webpage in isolation, which affects the calculation of matching degree and results in low search quality. Summary of the Invention

[0006] This application provides a webpage search method, apparatus, and storage medium, which improves the accuracy of webpage search by aggregating the semantic information of webpages with the same topic to construct the feature information of each webpage.

[0007] In a first aspect, embodiments of this application provide a webpage search method, comprising: obtaining a semantic vector of a query statement; determining the similarity between the query statement and each webpage based on the semantic vector of the query statement and feature information of each webpage among multiple webpages, wherein the feature information of each webpage is used to characterize first semantic aggregation information and at least one second semantic aggregation information of each webpage, wherein the first semantic aggregation information is obtained by semantic aggregation of the semantic information of multiple webpages, and at least one second semantic aggregation information is obtained by semantic aggregation of the semantic information of webpages among multiple webpages that have the same topic as each webpage, wherein the weight of each webpage is greater than the weight of other webpages participating in the semantic aggregation process; and obtaining a query result of the query statement based on the similarity between the query statement and each webpage, wherein the query result is at least one of the multiple webpages.

[0008] It should be noted that "in the process of semantic aggregation for each webpage" refers to the process of obtaining the first semantic aggregation information and at least one second semantic aggregation information for each webpage. That is, the process of semantically aggregating the semantic information of the multiple webpages to obtain the first semantic aggregation information of "each webpage", and the process of semantically aggregating the semantic information of webpages with the same theme as "each webpage" to obtain at least one second semantic aggregation information. In these two sub-processes, the weight of the webpage whose first semantic aggregation information and at least one second semantic aggregation information are calculated is greater than the weight of other webpages in the multiple webpages.

[0009] For example, regarding the first webpage, which can be any one of multiple webpages, in the process of semantically aggregating the semantic information of the multiple webpages to obtain the first semantic aggregation information of the first webpage, the semantic information of the multiple webpages can be semantically aggregated according to the weight of each webpage in the multiple webpages. It should be understood that in the process of semantic aggregation, the weight of the first webpage is greater than the weight of other webpages in the multiple webpages besides the first webpage. For example, the weight of the first webpage is 1, the weight of the webpages that have a direct link relationship with the first webpage is r (r is less than 1), and the weight of the webpages that do not have a link relationship with the first webpage (including direct links and indirect links) is 0. Similarly, in the process of obtaining the second semantic aggregation information of the first webpage, the weight of the first webpage is 1, the weight of other webpages with the same topic is less than 1, and then, based on the weight of the webpages with the same topic as the first webpage (including the first webpage), the semantic information of the webpages with the same topic is semantically aggregated to obtain the second semantic aggregation information of the first webpage.

[0010] As can be seen, in the embodiments of this application, the feature information of each webpage includes first semantic aggregation information obtained by semantic aggregation of multiple webpages, and second semantic aggregation information obtained by semantic aggregation of webpages with the same topic. Therefore, the feature information of each webpage is not composed of isolated semantic information of each webpage, but includes semantic information of webpages related to that webpage, thereby making the feature information of each webpage richer and more accurate, improving the matching accuracy between query statements and webpages, and thus improving the search accuracy of webpages.

[0011] In some possible implementations, for a first webpage, wherein the first webpage is any one of a plurality of webpages; the first semantic aggregation information of the first webpage is represented by a second vector, which is obtained by semantically aggregating the first vectors of each of the plurality of webpages, and the first vector of each webpage is used to represent the semantic information of each webpage; at least one second semantic aggregation information of the first webpage is represented by at least one third vector; each of the at least one third vector corresponds to a topic included in the first webpage, and the topics corresponding to each of the at least one third vector are different; wherein, in the at least one third vector, each third vector is obtained by semantically aggregating the first vector of the first webpage and the first vector of the second webpage, and the second webpage is a webpage among the plurality of webpages that contains the topic corresponding to each third vector.

[0012] As can be seen from the implementation, semantic aggregation is performed on the first vectors of multiple web pages to obtain the first vector of the first web page, i.e., the first semantic aggregation information; then, semantic aggregation is performed on the first vectors of multiple web pages that contain web pages with the same topic as the first web page to obtain at least one second vector of the first web page, i.e., at least one second semantic aggregation information. Therefore, when obtaining the second semantic aggregation information of each web page, only the first vectors of web pages with the same topic are aggregated, thereby avoiding the introduction of noise during aggregation, making the accuracy of the obtained second semantic aggregation information of each web page relatively high, and thus improving the accuracy of web page search.

[0013] In some possible implementations, at least one third vector of the first webpage is also related to a topology graph that indicates the relationships between multiple webpages.

[0014] As can be seen, in this embodiment, by constructing a topology map, web pages with the same theme as each web page can be quickly found based on the constructed topology map, and at least one second semantic aggregation information of each web page can be quickly constructed, thereby improving the construction efficiency of feature information of each web page.

[0015] In some possible implementations, the topology graph includes at least one sub-topology graph, that is, the sub-topology graph is formed by extracting web pages containing the topic of the first web page from the topology graph, and at least one sub-topology graph is obtained. Each of the at least one third vector corresponds to one of the at least one sub-topology graphs, and the sub-topology graphs corresponding to each of the at least one third vector are different. The web pages in the sub-topology graph corresponding to each third vector include the first web page and the second web page. Each third vector is obtained by semantically aggregating the first web page and the second web page in the sub-topology graph corresponding to each third vector.

[0016] As can be seen, in this embodiment, at least one sub-topology graph containing the topic of the first webpage is obtained from the topology graph. Then, semantic aggregation is performed on the webpages in each sub-topology graph to obtain at least one first semantic aggregation information of the first webpage. Sub-topology graphs can be directly extracted from the topology graph without reconstructing them, thus allowing for the rapid acquisition of at least one second semantic aggregation information of the first webpage.

[0017] In some possible implementations, each sub-topology graph in at least one sub-topology graph corresponds to a web page group in at least one web page group, and the web page groups corresponding to each sub-topology graph in at least one sub-topology graph are different. Each web page group in at least one web page group consists of multiple web pages containing web pages with a topic corresponding to each web page group. Each sub-topology graph in at least one sub-topology graph is extracted from the topology graph by extracting the web pages in the web page group corresponding to each sub-topology graph.

[0018] As can be seen, by first grouping multiple web pages by topic, we can obtain the second vector of each web page within each group. Finally, based on at least one web page group to which the first web page belongs, we can quickly obtain at least one second vector of the first web page. This avoids repeatedly constructing sub-topology graphs for multiple web pages, thus improving the efficiency of constructing web page feature information. For example, for web pages A and B, if we start from the topics contained in each web page to obtain the second vector of web page A under that topic, we need to first construct a sub-topology graph of web page A under that topic, and then perform semantic aggregation on the web pages under that sub-topology graph to obtain the second vector of web page A under that topic. When constructing the second vector of web page B under that topic, we need to construct the same sub-topology graph again and perform semantic aggregation on the web pages under that sub-topology graph again to obtain the second vector of web page B under that topic. Therefore, by first grouping multiple web pages by topic, we can directly obtain the sub-topology graph corresponding to the web page group to which that topic belongs, and simultaneously obtain the second vectors of web page A and web page B under that sub-topology graph, thus improving the efficiency of constructing web page feature information.

[0019] In some possible implementations, the feature information of each webpage also includes a first vector for each webpage, which indicates the semantic information of each webpage.

[0020] As can be seen, in this embodiment, the feature information of each webpage also includes the first vector of each webpage, which contains the semantic information of each webpage itself, thereby making the feature information of each webpage richer and more accurate, and thus improving the accuracy of subsequent webpage searches.

[0021] In some possible implementations, the feature information of each webpage is represented by a matrix. Before determining the similarity between the query statement and each webpage based on the semantic vector of the query statement and the feature information of each webpage among multiple webpages, the method further includes: transforming the matrix corresponding to each webpage into a target vector according to the weights of each vector in the matrix, the target vector indicating the feature information of each webpage; determining the similarity between the query statement and each webpage based on the semantic vector of the query statement and the feature information of each webpage among multiple webpages includes: calculating the similarity between the semantic vector of the query statement and the target vector, the similarity indicating the relevance between the query statement and the webpage corresponding to the target vector.

[0022] As can be seen, in this embodiment, the multiple vectors of each webpage are represented in the form of a matrix, which facilitates the subsequent conversion of the multiple vectors of each webpage into a target vector, thus creating conditions for calculating the matching degree between the query statement and the webpage.

[0023] In some possible implementations, before transforming the matrix corresponding to each webpage into a target vector based on the weights of the vectors in the matrix, the method further includes: determining the similarity between the semantic vector of the query statement and each vector in the matrix; and determining the weights of each vector in the matrix based on the similarity between the semantic vector of the query statement and each vector in the matrix.

[0024] As can be seen, in this embodiment, by determining the weight of each vector among the multiple vectors of each webpage through the self-attention mechanism, the vectors that match the semantic vector of the query statement can be retained, making the target vector of each webpage more closely match the query statement and improving the accuracy of webpage search.

[0025] Secondly, embodiments of this application provide a webpage search device, the beneficial effects of which are described in the first aspect and will not be repeated here. The webpage search device has the function of implementing the behavior in the method example of the first aspect. The function can be implemented by hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-mentioned function. In one possible design, the webpage search device includes an acquisition unit and a processing unit; the acquisition unit is used to acquire a query statement; the processing unit is used to acquire the semantic vector of the query statement; based on the semantic vector of the query statement and the feature information of each webpage in multiple webpages, the similarity between the query statement and each webpage is determined, the feature information of each webpage is used to characterize the first semantic aggregation information and at least one second semantic aggregation information of each webpage, wherein the first semantic aggregation information is obtained by semantic aggregation of the semantic information of multiple webpages, and at least one second semantic aggregation information is obtained by semantic aggregation of the semantic information of webpages with the same topic as each webpage in multiple webpages, and the weight of each webpage is greater than the weight of other webpages participating in the semantic aggregation process during the semantic aggregation process; based on the similarity between the query statement and each webpage, the query result of the query statement is obtained, and the query result is at least one of multiple webpages.

[0026] In some possible implementations, for a first webpage, wherein the first webpage is any one of a plurality of webpages; the first semantic aggregation information of the first webpage is represented by a second vector, which is obtained by semantically aggregating the first vectors of each of the plurality of webpages, and the first vector of each webpage is used to represent the semantic information of each webpage; at least one second semantic aggregation information of the first webpage is represented by at least one third vector; each of the at least one third vector corresponds to a topic included in the first webpage, and the topics corresponding to each of the at least one third vector are different; wherein, in the at least one third vector, each third vector is obtained by semantically aggregating the first vector of the first webpage and the first vector of the second webpage, and the second webpage is a webpage among the plurality of webpages that contains the topic corresponding to each third vector.

[0027] In some possible implementations, at least one third vector of the first webpage is also related to a topology graph that indicates the relationships between multiple webpages.

[0028] In some possible implementations, the topology graph includes at least one sub-topology graph, each of the at least one third vector corresponds to a sub-topology graph in the at least one sub-topology graph, and the sub-topology graphs corresponding to each of the at least one third vector are different; the web pages in the sub-topology graph corresponding to each third vector include a first web page and a second web page; each third vector is obtained by semantically aggregating the first web page and the second web page in the sub-topology graph corresponding to each third vector.

[0029] In some possible implementations, each sub-topology graph in at least one sub-topology graph corresponds to a web page group in at least one web page group, and the web page groups corresponding to each sub-topology graph in at least one sub-topology graph are different. Each web page group in at least one web page group consists of multiple web pages containing web pages with a topic corresponding to each web page group. Each sub-topology graph in at least one sub-topology graph is extracted from the topology graph by extracting the web pages in the web page group corresponding to each sub-topology graph.

[0030] In some possible implementations, the feature information of each webpage also includes a first vector for each webpage, which indicates the semantic information of each webpage.

[0031] In some possible implementations, the feature information of each webpage is represented by a matrix. Before the processing unit determines the similarity between the query statement and each webpage based on the semantic vector of the query statement and the feature information of each webpage among multiple webpages, the processing unit is further configured to convert the matrix corresponding to each webpage into a target vector according to the weights of each vector in the matrix. The target vector indicates the feature information of each webpage. In terms of determining the similarity between the query statement and each webpage based on the semantic vector of the query statement and the feature information of each webpage among multiple webpages, the processing unit is specifically configured to: calculate the similarity between the semantic vector of the query statement and the target vector. The similarity indicates the relevance between the query statement and the webpage corresponding to the target vector.

[0032] In some possible implementations, before the processing unit transforms the matrix corresponding to each webpage into a target vector based on the weights of each vector in the matrix, the processing unit is further configured to: determine the similarity between the semantic vector of the query statement and each vector in the matrix; and determine the weights of each vector in the matrix based on the similarity between the semantic vector of the query statement and each vector in the matrix.

[0033] Thirdly, embodiments of this application provide a web search device, including: a memory for storing a program; and a processor for executing the program stored in the memory; when the program stored in the memory is executed, the processor is used to implement the first aspect described above.

[0034] Fourthly, embodiments of this application provide a computer-readable medium storing program code for execution by a device, the program code including methods for implementing the methods in the first aspect described above.

[0035] Fifthly, embodiments of this application provide a computer program product containing instructions that, when run on a computer, cause the computer to implement the method described in the first aspect.

[0036] In a sixth aspect, embodiments of this application provide a chip that includes a processor and a data interface. The processor reads instructions stored in a memory through the data interface to implement the method in the first aspect described above.

[0037] Alternatively, as one implementation, the chip may also include a memory storing instructions, and a processor is used to execute the instructions stored in the memory. When the instructions are executed, the processor is used to implement the method in the first aspect described above. Attached Figure Description

[0038] Figure 1 An architecture diagram of a web search system provided in this application embodiment;

[0039] Figure 2 This application provides a schematic diagram of a process for constructing feature information of a webpage.

[0040] Figure 3 A schematic diagram of a topology graph provided in an embodiment of this application;

[0041] Figure 4 A schematic diagram of a sub-topology graph provided in an embodiment of this application;

[0042] Figure 5 A schematic diagram illustrating how to obtain the first vector of Albert Einstein's Wikipedia entry, provided as an embodiment of this application;

[0043] Figure 6 A schematic diagram illustrating the construction of a topological graph containing the Albert Einstein Wikipedia entry, provided as an embodiment of this application;

[0044] Figure 7 A schematic diagram of a subtopology graph containing Albert Einstein's Wikipedia entry, provided for an embodiment of this application;

[0045] Figure 8 A flowchart illustrating a webpage search method provided in an embodiment of this application;

[0046] Figure 9This is a schematic diagram of the structure of a web search device provided in an embodiment of this application;

[0047] Figure 10 This is a schematic diagram of another web search device provided in an embodiment of this application. Detailed Implementation

[0048] First, it should be noted that the webpages in this application can be understood as webpage documents that can establish relationships. For example, webpage documents can be webpages seen during information searches, or they can be documents, papers, etc., with citation relationships. This application mainly uses webpages seen during information searches as an example for illustration.

[0049] See Figure 1 , Figure 1 This is an architecture diagram of a web search system provided in an embodiment of this application. The web search system 10 includes an offline module 101 and an online module 102;

[0050] Offline module 101 includes one or more of the following functions:

[0051] Data cleaning, initial representation generation, full graph construction, subgraph construction, variable-length representation generation, and vector index construction;

[0052] Data cleaning refers to the offline module 101 cleaning the web pages and filtering out multiple high-quality web pages. These multiple web pages can be some or all of the web pages in the web page library, as well as cleaning the text in the web pages and filtering out high-quality text from the web pages.

[0053] Initial representation generation refers to the offline module 101 performing vectorization representation on each of the multiple web pages to obtain the first vector of each web page, where the first vector of each web page is used to represent the semantic information of each web page;

[0054] Full graph construction refers to the offline module 101 constructing a topology graph between multiple web pages based on the relationships between them;

[0055] Subgraph construction refers to the offline module 101 selecting web pages with the same theme from the topology graph of multiple web pages based on the theme of each web page, and constructing the relationship between these web pages with the same theme based on the relationship between them.

[0056] Variable-length representation generation refers to the offline module 101 calculating the semantic aggregation information of each webpage under the topology graph, and the semantic aggregation information of each webpage under the sub-topology graph corresponding to the topic contained in the webpage, and stacking the two kinds of semantic aggregation information to generate the variable-length representation of each webpage, thereby obtaining the feature information of each webpage.

[0057] Vector index construction refers to the offline module 101 building an index for the variable-length representation of each webpage, which facilitates efficient querying of the variable-length representation of each webpage.

[0058] Online module 102 mainly includes the following functions: preprocessing, query statement generation, variable-length representation fusion, similarity calculation, and webpage sorting.

[0059] Preprocessing refers to the online module 102 preprocessing the query statement (Query) input by the user to obtain a high-quality query statement. For example, preprocessing can be to remove special characters from the query statement. Special characters can be garbled characters or characters without semantic meaning, such as the characters "@", "#", "*", etc.

[0060] Query statement representation generation refers to the online module 102 vectorizing the preprocessed query statement to obtain the semantic vector of the preprocessed query statement;

[0061] Variable-length representation fusion refers to the process by which online module 102 fuses the variable-length representations of each webpage based on the semantic vector of the query statement to obtain the target vector of each webpage.

[0062] Similarity calculation refers to the online module 102 calculating the similarity between the semantic vector of the query statement and the target vector of each webpage. Among them, the similarity calculation methods for vectors include Euclidean distance, cosine similarity, etc.

[0063] Webpage sorting refers to the online module 102 sorting multiple webpages by their similarity to the query statement. For example, they can be sorted in descending order of similarity to facilitate the subsequent output of the webpage with the highest similarity.

[0064] The following detailed explanation, with reference to the accompanying diagram, describes the process of offline processing of web pages to obtain their feature information.

[0065] See Figure 2 , Figure 2 This is a schematic diagram illustrating a process for constructing feature information of a webpage, provided as an embodiment of this application. The method is applied to the aforementioned webpage search system. The method includes the following steps:

[0066] 201: Based on the relationships between web pages in multiple web pages, construct a topology graph corresponding to the multiple web pages.

[0067] The multiple web pages can be all the web pages in the web page library, or they can be a portion of the web page library; this application does not limit this.

[0068] Optionally, the relationship between web pages can be determined through hyperlinks between them. For example, if web page A contains a hyperlink to web page B, then there is a relationship between web page A and web page B.

[0069] Optionally, the relationship between web pages can also be determined through the text descriptions in the web pages. For example, if web page A contains text descriptions about web page B, then it is determined that there is a relationship between web page A and web page B.

[0070] Optionally, the relationship between web pages can also be determined based on their parent web pages. For example, if web page A and web page B both have web page C as their parent web page, then there is a relationship between web page A and web page B.

[0071] Therefore, this application does not limit the types of relationships between two web pages.

[0072] Optionally, each of the multiple web pages can be used as a node. If there is a relationship between two web pages, an edge can be constructed for the two nodes corresponding to the two web pages. If there is no relationship between two web pages, no edge can be constructed for the two nodes corresponding to the two web pages, thus obtaining a topology graph corresponding to the multiple web pages.

[0073] For example, multiple web pages include web page A, web page B, web page C, and web page D. Web page A and web page B are related, web page B and web page C are related, while web page D is not related to any web page. Therefore, a list of web pages can be constructed based on these relationships, such as... Figure 3 The topology diagram shown.

[0074] For example, Figure 3 The edges between related nodes in the illustrated topology graph can be directed or undirected; that is, the topology graph can be either directed or undirected. Furthermore, the sub-topologies mentioned later can be either directed or undirected, and this application does not impose any limitations on either. This application uses an undirected graph as an example for illustration.

[0075] 202: Based on the topology map and at least one topic included in each webpage, determine the feature information of each webpage.

[0076] For example, a topic recognition model is invoked to obtain at least one topic included in each webpage. The topic recognition model can be a Latent Dirichlet Allocation (LDA) model. For example, the at least one topic included in each webpage could be "politics," "economics," "education," "healthcare," etc. Understandably, identifying at least one topic for each webpage is the process of tagging each webpage, i.e., tagging each webpage with labels such as "politics," "economics," "education," and "healthcare." Therefore, at least one topic for each webpage can be indicated by the at least one tag attached to each webpage.

[0077] For example, the feature information of each webpage includes first semantic aggregation information of each webpage and at least one second semantic aggregation information corresponding to each webpage.

[0078] The following uses the first webpage as an example to illustrate the process of obtaining the first semantic aggregation information and at least one second semantic aggregation information of the first webpage. The process of obtaining the first semantic aggregation information and at least one second semantic aggregation information of other webpages is similar to that of the first webpage and will not be described again. The first webpage can be any one of multiple webpages.

[0079] Optionally, the first semantic aggregation information of the first webpage is represented by a second vector.

[0080] Specifically, semantic aggregation is performed on the first vectors of multiple web pages in the topology graph to obtain the second vector of each web page, that is, the second vector of the first web page. The first vector of each web page is used to represent the semantic information of each web page, and the first vector of each web page can be obtained by extracting semantic information from each web page through a trained semantic information extraction model. For example, the semantic information extraction model can be the BERT model.

[0081] For example, the semantic aggregation process for the first vector of multiple web pages involves semantically aggregating the first vectors of multiple web pages based on their weights in the topology graph. When obtaining the second vector of the first web page, the weight of the first web page is 1, and the weights of other web pages are determined based on their connection to the first web page and their distance from the first web page in the topology graph. Specifically, when a web page has no connection to the first web page, its weight is determined to be 0; when a web page has a connection to the first web page, its weight is determined based on its distance from the first web page in the topology graph. Web pages connected to the first web page include those with direct connections and those with indirect connections. For example, the first web page is... Figure 3 If webpage A is a page directly connected to webpage A, then webpage B is a page directly connected to webpage A, and webpage C is a page indirectly connected to webpage A. The distance between two webpages in the topology graph can be understood as the number of webpages separating them. For example, the distance between webpage C and webpage A is 1, meaning they are separated by one webpage B; the distance between webpage A and webpage B is 0, meaning they are not separated by any webpages.

[0082] Therefore, the second vector of the first webpage can be represented by formula (1):

[0083]

[0084] Where y is the second vector of the first webpage, α i Let e ​​be the weight of the i-th webpage among multiple webpages. iLet be the first vector of the i-th webpage, and n be the number of webpages.

[0085] For example, when the i-th webpage is the first webpage, then α i α is 1; when the i-th webpage is a webpage that is not related to the first webpage, α i α is 0; when the i-th webpage is a webpage related to the first webpage, α i =γ m+1 , where γ is a preset parameter, less than 1, and m is the number of pages between the i-th page and the first page.

[0086] It should be noted that the subsequent process of semantic aggregation of the first vector of the webpage in the sub-topology graph is similar to the process of semantic aggregation of the first vector of multiple webpages in the topology graph described above, and will not be described again.

[0087] Optionally, at least one second semantic aggregation information of the first webpage is represented by at least one third vector. This at least one third vector can be determined based on the aforementioned topology graph and the first vector of each of the multiple webpages. Each of the at least one third vector corresponds to a topic included in the first webpage, and the topics corresponding to each third vector are different; that is, at least one third vector corresponds one-to-one with the first topics included in the first webpage. Each third vector is obtained by semantically aggregating the first vector of the first webpage and the first vector of the second webpage, where the second webpage is a webpage containing the topic corresponding to each third vector among the multiple webpages.

[0088] In one embodiment of this application, the topics of each webpage in the topology graph are traversed to determine a second webpage containing topic E, where topic E is any one of the at least one topics contained in the first webpage. The second webpage containing topic E and the first webpage are extracted from the topology graph to obtain a sub-topology graph corresponding to topic E. Therefore, each webpage in the sub-topology graph includes the first webpage and a second webpage with the same topic as the first webpage. Similar operations to topic E are performed on at least one topic of the first webpage to obtain at least one sub-topology graph corresponding to the first webpage. Finally, semantic aggregation is performed on the first vector of each webpage in each sub-topology graph, that is, semantic aggregation is performed on the first vector of the first webpage and the first vector of the second webpage in each sub-topology graph to obtain a third vector of the first webpage in each sub-topology graph. This yields at least one third vector of the first webpage in at least one sub-topology graph, where each of the at least one third vectors corresponds to a sub-topology graph in at least one sub-topology graph, and the sub-topology graphs corresponding to each of the at least one third vectors are different, i.e., at least one third vector corresponds to at least one sub-topology graph. Figure 1 One-to-one correspondence.

[0089] In another embodiment of this application, all topics of multiple web pages are merged and deduplicated to obtain a topic set. Then, web pages containing a first topic are grouped into the same group to obtain multiple web page groups. The first topic is any one of the topics in the topic set. That is, similar to an inverted index, each topic in the topic set is used as a feature to group multiple web pages. For example, multiple web pages include web page 1 and web page 2, where web page 1 includes topics 1, 2, and 3, and web page 2 includes topics 1 and 2. Therefore, merging and deduplicating the topics yields topic sets 1, 2, and 3. The web page group containing topics 1 is web page 1 and web page 2, the web page group containing topics 2 is web page 1 and web page 2, and the web page group containing topics 3 is web page 1.

[0090] Then, based on at least one topic of the first webpage, at least one webpage group corresponding to the first webpage is determined from multiple webpage groups; then, the webpages contained in each webpage group in the at least one webpage group are extracted from the topology graph to obtain a sub-topology graph corresponding to each webpage group, thereby obtaining at least one sub-topology graph corresponding to at least one webpage group. Each sub-topology graph in the at least one sub-topology graph corresponds to one webpage group in the at least one webpage group, and the webpage groups corresponding to each sub-topology graph in the at least one sub-topology graph are different, that is, at least one webpage group and at least one sub-topology graph... Figure 1 One-to-one correspondence; finally, semantic aggregation is performed on the first vector of the webpage in each sub-topology graph, that is, semantic aggregation is performed on the vector of the first webpage and the second vector of the second webpage in each sub-topology graph to obtain the third vector of the first webpage in each sub-topology graph, and then at least one third vector of the first webpage in at least one sub-topology graph can be obtained.

[0091] Finally, the second vector of the first webpage is combined with at least one corresponding third vector of the first webpage to obtain the feature information of the first webpage. For example, the second vector and at least one third vector of the first webpage can be combined in matrix form, and the combined matrix can be used as the feature information of each webpage.

[0092] It should be noted that when extracting web pages from the topology graph to form sub-topology graphs, the relationships between the web pages in the topology graph are not changed.

[0093] For example, if a webpage group contains webpages A, B, and D, then webpages A, B, and D can be extracted from the topology graph to obtain a sub-topology graph corresponding to that webpage group, i.e., as shown below. Figure 4 The sub-topology diagram shown.

[0094] In one embodiment of this application, semantic aggregation of web page semantic information can be achieved through a graph neural network, such as a graph convolutional network (GCN) or a graph attention network (GAT), etc. For example, when semantically aggregating the first vectors of multiple web pages in a topology graph, the topology graph (i.e., the relationships between multiple web pages) and the first vector of each web page in the topology graph are used as input data to the graph neural network. The graph neural network performs semantic aggregation on the semantic information (i.e., the first vectors) of multiple web pages in the topology graph to obtain the second vector of each web page. The second vector of each web page is obtained by aggregating the first vector of each web page according to the weight of each web page mentioned above, which will not be described further.

[0095] It should be understood that in the process of semantic aggregation using graph neural networks, for each node, only the semantic information of nodes that have a direct relationship (direct connection) or an indirect relationship (indirect connection) with this node will be aggregated. For example... Figure 3 As shown, for webpage A in the topology graph, the semantic information of webpages B and C will be aggregated with the semantic information of webpage A to obtain the second vector of webpage A, but the semantic information of webpage D will not be aggregated. For completely isolated webpages in the topology graph, for example, the second vector of webpage D is the same as the first vector corresponding to webpage D.

[0096] In one embodiment of this application, the feature information of the first webpage further includes a first vector of the first webpage, namely a third vector of the first webpage, at least one second vector under at least one topic included in the first webpage, and the first vector of the first webpage, which together constitute the feature information of the first webpage. Since the feature information contains semantic information of each webpage itself, the constructed feature information is more accurate, further improving the accuracy of subsequent webpage searches.

[0097] In one embodiment of this application, before obtaining the first vector of each webpage, data cleaning is performed on each webpage to obtain high-quality text in each webpage. The high-quality text is then input into a semantic information extraction model to obtain the first vector of each webpage. The high-quality text in each webpage is the text in the webpage that is semantically complete and has a perplexity level below a threshold.

[0098] In one embodiment of this application, before constructing the feature information of each webpage, the webpages can be cleaned to filter out multiple high-quality webpages from the webpage library, namely the multiple webpages of this application.

[0099] The following example, using Wikipedia's page on Albert Einstein as its first page, illustrates the process of constructing the feature information of a webpage.

[0100] Step 1: Download the latest Wikipedia webpage data, resulting in multiple webpages.

[0101] Step 2: As Figure 5 As shown, through data processing, the text information of each webpage in multiple web pages is obtained; then, the text information of each webpage is input into the BERT model to obtain the first vector of each webpage.

[0102] Step 3: As Figure 6 As shown, a topology graph is constructed based on hyperlinks from multiple web pages. Figure 6 The underlined word is a hyperlink in Albert Einstein's Wikipedia page. Therefore, Albert Einstein's Wikipedia page is linked to other pages within multiple web pages via hyperlinks. Connecting the nodes of web pages that are hyperlinked to Albert Einstein's Wikipedia page creates a topology graph. Each node in this topology graph is a first vector representing the web page corresponding to that node, such as... Figure 6 The black nodes in the graph represent the first vector of Albert Einstein's Wikipedia entry. Hyperlinks between Albert Einstein's Wikipedia entry and other web pages are represented by edges connecting the nodes in the topology graph.

[0103] Step 4: Identify the topic of each webpage in the topology graph using the LDA topic recognition model.

[0104] Step 5: As Figure 7 As shown, sub-topologies are formed by extracting topics from the topology graph that contain Albert Einstein's Wikipedia entry. The number of sub-topologies is the same as the number of topics contained in Albert Einstein's Wikipedia entry. Figure 7 As shown, sub-topologies corresponding to topics 1, ..., n were extracted from the topology graph. Figure 1 The graph is divided into sub-topological graphs n, ..., n-n. A graph neural network is used to semantically aggregate the first vector of each webpage in the topological graph to obtain the third vector corresponding to Albert Einstein's Wikipedia entry. Then, a graph neural network is used to semantically aggregate the webpages in each sub-topological graph to obtain the second vector of Albert Einstein's Wikipedia entry in each sub-topological graph. Finally, the second vector of Albert Einstein's Wikipedia entry in the topological graph and the third vector in each sub-topological graph are combined to obtain the feature information of Albert Einstein's Wikipedia entry.

[0105] As can be seen from the feature information constructed from Albert Einstein's Wikipedia page, the variable-length representation (feature information) of a webpage is mainly reflected in the fact that the number of vectors contained in the feature information of the webpage is related to the number of topics of the webpage.

[0106] See Figure 8 , Figure 8 This is a flowchart illustrating a webpage search method provided in an embodiment of this application. The method is applied to the aforementioned webpage search system. The method includes the following steps:

[0107] 801: Get the semantic vector of the query statement.

[0108] For example, the query statement input by the user is obtained, and the query statement is represented by a vector to obtain the semantic vector of the query statement. The semantic vector of the query statement is used to represent the semantic information of the query statement. The vector representation of the query statement can be achieved by a semantic information extraction model, such as the BERT model mentioned above.

[0109] 802: Determine the similarity between the query statement and each webpage based on the semantic vector of the query statement and the feature information of each webpage in multiple webpages.

[0110] Among them, the feature information of each webpage in multiple webpages can be obtained through Figure 2 The method for constructing the feature information shown is not described further.

[0111] For example, based on the weights of each vector in the feature information of each webpage, the feature information of each webpage is transformed into a target vector. For instance, the vectors in the feature information are weighted according to their respective weights to obtain the target vector of each webpage. Then, the similarity between the semantic vector of the query statement and the target vector of each webpage is calculated to obtain the similarity between the query statement and each webpage. For instance, the cosine similarity between the semantic vector of the query statement and the target vector of each webpage can be calculated, and the cosine similarity is used as the similarity between the query statement and each webpage.

[0112] Specifically, the similarity between the semantic vector of the query statement and each semantic vector in the feature information of each webpage is determined. The similarity between the semantic vector of the query statement and each semantic vector is normalized, and the normalized result is used as the weight of each semantic vector.

[0113] Taking the feature information of the first webpage, including the second vector corresponding to the first webpage and at least one third vector, as an example, the process of determining the similarity between the query statement and the first webpage is explained.

[0114] For example, the similarity between the semantic vector of the query statement and the second vector, and the similarity between the semantic vector and each third vector are determined. Then, the similarity between the semantic vector of the query statement and the second vector, and the similarity between the semantic vector and each third vector are normalized to obtain the weights corresponding to the second vector and each third vector. Based on the weights corresponding to the second vector and each third vector, the second vector and at least one third vector are weighted to obtain the target vector corresponding to the first webpage. Finally, the similarity between the target vector of the first webpage and the semantic vector of the query statement is determined to obtain the similarity between the query statement and the first webpage.

[0115] 803: Based on the similarity between the query statement and each webpage, obtain the query results.

[0116] The query result consists of at least one of multiple web pages. For example, the multiple web pages are sorted in descending order of similarity between the query statement and each web page, and the top K web pages are used as the query result. This query result can be displayed in a visual interface, where K is an integer greater than or equal to 1.

[0117] As can be seen from the embodiments of this application, when constructing the feature information of each webpage, the semantic information of webpages associated with each webpage is also incorporated, rather than simply using the semantic information of each webpage itself to construct the feature information. Moreover, only the semantic information of webpages with the same theme as each webpage is incorporated, thus avoiding the introduction of noise during the information fusion process (e.g., incorporating the semantic information of irrelevant webpages), thereby resulting in higher accuracy of the constructed feature information. Because the constructed feature information is of higher accuracy, when matching query statements with webpages, the matching accuracy between query statements and webpages can be improved, thereby enhancing webpage search accuracy and the user's search experience.

[0118] See Figure 9 , Figure 9 This is a schematic diagram of the structure of a web search device provided in an embodiment of this application. Figure 9 As shown, the web search device 900 includes an acquisition unit 901 and a processing unit 902;

[0119] Unit 901 is used to obtain the semantic vector of the query statement;

[0120] Processing unit 902 is used to determine the similarity between the query statement and each webpage based on the semantic vector of the query statement and the feature information of each webpage in multiple webpages. The feature information of each webpage is used to characterize the first semantic aggregation information and at least one second semantic aggregation information of each webpage. The first semantic aggregation information is obtained by semantically aggregating the semantic information of multiple webpages, and the at least one second semantic aggregation information is obtained by semantically aggregating the semantic information of webpages with the same topic as each webpage in multiple webpages. In the process of semantic aggregation of each webpage, the weight of each webpage is greater than the weight of other webpages participating in the semantic aggregation process. Based on the similarity between the query statement and each webpage, the query result of the query statement is obtained, and the query result is at least one of the multiple webpages.

[0121] For a more detailed description of the acquisition unit 901 and the processing unit 902, please refer to the relevant descriptions in the above method embodiments, which will not be repeated here.

[0122] See Figure 10 , Figure 10 This is a schematic diagram of another web search device provided in an embodiment of this application. The web search device 1000 can be the web search device described above; or, it can be a chip or chip system in the web search device described above.

[0123] Figure 10 The web search device 1000 shown includes a memory 1001, a processor 1002, a communication interface 1003, and a bus 1004. The memory 1001, processor 1002, and communication interface 1003 are interconnected via the bus 1004.

[0124] The memory 1001 may be a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 1001 may store a program, and when the program stored in the memory 1001 is executed by the processor 1002, the processor 1002 and the communication interface 1003 are used to execute the various steps in the data stream transmission method of the embodiments of this application.

[0125] The processor 1002 may be a general-purpose central processing unit (CPU), microprocessor, application specific integrated circuit (ASIC), graphics processing unit (GPU), or one or more integrated circuits, used to execute relevant programs to achieve the functions required by the units in the audio feature compensation device or audio recognition device of the present application embodiments, or to execute the data stream transmission method of the method embodiments of the present application.

[0126] The processor 1002 can also be an integrated circuit chip with signal processing capabilities. In implementation, each step of the data stream transmission method of this application can be completed by the integrated logic circuits in the hardware of the processor 1002 or by instructions in software form. The processor 1002 described above can also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, or discrete hardware components. It can implement or execute the methods, steps, and logic block diagrams disclosed in the embodiments of this application. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the methods disclosed in the embodiments of this application can be directly embodied in the execution of a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software modules can be located in random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, or other mature storage media in the art. The storage medium is located in the memory 1001. The processor 1002 reads the information in the memory 1001 and, in conjunction with its hardware, performs the functions required by the units included in the user equipment or head-mounted device of this application embodiment, or executes the various steps in the data stream transmission method of the method embodiment of this application.

[0127] The communication interface 1003 can be a transceiver device such as a transceiver to enable communication between the web search device 1000 and other devices or communication networks; the communication interface 1003 can also be an input-output interface to enable data transmission between the web search device 1000 and input-output devices, wherein the input-output devices include, but are not limited to, keyboards, mice, displays, USB flash drives and hard drives.

[0128] Bus 1004 may include a pathway for transmitting information between various components of the device web search device 1000 (e.g., memory 1001, processor 1002, communication interface 1003).

[0129] It should be understood that the aforementioned processing unit 902 is equivalent to the processor 1002 in the web search device 1000.

[0130] It should be noted that, although Figure 10 The web search device 1000 shown only illustrates the memory, processor, and communication interface. However, those skilled in the art should understand that in specific implementations, the web search device 1000 may also include other components necessary for normal operation. Furthermore, depending on specific needs, those skilled in the art should understand that the web search device 1000 may also include hardware components for implementing other additional functions. Moreover, those skilled in the art should understand that the web search device 1000 may only include the components necessary for implementing the embodiments of this application, and may not necessarily include... Figure 10 All the devices shown.

[0131] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0132] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0133] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0134] In this application, "at least one" means one or more, and "more than one" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. In the textual description of this application, the character " / " generally indicates an "or" relationship between the preceding and following related objects; in the formulas of this application, the character " / " indicates a "division" relationship between the preceding and following related objects.

[0135] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application. The order of the process numbers described above does not imply the order of execution; the execution order of each process should be determined by its function and internal logic.

[0136] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0137] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A webpage search method, characterized in that, include: Obtain the semantic vector of the query statement; Based on the semantic vector of the query statement and the feature information of each webpage in the multiple webpages, the similarity between the query statement and each webpage is determined. The feature information of each webpage is used to characterize the first semantic aggregation information and at least one second semantic aggregation information of each webpage. The first semantic aggregation information is obtained by semantic aggregation of the semantic information of the multiple webpages. The at least one second semantic aggregation information is obtained by semantic aggregation of the semantic information of webpages with the same topic as each webpage in the multiple webpages. In the process of semantic aggregation of each webpage, the weight of each webpage is greater than the weight of other webpages participating in the semantic aggregation process. For a first webpage, wherein the first webpage is any one of the plurality of webpages; the first semantic aggregation information of the first webpage is represented by a second vector, wherein the second vector is obtained by semantically aggregating the first vector of each of the plurality of webpages, and the first vector of each webpage is used to represent the semantic information of each webpage; at least one second semantic aggregation information of the first webpage is represented by at least one third vector; each of the at least one third vector corresponds to a topic included in the first webpage, and the topics corresponding to each of the at least one third vector are different; wherein, in the at least one third vector, each third vector is obtained by semantically aggregating the first vector of the first webpage and the first vector of the second webpage, and the second webpage is a webpage among the plurality of webpages that contains the topic corresponding to each of the third vectors; The query result is obtained based on the similarity between the query statement and each webpage, and the query result is at least one of the multiple webpages.

2. The method according to claim 1, characterized in that, At least one third vector of the first webpage is also related to a topology graph that indicates the relationships between the plurality of webpages.

3. The method according to claim 2, characterized in that, The topology graph includes at least one sub-topology graph, and each of the at least one third vectors corresponds to a sub-topology graph in the at least one sub-topology graph, and the sub-topology graphs corresponding to each of the at least one third vectors are different; the web pages in the sub-topology graph corresponding to each third vector include the first web page and the second web page; Each third vector is obtained by semantic aggregation of the first webpage and the second webpage in the sub-topology graph corresponding to each third vector.

4. The method according to claim 3, characterized in that, Each of the at least one sub-topology graphs corresponds to a web page group in at least one web page group, and the web page groups corresponding to each of the at least one sub-topology graphs are different. Each web page group in the at least one web page group is composed of web pages containing a theme corresponding to each web page group. Each sub-topology graph in the at least one sub-topology graph is formed by extracting web pages from the web page group corresponding to each sub-topology graph.

5. The method according to any one of claims 1-4, characterized in that, The feature information of each webpage also includes a first vector of each webpage, which indicates the semantic information of each webpage.

6. The method according to claim 1, characterized in that, The feature information of each webpage is represented by a matrix. Before determining the similarity between the query statement and each webpage based on the semantic vector of the query statement and the feature information of each webpage in the multiple webpages, the method further includes: Based on the weights of each vector in the matrix, the matrix corresponding to each webpage is transformed into a target vector, and the target vector indicates the feature information of each webpage. The step of determining the similarity between the query statement and each webpage based on the semantic vector of the query statement and the feature information of each webpage in the multiple webpages includes: Calculate the similarity between the semantic vector of the query statement and the target vector, whereby the similarity indicates the relevance between the query statement and the webpage corresponding to the target vector.

7. The method according to claim 6, characterized in that, Before converting the matrix corresponding to each webpage into a target vector based on the weights of each vector in the matrix, the method further includes: Determine the similarity between the semantic vector of the query statement and each vector in the matrix; The weights of each vector in the matrix are determined based on the similarity between the semantic vector of the query statement and each vector in the matrix.

8. A webpage search device, characterized in that, Includes an acquisition unit and a processing unit; The acquisition unit is used to acquire the query statement; The processing unit is configured to acquire the semantic vector of the query statement; determine the similarity between the query statement and each webpage based on the semantic vector of the query statement and the feature information of each webpage in the plurality of webpages, wherein the feature information of each webpage is used to characterize the first semantic aggregation information and at least one second semantic aggregation information of each webpage, wherein the first semantic aggregation information is obtained by semantic aggregation of the semantic information of the plurality of webpages, and the at least one second semantic aggregation information is obtained by semantic aggregation of the semantic information of webpages in the plurality of webpages that have the same topic as each webpage, wherein the weight of each webpage is greater than the weight of other webpages participating in the semantic aggregation process; for the first webpage, wherein the first webpage is any one of the plurality of webpages; the first semantic aggregation information of the first webpage is transmitted through a second vector. The second vector is obtained by semantically aggregating the first vector of each webpage among the plurality of webpages, where the first vector of each webpage represents the semantic information of each webpage; at least one second semantic aggregation information of the first webpage is represented by at least one third vector; each of the at least one third vector corresponds to a topic included in the first webpage, and the topics corresponding to each of the at least one third vector are different; wherein, each of the at least one third vector is obtained by semantically aggregating the first vector of the first webpage and the first vector of the second webpage, and the second webpage is a webpage among the plurality of webpages that contains the topic corresponding to each third vector; based on the similarity between the query statement and each webpage, the query result of the query statement is obtained, and the query result is at least one of the plurality of webpages.

9. The apparatus according to claim 8, characterized in that, At least one third vector of the first webpage is also related to a topology graph that indicates the relationships between the plurality of webpages.

10. The apparatus according to claim 9, characterized in that, The topology graph includes at least one sub-topology graph, and each of the at least one third vectors corresponds to a sub-topology graph in the at least one sub-topology graph, and the sub-topology graphs corresponding to each of the at least one third vectors are different; the web pages in the sub-topology graph corresponding to each third vector include the first web page and the second web page; Each third vector is obtained by semantic aggregation of the first webpage and the second webpage in the sub-topology graph corresponding to each third vector.

11. The apparatus according to claim 10, characterized in that, Each of the at least one sub-topology graphs corresponds to a web page group in at least one web page group, and the web page groups corresponding to each of the at least one sub-topology graphs are different. Each web page group in the at least one web page group is composed of web pages containing a theme corresponding to each web page group. Each sub-topology graph in the at least one sub-topology graph is formed by extracting web pages from the web page group corresponding to each sub-topology graph.

12. The apparatus according to any one of claims 8-11, characterized in that, The feature information of each webpage also includes a first vector of each webpage, which indicates the semantic information of each webpage.

13. The apparatus according to claim 8, characterized in that, The feature information of each webpage is represented by a matrix. Based on the semantic vector of the query statement and the feature information of each webpage in the multiple webpages, before the processing unit determines the similarity between the query statement and each webpage, the processing unit is further configured to convert the matrix corresponding to each webpage into a target vector according to the weight of each vector in the matrix. The target vector indicates the feature information of each webpage. In determining the similarity between the query statement and each webpage based on the semantic vector of the query statement and the feature information of each webpage among multiple webpages, the processing unit is specifically used for: Calculate the similarity between the semantic vector of the query statement and the target vector, whereby the similarity indicates the relevance between the query statement and the webpage corresponding to the target vector.

14. The apparatus according to claim 13, characterized in that, Before the processing unit transforms the matrix corresponding to each webpage into a target vector according to the weights of each vector in the matrix, the processing unit is further configured to: Determine the similarity between the semantic vector of the query statement and each vector in the matrix; The weights of each vector in the matrix are determined based on the similarity between the semantic vector of the query statement and each vector in the matrix.

15. A webpage search device, characterized in that, include: Memory, used to store programs; A processor is used to execute programs stored in memory; When the program stored in the memory is executed, the processor is used to implement the method of any one of claims 1-7.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program code for execution by the device, the program code including methods for implementing any one of claims 1-7.

17. A computer program product, characterized in that, When the computer program product is run on a computer, it causes the computer to perform the method of any one of claims 1-7.

Images