Searching method and device based on artificial intelligence AI, medium and electronic equipment

Abstract

The invention provides an AI-based search method, an AI-based search device, a computer readable storage medium and electronic equipment, and is applied to the technical field of artificial intelligence. The method comprises the steps that according to vector representation of a search term and vector representation of a structured abstract of an ith article, the ith search term association degree between the search term and the ith article is determined, and i is a positive integer; calculating a correlation degree between portrait information of a target user corresponding to the search word and the ith article to obtain an ith personalized correlation degree; according to the ith search word association degree and the ith personalized association degree, determining a score about the ith article; and sorting the articles according to the scores corresponding to the articles to obtain a search result. According to the method and the device, personalized search experience can be provided for each search user while the search accuracy is improved.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence (AI) technology, and in particular to an AI-based search method, an AI-based search device, a computer-readable storage medium, and an electronic device. Background Technology

[0002] The information search solutions provided by related technologies can include the following steps: First, web page content can be crawled by browsing the internet. Then, the retrieved web page content is indexed, organizing the massive amounts of crawled data into a structured form, enabling the search engine to quickly respond to user queries. Next, when a user enters a query, the search engine parses the query statement, uses the index to quickly locate relevant documents, and employs various algorithms (such as Term Frequency-Inverse Document Frequency (TF-IDF), PageRank, or machine learning models) to evaluate the relevance of each stored document to the query, determining the document's ranking on the search results page. However, search results obtained using related technologies lack reflection of personalized user preferences, and search accuracy needs further improvement. Summary of the Invention

[0003] This application provides an AI-based search method, an AI-based search device, a computer-readable storage medium, and an electronic device, the search results of which can reflect the personalized preferences of the search user and help improve search accuracy.

[0004] Firstly, this application provides an AI-based search method, which includes: determining the relevance of a search term to an i-th article based on a search vector and an i-th summary vector, wherein the search vector is a vector representation of the search term, the i-th summary vector is a vector representation of the structured summary of the i-th article, and i is a positive integer; calculating the relevance between the profile information of the target user corresponding to the search term and the i-th article to obtain an i-th personalized relevance; determining a score for the i-th article based on the i-th search term relevance and the i-th personalized relevance; and sorting the multiple articles according to their respective scores to obtain search results.

[0005] In an exemplary embodiment, based on the above scheme, before determining the correlation degree between the search term and the i-th article based on the search vector and the i-th summary vector, the method further includes: inputting the i-th article into a pre-trained generative model, and obtaining the i-th summary suggestion based on the generative processing of the generative model; inputting the i-th article and the i-th summary suggestion into a pre-trained first encoder model, and obtaining the generation accuracy of the i-th summary suggestion based on the semantic similarity calculation of the first encoder model; inputting the i-th article into a pre-trained second encoder model, and obtaining the i-th set of entities based on the named entity recognition processing of the second encoder model, wherein the first encoder model and the second encoder model share a basic encoding part; if the generation accuracy of the i-th summary suggestion is greater than a first threshold, performing content splitting on the i-th summary suggestion based on the i-th set of entities, and labeling the split content to obtain the i-th structured summary; and storing the i-th structured summary in a structured summary pool.

[0006] In an exemplary embodiment, based on the above scheme, before determining the relevance of the i-th search term to the i-th article based on the search vector and the i-th summary vector, the method further includes: inputting the i-th article into a pre-trained generative model, and obtaining the i-th structured summary based on the generative processing of the generative model; inputting the i-th article and the i-th structured summary into a pre-trained first encoder model, and obtaining the generation accuracy of the i-th structured summary based on the semantic similarity calculation of the first encoder model; and storing the structured summaries with a generation accuracy greater than a second threshold in a structured summary pool; wherein the structured summaries in the structured summary pool are used to calculate the relevance of the search term.

[0007] In an exemplary embodiment, based on the above scheme, the above storage to the structured summary pool includes: storing the structured summary to the structured summary pool in a vertical manner according to the tags in the structured summary, so as to divide the structured summaries corresponding to multiple articles into multiple categories.

[0008] In an exemplary embodiment, based on the above scheme, the method further includes: obtaining a set of structured summaries belonging to the target category from the structured summary pool by means of vertical category retrieval; fine-tuning the generation model with the set of structured summaries belonging to the target category to generate a structured summary belonging to the target category through the fine-tuned generation model; and updating the structured summaries belonging to the target category to the structured summary pool.

[0009] In an exemplary embodiment, based on the above scheme, before determining the correlation degree between the search term and the i-th article based on the search vector and the i-th summary vector, the method further includes: encoding the search term using a machine reading comprehension MRC model to obtain the search vector; and encoding the structured summary of the i-th article using the MRC model to obtain the search vector and the i-th summary vector.

[0010] In an exemplary embodiment, based on the above scheme, the method further includes: inputting the search vector and the i-th summary vector into the MRC model to perform a post-check on the structured summary of the i-th article; wherein, if the post-check result is that the correlation between the structured summary of the i-th article and the search term is less than a third threshold, the structured summary of the i-th article is regenerated or the structured summary of the i-th article is adjusted.

[0011] In an exemplary embodiment, based on the above scheme, the above-mentioned encoding of search terms using a machine reading comprehension MRC model to obtain the above-mentioned search vector includes: expanding the received original search terms to obtain an expanded search term set; and encoding the search terms in the expanded search term set using the above-mentioned MRC model to obtain the above-mentioned search vector.

[0012] In an exemplary embodiment, based on the above scheme, before encoding the structured summary of the i-th article using the above MRC model to obtain the search vector and the i-th summary vector, the method further includes: collecting multi-dimensional data related to the target user; performing feature extraction on the collected information to determine at least one of the following information about the target user: basic attributes, interests and preferences, and behavioral habits, to obtain the profile information of the target user; and filtering all structured summaries in the structured summary pool according to the profile information of the target user corresponding to the search term to obtain a target structured summary with a correlation greater than a fourth threshold with the profile information.

[0013] In an exemplary embodiment, based on the above scheme, the calculation of the correlation between the profile information of the target user corresponding to the search term and the i-th article to obtain the i-th personalized correlation includes: determining at least one category to which the i-th article belongs; obtaining the sub-correlation between each category in the at least one category and the profile information; and determining the i-th personalized correlation between the target user and the i-th article based on the sub-correlation corresponding to each category in the at least one category.

[0014] In an exemplary embodiment, based on the above scheme, determining the score for the i-th article according to the i-th search term relevance and the i-th personalized relevance includes: obtaining a first weight corresponding to the i-th search term relevance and a second weight corresponding to the i-th personalized relevance, wherein the first weight and the second weight are determined by cross-validation; determining a first weighted value based on the first weight and the i-th search term relevance, and determining a second weighted value based on the second weight and the i-th personalized relevance; and calculating the sum of the first weighted value and the second weighted value to obtain the score for the i-th article.

[0015] In an exemplary embodiment, based on the above scheme, the process of sorting the multiple articles according to their respective scores to obtain search results includes: determining the display ranking of the i-th article based on the scores of the N articles, where N is a positive integer and i is a positive integer not greater than N; generating a visualization element for the i-th article based on its structured summary; determining at least the title of the i-th article and the visualization element as the display content at the display ranking of the i-th article; and determining the search results based on the display rankings of the multiple articles and the display content at the display rankings.

[0016] In an exemplary embodiment, based on the above scheme, determining the display ranking of the i-th article according to its score includes: determining the categories corresponding to the multiple articles respectively; and, for each category, determining the display ranking of the i-th article according to its score.

[0017] After determining the search results based on the display rankings and content displayed at the respective rankings of the multiple articles, the method further includes: obtaining the model of the target user's display device and determining the display format of the search results based on the model of the display device.

[0018] Secondly, this application provides an AI-based search device, which includes: a search term relevance determination module, a personalized relevance determination module, a score determination module, and a result generation module.

[0019] Specifically, the search term relevance determination module determines the relevance between the search term and the i-th article based on the search vector and the i-th summary vector, where the search vector is the vector representation of the search term, the i-th summary vector is the vector representation of the structured summary of the i-th article, and i is a positive integer; the personalized relevance determination module calculates the relevance between the profile information of the target user corresponding to the search term and the i-th article to obtain the i-th personalized relevance; the score determination module determines the score for the i-th article based on the i-th search term relevance and the i-th personalized relevance; and the result generation module sorts the multiple articles according to their respective scores to obtain the search results.

[0020] In an exemplary embodiment, based on the above scheme, the AI-based search device further includes: a summary pool generation module; the summary pool generation module includes: a summary suggestion generation unit, a similarity calculation unit, an entity generation unit, a structuring unit, and a storage unit. The summary suggestion generation unit is used to: before the search term relevance determination module determines the relevance between the search term and the i-th article based on the search vector and the i-th summary vector, input the i-th article into the pre-trained generation model, and obtain the i-th summary suggestion based on the generation processing of the generation model; the similarity calculation unit is used to: input the i-th article and the i-th summary suggestion into the pre-trained first encoder model, and obtain the generation accuracy of the i-th summary suggestion based on the semantic similarity calculation of the first encoder model; the entity generation unit is used to: input the i-th article and the i-th summary suggestion into the pre-trained first encoder model, and obtain the generation accuracy of the i-th summary suggestion based on the semantic similarity calculation of the first encoder model; the entity generation unit is used to: input the above... The first encoder model is input to the i-th article and pre-trained. The i-th entity set is obtained based on the named entity recognition processing of the second encoder model. The first encoder model and the second encoder model share the basic encoding part. The structuring unit is used to: split the i-th summary suggestion based on the i-th entity set and label the split content when the generation accuracy of the i-th summary suggestion is greater than a first threshold, so as to obtain the i-th structured summary. The storage unit is used to: store the i-th structured summary in the structured summary pool.

[0021] In an exemplary embodiment, based on the above scheme, the AI-based search device further includes: a summary pool generation module; the summary pool generation module includes: a structured summary generation unit, a similarity calculation unit, and a storage unit. The structured summary generation unit is used to: before the search term relevance determination module determines the relevance between the search term and the i-th article based on the search vector and the i-th summary vector, input the i-th article into a pre-trained generation model, and obtain the i-th structured summary based on the generation processing of the generation model; the similarity calculation unit is used to: input the i-th article and the i-th structured summary into a pre-trained first encoder model, and obtain the generation accuracy of the i-th structured summary based on the semantic similarity calculation of the first encoder model; and the storage unit is used to: store structured summaries with a generation accuracy greater than a second threshold in a structured summary pool; wherein the structured summaries in the structured summary pool are used to calculate the search term relevance.

[0022] In an exemplary embodiment, based on the above scheme, the storage unit is specifically used to: store the structured summary in a structured summary pool in a vertical manner according to the tags in the structured summary, so as to divide the structured summaries corresponding to multiple articles into multiple categories.

[0023] In an exemplary embodiment, based on the above scheme, the AI-based search device further includes: a vertical category retrieval module, a fine-tuning module, and an update module; wherein, the vertical category retrieval module is used to: obtain a set of structured summaries belonging to the target category from the structured summary pool through vertical category retrieval; the fine-tuning module is used to: fine-tune the generation model through the set of structured summaries belonging to the target category, so as to generate structured summaries belonging to the target category through the fine-tuned generation model; the update module is used to: update the structured summaries belonging to the target category to the structured summary pool.

[0024] In an exemplary embodiment, based on the above scheme, the AI-based search device further includes: a search vector generation module and a summary vector generation module; the search vector generation module is used to: encode the search term using a machine reading comprehension MRC model before the search term relevance determination module determines the relevance between the search term and the i-th article based on the search vector and the i-th summary vector, thereby obtaining the search vector; and the summary vector generation module is used to: encode the structured summary of the i-th article using the MRC model, thereby obtaining the search vector and the i-th summary vector.

[0025] In an exemplary embodiment, based on the above scheme, the AI-based search device further includes a post-checking module; wherein the post-checking module is used to: input the search vector and the i-th summary vector into the MRC model to perform a post-check on the structured summary of the i-th article; wherein, if the post-checking result is that the correlation between the structured summary of the i-th article and the search term is less than a third threshold, the structured summary of the i-th article is regenerated or the structured summary of the i-th article is adjusted.

[0026] In an exemplary embodiment, based on the above scheme, the search vector generation module is specifically used to: expand the received original search terms to obtain an expanded search term set; and encode the search terms in the expanded search term set using the above MRC model to obtain the search vector.

[0027] In an exemplary embodiment, based on the above scheme, the AI-based search device further includes: a summary filtering module; the summary filtering module is used to: collect multi-dimensional data related to the target user before the summary vector generation module encodes the structured summary of the i-th article through the MRC model to obtain the search vector and the i-th summary vector; perform feature extraction on the collected information to determine at least one of the following information about the target user: basic attributes, interests, and behavioral habits, to obtain the profile information of the target user; and filter all structured summaries in the structured summary pool according to the profile information of the target user corresponding to the search term to obtain a target structured summary with a correlation greater than a fourth threshold.

[0028] In an exemplary embodiment, based on the above scheme, the personalized correlation determination module is specifically used to: determine at least one category to which the i-th article belongs; obtain the sub-correlation degree between each category in the at least one category and the profile information; and determine the i-th personalized correlation degree between the target user and the i-th article based on the sub-correlation degree corresponding to each category in the at least one category.

[0029] In an exemplary embodiment, based on the above scheme, the score determination module is specifically used to: obtain the first weight corresponding to the relevance of the i-th search term and the second weight corresponding to the i-th personalized relevance, wherein the first weight and the second weight are determined by cross-validation; determine a first weighted value based on the first weight and the relevance of the i-th search term, and determine a second weighted value based on the second weight and the i-th personalized relevance; calculate the sum of the first weighted value and the second weighted value to obtain the score for the i-th article.

[0030] In an exemplary embodiment, based on the above scheme, the result generation module is specifically used to: determine the display ranking of the i-th article based on the scores corresponding to the N articles, where N is a positive integer and i is a positive integer not greater than N; generate a visualization element for the i-th article based on the structured summary of the i-th article; determine at least the title of the i-th article and the visualization element as the display content at the display ranking of the i-th article; and determine the search results based on the display rankings corresponding to the multiple articles and the display content at the display rankings.

[0031] In an exemplary embodiment, based on the above scheme, the result generation module is further specifically used to: determine the categories corresponding to the multiple articles respectively; and, for each category, determine the display ranking of the i-th article based on the score of the i-th article.

[0032] The aforementioned AI-based search device further includes a display format determination module; the display format determination module is used to: after the result generation module determines the search results based on the display rankings of the multiple articles and the content displayed at the display rankings, obtain the model of the target user's display device, and determine the display format of the search results based on the model of the display device.

[0033] Fourthly, an electronic device is provided, including a processor and a memory. The memory is used to store a computer program, and the processor is used to invoke and run the computer program stored in the memory to perform the AI-based search method described in the first aspect or its various implementations.

[0034] Fifthly, a chip is provided for implementing the methods of any one of the first aspects or their implementations. Specifically, the chip includes a processor for calling and running a computer program from a memory, causing a device equipped with the chip to perform an AI-based search method as described in the first aspect or its implementations.

[0035] In a sixth aspect, a computer-readable storage medium is provided for storing a computer program that causes a computer to perform the AI-based search method described in the first aspect or its various implementations.

[0036] In a seventh aspect, a computer program product is provided, including computer program instructions that cause a computer to execute the AI-based search method described in the first aspect or its implementations.

[0037] Eighthly, a computer program is provided that, when run on a computer, causes the computer to execute the AI-based search method described in the first aspect or its various implementations.

[0038] In summary, the AI-based search solution provided in this application converts search terms and structured summaries of each article into vector representations. By comparing the vector representation of the search term with the vector representation of the summary of the i-th article, the correlation between the search term and the i-th article is calculated. This demonstrates that the semantic understanding based on natural language processing technology improves the search engine's ability to understand query terms, thus providing more relevant search results. Furthermore, it calculates the user profile information corresponding to the search term and the correlation between articles to assess the personalized matching degree between the user and the article. Therefore, the search engine can provide personalized search results based on the user's historical search records, browsing habits, location, and other information. For example, for the same search term, users with different profiles may have different focuses, allowing the search engine to provide more accurate and personalized search results based on the user's profile information. Finally, the score of the i-th article is determined by combining the above two aspects. Therefore, the search results provided in this application not only reflect the correlation between search terms and articles after semantic understanding based on natural language processing technology but also reflect the personalized preferences of the search user, thereby improving search accuracy while providing a personalized search experience for each user. Attached Figure Description

[0039] To more clearly illustrate the technical solutions in the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0040] Figure 1 A schematic diagram of the system architecture for an application environment of an artificial intelligence-based search scheme provided in an embodiment of this application;

[0041] Figure 2 A flowchart illustrating an artificial intelligence-based search method provided in an embodiment of this application;

[0042] Figure 3A A schematic diagram illustrating information interaction in a method for determining a structured summary pool according to an embodiment of this application;

[0043] Figure 3B A schematic diagram of information interaction for a method of determining a structured summary pool provided in another embodiment of this application;

[0044] Figure 4 A schematic diagram illustrating a method for determining a target category corpus provided in an embodiment of this application;

[0045] Figure 5 A flowchart illustrating a method for vector representation of structured summaries provided in an embodiment of this application;

[0046] Figure 6 A flowchart illustrating a method for vector representation of search terms provided in an embodiment of this application;

[0047] Figure 7 A flowchart illustrating a method for calculating personalized correlation degree according to an embodiment of this application;

[0048] Figure 8 A schematic diagram of a search result display interface provided in an embodiment of this application;

[0049] Figure 9 A schematic diagram of the structure of an artificial intelligence-based search device provided in an embodiment of this application;

[0050] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0051] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0052] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in sequences other than those illustrated or described herein. In embodiments of this application, "B corresponding to A" means that B is associated with A. In one implementation, B can be determined based on A. However, it should also be understood that determining B based on A does not mean determining B solely based on A; B can also be determined based on A and / or other information. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or server that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to these processes, methods, products, or devices. In the description of this application, unless otherwise stated, "a plurality of" means two or more.

[0053] In this application, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0054] The following is through Figure 1 An exemplary system architecture of the embodiments of this application will be described.

[0055] For example, Figure 1 This is a schematic diagram of the system architecture for an application environment of an artificial intelligence-based search scheme provided in an embodiment of this application. For example... Figure 1 As shown, the system architecture of the implementation environment of this application embodiment may include: a terminal 102 for inputting search terms and viewing search results, a server 104 for executing the search process, and a database 106 for storing relevant data. The terminal 102 and server 104 can be connected via a network, and the server 104 and database 106 can also be connected via a network. Exemplarily, the network can be a wired communication link, a wireless communication link, or a fiber optic cable, etc., and this application embodiment does not impose limitations. For example, it can be a communication medium of various connection types capable of providing a communication link between the terminal 102 and the server 104.

[0056] In an exemplary embodiment, reference is made to Figure 1In step S10, the user inputs a search term through terminal 120, and terminal stick 102 sends the search term to server 104. Upon receiving the search term, server 104 performs the following two relevance calculations. Firstly, in step S11, server 104 calculates the relevance of the search term based on the vector corresponding to the search term. For example, a structured summary of each article can be predetermined, and the summary vector corresponding to the structured summary of each article can be obtained. Specifically, the structured summary of each article and its corresponding summary vector are stored in database 106. Thus, server 104 calculates the similarity between the search vector corresponding to the search term and the summary vector, thereby determining the relevance between the search term and each article. Secondly, in step S12, server 104 also determines the i-th personalized relevance between the target user and the i-th article based on the relevance between the target user's profile information corresponding to the search term and the i-th article. This allows the search structure to reflect the personalized preferences of search users. For example, if the search term is "hot pot," the search engine can prioritize recommending content that matches the user's personalized preferences, such as placing hot pot types that match personalized preferences at the top of the rankings. It is evident that for the same search term, users with different profiles focus on different aspects, and the search engine can provide more accurate and personalized search results based on user profile information. Further, in step S13, server 104 combines search term relevance and personalized relevance to determine the score of each article. Server 104 can then sort the multiple articles based on their respective scores to obtain search results. In step S14, server 104 sends the search results to terminal 102, which can then display the search results to the user. Therefore, the search results provided in this embodiment not only reflect the relevance between search terms and articles after semantic understanding based on natural language processing technology, but also reflect the personalized preferences of search users, thereby improving search accuracy while providing a personalized search experience for each user.

[0057] For example, server 104 can be a cloud server, specifically providing basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, and big data and artificial intelligence platforms. Additionally, server 104 can be an independent physical server, a server cluster composed of multiple physical servers, or a distributed system. For example, terminal 102 can also be used to display model training progress and training datasets. For example, terminal 102 can be a computer, smartphone, tablet, smart voice interaction device, smart home appliance, vehicle terminal, aircraft, wearable smart device, medical device, etc. Devices are often equipped with display devices, which can be monitors, displays, touchscreens, etc., and touchscreens can be touchscreens, touch panels, etc., but are not limited to these. For example, the aforementioned database 106 can be a cloud storage database. The type of database is not limited. For example, it can be a Structured Query Language (SQL) database, or a database stored in a Remote Dictionary Server (Redi) database, etc., but is not limited to these.

[0058] It should be noted that, Figure 1 The system architecture of the implementation environment of the embodiments of this application is illustrated, but the system construction of the implementation environment of the embodiments of this application is not limited to... Figure 1 As shown.

[0059] The technical solutions of the embodiments of this application will be described in detail below through some examples. The following embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0060] Figure 2 This is a flowchart illustrating an artificial intelligence-based search method P200 provided in an embodiment of this application. The executing entity of method P200 can be a search server deployed with a search engine, such as... Figure 1 The Chinese server is 104, or it could be like this: Figure 10 The electronic device shown is not limited to the embodiments described in this application. (See reference...) Figure 2 Method P200 includes S202 to S208.

[0061] In step S202, the correlation degree between the search term and the i-th article is determined based on the search vector and the i-th summary vector, wherein the search vector is the vector representation of the search term, the i-th summary vector is the vector representation of the structured summary of the i-th article, and i is a positive integer.

[0062] In an exemplary embodiment, the search server uses a web crawler to automatically browse web pages on the Internet. The crawler starts from some preset seed URLs and then continuously crawls new pages along the links in the web pages. It should be noted that the search server's data collection process follows the relevant protocols and crawling rules of the website. For example, the crawler puts newly discovered links into a queue of Uniform Resource Locators (URLs) to be crawled, and determines the next page to crawl according to a certain strategy (such as depth-first, breadth-first, etc.) to achieve efficient traversal.

[0063] In an exemplary embodiment, in order to quickly and accurately determine the search results, natural language processing technology is used to process the collected documents. Specifically, this application embodiment further includes step S201: generating a structured summary corresponding to each article based on a generative model to obtain a structured summary pool.

[0064] Figure 3A This diagram illustrates the information interaction of a method P300 for determining a structured summary pool according to an embodiment of this application. Exemplarily, method P300 is implemented based on information interaction between various models and program modules in a search server. (See reference...) Figure 3A The figure illustrates an exemplary embodiment of step S201, which provides a scheme for determining the structured summary pool, including steps S31-S310.

[0065] In step S31, the search server preprocesses the i-th article using preprocessing procedure 301.

[0066] For example, to facilitate subsequent text processing by the AI ​​model, the original web page document is preprocessed using the aforementioned preprocessing procedure 301. For instance, text extraction may involve removing HyperText Markup Language (HTML) tags and ignoring non-content information such as Cascading Style Sheets (CSS) and JavaScript. Noise reduction may involve removing non-core content that doesn't significantly aid in understanding the web page's theme, such as advertisements, navigation bars, and copyright information. Furthermore, word segmentation is necessary for languages ​​like Chinese that are written continuously without clear delimiters, breaking the text down into meaningful word units. And so on.

[0067] In step S32, the preprocessed i-th article is sent to the generation model 302, the first encoder model 304, and the second encoder model 305 respectively; and in step S33, the generation model 302 performs the summary suggestion generation process.

[0068] For example, the preprocessed text is input into a pre-trained generative model (such as the GPT model) 302, where the prompt word of the generative model 302 can be a suggestion to generate a summary of the input article. The summary suggestion refers to a preliminary, relatively free-form summary text generated by the GPT model based on the input text content. The pre-trained GPT model, by understanding and learning from a large amount of text data, can creatively generate coherent and fluent sentences to summarize the main information of the input text. Specifically, the summary suggestion contains the key points of the article, helping users quickly obtain potential key information points. For example, a summary suggestion for an article might be as follows:

[0069] "With the advancement of technology, artificial intelligence has achieved remarkable results in many fields. Among them, Natural Language Processing (NLP) is an important branch of artificial intelligence, aiming to enable computers to understand, generate, and process human language. In recent years, NLP research has made groundbreaking progress, especially pre-trained models such as BERT (Bidirectional Encoder Representations from Transformers) and Generative Pre-trained Transformer (GPT), which have performed exceptionally well in various NLP tasks. These models learn rich linguistic knowledge through pre-training on large amounts of text data, thus achieving superior performance in various NLP tasks. However, these models have very high computational and storage requirements, thus requiring continuous optimization to adapt to different application scenarios. In addition, to further improve the performance of NLP models, researchers are striving to explore more efficient training methods and more powerful model structures."

[0070] In step S34, the generation model 302 sends the generated i-th summary suggestion to the structured summary pool 303 and the first encoder model 304, respectively. In step S35, the first encoder model 304 performs semantic similarity calculation based on the received i-th article and its i-th summary suggestion to obtain the generation accuracy of the i-th summary suggestion.

[0071] In this embodiment, the semantic similarity between the original text and the summary suggestions generated by the generation model 302 is calculated using the first encoder model 304, denoted as the generation accuracy, and used to measure the accuracy of the generated text. If the semantic similarity (generation accuracy) is greater than a first threshold, it indicates that the suggestions generated by the generation model 302 have captured the semantics of the original text well; if the semantic similarity (generation accuracy) is not greater than the first threshold, the suggestions can be modified or regenerated.

[0072] Understandably, to ensure high accuracy in summary generation, the lower limit of the aforementioned first threshold range should be greater than 50%, for example, the range of the first threshold could be [75%, 1]. Within this range, the actual value of the first threshold can be determined based on actual needs. For instance, in some use cases where high accuracy in summary generation is required, the first threshold can be set to a larger value (e.g., 95%), while in others where lower accuracy is required, the first threshold can be set to a smaller value (e.g., 80%).

[0073] For example, the first encoder model 304 mentioned above can be a BERT model or other encoder models, and this application embodiment does not limit this. Taking the BERT model as an example: S34-1, the BERT model encodes the articles and summary suggestions to be compared into vector representations respectively; wherein, in the input encoding stage of BERT, each sentence is converted into a fixed-length vector, which contains the deep semantic information of the sentence. S34-2, the sentence vector is extracted next; if the summary suggestion is a single sentence, the BERT model usually takes the output of the last few layers of the Transformer model (such as the average of the last layer or multiple layers) as the vector representation of the sentence; if the i-th article is a sentence pair task, a special method can be used, such as concatenating the vectors of the two sentences or using the Siamese network structure to encode them separately before operation. S34-3, having obtained the vector representations of two sentences, allows for the calculation of their similarity. Exemplary similarity metrics include cosine similarity and Euclidean distance. Alternatively, multilayer perceptrons or other machine learning models can be used to transform the vectors before calculating similarity, or more complex models (such as Siamese networks) can be used directly in the vector space to optimize similarity calculation. In summary, the BERT model captures the contextual semantic information of text through deep learning and achieves quantitative evaluation of semantic similarity through vectorized representation and similarity calculation methods. This process leverages BERT's powerful pre-training capabilities and ensures accuracy in specific application scenarios through task-specific fine-tuning.

[0074] In step S36, the first encoder model 304 sends the above-mentioned generation accuracy of the i-th summary suggestion to the structured summary pool.

[0075] In this embodiment, summary suggestions with an accuracy greater than a second threshold are subjected to structuring processing to generate structured summaries. Structured summaries can present key information from summary suggestions in an organized manner according to a certain structure or format, which is beneficial for matching efficiency with search vectors.

[0076] Similar to the first threshold, to ensure high accuracy in summary generation, the lower limit of the second threshold's value range should be greater than 50%, for example, the first threshold's value range is [80%, 1). Within the set value range, the second threshold's actual value can be determined based on actual needs.

[0077] In step S37, the second encoder model 305 performs named entity recognition processing on the received i-th article to obtain the i-th set of entities.

[0078] For example, the first encoder model 304 and the second encoder model 305 can share a basic encoding part and adapt to different tasks by setting different output layers. Sharing the encoding part can reduce the model size, requiring only one large pre-trained model to maintain, thus reducing storage and computational costs. Simultaneously, the language knowledge in the pre-trained model can be applied to different tasks, which is beneficial for improving the model's generalization ability. Furthermore, since the encoder part has already been pre-trained on a large amount of data, fine-tuning for a specific task can mainly focus on adjusting the two output layers, accelerating the training process.

[0079] In step S38, the second encoder model 305 sends the i-th entity set to the structured summary pool 303. And in step S39, if the generation accuracy of the summary suggestion for the i-th article is greater than a first threshold, the summary suggestion for the i-th article is split based on the i-th entity set, and the split content is labeled to obtain the i-th structured summary.

[0080] Since the summary suggestions generated by the generative model 302 can be regarded as a preliminary, relatively free-flowing text overview, while the structured summary is an ordered, structured information overview formed after further organizing and classifying these key points, in order to improve the standardization of the summary suggestions and thus improve the matching efficiency with the search vector, this embodiment of the application, in the structured summary pool 303, splits the content of the summary suggestions with an accuracy greater than a first threshold according to the entity set of the same article, and then labels the split content to obtain the above-mentioned structured summary.

[0081] Specifically, the entity set generated by the second encoder model 305 facilitates the understanding of summary suggestions for the same article, thereby enabling the precise decomposition of summary suggestions into several clearly defined parts, including content about the topic (such as "Advances in Artificial Intelligence in Natural Language Processing (NLP)" in the above summary suggestion), content about achievements (such as "Pre-trained models such as BERT and GPT perform well in various NLP tasks" in the above summary suggestion), content about methods (such as "Pre-training on large amounts of text data to learn language knowledge" in the above summary suggestion), content about challenges (such as "High computational and storage requirements of models" in the above summary suggestion), and content about future directions (such as "Exploring efficient training methods and robust model structures to improve performance" in the above summary suggestion), etc. Furthermore, labels are assigned to each of the above-mentioned parts. In this embodiment, the labels include: topic, achievements, methods, challenges, and future directions. It is understood that the labels are summary terms of the content corresponding to the labels, and are not limited to the labels listed above.

[0082] For example, structured summaries can present information using explicit lists, tables, etc. As can be seen, structured summaries emphasize the order and logic of information, facilitating quick browsing and understanding by users. The structured summary obtained by structuring the above exemplary summary suggestions is as follows:

[0083] Topic: Advances of Artificial Intelligence in Natural Language Processing (NLP);

[0084] Results: Pre-trained models such as BERT and GPT perform well in various NLP tasks;

[0085] Method: Pre-training on a large amount of text data to learn language knowledge;

[0086] Challenge: High computational and storage requirements for the model;

[0087] Future directions: Explore efficient training methods and robust model structures to improve performance.

[0088] In the process described above in the embodiments of this application, the summary suggestions generated by the generation model 302 (such as the GPT model) are first subjected to auxiliary analysis by the first encoder model 304 (such as BERT) (such as the semantic similarity calculation mentioned above). Then, the summary suggestions with semantic similarity greater than a first threshold are subjected to structuring processing to obtain more standardized structured summaries. Specifically, in the process of converting summary suggestions into structured summaries, the entity set of the same article is used to help understand the content of the summary suggestions, thereby improving the efficiency of converting summary suggestions into structured summaries.

[0089] In step S310, the structured summaries corresponding to each article are stored in the structured summary pool in a vertical category manner.

[0090] For example, based on the tags (e.g., topic, results, methods, challenges, future directions, etc.) in the structured summaries corresponding to multiple articles, the structured summaries are categorized into corresponding vertical categories. For instance, all summaries containing the tag "machine learning" can be categorized under the "machine learning" vertical category. This allows the structured summaries corresponding to the multiple articles to be divided into multiple categories, enabling the information in the structured summary pool to be stored in a vertical category manner. The result of vertical category storage not only enables automatic classification of structured summaries but also facilitates rapid information location and relevance retrieval.

[0091] Method P300 provides a scheme for determining structured summaries. It generates summary suggestions for an article using a generative model and measures the accuracy of these suggestions. For highly accurate suggestions, a second encoder model generates entity vectors from the same article to perform semantic understanding, accurately splitting the content of the suggestions. The split content is then labeled to obtain the structured summary. This embodiment, through the accuracy verification step, can, on the one hand, filter and correct errors in the summary suggestions, improving the accuracy of the final structured summary; on the other hand, by adding a review process after generating the suggestions, it can ensure that the summary content better meets specific structured requirements, increasing the controllability of structured summary generation.

[0092] Figure 3B This is a schematic diagram illustrating the information interaction of a method P300' for determining a structured summary pool, provided in another embodiment of this application. Exemplarily, method P300' is implemented based on information interaction between various models and program modules in a search server. (See reference...) Figure 3B The figure illustrates an exemplary embodiment of step S201, which provides a scheme for determining the structured summary pool, including steps S31'-S37'.

[0093] In step S31', the search server preprocesses the i-th article using preprocessing procedure 301.

[0094] For a detailed implementation of this step, please refer to the embodiment corresponding to step S31, which will not be repeated here.

[0095] In step S32', the preprocessed i-th article is sent to the generation model 302 and the first encoder model 304 respectively; and in step S33', the generation model 302 performs the structured summary generation process.

[0096] For example, the preprocessed text is input into a pre-trained generative model (such as the GPT model) 302, where the prompt for the generative model 302 can be a requirement for the model to generate a structured summary of the input article. A structured summary refers to presenting the key information of the original text of the i-th article in an organized manner according to a certain structure or format, including decomposing the content into several clearly defined parts, where tags can include topics, achievements, methods, challenges, future directions, etc. Therefore, structured summaries facilitate quick browsing and understanding, and also improve the matching efficiency with search vectors.

[0097] In step S34', the generation model 302 sends the generated i-th structured summary to the structured summary pool 303 and the first encoder model 304 respectively. And, in step S35', the first encoder model 304 performs semantic similarity calculation based on the received i-th article and its i-th structured summary to obtain the generation accuracy of the i-th structured summary.

[0098] In this embodiment, the semantic similarity between the original text and the structured summary generated by the generation model 302 is calculated using the first encoder model 304, denoted as the generation accuracy, and used to measure the accuracy of the generated text. If the semantic similarity (generation accuracy) is greater than a second threshold (e.g., greater than 95%), it indicates that the structured summary generated by the generation model 302 has captured the semantics of the original text well; if the semantic similarity (generation accuracy) is less than or equal to the second threshold, the structured summary can be modified or regenerated.

[0099] For example, the first encoder model mentioned above can be the BERT model or other encoder models, and this application embodiment does not limit this. The semantic similarity calculation process using the BERT model as an example can be referred to the embodiment in supplementary S34, and will not be repeated here.

[0100] In step S36', the first encoder model sends the above-mentioned generation accuracy of the i-th structured summary to the structured summary pool.

[0101] In step S37', multiple structured summaries with an accuracy greater than the second threshold are generated and stored in the structured summary pool in a vertical category manner.

[0102] For example, based on the tags (e.g., topic, results, methods, challenges, etc.) in the structured abstracts corresponding to multiple articles, the structured abstracts corresponding to the multiple articles can be divided into multiple categories, so that the information in the structured abstract pool is stored in a vertical category manner.

[0103] The structured summary determination scheme provided by method P300 directly generates structured summaries of articles through a generative model, which improves the efficiency of structured summary generation and saves on generation costs. Measuring the accuracy of the generated structured summaries and storing those with high accuracy helps ensure overall accuracy. Therefore, this embodiment requires less model maintenance while meeting the required accuracy for the generated structured summaries, thus saving costs.

[0104] In an exemplary embodiment, the structured summary determined by method P300 or method P300' is vectorized, and similarity is calculated with the vector identifier corresponding to the search term to determine the relevance between the search term and the corresponding article.

[0105] In exemplary embodiments, to better adapt to domain-specific structured summary generation tasks, such as... Figure 3A and Figure 3B The illustrated generative model and encoder can be fine-tuned on specific datasets. Specifically, based on the pre-trained model, the model parameters are adjusted using a dataset containing semantic similarity tags (such as structured summary data stored in a structured summary pool) so that the output of the fine-tuned model more accurately reflects the actual semantics. This allows the fine-tuned model to generate corpora corresponding to each category, and further, the relevance can be calculated based on the structured summaries in each type of corpus. The following section first introduces an example of fine-tuning the model to generate a target category corpus.

[0106] Figure 4 A schematic diagram illustrating a method for determining a target category corpus provided in an embodiment of this application. Specifically, through... Figure 4 The illustrated embodiments can be used to build corpora for vertical domains.

[0107] refer to Figure 4 A query seed is a set of initial queries or keywords used to initiate or guide the content generation process. Query seeds can represent a topic or content area that you want to explore, allowing you to target or filter relevant information.

[0108] Next, a vertical category check is performed on the structured summary pool based on the query seed (step S41). Specifically, the vertical category check is the process of classifying and filtering data in the structured summary pool according to a specific domain (such as science and technology, sports, entertainment, etc.), thereby finding structured summaries in the pool that belong to the same category as the query seed, i.e., obtaining similar samples to the query seed. Through the vertical category check, content that conforms to a specific category or theme can be dynamically identified and selected from a large amount of data, ensuring that the generated corpus is highly relevant to the target domain.

[0109] The above-described vertical category check dynamically selects a small number of representative samples, best_few_shots (step S42). Dynamic selection means that this process is automatically adjusted based on current task requirements and data changes; the selected samples are not a fixed set. The small number of representative samples, best_few_shots, are a few typical samples selected from the target categories that best represent that category. These small number of representative samples, best_few_shots, serve as exemplary input, helping the generative model quickly understand how to generate high-quality content for a specific context.

[0110] A small number of representative samples are input into the generative model, which is then fed into the model to be fine-tuned (such as the generative model mentioned above). Based on these input samples, the generative model learns and generates new, diverse text content that meets the requirements of the target vertical category, thereby efficiently and accurately generating high-quality text content for a specific domain and constructing a corpus containing structured summaries for each category. Furthermore, in an exemplary embodiment, the structured summaries in the aforementioned corpus can be updated to the aforementioned structured summary pool, and similarity calculations can be performed with the vector identifiers corresponding to the search terms to determine the relevance between the search terms and the corresponding articles.

[0111] The following examples illustrate how to implement the vector representation of structured summaries.

[0112] Figure 5 The flowchart of a method for vector representation of structured summaries provided in an embodiment of this application can be used as a specific implementation of determining the i-th summary vector in step S202.

[0113] refer to Figure 5 In step S202-1, based on the profile information of the target user corresponding to the search term, all structured summaries in the structured summary pool are filtered to obtain target structured summaries with a correlation degree greater than the fourth threshold of the profile information.

[0114] In an exemplary embodiment, after receiving the search term, the user profile information corresponding to the search term (denoted as the target user) is also obtained to construct the user profile. The user profile information helps us understand the user's interests and needs, thereby providing more personalized recommendations. For example, firstly, multi-dimensional data related to the user needs to be collected, including: 1) Browsing history; websites visited, types of articles read, video content watched, etc. 2) Interaction behavior; social interactions such as likes, comments, shares, and subscriptions. 3) Search history; the user's search keywords can reflect their immediate interests and long-term preferences. 4) Purchase history; categories of goods and services purchased on e-commerce platforms; etc. Next, feature extraction is performed on the collected information to extract features meaningful to the user profile, such as: frequency statistics of access to various types of content; determining the user's time series pattern based on the user's active periods throughout the day or week, and periodic preferences for specific categories; extracting keywords from search and browsing content to reflect their interests; understanding the interests of their social circle based on participating communities, followed KOLs (Key Opinion Leaders), or circles; etc. Based on the features extracted above, user profiles can be constructed, including basic attributes, interests and preferences, and behavioral habits.

[0115] It should be noted that the collection, use, processing, transmission, provision, disclosure, and storage of user personal information involved in this application embodiment all comply with the provisions of relevant laws and regulations and do not violate public order and good morals; the user's authorization or consent was obtained before obtaining or collecting user personal information.

[0116] In some embodiments, upon receiving a search term, all structured summaries in the structured summary pool can be represented as vectors. In some embodiments, to reduce data processing volume and improve search response speed, all structured summaries in the structured summary pool can be filtered based on the profile information of the target user corresponding to the search term, to obtain target structured summaries whose relevance to the target user's profile information is greater than the aforementioned fourth threshold, and then the target structured summaries can be specifically represented as vectors. For example, the relevance is measured based on the similarity between the profile information and the structured summaries; structured summaries with a similarity greater than the fourth threshold can be considered target structured summaries. To ensure a high degree of correlation between the target structured summaries and the target user's profile information, the lower limit of the fourth threshold's value range should be greater than 50%, for example, the value range of the fourth threshold is [75%, 1]. Within the set value range, the actual value of the fourth threshold can be determined according to actual needs.

[0117] For example, user A's search term is "lunch recommendation." Based on user A's historical order records, there are many orders for Chinese food and almost no orders for Western food. Therefore, information with little relevance to Chinese food in the structured summary can be filtered out, and the remaining structured summary can be used as the target structured summary. It is evident that this filtering method for determining the target structured summary not only reduces computation and improves search response speed but also helps to produce more targeted search results, thus enhancing the personalization of the search results.

[0118] In step S202-2, the target structured summary is encoded using a machine reading comprehension MRC model to obtain the summary vector corresponding to the target structured summary.

[0119] In exemplary embodiments, the Machine Reading Comprehension (MRC) model can understand the deep semantics of a given text, going beyond surface word matching to grasp the logical relationships, implicit meanings, and contextual dependencies within the text. Furthermore, the MRC model can be fine-tuned to adapt to data from different domains, such as news, scientific literature, and conversation logs. Embodiments of this application can employ the MRC model to perform vector representations of structured summaries and search terms, respectively.

[0120] For example, the MRC models that can be used include any one of BERT, the Bidirectional Attention Flow (BiDAF) model, and the lightweight ALBERT based on BERT. For example, a pre-trained BERT model is used as the vector representation model. For example, the model can be fine-tuned according to actual needs to adapt to specific categories of text summarization tasks. Specifically, the fine-tuning implementation of the vector representation model is introduced using a text summarization task of the target category as an example. To improve the robustness and generalization ability of the model, adversarial training methods can be used to optimize the parameters of the above vector representation model. First, the sample set of the target category is determined, specifically including normal samples and adversarial samples. Normal samples can be obtained from the summary text of the target category, and adversarial samples can be mined using the MRC model. During training, a combination of normal samples and adversarial samples can be used as model input to perform adversarial training on the above vector representation model.

[0121] For example, by inputting the above target structured summary into the fine-tuned vector representation model, the summary vector corresponding to the target structured summary can be obtained.

[0122] Figure 6This is a flowchart illustrating a method for vector representation of search terms provided in an embodiment of this application, which can serve as a specific implementation of determining the search vector in step S202.

[0123] refer to Figure 6 In step S202-1', the received original search terms are expanded to obtain an expanded search term set.

[0124] In an exemplary embodiment, to broaden the coverage of search terms and improve the relevance and comprehensiveness of search results, after receiving a search term (denoted as the original search term), it can be expanded to obtain an expanded search term set. Exemplary expansion methods may include one or more of the following methods.

[0125] Method 1: Incorporate synonyms and related words; Search engines identify keywords in user input information and then add related synonyms or near-synonyms to the search request to help capture information that uses different expressions but has similar meanings.

[0126] Method 2, Root word expansion: Search engines can use stemming or word form restoration techniques to restore the original search terms to their basic root words, thereby matching more words with different forms containing the same root word.

[0127] Method 3: Spelling correction; The search engine identifies possible spelling errors in the user's input query information and then provides corrected suggestions, or uses the corrected words to search in the background to ensure that relevant information is not missed due to spelling errors.

[0128] Method 4, Contextual Understanding: Search engines use natural language processing technology to try to understand the intent and context behind a query, thereby providing more accurate and relevant search results. Examples include understanding factors such as entities, time, and location in the query.

[0129] Method 5: Adjust the original search terms based on user profile information; Based on user profile information, the search engine can infer the user's interests and needs, thereby adjusting the search results and providing a more personalized expanded vocabulary search.

[0130] Method 6: Expand based on popular and trending topics; search engines can combine current popular topics and trends to add the latest or popular content related to the query topic to the search results.

[0131] It is understood that the methods for expanding the original search terms are not limited to those mentioned above, and other expansion methods achievable in this field are also possible. This application embodiment does not limit these methods. By reasonably expanding the query information entered by the user, this application embodiment can broaden the coverage of search terms, improve the relevance and comprehensiveness of search results, thereby helping to ensure that users quickly find the information they need and enhancing their search experience.

[0132] Continue to refer to Figure 6 In step S202-2', the search terms in the extended search term set are encoded using the machine reading comprehension MRC model to obtain search vectors.

[0133] For example, the search terms in the expanded search term set are input into the fine-tuned vector representation model, so that each search term is encoded by the vector representation model to obtain the expanded search vector set.

[0134] In one implementation, for each search vector in the extended search vector set, a similarity score is calculated between it and the summary vector. Then, based on the similarity scores of each search vector in the extended search vector set, the search term relevance between the search term and the article is determined. For example, the extended search vector set for search term x includes search vectors a, b, and c. To calculate the search term relevance between search term x and article X, search vectors a, b, and c can be compared with the summary vector corresponding to article X, resulting in three similarity scores s. a s b and s c Then based on three similarity scores s a s b and s c The statistical values ​​determine the relevance between search term x and article X. This implementation processes each expanded term individually, ensuring coverage of every relevant vocabulary and reducing the possibility of missed detections. Simultaneously, the search results for each term can be evaluated and ranked independently, facilitating adjustments later based on term importance or relevance.

[0135] In one implementation, search vectors from the extended search vector sets for the same search term are fused, and the similarity between the fused search vector and the summary vector is calculated to obtain the search term relevance between the search term and the article. For example, merging the vector representations corresponding to multiple extended search terms into a single vector representation can be achieved using any of the following methods: average pooling; max pooling; weighted average; concatenating the vectors of all extended search terms in terms of dimension to form a higher-dimensional vector, which can then be compressed to a lower dimension using dimensionality reduction techniques (such as Principal Component Analysis (PCA) or an additional neural network layer) to retain more original information; etc. For example, the extended search vector set for search term x contains search vectors a, b, and c; to calculate the search term relevance between search term x and article X, search vectors a, b, and c can be fused to obtain a fused vector x', and the similarity between the fused vector x' and the summary vector corresponding to article X is calculated to obtain the similarity s. x’ This determines the correlation between search term x and article X. This implementation uses a fused search vector, which integrates vectors from an expanded search vector set. This reduces the number of actual searches performed and lowers computational costs. Furthermore, combining information from multiple words, such as "refrigerator" and "in the refrigerator," can generate richer and more accurate expressions of search intent, making it suitable for handling searches of related terms. Additionally, vector fusion avoids duplicate search results for individual terms, improving the user experience.

[0136] In one implementation, the two methods described above can be combined. For example, a moderate vector fusion can be performed first to reduce the search volume, and then the expanded search terms that are highly relevant to the original search terms can be processed separately, thus combining the advantages of both methods.

[0137] In an exemplary embodiment, a post-check can also be performed on the structured summary based on the vector generation model described above. For example, the search vector and the i-th summary vector are input into the MRC model to check whether the structured summary of the i-th article contains key information from the search terms. Specifically, if the post-check result indicates that the correlation between the structured summary of the i-th article and the search terms is less than a third threshold, the structured summary of the i-th article needs to be regenerated or the existing structured summary of the i-th article needs to be adjusted. For example, this applies when a user continuously inputs related search terms. To facilitate efficient search results, a post-check can be performed on the structured summaries of related articles. This post-check can specifically detect whether the current structured summary contains the key information the user is searching for, i.e., whether the current article is the article the user is looking for. This post-check facilitates efficient search results, improves search targeting and efficiency, and enhances the user's search experience.

[0138] To ensure a high degree of relevance between the structured summary and the search terms, the lower limit of the third threshold value range should be greater than 50%, for example, the value range of the third threshold is [75%, 1). Within the set value range, the actual value of the third threshold can be determined according to actual needs.

[0139] Continue to refer to Figure 2 In step S204, the correlation between the profile information of the target user corresponding to the search term and the i-th article is calculated to obtain the i-th personalized correlation.

[0140] In this embodiment of the application, when sorting articles in the search results, not only is the relevance between the search terms and the articles (i.e., the search term relevance mentioned above) considered, but also the relevance between the user profile information and the articles (denoted as personalized relevance). By combining the two aspects of relevance, more accurate and personalized search results are provided to the user.

[0141] Figure 7 The flowchart of a method for calculating personalized relevance provided in an embodiment of this application can be used as a specific implementation of determining the search vector in step S204.

[0142] refer to Figure 7 In step S204-1, at least one category to which the i-th article belongs is determined.

[0143] It is understandable that the same article can belong to different categories simultaneously; for example, article A may belong to both the science and technology category and the education category. For instance, the categories to which an article belongs may include multiple broad categories, and further subcategories within these broad categories. For example, broad categories may include: news, academic papers, literature, education, etc.; among which, subcategories of the news category may include international news, domestic news, sports news, financial news, etc.; and subcategories of the literature category may include novels, essays, poetry, drama, etc. This application embodiment does not limit the number of categories or the method of classification for an article. Specifically, the category to which an article belongs can also be updated based on the article content to ensure the accuracy of personalized relevance.

[0144] In an exemplary embodiment, the category to which the i-th article belongs can be predicted using a trained multi-label classification model. For example, the full text, summary, or summary suggestion output by the aforementioned generation model of the i-th article can be used as input to the multi-label classification model, and the at least one category to which the i-th article belongs can be determined based on the model's output. In multi-label classification, each category is independent, and the model's goal is to predict the probability of a sample belonging to each category. Unlike traditional multi-class problems, which assume a sample can only belong to one category, multi-label classification allows a sample to belong to multiple categories simultaneously. For example, the multi-label classification model can predict that the i-th article belongs to both the "technology" and "medical" categories. The model's output can be a binary vector or a probability vector, where each element corresponds to a category, with a value of 1 indicating that the article belongs to that category, a value of 0 indicating that it does not, or a higher probability value indicating that the article is more likely to belong to that category.

[0145] In an exemplary embodiment, the category to which the i-th article belongs can also be determined based on the source of the i-th article. For example, if the i-th article comes from a special issue on health, then the category to which the i-th article belongs can be determined to include "health".

[0146] The embodiments of this application do not limit the method for determining the category to which the i-th article belongs; it can be any implementation method that can determine the category to which the i-th article belongs.

[0147] In step S204-2, the sub-association degree between each category in the at least one category and the portrait information is obtained.

[0148] In this embodiment, after obtaining user profile information, the correlation degree (denoted as sub-correlation degree) between the user profile information and different categories can be determined. For example, the implementation method for determining the sub-correlation degree based on the user profile constructed in the aforementioned content includes: Step S1, using machine learning or statistical methods to establish a correlation model between the user profile and categories; for example, a classification model or clustering model, which can specifically classify users according to their interests and identify user groups highly correlated with specific categories; a collaborative filtering model, which, based on user-content interaction, is a commonly used algorithm in recommendation systems and can reveal the user's potential preferences for a certain category; a content-based algorithm, used to analyze the features of content with known user preferences and match other categories of content with similar features; a deep learning model, which can utilize neural networks, such as convolutional neural networks or recurrent neural networks, to automatically extract high-level features and predict the degree of correlation between the user and each category. Step S2, calculating the sub-correlation degree; the similarity or relevance score between the user's profile features and the content of each category can be calculated using the above correlation model, for example, through methods such as cosine similarity, Jaccard coefficient, and Pearson correlation coefficient. The above process can effectively determine the correlation between user profile information and different categories, thereby providing users with more personalized and accurate content recommendations.

[0149] In step S204-3, the i-th personalized correlation degree between the target user and the i-th article is determined based on the sub-correlation degree corresponding to each category in the at least one category.

[0150] In an exemplary embodiment, if article A belongs to category A1-Category A3, and the sub-associations between user S and categories A1-A3 are a1, a2, and a3 respectively, then the personalized association degree between user S and article A can be determined based on the sub-associations a1, a2, and a3. For example, the personalized association degree between a user and an article can be determined by statistical values ​​(such as average values) of multiple sub-associations corresponding to multiple categories to which the same article belongs.

[0151] Continue to refer to Figure 2 In step S206, the score for the i-th article is determined based on the relevance of the i-th search term and the i-th personalized relevance.

[0152] In an exemplary embodiment, in step S206-1, a first weight corresponding to the relevance of the i-th search term and a second weight corresponding to the i-th personalized relevance are obtained, wherein the first weight and the second weight are determined by cross-validation.

[0153] The first and second weights are used to balance the impact of two factors (search term relevance and personalized relevance) on the score of the i-th article. By setting these weights appropriately, search results can more accurately reflect user needs and preferences. An exemplary implementation of determining the first and second weights based on cross-validation includes the following.

[0154] S51. Set the value range for the first and second weights to be verified (denoted as W1 and W2 respectively), for example, from 0.1 to 0.9, with a step size of 0.1. Note that since the first and second weights both affect the article's score, the sum of the first and second weights equals 1. For example, a pair (W1, W2) can be represented as (0.1, 0.9), (0.2, 0.8), or (0.3, 0.7), etc.

[0155] S52. For each pair (W1, W2), perform the cross-validation process:

[0156] Specifically, multiple sets of sample user data are obtained. Each set of sample user data includes the user's profile information, search terms, and a recommendation list of multiple test articles for the user. For example, if user A's search term is B, the recommendation list for user A would be: Article X1, Article X2, and Article X3. Using the m-th set of sample user data and the n-th article, the similarity of the search term is determined. n And personalization correlation wight n Calculate the score for the nth article based on the current values ​​of (W1, W2). n =W1*similarity n +W2*weight n Therefore, based on the scores, a recommendation list for users in the m-th sample user data can be determined.

[0157] Next, we evaluate the quality of the articles recommended to the i-th sample user with values ​​(W1, W2). For example, evaluation metrics that can be used include Mean Average Precision (MAP). MAP primarily measures the average precision of a set of queries, where precision refers to the proportion of relevant items retrieved out of all retrieved items. A higher MAP indicates that relevant items rank higher in the results list for most queries; a lower MAP value indicates that relevant items rank lower, or that there are more irrelevant items retrieved, indicating poor system performance.

[0158] Taking the above sample user A as an example, if we take three pairs (W1, W2) as an example, they are specifically represented as follows: (W 11 W 21 ), (W12 W 22 ), (W 13 W 23 We can determine the article recommendation lists for sample user A under three pairs (W1, W2), for example: (article X1, article X4, article X5), (article X2, article X3, article X5), (article X1, article X2, article X3). Obviously, in this example (W1, W2) 13 W 23 The weight that yields the best recommendation results is 0.

[0159] S53. Record the performance index of each pair (W1,W2), and select the pair (W1,W2) that makes the recommendation effect the best (such as the highest MAP) as the first weight w1 and the second weight w2.

[0160] By determining the first weight w1 and the second weight w2 through the cross-validation method described above, the optimal weights for search term relevance and personalized relevance can be systematically determined, thereby improving the accuracy of search results and user satisfaction.

[0161] It is understandable that the first weight w1 and the second weight w2 are not limited to being determined by the cross-validation method described above. They can also be set according to the actual needs of the recommendation system. For example, if the current recommendation system requires that the recommended articles should better reflect the personal relevance, then the second weight w2 can be set to be greater than the first weight w1. For example, w2 can be set to 0.7 and w1 to 0.3, etc.

[0162] In step S206-2, a first weighted value is determined based on the first weight and the relevance of the i-th search term, and a second weighted value is determined based on the second weight and the i-th personalized relevance. In step S206-3, the sum of the first weighted value and the second weighted value is calculated to obtain the score for the i-th article. For example, the score for the i-th article is calculated according to formula (1). i .

[0163] score i = w1 * similarity i + w2 * weight i (1)

[0164] Among them, similarity i The relevance between the search term and the i-th article is represented as the relevance of the i-th search term; weight i The degree of relevance between the user profile information and the i-th article is represented as the i-th personalized relevance; w1 represents the first weight corresponding to the i-th search term relevance, and w2 represents the second weight corresponding to the i-th personalized relevance.

[0165] According to formula (1), the score corresponding to the article is determined through the embodiments of this application. It takes into account the correlation between the article's summary vector and the vector representation of the search term. Semantic understanding based on natural language processing technology is beneficial to improving the search engine's ability to understand the semantics of the query term, thereby providing more relevant search results. It also takes into account the correlation between the article's category and the search user's preferences, so that the search results can reflect the search user's personalized preferences, thereby improving search accuracy while providing a personalized search experience for each search user.

[0166] Continue to refer to Figure 2 In step S208, the multiple articles are sorted according to their respective scores to obtain search results.

[0167] In an exemplary embodiment, after determining the score corresponding to an article, multiple articles are sorted according to their scores, and the top-ranked summaries can be displayed to the user as search results.

[0168] In step S208-1, the display ranking of the i-th article is determined based on the scores of the N articles, where N is a positive integer and i is a positive integer not greater than N. For example, a higher article score indicates a higher similarity between the article and the search term and the user's profile, and therefore a higher ranking; conversely, a lower article score indicates a lower similarity between the article and the search term or the user's profile, and therefore a lower ranking.

[0169] For example, to facilitate users' access to targeted information, text classification algorithms (such as convolutional neural networks, recurrent neural networks, etc.) can be used to categorize search results, organizing the results according to different categories (such as news, blogs, forums, academic papers, etc.). Therefore, when determining article rankings, the categories corresponding to multiple articles can be determined first; then, for each category, the display ranking of the i-th article is determined based on its score.

[0170] In step S208-2, visualization elements about the i-th article are generated based on the structured summary of the i-th article or based on the i-th summary vector.

[0171] For example, Generative Adversarial Networks (GANs) or other generative models can be used to generate richer visualization elements based on the structured summary or summary vector of the article. Figure 8 This is a schematic diagram of a search result display interface provided in an embodiment of this application, with reference to... Figure 8The search results for the search term "Chongqing hotpot" are shown, including multiple generated images, such as image 11 with the "characteristic" being "numbing". In this embodiment, the search engine generates images that match the search result keywords, thereby improving the intuitiveness of information retrieval for users.

[0172] In step S208-3, at least the title of the i-th article and the visualization element are determined as the display content of the i-th article's display ranking.

[0173] For example, the search results include the display content corresponding to articles ranked in each category. Specifically, article A is displayed as the first ranked article, and the content about article A is displayed in the first position. The display content for each article includes the article title, URL, suggested summary, and keywords, and the classification results can also be visually presented by adding visualization elements such as maps, charts, and infographics.

[0174] In step S208-4, the search results are determined based on the display rankings of the multiple articles and the content displayed at the display rankings.

[0175] The content displayed in each ranking section is arranged sequentially to form the search results for the search term. For example, the hierarchical relationship between different objects can also be determined, and the search results can then be organized based on this hierarchical relationship. Figure 8 The webpage sample (demonstration, Demo) for searching "Chongqing hotpot" uses its basic categories, specialty foods, and seasonings and dipping sauces as primary categories, and further subdivides under these primary categories to create a mind map for this search term.

[0176] In an exemplary embodiment, after determining the search results based on the display rankings of the multiple articles and the content displayed at those rankings, the model of the target user's display device can also be obtained, and the display format of the search results can be determined based on the display device model. For example, the search engine utilizes a generative model to generate a personalized search results page layout based on the user's device and preferences. For instance, if the display device is a smartphone terminal with model number 'a', the search results page will automatically adapt to the screen size of model 'a', adopting an interface layout more suitable for touch operation, such as increasing the size and spacing of touch buttons to ensure clear and readable text. Simultaneously, content summaries and lightweight pages suitable for mobile reading are prioritized, reducing loading time and data consumption. For instance, if the display device is a high-resolution desktop monitor, the search engine will provide higher-resolution images and more detailed graphic elements, such as high-definition charts and infographics, controlling the search results page to display more columns and detailed information, making full use of the large screen space without appearing cluttered. Furthermore, video search results can be directly previewed as high-definition thumbnails or clips, enhancing the browsing experience. For example, when the display device is a smartwatch, given the limited screen size, the search engine will only provide the most concise information, such as brief answer summaries, voice search result feedback, or quick access to frequently used functions (such as navigation start point settings). The interface design is simplified while ensuring access to key information, and it also supports voice command operation, reducing the need for manual input. Therefore, this embodiment of the application customizes the display format of search results according to the target user's display device model. By dynamically layouting the search result display interface, it can effectively improve the user experience across platforms and devices, making information acquisition more convenient and efficient.

[0177] In the AI-based search solution provided in this application embodiment, the relevance between the search term and the i-th article is calculated by comparing the vector representation of the search term with the vector representation of the i-th article summary. It is evident that this application embodiment, based on natural language processing technology for semantic understanding, improves the search engine's ability to understand the semantics of query terms, thereby providing more relevant search results. The search server can provide more personalized search results based on the user's historical search records, browsing habits, location, and other information. For example, for the same search term, different users may have different focuses; the search engine can provide more accurate results based on the user's personalized information, effectively improving the targeting and personalization of search results. The search results provided in this application embodiment not only reflect the relevance between the search term and the article after semantic understanding based on natural language processing technology, but also reflect the personalized preferences of the search user, thereby improving search accuracy while providing a personalized search experience for each search user.

[0178] Furthermore, regarding the display of search results, embodiments of this application also provide search results including visual elements, making information retrieval more intuitive. For some complex queries, the search server can organize the results according to different categories, such as news, blogs, forums, academic papers, etc., so that users can more easily find the information they need.

[0179] The above text combined Figures 1 to 8 The present application describes in detail embodiments of the artificial intelligence-based search method, which are illustrated below in conjunction with... Figure 9 This application describes in detail embodiments of an AI-based search device.

[0180] Figure 9 A schematic diagram of the structure of an AI-based search device 900 provided in one embodiment of this application. (See reference) Figure 9 The AI-based search device 900 includes: a search term relevance determination module 910, a personalized relevance determination module 920, a score determination module 930, and a result generation module 940.

[0181] Specifically, the search term relevance determination module 910 is used to determine the relevance between the search term and the i-th article to be searched based on the search vector and the i-th summary vector, where the search vector is the vector representation of the search term, the i-th summary vector is the vector representation of the structured summary of the i-th article to be searched, and i is a positive integer; the personalized relevance determination module 920 is used to calculate the relevance between the profile information of the target user corresponding to the search term and the i-th article to obtain the i-th personalized relevance; the score determination module 930 is used to determine the score of the i-th article to be searched based on the i-th search term relevance and the i-th personalized relevance; and the result generation module 940 is used to sort the multiple articles according to the scores corresponding to the multiple articles to obtain the search results.

[0182] In an exemplary embodiment, based on the above scheme, the AI-based search device 900 further includes: a summary pool generation module; the summary pool generation module includes: a summary suggestion generation unit, a similarity calculation unit, an entity generation unit, a structuring unit, and a storage unit. The summary suggestion generation unit is used to: before the search term relevance determination module 910 determines the relevance between the search term and the i-th article based on the search vector and the i-th summary vector, input the i-th article into the pre-trained generation model, and obtain the i-th summary suggestion based on the generation processing of the generation model; the similarity calculation unit is used to: input the i-th article and the i-th summary suggestion into the pre-trained first encoder model, and obtain the generation accuracy of the i-th summary suggestion based on the semantic similarity calculation of the first encoder model; the entity generation unit is used to: input the i-th article and the i-th summary suggestion into the pre-trained first encoder model, and obtain the generation accuracy of the i-th summary suggestion based on the semantic similarity calculation of the first encoder model; the entity generation unit is used to: The i-th article is input into the pre-trained second encoder model. Based on the named entity recognition processing of the second encoder model, the i-th entity set is obtained. The first encoder model and the second encoder model share the basic encoding part. The structuring unit is used to: when the generation accuracy of the i-th summary suggestion is greater than the first threshold, split the i-th summary suggestion based on the i-th entity set, and label the split content to obtain the i-th structured summary. The storage unit is used to: store the i-th structured summary into the structured summary pool.

[0183] In an exemplary embodiment, based on the above scheme, the AI-based search device 900 further includes: a summary pool generation module; the summary pool generation module includes: a structured summary generation unit, a similarity calculation unit, and a storage unit. The structured summary generation unit is used to: before the search term relevance determination module 910 determines the relevance between the search term and the i-th article based on the search vector and the i-th summary vector, input the i-th article into a pre-trained generation model, and obtain the i-th structured summary based on the generation processing of the generation model; the similarity calculation unit is used to: input the i-th article and the i-th structured summary into a pre-trained first encoder model, and obtain the generation accuracy of the i-th structured summary based on the semantic similarity calculation of the first encoder model; and the storage unit is used to: store structured summaries with a generation accuracy greater than a second threshold in a structured summary pool; wherein the structured summaries in the structured summary pool are used to calculate the search term relevance.

[0184] In an exemplary embodiment, based on the above scheme, the storage unit is specifically used to: store the structured summary in a structured summary pool in a vertical manner according to the tags in the structured summary, so as to divide the structured summaries corresponding to multiple articles into multiple categories.

[0185] In an exemplary embodiment, based on the above scheme, the AI-based search device 900 further includes: a vertical category retrieval module, a fine-tuning module, and an update module; wherein, the vertical category retrieval module is used to: obtain a set of structured summaries belonging to the target category from the structured summary pool through vertical category retrieval; the fine-tuning module is used to: fine-tune the generation model through the set of structured summaries belonging to the target category, so as to generate a structured summary belonging to the target category through the fine-tuned generation model; the update module is used to: update the structured summaries belonging to the target category to the structured summary pool.

[0186] In an exemplary embodiment, based on the above scheme, the AI-based search device 900 further includes: a search vector generation module and a summary vector generation module; the search vector generation module is used to: before the search term relevance determination module 910 determines the relevance between the search term and the i-th article based on the search vector and the i-th summary vector, encode the search term using a machine reading comprehension MRC model to obtain the search vector; and the summary vector generation module is used to: encode the structured summary of the i-th article using the MRC model to obtain the search vector and the i-th summary vector.

[0187] In an exemplary embodiment, based on the above scheme, the AI-based search device 900 further includes a post-checking module; wherein the post-checking module is used to: input the search vector and the i-th summary vector into the MRC model to perform a post-check on the structured summary of the i-th article; wherein, if the post-checking result is that the correlation between the structured summary of the i-th article and the search term is less than a third threshold, the structured summary of the i-th article is regenerated or the structured summary of the i-th article is adjusted.

[0188] In an exemplary embodiment, based on the above scheme, the search vector generation module is specifically used to: expand the received original search terms to obtain an expanded search term set; and encode the search terms in the expanded search term set using the above MRC model to obtain the search vector.

[0189] In an exemplary embodiment, based on the above scheme, the AI-based search device 900 further includes: a summary filtering module; the summary filtering module is used to: collect multi-dimensional data related to the target user before the summary vector generation module encodes the structured summary of the i-th article through the MRC model to obtain the search vector and the i-th summary vector; perform feature extraction on the collected information to determine at least one of the following information about the target user: basic attributes, interests, and behavioral habits, to obtain the profile information of the target user; and filter all structured summaries in the structured summary pool according to the profile information of the target user corresponding to the search term to obtain a target structured summary with a correlation greater than a fourth threshold.

[0190] In an exemplary embodiment, based on the above scheme, the personalized correlation determination module 920 is specifically used to: determine at least one category to which the i-th article belongs; obtain the sub-correlation degree between each category in the at least one category and the profile information; and determine the i-th personalized correlation degree between the target user and the i-th article based on the sub-correlation degree corresponding to each category in the at least one category.

[0191] In an exemplary embodiment, based on the above scheme, the score determination module 930 is specifically used to: obtain the first weight corresponding to the relevance of the i-th search term and the second weight corresponding to the i-th personalized relevance, wherein the first weight and the second weight are determined by cross-validation; determine a first weighted value based on the first weight and the relevance of the i-th search term, and determine a second weighted value based on the second weight and the i-th personalized relevance; calculate the sum of the first weighted value and the second weighted value to obtain the score for the i-th article.

[0192] In an exemplary embodiment, based on the above scheme, the result generation module 940 is specifically used to: determine the display ranking of the i-th article based on the scores corresponding to the N articles, where N is a positive integer and i is a positive integer not greater than N; generate a visualization element about the i-th article based on the structured summary of the i-th article; determine at least the title of the i-th article and the visualization element as the display content at the display ranking of the i-th article; and determine the search results based on the display rankings corresponding to the multiple articles and the display content at the display rankings.

[0193] In an exemplary embodiment, based on the above scheme, the result generation module 940 is further specifically used to: determine the categories corresponding to the multiple articles respectively; and, for each category, determine the display ranking of the i-th article based on the score of the i-th article.

[0194] The aforementioned AI-based search device 900 further includes a display format determination module; the display format determination module is used to: after the result generation module 940 determines the search results based on the display rankings of the multiple articles and the content displayed at the display rankings, obtain the model of the target user's display device, and determine the display format of the search results based on the model of the display device.

[0195] It should be understood that the device embodiments and method embodiments can correspond to each other, and similar descriptions can be referred to the method embodiments. To avoid repetition, further details will not be provided here. Specifically, Figure 9 The apparatus shown can execute the above-described embodiments of the artificial intelligence-based search method, and for the sake of brevity, the method embodiments corresponding to the foregoing and other operations and / or functions of each module in the apparatus will not be described again here.

[0196] The apparatus of this application embodiment has been described above from the perspective of functional modules in conjunction with the accompanying drawings. It should be understood that this functional module can be implemented in hardware, in software instructions, or in a combination of hardware and software modules. Specifically, the steps of the method embodiments in this application can be completed by integrated logic circuits in the processor's hardware and / or by software instructions. The steps of the method disclosed in this application embodiment can be directly embodied as being executed by a hardware decoding processor, or by a combination of hardware and software modules in the decoding processor. Optionally, the software module can reside in a mature storage medium in the art, such as random access memory, flash memory, read-only memory, programmable read-only memory, electrically erasable programmable memory, registers, etc. This storage medium is located in memory, and the processor reads information from the memory and, in conjunction with its hardware, completes the steps in the above method embodiments.

[0197] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Figure 10 The electronic device can be used to perform the aforementioned artificial intelligence-based search method, and the electronic device can be the aforementioned network device.

[0198] like Figure 10 As shown, the electronic device 1000 may include:

[0199] The system includes a memory 1010 and a processor 1020. The memory 1010 stores the computer program 33 and transfers the program code 33 to the processor 1020. In other words, the processor 1020 can call and run the computer program 1030 from the memory 1010 to implement the methods in the embodiments of this application.

[0200] For example, the processor 1020 can be used to execute the steps in the above method according to the instructions in the computer program 1030.

[0201] In some embodiments of this application, the processor 1020 may include, but is not limited to:

[0202] General-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0203] In some embodiments of this application, the memory 1010 includes, but is not limited to:

[0204] Volatile memory and / or non-volatile memory. Non-volatile memory can be read-only memory (ROM), programmable read-only memory (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), which is used as an external cache. By way of example, but not limitation, many forms of RAM are available, such as Static RAM (SRAM), Dynamic RAM (DRAM), Synchronous DRAM (SDRAM), Double Data Rate SDRAM (DDR SDRAM), Enhanced Synchronous DRAM (ESDRAM), Synchronous Link DRAM (SLDRAM), and Direct Rambus RAM (DR RAM).

[0205] In some embodiments of this application, the computer program 1030 may be divided into one or more modules, which are stored in the memory 1010 and executed by the processor 1020 to complete the page recording method provided in this application. The one or more modules may be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program 1030 in the electronic device.

[0206] like Figure 10 As shown, the electronic device 1000 may further include:

[0207] Transceiver 1040, which can be connected to processor 1020 or memory 1010.

[0208] The processor 1020 can control the transceiver 1040 to communicate with other devices; specifically, it can send information or data to other devices or receive information or data sent by other devices. The transceiver 1040 may include a transmitter and a receiver. The transceiver 1040 may further include antennas, and the number of antennas may be one or more.

[0209] It should be understood that the various components in the electronic device 1000 are connected through a bus system, which includes a data bus, a power bus, a control bus, and a status signal bus.

[0210] According to one aspect of this application, a computer storage medium is provided that stores a computer program thereon, which, when executed by a computer, enables the computer to perform the methods of the above-described method embodiments. Alternatively, embodiments of this application also provide a computer program product containing instructions that, when executed by a computer, cause the computer to perform the methods of the above-described method embodiments.

[0211] According to another aspect of this application, a computer program product or computer program is provided, comprising computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the method described in the above-described method embodiments.

[0212] In other words, when implemented using software, it can be implemented wholly or partially in the form of a computer program product. This computer program product includes one or more computer instructions. When these computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., digital video disc (DVD)), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0213] Those skilled in the art will recognize that the modules and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0214] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of modules is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple modules or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or modules may be electrical, mechanical, or other forms.

[0215] The modules described as separate components may or may not be physically separate. The components shown as modules may or may not be physical modules; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this embodiment according to actual needs. For example, the functional modules in the various embodiments of this application may be integrated into one processing module, or each module may exist physically separately, or two or more modules may be integrated into one module.

[0216] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. An artificial intelligence-based search method, characterized by, The method comprises: determining an ith search term relevance between a search term and an ith article according to a search vector and an ith summary vector, wherein the search vector is a vector representation of the search term, the ith summary vector is a vector representation of a structured summary of the ith article, and i is a positive integer; calculating a relevance between portrait information of a target user corresponding to the search term and the ith article to obtain an ith personalized relevance; determining a score about the ith article according to the ith search term relevance and the ith personalized relevance; ranking a plurality of articles according to respective scores of the plurality of articles to obtain a search result.

2. The method of claim 1, wherein, Before the determining the ith search term relevance between the search term and the ith article according to the search vector and the ith summary vector, the method further comprises: inputting the ith article into a pre-trained generation model to obtain an ith summary suggestion based on a generation processing of the generation model; inputting the ith article and the ith summary suggestion into a pre-trained first encoder model to obtain a generation accuracy about the ith summary suggestion based on a semantic similarity calculation of the first encoder model; inputting the ith article into a pre-trained second encoder model to obtain an ith entity set based on a named entity recognition processing of the second encoder model, wherein the first encoder model and the second encoder model share a basic encoding part; in a case where the generation accuracy about the ith summary suggestion is greater than a first threshold, performing content splitting on the ith summary suggestion based on the ith entity set and performing label marking on the split content to obtain an ith structured summary; storing the ith structured summary into a structured summary pool; wherein structured summaries in the structured summary pool are used to calculate search term relevance.

3. The method of claim 1, wherein, Before the determining the ith search term relevance between the search term and the ith article according to the search vector and the ith summary vector, the method further comprises: inputting the ith article into a pre-trained generation model to obtain an ith structured summary based on a generation processing of the generation model; inputting the ith article and the ith structured summary into a pre-trained first encoder model to obtain a generation accuracy about the ith structured summary based on a semantic similarity calculation of the first encoder model; storing structured summaries with a generation accuracy about the ith structured summary greater than a second threshold into a structured summary pool; wherein structured summaries in the structured summary pool are used to calculate search term relevance.

4. The method according to claim 2 or 3, characterized in that, The storing into the structured summary pool comprises: storing the structured summaries into the structured summary pool in a vertical category manner according to labels in the structured summaries to divide structured summaries of a plurality of articles into a plurality of categories.

5. The method of claim 4, wherein, The method further comprises: obtaining a structured summary set belonging to a target category in the structured summary pool in a vertical category retrieval manner; fine-tune the generative model by a set of structured summaries belonging to the target category, to generate a structured summary belonging to the target category by the fine-tuned generative model; update the structured summary belonging to the target category to the structured summary pool.

6. The method according to any one of claims 1 to 3, characterized in that, Before the method determines the i-th search term relevance between the search term and the i-th article according to the search vector and the i-th summary vector, the method further comprises: encoding the search term by a machine reading comprehension (MRC) model to obtain the search vector; encoding the structured summary of the i-th article by the MRC model to obtain the search vector and the i-th summary vector.

7. The method of claim 6, wherein, The method further comprises: inputting the search vector and the i-th summary vector into the MRC model to perform a post-check on the structured summary of the i-th article; wherein, in the case that the post-check result is that the structured summary of the i-th article and the search term have a relevance less than a third threshold, re-generating the structured summary of the i-th article or adjusting the structured summary of the i-th article.

8. The method of claim 6, wherein, The encoding the search term by the MRC model to obtain the search vector comprises: extending the received original search term to obtain a set of extended search terms; encoding the search terms in the set of extended search terms by the MRC model to obtain the search vector.

9. The method of claim 6, wherein, Before the method encodes the structured summary of the i-th article by the MRC model to obtain the search vector and the i-th summary vector, the method further comprises: collecting multi-dimensional data related to a target user; extracting features from the collected information to determine at least one of the following information about the target user: basic attributes, interest preferences, and behavior habits, to obtain portrait information of the target user; filtering all structured summaries in the structured summary pool according to the portrait information of the target user corresponding to the search term, to obtain a target structured summary having a relevance greater than a fourth threshold with the portrait information.

10. The method according to any one of claims 1 to 3, characterized in that, The calculating the relevance between the portrait information of the target user corresponding to the search term and the i-th article to obtain an i-th personalized relevance comprises: determining at least one category to which the i-th article belongs; obtaining a sub-relevance between each category in the at least one category and the portrait information; determining an i-th personalized relevance between the target user and the i-th article according to the sub-relevance corresponding to each category in the at least one category.

11. The method according to any one of claims 1 to 3, characterized in that, The determining a score about the i-th article according to the i-th search term relevance and the i-th personalized relevance comprises: obtaining a first weight corresponding to the i-th search term relevance and a second weight corresponding to the i-th personalized relevance, wherein the first weight and the second weight are determined by a cross-validation method; determining a first weighted value according to the first weight and the i-th search term relevance, and determining a second weighted value according to the second weight and the i-th personalized relevance; computing a sum of the first weight value and the second weight value to obtain a score about the ith article.

12. The method according to any one of claims 1 to 3, characterized in that, The method further comprises: determining a display rank of the ith article according to scores respectively corresponding to the N articles, N being a positive integer and i being a positive integer not greater than N; generating a visual element about the ith article according to the structured abstract of the ith article; determining, as display content at the display rank of the ith article, at least a title of the ith article and the visual element; determining a search result according to the display ranks respectively corresponding to the articles and the display content at the display ranks.

13. The method of claim 12, wherein, The method further comprises: determining a category respectively corresponding to each of the articles; determining a display rank of the ith article according to the score of the ith article for each category; After the search result is determined according to the display ranks respectively corresponding to the articles and the display content at the display ranks, the method further comprises: obtaining a model of a display device of the target user and determining a display form of the search result according to the model of the display device.

14. An artificial intelligence, AI, based search apparatus, characterized by, The apparatus comprises: a search term relevance determination module configured to determine an ith search term relevance between a search term and an ith article to be searched according to a search vector and an ith abstract vector, wherein the search vector is a vector representation of the search term, the ith abstract vector is a vector representation of a structured abstract of the ith article to be searched, and i is a positive integer; a personalized relevance determination module configured to calculate a relevance between portrait information of a target user corresponding to the search term and the ith article to obtain an ith personalized relevance; a score determination module configured to determine a score about the ith article to be searched according to the ith search term relevance and the ith personalized relevance; a result generation module configured to sort the articles according to the scores respectively corresponding to the articles to obtain a search result.

15. An electronic device, comprising: comprises a processor and a memory; the memory is configured to store a computer program; the processor is configured to execute the computer program to implement the AI-based search method according to any one of claims 1 to 13.

16. A computer-readable storage medium, characterized in that, for storing a computer program; the computer program enables a computer to execute the AI-based search method according to any one of claims 1 to 13.

Images