Content search method and device, electronic equipment and storage medium

Abstract

This application relates to a content search method, apparatus, electronic device, and storage medium. The method includes: responding to a search request for a target object, using a generative large model to perform text correction processing on initial search terms carried in the search request to obtain corrected search terms; using the generative large model to search for target media content matching the corrected search terms from a preset vectorized knowledge base as first search context information, the preset vectorized knowledge base being used to store vectors of content vertical domain knowledge; guided by the first search context information, the generative large model performing search term update and rewriting processing on the corrected search terms to obtain corresponding rewritten search terms; and performing content search based on the rewritten search terms to obtain content search results. According to the technical solution provided in this application, the generative large model is made more lightweight, and the rewritten reference is more comprehensive, thereby making the search rewriting more accurate.

Description

Technical Field

[0001] This application relates to the field of computer technology, and in particular to a content search method, apparatus, electronic device, and storage medium. Background Technology

[0002] With the development of internet applications, users have increasingly higher demands for content search based on these applications. However, the increasing richness of content makes search results less accurate. Related technologies often expand search terms to include more words to retrieve more comprehensive content. However, many of these newly introduced words do not match the user's search intent. While this expands the search results from multiple sources, the lack of precision in term expansion leads to significant noise in the search results, resulting in poor accuracy. Summary of the Invention

[0003] This application provides a content search method, apparatus, electronic device, and storage medium to at least make generative large models more lightweight and to make the reference knowledge for search term rewriting more comprehensive, thereby improving the accuracy of search term rewriting. The technical solution of this application is as follows.

[0004] According to a first aspect of the embodiments of this application, a content search method is provided, including:

[0005] In response to a search request for a target object, a generative large model is used to perform text correction processing on the initial search terms carried in the search request to obtain the corrected search terms corresponding to the initial search terms after correction.

[0006] The generative large model is used to search for the target media content that matches the corrected search terms from a preset vectorized knowledge base as the first search context information. The preset vectorized knowledge base is used to store vectors of content vertical domain knowledge.

[0007] Guided by the first search context information, the generative large model performs search term update and rewriting processing on the corrected search terms to obtain the corresponding rewritten search terms;

[0008] Based on the rewritten search terms, a content search is performed to obtain the content search results.

[0009] According to a second aspect of the embodiments of this application, a content search device is provided, comprising:

[0010] The search term correction module is used to respond to the search request of the target object, and use a generative large model to perform text correction processing on the initial search terms carried in the search request to obtain the corrected search terms corresponding to the initial search terms after correction.

[0011] The first search context acquisition module is used to search for the target media content matching the corrected search term from the preset vectorized knowledge base using the generative large model as the first search context information. The preset vectorized knowledge base is used to store vectors of content vertical domain knowledge.

[0012] The search term update module is used by the generative large model to perform search term update and rewriting processing on the corrected search terms under the guidance of the first search context information, so as to obtain the corresponding rewritten search terms.

[0013] The content search module is used to perform content searches based on the rewritten search terms and obtain content search results.

[0014] In one possible implementation, the device further includes:

[0015] An object association information acquisition module is used to acquire object association information corresponding to the target object, wherein the object association information is at least one of the historical search information of the target object and the object description information of the target object;

[0016] The second search context acquisition module is used to predict the search intent of the corrected search term based on the object association information characterized by the generative big model, and obtain the second search context information.

[0017] The search term update module is further used to perform search term update and rewriting processing on the corrected search term under the guidance of the first search context information and / or the second search context information, so as to obtain the rewritten search term.

[0018] In one possible implementation, the device further includes:

[0019] The search key information extraction and rewriting module is also used to extract search key information related to the search intent of the corrected search term from the second search context information and the object association information, and to supplement or modify the corrected search term based on the search key information to obtain the initial rewritten search term;

[0020] The search term update module includes a rewriting unit, used to search for target media content matching the initial rewritten search term from a preset vectorized knowledge base using the generative large model as the first search context information.

[0021] In one possible implementation, the search term update module includes:

[0022] The first search term update unit is used to input the first search context information and / or the second search context information into the search term rewriting module of the generative big model. Under the guidance of the first search context information and / or the second search context information, the generative big model performs search term update and rewriting processing on the corrected search term to obtain the rewritten search term.

[0023] In one possible implementation, the search term update module includes:

[0024] The content clustering unit is used to cluster the target media content based on the first search context information to obtain content clustering information;

[0025] The search key information extraction unit is used to extract search key information related to the object association information and characterizing the search intent of the corrected search term from the second search context information;

[0026] The second search term update unit is used to supplement or modify the corrected search term based on the content clustering information and / or the search key information to obtain the rewritten search term.

[0027] In one possible implementation, the first search context acquisition module includes:

[0028] The search term preprocessing unit is used to preprocess the corrected search terms to obtain the text to be retrieved;

[0029] The first search context acquisition unit is used to search for target media content matching the text to be retrieved from a preset vectorized knowledge base using the generative large model as the first search context information.

[0030] In one possible implementation, the content search module includes:

[0031] The target vector matching unit is used to search for matching target vectors in the preset vectorized knowledge base based on the rewritten search terms.

[0032] An object association information acquisition unit is used to acquire multiple media contents corresponding to the target vector and object association information corresponding to the target object, wherein the object association information is at least one of the historical search information of the target object and the object description information of the target object;

[0033] The first content search result acquisition unit is used to select target media content from the multiple media contents as the content search result based on the object association information.

[0034] In one possible implementation, the content search module includes:

[0035] The first media content acquisition unit is used to search for matching first media content in the preset vectorized knowledge base based on the rewritten search terms.

[0036] The second media content acquisition unit is used to perform multi-way content retrieval and matching in the search content index based on the rewritten search terms to obtain the second media content;

[0037] The second content search result acquisition unit is used to sort the first media content and the second media content to obtain the content search result.

[0038] According to a third aspect of the embodiments of this application, an electronic device is provided, comprising: a processor; a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the method as described in any one of the first aspects above.

[0039] According to a fourth aspect of the present application, a computer-readable storage medium is provided, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform any of the methods described in the first aspect of the present application.

[0040] According to a fifth aspect of the embodiments of this application, a computer program product is provided, including computer instructions that, when executed by a processor, cause a computer to perform the method described in any one of the first aspects of the embodiments of this application.

[0041] The technical solutions provided by the embodiments of this application bring at least the following beneficial effects:

[0042] By using a generative large-scale model to perform text correction on the initial search terms carried in the search request, corrected search terms are obtained, making the semantics of the corrected search terms more accurate and thus more accurately and effectively expressing the search intent of the target object. Furthermore, the generative large-scale model searches for target media content matching the corrected search terms from a pre-defined vectorized knowledge base, which stores vectors of content vertical domain knowledge, as the first search context information. Guided by the first search context information, the generative large-scale model updates and rewrites the corrected search terms, obtaining corresponding rewritten search terms. This first search context information indicates the preference of the corrected search terms in the vertical sub-domain knowledge, thus guiding the generative large-scale model in updating and rewriting the corrected search terms. This ensures that the rewritten search terms match the preferences of the corrected search terms in the vertical sub-domain knowledge, thereby improving the accuracy of the rewritten search terms in expressing the search intent of the target object. This not only expands the search results of multi-path recall but also effectively reduces the noise of search results, resulting in better accuracy of content search results.

[0043] Furthermore, by setting up a pre-defined vectorized knowledge base, relevant information can be retrieved and matched in a vectorized manner, eliminating the need to input too much knowledge into the generative large model at once. Most knowledge can be carried by the vector database (i.e., the pre-defined vectorized knowledge base), solving the problem that the dialogue length of the current generative large model is limited by the token, and realizing the lightweighting of the generative large model. At the same time, it can make the latest knowledge available for application without retraining the basic language model, effectively reducing the cost of model deployment and application, and improving the scalability of search application scenarios.

[0044] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0045] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application, and do not constitute an undue limitation of this application.

[0046] Figure 1 This is a schematic diagram illustrating an application environment according to an exemplary embodiment.

[0047] Figure 2 This is a flowchart illustrating a content search method according to an exemplary embodiment.

[0048] Figure 3 This is a schematic diagram illustrating a content search process architecture according to an exemplary embodiment.

[0049] Figure 4 This is a flowchart illustrating a content search method according to an exemplary embodiment.

[0050] Figure 5 This is a schematic diagram illustrating the input and output of a generative large model according to an exemplary embodiment.

[0051] Figure 6 This is a block diagram illustrating a content search device according to an exemplary embodiment.

[0052] Figure 7 This is a block diagram illustrating an electronic device for content search according to an exemplary embodiment.

[0053] Figure 8 This is a block diagram illustrating an electronic device for content search based on an exemplary embodiment. Detailed Implementation

[0054] Various exemplary embodiments, features, and aspects of this application will now be described in detail with reference to the accompanying drawings. The same reference numerals in the drawings denote elements that have the same or similar functions. Although various aspects of the embodiments are shown in the drawings, they are not necessarily drawn to scale unless specifically indicated otherwise.

[0055] The term “exemplary” as used herein means “serving as an example, embodiment, or illustration.” Any embodiment illustrated herein as “exemplary” is not necessarily to be construed as superior to or better than other embodiments.

[0056] In this application embodiment, the terms "module" or "unit" refer to a computer program or part of a computer program that has a predetermined function and works with other related parts to achieve a predetermined goal, and can be implemented wholly or partially using software, hardware (such as processing circuitry or memory), or a combination thereof. Similarly, a processor (or multiple processors or memory) can be used to implement one or more modules or units. Furthermore, each module or unit can be part of an overall module or unit that includes the functionality of that module or unit.

[0057] Furthermore, to better illustrate this application, numerous specific details are provided in the following detailed embodiments. Those skilled in the art should understand that this application can be implemented without certain specific details. In some instances, methods, means, components, and circuits well-known to those skilled in the art have not been described in detail in order to highlight the main points of this application.

[0058] Artificial intelligence (AI) is the theory, methods, technology, and application systems that use digital computers or computers-controlled machines to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. AI software technology mainly includes computer vision, speech processing, natural language processing, and machine learning / deep learning.

[0059] In recent years, with the research and progress of artificial intelligence technology, artificial intelligence technology has been widely used in many fields. The solutions provided in this application involve technologies such as natural language processing, which are specifically illustrated through the following embodiments.

[0060] Please see Figure 1 , Figure 1 This diagram illustrates an application system according to an embodiment of this application. The application system can be used in the content search method of this application. Figure 1 As shown, the application system may include at least server 01 and terminal 02.

[0061] In this embodiment of the application, the server 01 can be used for content search processing. The server 01 may include an independent physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, CDN (Content Delivery Network), and big data and artificial intelligence platforms.

[0062] In this embodiment, the terminal 02 can be used to provide a front-end visual interface for content search, thereby enabling users to trigger search requests and display content search results. The terminal 02 may include physical devices such as smartphones, desktop computers, tablets, laptops, smart speakers, digital assistants, augmented reality (AR) / virtual reality (VR) devices, and smart wearable devices. Physical devices may also include software running on them, such as applications. In this embodiment, the operating system running on the terminal 02 may include, but is not limited to, Android, iOS, Linux, and Windows.

[0063] In addition, it should be noted that, Figure 1 The example shown is merely one application environment of the content search method provided in this application.

[0064] In the embodiments described in this specification, the terminal 02 and the server 01 can be directly or indirectly connected through wired or wireless communication, and this application does not limit this connection.

[0065] It should be noted that in the specific implementation of this application, user-related data is involved. When the following embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions.

[0066] Before introducing the method embodiments provided in this application, a brief introduction will be given on the application scenarios, related terms or nouns that may be involved in the method embodiments of this application, so as to facilitate the understanding of those skilled in the art.

[0067] LLM: Large Language Model (LLM) refers to a computer model capable of processing and generating natural language. It represents a significant advancement in the field of artificial intelligence and holds the promise of transforming the field through learned knowledge. LLMs can predict the next word or sentence by learning statistical patterns and semantic information from language data. As the input dataset and parameter space expand, the capabilities of LLMs also increase. It is used in various application areas such as robotics, machine learning, machine translation, speech recognition, and image processing, and can be referred to as a multimodal large language model.

[0068] Instruction Tuning refers to generating instructions individually for each task, fine-tuning them on several full-shot tasks, and then evaluating their generalization ability on a specific task (zero shot). It can usually be fine-tuned on a large set of NLP (Natural Language Processing) task datasets to enhance the language model's understanding ability by providing more explicit instructions so that the model can understand and make the correct responses.

[0069] Prompt tuning, or cue learning, is a type of learning method in machine learning that significantly improves the performance of a pre-trained language model by adding "cues" to the input as a form of information enhancement without significantly changing the structure and parameters of the pre-trained model. It can be seen as an instruction for the task and a reuse of the pre-trained objective. Essentially, it can be seen as an enhancement of parameter effectiveness training by generating separate prompt templates and then performing full-shot fine-tuning and evaluation on each task.

[0070] Figure 2 This is a flowchart illustrating a content search method according to an exemplary embodiment. For example... Figure 2 As shown, the method may include the following steps.

[0071] In step S201, in response to the search request for the target object, the generative large model is used to perform text correction processing on the initial search terms carried in the search request to obtain the corrected search terms corresponding to the initial search terms after correction.

[0072] In the embodiments described in this specification, the target object can refer to any object performing content search, and this object can be a user. The target object can trigger a search request by entering search information in the search interface, i.e., the target object's search request. Accordingly, in response to the target object's search request, a generative large model can be used to perform text correction processing on the initial search terms carried in the search request to obtain the corrected search terms corresponding to the corrected initial search terms. The initial search terms can be obtained based on search information, such as search speech, search terms, etc., and the initial search terms can be the text corresponding to the search information.

[0073] In one example, the generative large model can be a Large Language Model (LLM), which may include a text correction module for the aforementioned text correction processing. Optionally, this text correction module can be obtained by fine-tuning samples using correction data, for example, through SFT (Supervised Fine-Tun-ing) training. Exemplarily, the correction data sample may include a large number of correction samples accumulated using a preset correction method, as well as fine-tuned samples. These fine-tuned samples can be manually selected semantically incorrect query samples from the user's input query, which are then manually corrected in small amounts. For example, a correction data sample may be: search term - corrected search term, i.e., a data pair before and after correction.

[0074] Exemplarily, the above-mentioned preset error correction method can be implemented by at least one of the following methods. The at least one method may include, but is not limited to, a rule-based error correction algorithm, which can use predefined rules to detect and correct errors. For example, by comparing the input text with a dictionary, finding words not in the dictionary, and providing possible correct spelling suggestions; or, a statistical model error correction algorithm. Commonly used statistical models may include n-gram models and sequence-to-sequence models, etc.; or a machine learning-based error correction algorithm, which can use machine learning techniques to train a model so as to be able to judge whether there are errors in the input text and provide correction suggestions. For example, it may include a naive Bayes classifier, a support vector machine, etc. Or, a semantics-based error correction algorithm, which attempts to understand the semantic meaning of the text and judges whether there are errors through context analysis. For example, during the process of searching for content and channels in a social network, when the user inputs, it is very easy to have various errors in the Query, including but not limited to spelling errors, such as pinyin-related errors (same pronunciation, same syllable but different tones, similar syllables with the same tone, similar syllables with different tones), grammar errors, and semantic errors, etc. For example, easily misspelled words like "哈蜜" -> "哈密", "度假" -> "渡假", "配副眼睛" -> "配副眼镜", etc., can be effectively corrected to improve the accuracy of the search term.

[0075] In step S203, a generative large model is used to search for target media content matching the corrected search term from a preset vectorized knowledge base as the first search context information.

[0076] In the embodiments of this specification, a preset vectorized knowledge base can be used to store vectors of content vertical domain knowledge. This content vertical domain knowledge refers to content knowledge corresponding to a vertical domain. For example, the content vertical domain knowledge here can be vectors corresponding to web pages or documents in a vertical domain. The vertical domain can be a content type, such as sports or food; this application does not limit this. For example, after obtaining these documents or web pages, the data can be cleaned and processed, and some metadata, such as filenames, timestamps, chapters, and images, can be extracted and vectorized to obtain vectors of content vertical domain knowledge. Optionally, when the document is large, it can be further divided into text blocks. For example, it can be split according to the document's structure, based on chapters and paragraphs. If there are no document paragraphs, it can usually be divided into fixed text blocks (the segmentation size can be the single text length supported by the vectorization model, such as 512 or a related multiple), resulting in multiple text blocks. Thus, multiple text blocks can be vectorized to obtain vectors of content vertical domain knowledge. For subsequent user search queries, relevant information can be retrieved and matched using vectorization, eliminating the need to input excessive knowledge into the generative large model at once. Most knowledge can be carried by a vector database (i.e., a pre-defined vectorized knowledge base), solving the problem of the current generative large model's dialogue length being limited by tokens. At the same time, it allows the latest knowledge to be applied without retraining the basic language model, effectively reducing the cost of model deployment and application.

[0077] In an optional implementation, the aforementioned method of using a generative large model to search for target media content matching the corrected search terms from a preset vectorized knowledge base as the first search context information may include: preprocessing the corrected search terms to obtain the text to be retrieved. This preprocessing may include, but is not limited to, text cleaning and standardization, etc., and this application does not limit this. Furthermore, a generative large model can be used to search for target media content matching the text to be retrieved from a preset vectorized knowledge base as the first search context information. For example, the target media content may be the content corresponding to a matching document, webpage, etc. By preprocessing the corrected search terms before retrieving from the knowledge vector base, the accuracy of target media content retrieval and matching can be improved, thereby improving the precision of content search.

[0078] In step S205, the generative large model, guided by the first search context information, performs search term update and rewriting processing on the corrected search terms to obtain the corresponding rewritten search terms.

[0079] In the embodiments of this specification, the generative big data model, guided by the first search context information, performs search term update and rewriting processing on the corrected search terms to obtain the corresponding rewritten search terms. For example, the first search context information can be input into the generative big data model. This first search context information can indicate the preference of the corrected search terms in the vertical sub-domain knowledge, thereby guiding the generative big data model to perform the search term update and rewriting process on the corrected search terms, so that the obtained rewritten search terms can match the preference of the corrected search terms in the vertical sub-domain knowledge, thereby improving the accuracy of search term rewriting.

[0080] The generative large model may include a rewriting module, which can be trained through prompting based on rewritten samples. Each rewritten sample may include a sample search term and the vertical domain content matched by that search term. The tag corresponding to the rewritten sample can be the rewritten search term. Based on the rewritten samples and the tags corresponding to each rewritten sample, the rewriting module of the generative large model can be fine-tuned to obtain the rewriting module used for rewriting search terms. This application does not limit this. For example, if the sample search term is "basketball," the vertical domain content matched by the sample search term may include, but is not limited to, players, events, and sportswear related to the sports vertical domain corresponding to basketball. This application does not limit this.

[0081] In step S207, a content search is performed based on the rewritten search terms to obtain the content search results.

[0082] In this specification, rewritten search terms can be used for multi-channel content retrieval, thereby allowing the multi-channel retrieved media content to be rearranged to obtain content search results. For example, the content search results may include media content arranged in order.

[0083] For example, content search can be performed from a search content index and / or a preset vectorized knowledge base. That is, content search can be performed from the search content index, or from the preset vectorized knowledge base, or from both. Accordingly, when combining content search with the preset vectorized knowledge base, it is unnecessary to input excessive knowledge into the generative large model. Most knowledge can be carried by the vector database, reducing the deployment and inference costs of the generative large model itself. While ensuring the lightweight nature of the generative large model, the search content can be more comprehensive, and the illusion problem of the generative large model in specific domains can be mitigated, thus serving more scenarios. Based on this, the above-mentioned content search based on rewritten search terms to obtain content search results can include: searching for matching first media content in the preset vectorized knowledge base based on the rewritten search terms; and performing multi-path content retrieval and matching based on the rewritten search terms in the search content index to obtain second media content. The first and second media content can then be sorted to obtain the content search results.

[0084] Optionally, when searching for content in a preset vectorized knowledge base, the searched content can be filtered based on the object association information corresponding to the target object to obtain the content search results. In the case of a question-and-answer search scenario, there can be only one content search result, so users do not need to filter the search results again, making the presentation of question-and-answer search results more convenient and efficient. Based on this, the above-mentioned content search based on rewritten search terms to obtain content search results may include: searching for matching target vectors in the preset vectorized knowledge base based on the rewritten search terms, thereby obtaining multiple media contents corresponding to the target vectors, and obtaining object association information corresponding to the target object. This object association information can be at least one of the target object's historical search information and the target object's object description information. For example, historical search information can be extracted from the target object's historical search records, such as search terms, search domains, and search result viewing records; object description information can refer to information that can express the target object's content preferences, such as basic object information, etc., and this application does not limit these. Furthermore, target media content can be selected from multiple media content as the content search result based on object association information. For example, the target media content that matches the object association information can be selected from multiple media content as the content search result, such as selecting the media content with the highest matching degree (target media content) as the content search result.

[0085] This specification's embodiments fully utilize generative big models to process user query search requests within the main framework and process of search technology. It directly introduces query correction, intent recognition, and query intent expansion enhancement based on generative big models at the query input end. It leverages generative big models to process user query search requests and optimizes queries. For example, it incorporates historical search information of the target object, basic object information, and vertical domain knowledge retrieval as context for query expansion. When rewriting queries, it considers the coherence of the context, such as extracting information from the context to supplement and rewrite rewritten search terms, increasing the personalized representation capability and deep intent understanding of the query itself, and significantly improving the recall and matching accuracy of content search results.

[0086] By using a generative large-scale model to perform text correction on the initial search terms carried in the search request, corrected search terms are obtained, making the semantics of the corrected search terms more accurate and thus more accurately and effectively expressing the search intent of the target object. Furthermore, the generative large-scale model searches for target media content matching the corrected search terms from a pre-defined vectorized knowledge base, which stores vectors of content vertical domain knowledge, as the first search context information. Guided by the first search context information, the generative large-scale model updates and rewrites the corrected search terms, obtaining corresponding rewritten search terms. This first search context information indicates the preference of the corrected search terms in the vertical sub-domain knowledge, thus guiding the generative large-scale model in updating and rewriting the corrected search terms. This ensures that the rewritten search terms match the preferences of the corrected search terms in the vertical sub-domain knowledge, thereby improving the accuracy of the rewritten search terms in expressing the search intent of the target object. This not only expands the search results of multi-path recall but also effectively reduces the noise of the search results, resulting in better search result accuracy.

[0087] In one possible implementation, the search context can be determined by combining object association information and / or search history to characterize the user's personalized search intent, thereby improving the matching degree between content search and the target object. Based on this, the method may further include: obtaining object association information corresponding to the target object, which is at least one of the target object's historical search information and the target object's object description information. This allows for the prediction of the search intent of corrected search terms based on a generative large model, thus obtaining the second search context information.

[0088] Accordingly, guided by the first search context information, the aforementioned generative large model performs search term update and rewriting processing on the corrected search term to obtain the corresponding rewritten search term. That is, step S205 may include: the generative large model can perform search term update and rewriting processing on the corrected search term under the guidance of the first search context information and / or the second search context information to obtain the rewritten search term. For example, the first search context information and / or the second search context information, along with the corrected search term, can be input into the search term rewriting module of the generative large model for search term rewriting processing to obtain the rewritten search term. This search term rewriting module can be trained by prompting learning on the rewriting module of the generative large model based on a large number of rewritten data samples. The rewritten data samples may include sample search terms, a first sample context, and a second sample context; the labels corresponding to the rewritten data samples can be the rewritten sample search terms.

[0089] In one example, the aforementioned generative large model, guided by the first and / or second search context information, performs search term update and rewriting processing on the corrected search terms to obtain rewritten search terms. This may include: clustering the target media content based on the first search context information to obtain content clustering information. Exemplarily, a preset clustering algorithm can be used for this target media content clustering processing; this application does not limit the preset clustering algorithm. The content clustering information can be the content information ranked in the top M after clustering, such as content keywords, where M can be an integer greater than or equal to 1. Furthermore, search key information related to the search intent of the corrected search terms can be extracted from the second search context information, representing the object association information. That is, keywords can be extracted from the second search context information as the search key information, such as frequently used words or words with a high degree of matching with the search intent; this application does not limit this. Further, the corrected search terms can be supplemented or modified based on the content clustering information and / or search key information to obtain rewritten search terms. For example, the semantic understanding and text generation capabilities of generative large models can be utilized to extract information from content clustering information and / or search key information to supplement the corrected search terms, or to adjust and modify the corrected search terms based on content clustering information and / or search key information, thereby obtaining rewritten search terms.

[0090] In one possible implementation, the generative big model, guided by the first search context information and / or the second search context information, performs search term update and rewriting processing on the corrected search term to obtain the rewritten search term. This may include: inputting the first search context information and / or the second search context information into the search term rewriting module (or simply rewriting module) of the generative big model; and, guided by the first search context information and / or the second search context information, performing search term update and rewriting processing on the corrected search term to obtain the rewritten search term.

[0091] In an optional implementation, the method may further include: extracting key search information related to the search intent of the corrected search term from the second search context information and representing the object association information, and supplementing or modifying the corrected search term based on the key search information to obtain the initial rewritten search term. That is, before retrieving the preset vectorized knowledge base, the corrected search term can be rewritten based on the search intent represented by the object association information, so that the initial rewritten search term can effectively express the search intent of the target object, thereby improving the accuracy of vertical domain knowledge retrieval. Based on this, the above-mentioned use of a generative large model to search for target media content matching the corrected search term from the preset vectorized knowledge base as the first search context information may include: using a generative large model to search for target media content matching the initially rewritten search term from the preset vectorized knowledge base as the first search context information. By first rewriting the corrected search term based on the object association information, and then retrieving it from the vector base, the retrieval of vector knowledge can be more accurate, and the first search context information obtained from the retrieval has a higher degree of personalized matching with the target object, thereby improving the matching degree of subsequent content searches.

[0092] As an example, refer to Figure 4 and Figure 5In response to a search request for a target object, initial search terms can be extracted from the request. These initial search terms can then be input into a generative big data model. The model's error correction module performs text correction on the initial search terms, resulting in corrected search terms with more accurate semantics. Next, object association information corresponding to the target object can be obtained, such as historical search information and basic object information. This corrected search term and object association information can then be input into the generative big data model's search term rewriting module. This module can search a pre-defined vectorized knowledge base for domain knowledge matching the corrected search term as first search context information and understand the target object's search intent based on the object association information. This search intent serves as second search context information. Furthermore, guided by the first and second search context information, the search term rewriting module updates and rewrites the corrected search term, resulting in rewritten search terms. These rewritten search terms can then be input into the generative big data model's content search module, for example... Figure 3 As shown, this content search module can search for matching content indexes from the search content index system based on the search multi-path recall system to obtain the second media content. Furthermore, this content search module can search for matching first media content again from the preset vectorized knowledge base based on rewritten search terms. Next, you can refer to... Figure 3 The system can sort the first and second media content to obtain content search results. Alternatively, in scenarios where the initial search term is a question-and-answer search, the content search module can also output a search answer. For example, the search answer can be one of the first and second media content, or the content of the first and second media after fusion processing; this application does not limit this. Exemplarily, the content search results can be in the form of text, images, etc., and this application does not limit this.

[0093] Optionally, the generative big model can output rewritten search terms, and subsequent content search matching based on the rewritten search terms (such as the search matching of the first media content and / or the second media content mentioned above) can be performed based on search services outside the generative big model. For example, media content that matches the rewritten search terms can be found from the search content index system based on the search multi-path recall system and sorted to obtain content search results.

[0094] The embodiments described in this specification can enhance search scenarios and search business functions within social networks. They also seamlessly integrate with existing mainstream search system frameworks. Based on a large language model, they effectively correct and specifically expand user search queries. The large language model introduces external knowledge beyond the query, such as object association information and vertical domain knowledge, which is highly relevant to the query, thus enriching it and strengthening the understanding of the user's true intent. Furthermore, it fully utilizes users' historical queries and object basic characteristics and other preference information to obtain personalized search query enhancements, improving the enhanced matching ability of content search results, thereby increasing the personalization level of content search results and user click-through rates. It can seamlessly incorporate vertical domain knowledge bases and combine them with search queries. During a search, it can retrieve information related to the user's question from a large knowledge base, accessing more data points to provide richer and more accurate answers. For example, when a user asks about details of a historical event, it can retrieve relevant information from multiple sources, ensuring the comprehensiveness and accuracy of the answer. This also mitigates the model illusion problem to some extent and reduces the deployment and inference costs of the large model itself, enabling the content search processing of the large language model to serve more scenarios.

[0095] Figure 6 This is a block diagram illustrating a content search device according to an exemplary embodiment. (Refer to...) Figure 6 The device may include:

[0096] The search term correction module 601 is used to respond to the search request of the target object, and use a generative large model to perform text correction processing on the initial search term carried in the search request to obtain the corrected search term corresponding to the initial search term after correction.

[0097] The first search context acquisition module 603 is used to search for the target media content of the corrected search term matching from the preset vectorized knowledge base using the generative large model as the first search context information. The preset vectorized knowledge base is used to store vectors of content vertical domain knowledge.

[0098] The search term update module 605 is used to update and rewrite the corrected search term under the guidance of the first search context information to obtain the corresponding rewritten search term.

[0099] The content search module 607 is used to perform content search based on the rewritten search terms and obtain content search results.

[0100] By using a generative large-scale model to perform text correction on the initial search terms carried in the search request, corrected search terms are obtained, making the semantics of the corrected search terms more accurate and thus more accurately and effectively expressing the search intent of the target object. Furthermore, the generative large-scale model searches for target media content matching the corrected search terms from a pre-defined vectorized knowledge base, which stores vectors of content vertical domain knowledge, as the first search context information. Guided by the first search context information, the generative large-scale model updates and rewrites the corrected search terms, obtaining corresponding rewritten search terms. This first search context information indicates the preference of the corrected search terms in the vertical sub-domain knowledge, thus guiding the generative large-scale model in updating and rewriting the corrected search terms. This ensures that the rewritten search terms match the preferences of the corrected search terms in the vertical sub-domain knowledge, thereby improving the accuracy of the rewritten search terms in expressing the search intent of the target object. This not only expands the search results of multi-path recall but also effectively reduces the noise of the search results, resulting in better search result accuracy.

[0101] Furthermore, by setting up a pre-defined vectorized knowledge base, relevant information can be retrieved and matched in a vectorized manner, eliminating the need to input too much knowledge into the generative large model at once. Most knowledge can be carried by the vector database (i.e., the pre-defined vectorized knowledge base), solving the problem that the dialogue length of the current generative large model is limited by the token, and realizing the lightweighting of the generative large model. At the same time, it can make the latest knowledge available for application without retraining the basic language model, effectively reducing the cost of model deployment and application, and improving the scalability of search application scenarios.

[0102] In one possible implementation, the device may further include:

[0103] An object association information acquisition module is used to acquire object association information corresponding to the target object, wherein the object association information is at least one of the historical search information of the target object and the object description information of the target object;

[0104] The second search context acquisition module is used to predict the search intent of the corrected search term based on the object association information characterized by the generative big model, and obtain the second search context information.

[0105] The search term update module 605 is further configured to perform search term update and rewriting processing on the corrected search term under the guidance of the first search context information and / or the second search context information, so as to obtain the rewritten search term.

[0106] In one possible implementation, the device may further include:

[0107] The search key information extraction and rewriting module is also used to extract search key information related to the search intent of the corrected search term from the second search context information and the object association information, and to supplement or modify the corrected search term based on the search key information to obtain the initial rewritten search term;

[0108] The search term update module 605 may include: a rewriting unit, used to search for target media content matching the initial rewritten search term from a preset vectorized knowledge base using the generative large model as the first search context information.

[0109] In one possible implementation, the search term update module may include:

[0110] The first search term update unit is used to input the first search context information and / or the second search context information into the search term rewriting module of the generative big model. Under the guidance of the first search context information and / or the second search context information, the generative big model performs search term update and rewriting processing on the corrected search term to obtain the rewritten search term.

[0111] In one possible implementation, the search term update module 605 may include:

[0112] The content clustering unit is used to cluster the target media content based on the first search context information to obtain content clustering information;

[0113] The search key information extraction unit is used to extract search key information related to the object association information and characterizing the search intent of the corrected search term from the second search context information;

[0114] The second search term update unit is used to supplement or modify the corrected search term based on the content clustering information and / or the search key information to obtain the rewritten search term.

[0115] In one possible implementation, the first search context acquisition module 603 may include:

[0116] The search term preprocessing unit is used to preprocess the corrected search terms to obtain the text to be retrieved;

[0117] The first search context acquisition unit is used to search for target media content matching the text to be retrieved from a preset vectorized knowledge base using the generative large model as the first search context information.

[0118] In one possible implementation, the content search module 607 may include:

[0119] The target vector matching unit is used to search for matching target vectors in the preset vectorized knowledge base based on the rewritten search terms.

[0120] An object association information acquisition unit is used to acquire multiple media contents corresponding to the target vector and object association information corresponding to the target object, wherein the object association information is at least one of the historical search information of the target object and the object description information of the target object;

[0121] The first content search result acquisition unit is used to select target media content from the multiple media contents as the content search result based on the object association information.

[0122] In one possible implementation, the content search module 607 may include:

[0123] The first media content acquisition unit is used to search for matching first media content in the preset vectorized knowledge base based on the rewritten search terms.

[0124] The second media content acquisition unit is used to perform multi-way content retrieval and matching in the search content index based on the rewritten search terms to obtain the second media content;

[0125] The second content search result acquisition unit is used to sort the first media content and the second media content to obtain the content search result.

[0126] Regarding the apparatus in the above embodiments, the specific manner in which each module performs its operation has been described in detail in the embodiments related to the method, and will not be elaborated upon here.

[0127] Figure 7 This is a block diagram illustrating an electronic device for content search according to an exemplary embodiment. The electronic device may be a terminal, and its internal structure diagram may be as follows: Figure 7As shown, the electronic device includes a processor, memory, network interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When the computer program is executed by the processor, it implements a content search method. The display screen can be a liquid crystal display (LCD) or an e-ink display. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad mounted on the device's casing, or an external keyboard, touchpad, or mouse.

[0128] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0129] Figure 8 This is a block diagram illustrating an electronic device for content search based on an exemplary embodiment. The electronic device may be a server, and its internal structure diagram may be as follows: Figure 8 As shown, this electronic device includes a processor, memory, and a network interface connected via a system bus. The processor provides computing and control capabilities. The memory includes a non-volatile storage medium and internal memory. The non-volatile storage medium stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage medium. The network interface is used to communicate with external terminals via a network connection. When executed by the processor, the computer program implements a content search method.

[0130] Those skilled in the art will understand that Figure 8 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the electronic device to which the present application is applied. The specific electronic device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0131] In an exemplary embodiment, an electronic device is also provided, including: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the content search method as described in the embodiments of this application.

[0132] In an exemplary embodiment, a computer-readable storage medium is also provided, which, when executed by a processor of an electronic device, enables the electronic device to perform the content search method of the embodiments of this application. The computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, or optical data storage device, etc.

[0133] In an exemplary embodiment, a computer program product containing instructions is also provided, which, when run on a computer, causes the computer to perform the content search method described in the embodiments of this application.

[0134] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. This computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0135] Other embodiments of this application will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this application that follow the general principles of this application and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this application are indicated by the following claims.

[0136] It should be understood that this application is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this application is limited only by the appended claims.

Claims

1. A content search method, characterized in that, include: In response to a search request for a target object, a generative large model is used to perform text correction processing on the initial search terms carried in the search request to obtain the corrected search terms corresponding to the initial search terms after correction. The generative large model is used to search for the target media content that matches the corrected search terms from a preset vectorized knowledge base as the first search context information. The preset vectorized knowledge base is used to store vectors of content vertical domain knowledge. Guided by the first search context information, the generative large model performs search term update and rewriting processing on the corrected search terms to obtain the corresponding rewritten search terms; Based on the rewritten search terms, a content search is performed to obtain the content search results.

2. The method according to claim 1, characterized in that, The method further includes: Obtain object association information corresponding to the target object, wherein the object association information is at least one of the target object's historical search information and the target object's object description information; Based on the generative large model, the search intent of the corrected search term is predicted by the object association information to obtain the second search context information; Guided by the first search context information, the generative large model performs search term update and rewriting processing on the corrected search terms to obtain the corresponding rewritten search terms, including: Guided by the first search context information and / or the second search context information, the generative large model performs search term update and rewriting processing on the corrected search terms to obtain the rewritten search terms.

3. The method according to claim 1 or 2, characterized in that, The method further includes: Extract key search information related to the object association information to characterize the search intent of the corrected search term from the second search context information, and supplement or modify the corrected search term based on the key search information to obtain the initial rewritten search term; The step of using the generative large model to search for the target media content matching the corrected search term from the preset vectorized knowledge base as the first search context information includes: using the generative large model to search for the target media content matching the initial rewritten search term from the preset vectorized knowledge base as the first search context information.

4. The method according to claim 2, characterized in that, Guided by the first search context information and / or the second search context information, the generative large model performs search term update and rewriting processing on the corrected search terms to obtain the rewritten search terms, including: The first search context information and / or the second search context information are input into the search term rewriting module of the generative large model. Under the guidance of the first search context information and / or the second search context information, the generative large model performs search term update and rewriting processing on the corrected search term to obtain the rewritten search term.

5. The method according to claim 2, characterized in that, Guided by the first search context information and / or the second search context information, the generative large model performs search term update and rewriting processing on the corrected search terms to obtain the rewritten search terms, including: The first search context information is used to perform media content clustering to obtain content clustering information; Extract key search information related to the object association information from the second search context information to characterize the search intent of the corrected search term; Based on the content clustering information and / or the search key information, the corrected search terms are supplemented or modified to obtain the rewritten search terms.

6. The method according to claim 1, characterized in that, The step of using the generative large model to search for the target media content matching the corrected search term from the preset vectorized knowledge base as the first search context information includes: The corrected search terms are preprocessed to obtain the text to be retrieved; The generative large model is used to search for target media content that matches the text to be retrieved from a preset vectorized knowledge base as the first search context information.

7. The method according to claim 1 or 2, characterized in that, The content search based on the rewritten search terms, to obtain content search results, includes: Based on the rewritten search terms, a matching target vector is searched in the preset vectorized knowledge base; Obtain multiple media contents corresponding to the target vector and object association information corresponding to the target object, wherein the object association information is at least one of the historical search information of the target object and the object description information of the target object; The target media content is selected from the multiple media contents based on the object association information as the content search result.

8. The method according to claim 1, characterized in that, The content search based on the rewritten search terms, to obtain content search results, includes: Based on the rewritten search terms, search for matching first media content in the preset vectorized knowledge base; Based on the rewritten search terms, a multi-path content retrieval and matching process is performed in the search content index to obtain the second media content; The first media content and the second media content are sorted to obtain the content search results.

9. A content search device, characterized in that, include: The search term correction module is used to respond to the search request of the target object, and use a generative large model to perform text correction processing on the initial search terms carried in the search request to obtain the corrected search terms corresponding to the initial search terms after correction. The first search context acquisition module is used to search for the target media content matching the corrected search term from the preset vectorized knowledge base using the generative large model as the first search context information. The preset vectorized knowledge base is used to store vectors of content vertical domain knowledge. The search term update module is used by the generative large model to perform search term update and rewriting processing on the corrected search terms under the guidance of the first search context information, so as to obtain the corresponding rewritten search terms. The content search module is used to perform content searches based on the rewritten search terms and obtain content search results.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the content search method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is enabled to perform the content search method as described in any one of claims 1 to 8.

12. A computer program product, characterized in that, Includes computer instructions, which, when executed by a processor, cause the computer to perform the content search method as described in any one of claims 1 to 8.

Images