Content searching method and device and computer readable storage medium

Abstract

The embodiment of the invention discloses a content searching method and device and a computer readable storage medium. A search text is obtained; performing keyword processing on the search text by adopting the trained keyword rewriting model to obtain a search keyword corresponding to the search text and a search response type corresponding to the search keyword; when the search response type is a search engine response type, searching a search result matched with the search text through a search engine based on the search keyword; when the search response type is a dialogue model response type, based on the search keyword, reply information matched with the search text is generated through a dialogue generation model to serve as a search result; and sending the search result to a sender corresponding to the search text. In this way, the situation that the user inputs the search text and does not search the matched search content is avoided, unnecessary resource and time consumption caused when the search content is not searched is avoided, the content search efficiency is improved, and then the content search experience is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a content search method, device and computer-readable storage medium. Background Art

[0002] With the rapid development of computer technology, a huge amount of content has been generated on the Internet. Users can search for the required content in the huge amount of content through search engines. In the existing content search method, search keywords (query) are generally extracted from the search text input by the user, and then the search content matching the search text is searched according to the extracted query.

[0003] In the process of research and practice of existing technologies, it is found that in existing content search methods, it is very easy for users to enter search text but fail to find matching search content, resulting in low content search efficiency and a poor content search experience. Summary of the invention

[0004] The embodiments of the present application provide a content search method, device and computer-readable storage medium to avoid the situation where a user enters a search text but fails to find matching search content, and to avoid unnecessary resource and time consumption when the search content is not found, thereby improving content search efficiency and thereby improving content search experience.

[0005] The present application provides a content search method, including:

[0006] Get the search text;

[0007] Perform keyword processing on the search text using a trained keyword rewriting model to obtain a search keyword corresponding to the search text and a search response type corresponding to the search keyword;

[0008] When the search response type is a search engine response type, searching for search results matching the search text through a search engine based on the search keyword;

[0009] When the search response type is a dialogue model response type, based on the search keyword, a dialogue generation model is used to generate reply information matching the search text as a search result;

[0010] The search result is sent to a sender corresponding to the search text.

[0011] Accordingly, an embodiment of the present application provides a content search device, including:

[0012] An acquisition unit, used to acquire the search text;

[0013] A processing unit, configured to perform keyword processing on the search text using a trained keyword rewriting model to obtain a search keyword corresponding to the search text and a search response type corresponding to the search keyword;

[0014] A search unit, configured to search for search results matching the search text through a search engine based on the search keyword when the search response type is a search engine response type;

[0015] a generating unit, configured to generate, based on the search keyword, reply information matching the search text as a search result by using a dialogue generation model when the search response type is a dialogue model response type;

[0016] A sending unit is used to send the search result to a sender corresponding to the search text.

[0017] In one embodiment, the processing unit is used to:

[0018] The trained keyword rewriting model is used to rewrite the search text into a search keyword that matches the search input requirement of the search engine, and the search response type corresponding to the search keyword is predicted.

[0019] In one embodiment, the search unit is used to:

[0020] Based on the search keyword, searching for search content matching the search text through a search engine;

[0021] Extracting a summary of the search content to generate summary information corresponding to the search content;

[0022] The summary information and the search content are used as search results matching the search text.

[0023] In one embodiment, the content search device further includes:

[0024] A sample acquisition unit, used to acquire a training sample corresponding to a preset keyword rewriting model, wherein the training sample includes a search text sample, an expected search keyword corresponding to the search text sample, and an expected search response type;

[0025] A sample processing unit, configured to extract keywords and predict search response types from the search text sample using the preset keyword rewriting model, and obtain predicted search keywords and predicted search response types corresponding to the search text sample;

[0026] A loss calculation unit, configured to calculate loss information corresponding to the preset keyword rewriting model according to the expected search keyword and the predicted search keyword, as well as the expected search response type and the predicted search response type;

[0027] The model training unit is used to converge the preset keyword rewriting model based on the loss information to obtain a trained keyword rewriting model.

[0028] In one embodiment, the sample acquisition unit is used to:

[0029] Obtaining a training sample data set corresponding to a preset keyword rewriting model, and dividing the training sample data set into a simple training sample set and a difficult training sample set;

[0030] Obtaining a first weight parameter corresponding to each training round of the preset keyword rewriting model for the simple training sample set, and a second weight parameter corresponding to each training round of the preset keyword rewriting model for the difficult training sample set;

[0031] According to the first weight parameter and the second weight parameter, training samples of the preset keyword rewriting model in each training round are screened out from the simple training sample set and the difficult training sample set.

[0032] In one embodiment, the first weight parameter decreases with the number of training iterations of the preset keyword rewriting model, and the second weight parameter increases with the number of training iterations of the preset keyword rewriting model.

[0033] In one embodiment, the loss calculation unit is used to:

[0034] Calculating keyword prediction loss information corresponding to the preset keyword rewriting model according to the expected search keyword and the predicted search keyword;

[0035] Calculating response type prediction loss information corresponding to the preset keyword rewriting model according to the expected search response type and the predicted search response type;

[0036] The keyword prediction loss information and the response type prediction loss information are fused to obtain the loss information corresponding to the preset keyword rewriting model.

[0037] In addition, an embodiment of the present application further provides a computer-readable storage medium, which stores a plurality of instructions, and the instructions are suitable for a processor to load to execute the steps in any content search method provided in the embodiment of the present application.

[0038] In addition, an embodiment of the present application further provides a computer device, including a processor and a memory, wherein the memory stores an application program, and the processor is used to run the application program in the memory to implement the content search method provided in the embodiment of the present application.

[0039] The embodiment of the present application also provides a computer program product or a computer program, wherein the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs the steps in the content search method provided in the embodiment of the present application.

[0040] The embodiment of the present application obtains the search text; uses the trained keyword rewriting model to perform keyword processing on the search text to obtain the search keyword corresponding to the search text and the search response type corresponding to the search keyword; when the search response type is the search engine response type, based on the search keyword, the search results matching the search text are searched out through the search engine; when the search response type is the dialogue model response type, based on the search keyword, the dialogue generation model generates reply information matching the search text as the search result; and the search result is sent to the sender corresponding to the search text. In this way, when performing content search for the search text, the search keyword corresponding to the search text and the search response type corresponding to the search keyword can be predicted by using the trained keyword rewriting model, so as to directly determine whether it is necessary to search through the search engine or directly use the dialogue generation model to generate the reply information according to the search keyword corresponding to the search text and the search response type, so as to avoid the situation where the user enters the search text but fails to search for the matching search content, and the unnecessary resource and time consumption caused when the search content cannot be found, thereby improving the content search efficiency and thus improving the content search experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0042] Figure 1 This is a schematic diagram of an implementation scenario of a content search method provided in an embodiment of the present application;

[0043] Figure 2 It is a flowchart of a content search method provided in an embodiment of the present application;

[0044] Figure 3 It is a schematic diagram of the overall process of a content search method provided by an embodiment of the present application;

[0045] Figure 4 is a schematic diagram of the structure of a content search device provided in an embodiment of the present application;

[0046] Figure 5 It is a schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0048] The embodiments of the present application provide a content search method, device and computer-readable storage medium. The content search device can be integrated in a computer device, which can be a server or a terminal.

[0049] Among them, the server can be an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, network acceleration services (Content Delivery Network, CDN), and basic cloud computing services such as big data and artificial intelligence platforms. Terminals may include but are not limited to mobile phones, computers, intelligent voice interaction devices, smart home appliances, vehicle-mounted terminals, aircraft, etc. Terminals and servers can be directly or indirectly connected via wired or wireless communications, and this application does not limit this.

[0050] See also Figure 1 , taking the case where the content search device is integrated into a computer device, Figure 1A schematic diagram of an implementation scenario of the content search method provided in an embodiment of the present application, wherein the computer device can be a server or a terminal, and the computer device can obtain a search text; use a trained keyword rewriting model to perform keyword processing on the search text to obtain a search keyword corresponding to the search text, and a search response type corresponding to the search keyword; when the search response type is a search engine response type, based on the search keyword, search results matching the search text are searched through a search engine; when the search response type is a dialogue model response type, based on the search keyword, reply information matching the search text is generated as a search result through a dialogue generation model; and the search results are sent to a sender corresponding to the search text.

[0051] It should be noted that Figure 1 The implementation environment scenario diagram of the content search method shown is only an example. The implementation environment scenario of the content search method described in the embodiment of the present application is to more clearly illustrate the technical solution of the embodiment of the present application, and does not constitute a limitation on the technical solution provided by the embodiment of the present application. It is known to those skilled in the art that with the evolution of data processing and the emergence of new business scenarios, the technical solution provided by the present application is also applicable to similar technical problems.

[0052] The solutions provided by the embodiments of the present application are specifically described by the following embodiments. It should be noted that the description order of the following embodiments is not intended to limit the preferred order of the embodiments.

[0053] This embodiment will be described from the perspective of a content search device. The content search device may be integrated into a computer device, which may be a server. This application does not limit this.

[0054] See also Figure 2 , Figure 2 : is a flow chart of a content search method provided in an embodiment of the present application. The content search method includes:

[0055] In step 101, a search text is obtained.

[0056] The search text may be a text for searching. For example, a user may input the search text "How high is Mount Everest?" to search for and obtain the height of Mount Everest, or input the search text "How can I climb Mount Everest?" to search for and obtain the guide for climbing Mount Everest.

[0057] In step 102, the trained keyword rewriting model is used to perform keyword processing on the search text to obtain the search keyword corresponding to the search text and the search response type corresponding to the search keyword.

[0058] Among them, the trained keyword rewriting model can be a keyword rewriting model used to rewrite keywords for search text and predict the search response mode of the search text. The search keyword can be a keyword obtained by keyword processing the search text using the trained keyword rewriting model, which is used to provide the search engine for content search. The search response type can be a type of response to the search text, for example, it can include a search engine response type and a dialogue model response type. The search engine response type can be a response type that requires the use of a search engine to determine the search results of the search text. For example, when the search text is "How can I climb Mount Everest?", at this time, a search engine is needed to obtain related content. Therefore, the search response type can be a search engine response type, and the response mode of the search engine response type can be that a search engine needs to be used to search for content that matches the search text, thereby determining the search results of the search text. The dialogue model response type can be a response type that uses a dialogue generation model to generate search results for the search text without using a search engine. For example, when the search text is "write me a breakfast copy of more than 500 words", the search response type can be a dialogue model response type, and the response method of the dialogue generation model can be to directly generate a reply that matches the search text as the search result corresponding to the search text through a dialogue generation model without using a search engine.

[0059] Optional, please refer to Figure 3 , Figure 3 It is a schematic diagram of the overall process of a content search method provided by an embodiment of the present application. After obtaining the search text, the keyword rewriting model can be used to rewrite the search text to obtain the search keyword corresponding to the search text, and predict the search response type of the search text. Among them, the search keyword can be a keyword extracted from the search text. For example, when the search text is "Who is the founder of XX company?", the keyword rewriting model can be used to rewrite the search text into search keywords "XX company" and "founder", etc. It can also be a new keyword obtained by correcting, rewriting, expanding, converting, etc. the keywords extracted from the search text. For example, when the search text is "How to mosaic the face in the picture", the keyword rewriting model can be used to rewrite the search text into search keywords "picture" or "image", "face" or "face" and "mosaic", etc.

[0060] Among them, there are many ways to use a trained keyword rewriting model to perform keyword processing on the search text to obtain the search keywords corresponding to the search text, and the search response type corresponding to the search keywords. For example, a trained keyword rewriting model can be used to rewrite the search text into search keywords that match the search input requirements of the search engine, and predict the search response type corresponding to the search keywords.

[0061] The search input requirement may be a search engine's input requirement. For example, the search engine's input tends to be identified and searched in the form of multiple search keywords. Thus, the search text may be rewritten according to the search input requirement of the search engine. The search text that is easy for users to understand may be converted into search keywords that are easy for the search engine to understand, so that the search engine can quickly and accurately identify the semantics of the search text, thereby quickly and accurately searching for content that matches the semantics of the search text. For example, when the search text is "Which is the best coffee shop in XX City? How is the consumption? How many people are suitable for going?", the trained keyword rewriting model may be used to rewrite the search text into search keywords that match the search input requirement of the search engine, for example, it may be rewritten into search keywords "XX City Coffee Shop Guide" and the like. For another example, when the search text is "Who is the founder of XX Company?", the trained keyword rewriting model may be used to rewrite the search text into search keywords that match the search input requirement of the search engine, for example, it may be rewritten into search keywords "XX Company Founder" and the like.

[0062] In one embodiment, there may be multiple ways to train the keyword rewriting model. For example, a training sample corresponding to a preset keyword rewriting model may be obtained, the training sample including a search text sample, an expected search keyword corresponding to the search text sample, and an expected search response type; the preset keyword rewriting model is used to extract keywords and predict search response types for the search text sample to obtain predicted search keywords and predicted search response types corresponding to the search text sample; loss information corresponding to the preset keyword rewriting model is calculated based on the expected search keywords and predicted search keywords, as well as the expected search response type and predicted search response type; and the preset keyword rewriting model is converged based on the loss information to obtain a trained keyword rewriting model.

[0063] Among them, the preset keyword rewriting model can be an untrained model for keyword rewriting. Optionally, the preset keyword rewriting model can use a text processing model such as a generative pre-trained Transformer model (GPT) as a basic model structure. The training sample can be a sample for training the preset keyword rewriting model, the search text sample can be a search text as a sample, the expected search keyword and the expected search response type can be the training target of the preset keyword rewriting model, the expected search keyword can be a search keyword that the preset keyword rewriting model is expected to rewrite in the search text sample, the expected search response type can be a search response type corresponding to the search text sample predicted by the preset keyword rewriting model, the predicted search keyword can be a search keyword obtained by the preset keyword rewriting model performing keyword processing on the search text sample, and the predicted search response type can be a search response type predicted by the preset keyword rewriting model on the search text sample. The loss information can be information for measuring the gap between expected search keywords and predicted search keywords, and for measuring the gap between expected search response types and predicted search response types, and can be used to evaluate the gap between the keyword rewriting ability of a preset keyword rewriting model and the expected keyword rewriting ability.

[0064] Among them, there can be multiple ways to obtain training samples corresponding to the preset keyword rewriting model. For example, you can obtain a training sample data set corresponding to the preset keyword rewriting model, divide the training sample data set into a simple training sample set and a difficult training sample set; obtain a first weight parameter corresponding to the simple training sample set in each training round of the preset keyword rewriting model, and a second weight parameter corresponding to the difficult training sample set in each training round of the preset keyword rewriting model; according to the first weight parameter and the second weight parameter, filter out the training samples of the preset keyword rewriting model in each training round in the simple training sample set and the difficult training sample set.

[0065] The training sample data set may be a sample data set used to train the preset keyword rewriting model, the simple training sample set may be the whole of simple samples in the training sample data set, and the difficult training sample set may be the whole of difficult samples in the training sample data set. Optionally, the simple training sample set may be divided into simple samples of at least one degree of simplicity, and the difficult training sample set may be divided into difficult samples of at least one degree of difficulty. The training round (epoch) may refer to a process of inputting a training sample data set completely into the preset keyword rewriting model once, and the model parameters of the preset keyword rewriting model are iterated once during this epoch. The first weight parameter may be a weight parameter corresponding to the simple training sample set in each training round of the preset keyword rewriting model, and the first weight parameter is used to determine the number of simple samples in the training samples used in one training round. The second weight parameter may be a weight parameter corresponding to the difficult training sample set in each training round of the preset keyword rewriting model, and the second weight parameter is used to determine the number of difficult samples in the training samples used in one training round.

[0066] Optionally, the first weight parameter may decrease with the number of training iterations of the preset keyword rewriting model, and the second weight parameter may increase with the number of training iterations of the preset keyword rewriting model. For example, assuming that during the training of the preset keyword rewriting model, the number of training rounds epoch is set to 10 times, the first weight parameter may be set to 90% and the second weight parameter may be set to 10% in the first epoch. Then, 90% of the training samples used in the first training round are simple samples and 10% of the training samples are difficult samples. In the second epoch, the first weight parameter may be 80% and the second weight parameter may be 20%. Then, 80% of the training samples used in the second training round are simple samples and 20% of the training samples are difficult samples. In the third epoch, the first weight parameter may be 80% and the second weight parameter may be 20%. Then, 80% of the training samples used in the second training round are simple samples and 20% of the training samples are difficult samples. In poch, the first weight parameter can be 70% and the second weight parameter can be 30%. In the third training round, 70% of the training samples are simple samples and 30% of the training samples are difficult samples. In the fourth epoch, the first weight parameter can be 60% and the second weight parameter can be 40%. In the fourth training round, 60% of the training samples are simple samples and 40% of the training samples are difficult samples. In the fifth epoch, the first weight parameter can be 50% and the second weight parameter can be 50%. In the fifth training round, 60% of the training samples are simple samples and 40% of the training samples are difficult samples. In this case, 50% of the training samples are simple samples and 50% of the training samples are difficult samples. In the sixth epoch, the first weight parameter can be 40% and the second weight parameter can be 40%. Then, 40% of the training samples used in the sixth training round are simple samples and 60% of the training samples are difficult samples. In the seventh epoch, the first weight parameter can be 30% and the second weight parameter can be 70%. Then, 30% of the training samples used in the seventh training round are simple samples and 70% of the training samples are difficult samples. In the eighth epoch, the first weight parameter can be 40% and the second weight parameter can be 40%. The first weight parameter can be 20%, and the second weight parameter can be 80%. Then, 20% of the training samples used in the eighth training round are simple samples, and 80% of the training samples are difficult samples. In the ninth epoch, the first weight parameter can be 10%, and the second weight parameter can be 90%. Then, 10% of the training samples used in the ninth training round are simple samples, and 90% of the training samples are difficult samples. By the tenth epoch, the first weight parameter can be 0, and the second weight parameter can be 100%, that is, all the training samples used in the tenth training round are difficult samples, and so on.In this way, as the number of training iterations of the preset keyword rewriting model increases, the first weight parameter is decreased and the second weight parameter is increased, so that in the training rounds of the preset keyword rewriting model, the proportion of simple samples is gradually reduced and the proportion of difficult samples is gradually increased. In this way, the preset keyword rewriting model can be trained from simple to difficult, which makes it easy for the preset keyword rewriting model to find better local optimal parameters, and at the same time can speed up the training speed and improve the model training efficiency.

[0067] Optionally, in the preset keyword rewriting model, the rate of obtaining the trained keyword rewriting model can be accelerated by locking some model layers and only training some model layers. For example, assuming that there is a 48-layer decoder network in the preset keyword rewriting model, the first 36 layers of the decoder network can be frozen, and then only the last 12 layers of the decoder network can be trained, so that a trained keyword rewriting model can be quickly obtained.

[0068] Among them, there are many ways to calculate the loss information corresponding to the preset keyword rewriting model based on the expected search keywords and predicted search keywords, as well as the expected search response types and the predicted search response types. For example, the keyword prediction loss information corresponding to the preset keyword rewriting model can be calculated based on the expected search keywords and the predicted search keywords, and the response type prediction loss information corresponding to the preset keyword rewriting model can be calculated based on the expected search response type and the predicted search response type. The keyword prediction loss information and the response type prediction loss information are fused to obtain the loss information corresponding to the preset keyword rewriting model.

[0069] Among them, the keyword prediction loss information can be information measuring the gap between the expected search keyword and the predicted search keyword, and the response type prediction loss information can be information measuring the gap between the expected search response type and the predicted search response type.

[0070] Among them, there are many ways to fuse the keyword prediction loss information and the response type prediction loss information. For example, the keyword prediction loss information and the response type prediction loss information can be summed to obtain the loss information corresponding to the preset keyword rewriting model.

[0071] Optionally, the loss information may be calculated by calculating a loss function, and the loss function may be a softmax function, a cross entropy loss function, or other loss functions.

[0072] In step 103, when the search response type is a search engine response type, based on the search keyword, a search engine is used to search for search results that match the search text.

[0073] The search result may be a result obtained by searching based on the search text, and may include content searched through a search engine, or may include content such as a reply corresponding to the search text generated by a dialogue generation model.

[0074] Among them, there are many ways to search for search results that match the search text through a search engine based on search keywords. For example, based on the search keywords, you can search for search content that matches the search text through a search engine, extract a summary of the search content, generate summary information corresponding to the search content, and use the summary information and the search content as search results that match the search text.

[0075] The search content may be content matching the search text found through a search engine, and the summary information may be a content summary extracted from the search content, which may be used to summarize the content included in the search content.

[0076] There are many ways to extract the summary of the search content and generate summary information corresponding to the search content. For example, a trained first summary model can be used to extract key text from the search content to obtain the key text corresponding to the search content, and a trained second summary model can be used to generate a summary of the key text to obtain summary information corresponding to the search content.

[0077] Among them, the trained first summary model can be a summary model trained to extract key text from the search content, and can be an extractive summary model. For example, the bidirectional encoder representation from the transformer (Bidirectional Encoder Representations from Transformers, referred to as BERT) can be used as the basic model of the first summary model. The key text can be the key text in the search content, or it can be a more important sentence in the search content. The trained second summary model can be a summary model trained to generate a summary of the key text, and can be a generative summary model. For example, the generative pre-trained Transformer model (Generative Pre-Trained Transformer, referred to as GPT) can be used as the basic model of the second summary model.

[0078] Optionally, there may be multiple ways to train the first summary model and the second summary model. For example, a first sample pair of a first language type may be obtained, the sample pair of the first language type may be translated to obtain a second sample pair of the second language type, model training may be performed on a preset first summary model and a preset second summary model based on the first sample pair to obtain the trained first summary model and the second summary model in the first language type, and language transfer training may be performed on the trained first summary model and the second summary model in the first language type according to the second sample pair to obtain the trained first summary model and the trained second summary model in the second language type.

[0079] The first language type may be the language type of the acquired data set, for example, it may be the English type, the data set may be a sample set for training the first summary model and the second summary model, the first sample pair may be a sample pair in the data set of the first language type, that is, a sample pair of the first language type corresponding to the first summary model and the second summary model. The second language type may be the same as the language type corresponding to the search text, for example, it may be the Chinese type. The second sample pair may be a sample pair of the second language type. The trained first summary model and second summary model in the first language type may be the first summary model and second summary model obtained by training with the sample pair of the first language type, and the first summary model and the second summary model at this time may extract and generate a summary of the search content corresponding to the search text of the first language type.

[0080] There are many ways to translate the sample pairs of the first language type to obtain the second sample pairs of the second language type. For example, some sample pairs of the sample pairs of the first language type can be translated to obtain the second sample pairs of the second language type. For example, assuming that the number of first sample pairs of the first language type is 1.8 million, 170,000 sample pairs can be selected from the first sample pairs of the first language type and translated into the second language type to obtain the second sample pairs of the second language type.

[0081] There are multiple ways to perform model training on the preset first summary model and the preset second summary model based on the first sample to obtain the trained first summary model and the second summary model under the first language type. For example, the first sample pair may include a text sample of the first language type, a content sample matched by the text sample, and target summary information. The text sample may be a query, a keyword, or a text. The content sample may be content matched by the text sample that can be searched by a search engine. In this way, the preset first summary model may be used to extract features from the sentence text in the content sample to obtain text features corresponding to the sentence text. Based on the text features, sample key texts are screened out from the sentence text. The preset second summary model is used to generate a summary of the sample key text to obtain sample summary information matching the text sample. Based on the sample summary information, the text sample, and the target summary information, the summary generation loss information corresponding to the preset first summary model and the preset second summary model is determined. Based on the summary generation loss information, the preset first summary model and the preset second summary model are converged to obtain the trained first summary model and the second summary model under the first language type.

[0082] Among them, the target summary information can be the summary information expected to be extracted from the content sample, that is, it can be used as a label in the model training process. The preset first summary model can be a preset untrained first summary model, the sentence text can be a sentence in the content sample, the text feature can be a feature that characterizes the sentence text, the sample key text can be a sentence text with a high degree of importance screened out from the sentence text, the preset second summary model can be a preset untrained second summary model, and the sample summary information can be the summary information generated in the content sample. The summary generation loss information can be information that measures the gap between the summary generation capabilities of the preset first summary model and the preset second summary model and the expected summary generation capabilities, that is, it can be information that measures the gap between the sample summary information extracted from the content sample by the preset first summary model and the preset second summary model and the expected target summary information of the content sample.

[0083] There are multiple ways to determine the summary generation loss information corresponding to the preset first summary model and the preset second summary model based on the sample summary information, the text sample and the target summary information. For example, the similarity between the sample summary information and the text sample and the target summary information can be calculated, and the summary generation loss information corresponding to the preset first summary model and the preset second summary model can be calculated based on the similarity. For example, a cross entropy loss function (Softmax function) can be used to calculate the summary generation loss information corresponding to the preset first summary model and the preset second summary model based on the similarity between the sample summary information and the text sample and the target summary information.

[0084] After the preset first summary model and the preset second summary model are trained based on the first sample, language transfer training can be performed on the trained first summary model and the second summary model in the first language type according to the second sample pair to obtain the trained first summary model and the trained second summary model in the second language type. There are multiple ways to perform language transfer training on the trained first summary model and the second summary model in the first language type according to the second sample pair. For example, the second sample pair may include a text sample in the second language type, and the text sample matches a content sample and target summary information in the second language type. Therefore, in the second stage training of domain transfer, the first summary model trained in the first language type can be used to extract features of the sentence text in the content sample of the second sample pair to obtain text features corresponding to the sentence text in the second language type, and the sample key text of the second language type is screened out from the sentence text based on the text features of the second language type. The second summary model trained in the first language type is used to generate a summary of the sample key text of the second language type to obtain sample summary information of the second language type that matches the text sample, and the loss of the first summary model trained in the first language type and the preset second summary model in the second language type corresponding to the sample summary information, the text sample and the target summary information of the second language type is determined, and the first summary model and the second summary model trained in the first language type are converged based on the loss in the second language type to obtain the trained first summary model and the trained second summary model in the second language type.

[0085] Since most of the existing data sets are English data sets, in order to obtain the trained first summary model and the second summary model that match the currently required language type (such as Chinese type), a two-stage model training method can be used to train the first summary model and the second summary model. Specifically, in the first stage of training, a large number of sample pairs of the first language type (English type) can be used to train the preset first summary model and the second summary model to obtain a trained summary model with good summary generation ability for the search content of the first language type, so that in the second stage of training, the sample pairs of the second language type with a small amount of data can be used to train the trained first summary model and the second summary model of the first language type, so that the trained summary model with good summary generation ability for the search content of the second language type can be obtained. In this way, when the number of sample pairs of the second language type is small or missing, the first summary model and the second summary model can be trained in two stages by using the sample pairs of the first language type and the sample pairs of the second language type obtained by translating the sample pairs of the first language type, so that the first summary model and the second summary model with good summary generation ability suitable for the second language type can be obtained, thereby improving the model training efficiency and thus improving the summary generation efficiency.

[0086] In step 104, when the search response type is a dialogue model response type, based on the search keyword, reply information matching the search text is generated as a search result through a dialogue generation model.

[0087] Among them, the dialogue generation model can be a model that generates reply information corresponding to the search text in the form of a dialogue. The dialogue generation model can use a generative pre-trained model such as GPT as a basic model.

[0088] Please continue to refer to Figure 3 When the search response type is a dialogue model response type, the reply information matching the search text can be generated by the dialogue generation model based on the search keywords, so that the reply information can be used as the search result, avoiding the situation where the user enters the search text but fails to find the matching search content, and the unnecessary resource and time consumption when the search content cannot be found, thereby improving the content search efficiency. For example, assuming that the search text is "write me a breakfast copy of less than 100 words", the dialogue generation model can be used to generate a reply information containing a breakfast copy of less than 100 words based on the search keyword "breakfast copy of less than 100 words" corresponding to the search text. For example, the reply information can include "I have carefully written a breakfast copy of less than 100 words for you. Please refer to it and look forward to your feedback: a year's plan starts with spring, a day's plan starts with the morning, and what I look forward to most in the morning is a delicious and nutritious breakfast..." and other reply information.

[0089] In step 105, the search results are sent to a sender corresponding to the search text.

[0090] The sender may be a terminal or a client that sends the search text, and the sender receives the search results to display the search results in a search result page corresponding to the terminal or the client.

[0091] In one embodiment, there may be multiple ways to display search results in the search results page of the sender. For example, a conversational search page may be displayed in the sender, and in response to a search text input operation for the conversational search page, the search text may be obtained and the conversational search results page may be displayed.

[0092] The conversational search page may be a page for implementing conversational search, and conversational search may refer to a scenario in which search triggering and search result feedback are implemented in the form of a conversation. The conversational search page may provide a conversational search function, and the conversational search function may refer to a function for implementing search triggering and search result feedback in the form of a conversation. The search text input operation may be an operation for inputting a search text, for example, an operation triggered by a user inputting a search text in an input box provided by the conversational search page. The search text may be information to be searched, may be information of a text type input by a user, may be information of a voice type input by a user, etc. For example, when a user needs to search for a good coffee shop nearby, a search text such as "recommendation of coffee shops in XX area" may be input in the conversational search page. The conversational search result page may be a page for displaying search results corresponding to the search text in a conversational manner, and the conversational search result page may include reply information corresponding to the search text, and the reply information may be a reply to the search text generated according to the searched search results, and the reply information may include identification information of the search content searched based on the search text, and conversation reply information for the search text, and the conversation reply information is generated based on summary information extracted from the search content.

[0093] The search content may be content that a search engine searches for based on the search text, and the content may be articles, notes, pictures, videos, and other types of content. The identification information may be information that identifies a search content, may indicate a search content, and may also enable viewing of the search content by triggering the identification information. For example, the identification information may be a mark of a link corresponding to the search content, and identification information carrying a link corresponding to the search content may be generated in the form of a hyperlink. The identification information may use the title or summary of the search content as a carrier of the link to the search content, and the user may click on the identification information in the form of a link corresponding to the search content to jump to the details of the search content for viewing. The dialogue reply information may be information that replies to the search text in the form of a dialogue based on summary information extracted from the search content, and the summary information may be a summary of the content extracted from the search content corresponding to the search text. For example, when the search text is "recommendations for coffee shops in XX area", the dialogue reply information may be "There are many delicious coffee shops in XX area. For example, there is a XX coffee shop on XX Road, which is closest to you. This shop has many repeat customers. Their coffee beans are all produced from XX coffee shops in XX area and are rich in flavor. In addition, there is a XX coffee shop on XX Road. This shop has many people queuing up every day and uses the latest XX method to make coffee..." etc. Specific dialogue reply information may be generated according to actual needs and actual search conditions. In this way, users can intuitively and conveniently obtain search results that match the input search text according to the concise and clear search results indicated in the dialogue reply information, so that they can quickly obtain the required information in the search results, avoiding the need to check each search content to find information that meets the user's needs in the prior art, improving content search efficiency, and thus improving the user's search experience.

[0094] Optionally, after the conversational search result page is displayed, in response to a detail viewing operation for the target identification information in the identification information, a content detail page of the target search content corresponding to the target identification information may be displayed.

[0095] The target identification information may be the identification information selected for detailed viewing in the identification information, the detailed viewing operation may be an operation for viewing the detailed content of the search content, the target search content may be the search content corresponding to the target identification in detail, and the content details page may include the detailed content of the target search content. For example, in the area displaying the identification information of the search content, the user may determine the target identification information in the identification information of the search content according to the information of interest, and may perform a trigger operation such as clicking on the target identification information, thereby triggering a detailed viewing operation for the target identification information, and displaying the content details page of the target search content corresponding to the target identification information. Optionally, the content details page of the target search content may be directly opened, or a new dialogue message may be generated in the dialogue search result page in the form of a dialogue, and the dialogue information may include the content details of the target search content, etc.

[0096] As can be seen from the above, the embodiment of the present application obtains the search text; uses the trained keyword rewriting model to perform keyword processing on the search text to obtain the search keyword corresponding to the search text, and the search response type corresponding to the search keyword; when the search response type is the search engine response type, based on the search keyword, the search results matching the search text are searched out through the search engine; when the search response type is the dialogue model response type, based on the search keyword, the dialogue generation model generates reply information matching the search text as the search result; and the search result is sent to the sender corresponding to the search text. In this way, when performing content search for the search text, the search keyword corresponding to the search text and the search response type corresponding to the search keyword can be predicted by using the trained keyword rewriting model, so as to directly determine whether it is necessary to search through the search engine or directly use the dialogue generation model to generate the reply information according to the search keyword corresponding to the search text and the search response type, so as to avoid the situation where the user enters the search text but fails to search for the matching search content, and the unnecessary resource and time consumption caused when the search content cannot be found, thereby improving the content search efficiency and thus improving the content search experience.

[0097] In order to better implement the above method, an embodiment of the present invention further provides a content search device, which can be integrated in a computer device, and the computer device can be a server.

[0098] For example, Figure 4 As shown, it is a schematic diagram of the structure of the content search device provided in an embodiment of the present application. The content search device may include an acquisition unit 201, a processing unit 202, a search unit 203, a generation unit 204 and a sending unit 205, as follows:

[0099] An acquisition unit 201 is used to acquire a search text;

[0100] The processing unit 202 is used to perform keyword processing on the search text using the trained keyword rewriting model to obtain the search keyword corresponding to the search text and the search response type corresponding to the search keyword;

[0101] A search unit 203, configured to search for search results matching the search text through a search engine based on the search keyword when the search response type is a search engine response type;

[0102] A generating unit 204, configured to generate, based on the search keyword, reply information matching the search text as a search result by using a dialogue generation model when the search response type is a dialogue model response type;

[0103] The sending unit 205 is used to send the search result to the sender corresponding to the search text.

[0104] In one embodiment, the processing unit 202 is configured to:

[0105] The trained keyword rewriting model is used to rewrite the search text into search keywords that match the search engine's search input requirements, and predict the search response type corresponding to the search keywords.

[0106] In one embodiment, the search unit 203 is configured to:

[0107] Based on the search keywords, search for search content matching the search text through the search engine;

[0108] Extracting a summary of the search content and generating summary information corresponding to the search content;

[0109] The summary information and search results are included as search results that match the search text.

[0110] In one embodiment, the content search device further includes:

[0111] A sample acquisition unit, used to acquire a training sample corresponding to a preset keyword rewriting model, wherein the training sample includes a search text sample, an expected search keyword corresponding to the search text sample, and an expected search response type;

[0112] A sample processing unit, used to extract keywords and predict search response types from search text samples using a preset keyword rewriting model, and obtain predicted search keywords and predicted search response types corresponding to the search text samples;

[0113] A loss calculation unit, used to calculate loss information corresponding to a preset keyword rewriting model according to expected search keywords and predicted search keywords, and expected search response types and predicted search response types;

[0114] The model training unit is used to converge the preset keyword rewriting model based on the loss information to obtain a trained keyword rewriting model.

[0115] In one embodiment, the sample acquisition unit is used to:

[0116] Obtain a training sample data set corresponding to a preset keyword rewriting model, and divide the training sample data set into a simple training sample set and a difficult training sample set;

[0117] Obtaining a first weight parameter corresponding to each training round of the preset keyword rewriting model for the simple training sample set, and a second weight parameter corresponding to each training round of the preset keyword rewriting model for the difficult training sample set;

[0118] According to the first weight parameter and the second weight parameter, training samples of the preset keyword rewriting model in each training round are screened out from the simple training sample set and the difficult training sample set.

[0119] In one embodiment, the first weight parameter decreases with the number of training iterations of the preset keyword rewriting model, and the second weight parameter increases with the number of training iterations of the preset keyword rewriting model.

[0120] In one embodiment, the loss calculation unit is used to:

[0121] Calculate keyword prediction loss information corresponding to a preset keyword rewriting model based on expected search keywords and predicted search keywords;

[0122] Calculate the response type prediction loss information corresponding to the preset keyword rewriting model according to the expected search response type and the predicted search response type;

[0123] The keyword prediction loss information and the response type prediction loss information are fused to obtain the loss information corresponding to the preset keyword rewriting model.

[0124] In specific implementation, the above units can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units can refer to the previous method embodiments, which will not be repeated here.

[0125] As can be seen from the above, the embodiment of the present application obtains the search text through the acquisition unit 201; the processing unit 202 uses the trained keyword rewriting model to perform keyword processing on the search text to obtain the search keyword corresponding to the search text and the search response type corresponding to the search keyword; the search unit 203 searches for the search results matching the search text through the search engine based on the search keyword when the search response type is the search engine response type; the generation unit 204 generates the reply information matching the search text through the dialogue generation model based on the search keyword when the search response type is the dialogue model response type; the sending unit 205 sends the search results to the sender corresponding to the search text. In this way, when performing content search for the search text, the search keyword corresponding to the search text and the search response type corresponding to the search keyword can be predicted by using the trained keyword rewriting model, so as to directly determine whether to search through the search engine or directly use the dialogue generation model to generate the reply information according to the search keyword corresponding to the search text and the search response type, so as to avoid the situation where the user enters the search text but fails to search for the matching search content, and the unnecessary resource and time consumption caused when the search content cannot be searched, thereby improving the content search efficiency and thus improving the content search experience.

[0126] The present application also provides a computer device, such as Figure 5 As shown, it shows a schematic diagram of the structure of a computer device involved in an embodiment of the present application, and the computer device may be a server, specifically:

[0127] The computer device may include one or more processing core processors 301, one or more computer readable storage media memories 302, a power supply 303, an input unit 304 and other components. Those skilled in the art will appreciate that Figure 5 The computer device structure shown in the figure does not constitute a limitation on the computer device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0128] The processor 301 is the control center of the computer device. It uses various interfaces and lines to connect various parts of the entire computer device. It executes various functions of the computer device and processes data by running or executing software programs and / or modules stored in the memory 302, and calling data stored in the memory 302. Optionally, the processor 301 may include one or more processing cores; preferably, the processor 301 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, and the modem processor mainly processes wireless communications. It is understandable that the above-mentioned modem processor may not be integrated into the processor 301.

[0129] The memory 302 can be used to store software programs and modules. The processor 301 executes various functional applications and data processing by running the software programs and modules stored in the memory 302. The memory 302 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the computer device, etc. In addition, the memory 302 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 302 may also include a memory controller to provide the processor 301 with access to the memory 302.

[0130] The computer device also includes a power supply 303 for supplying power to various components. Preferably, the power supply 303 can be logically connected to the processor 301 through a power management system, so as to manage charging, discharging, and power consumption through the power management system. The power supply 303 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0131] The computer device may further include an input unit 304, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0132] Although not shown, the computer device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 301 in the computer device will load the executable files corresponding to the processes of one or more application programs into the memory 302 according to the following instructions, and the processor 301 will run the application programs stored in the memory 302, thereby realizing various functions, as follows:

[0133] Obtain the search text; perform keyword processing on the search text using a trained keyword rewriting model to obtain the search keyword corresponding to the search text and the search response type corresponding to the search keyword; when the search response type is a search engine response type, search for search results that match the search text through a search engine based on the search keyword; when the search response type is a dialogue model response type, generate reply information that matches the search text as a search result based on the search keyword through a dialogue generation model; and send the search result to the sender corresponding to the search text.

[0134] The specific implementation of each of the above operations can be found in the previous embodiments, which will not be described in detail here. It should be noted that the computer device provided in the embodiment of this application and the content search method in the above embodiment belong to the same concept, and its specific implementation process is detailed in the above method embodiment, which will not be described in detail here.

[0135] A person of ordinary skill in the art will appreciate that all or part of the steps in the various methods of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0136] To this end, an embodiment of the present application provides a computer-readable storage medium, in which a plurality of instructions are stored, and the instructions can be loaded by a processor to execute the steps in any of the content search methods provided in the embodiments of the present application. For example, the instructions can execute the following steps:

[0137] Obtain the search text; perform keyword processing on the search text using a trained keyword rewriting model to obtain the search keyword corresponding to the search text and the search response type corresponding to the search keyword; when the search response type is a search engine response type, search for search results that match the search text through a search engine based on the search keyword; when the search response type is a dialogue model response type, generate reply information that matches the search text as a search result based on the search keyword through a dialogue generation model; and send the search result to the sender corresponding to the search text.

[0138] The computer-readable storage medium may include: a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.

[0139] Since the instructions stored in the computer-readable storage medium can execute the steps in any content search method provided in the embodiments of the present application, the beneficial effects that can be achieved by any content search method provided in the embodiments of the present application can be achieved. Please refer to the previous embodiments for details and will not be repeated here.

[0140] According to one aspect of the present application, a computer program product or a computer program is provided, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementations provided in the above embodiments.

[0141] The above is a detailed introduction to a content search method, device and computer-readable storage medium provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A content search method, characterized in that: include: Get the search text; Perform keyword processing on the search text using a trained keyword rewriting model to obtain a search keyword corresponding to the search text and a search response type corresponding to the search keyword; When the search response type is a search engine response type, searching for search results matching the search text through a search engine based on the search keyword; When the search response type is a dialogue model response type, based on the search keyword, a dialogue generation model is used to generate reply information matching the search text as a search result; The search result is sent to a sender corresponding to the search text.

2. The content search method according to claim 1, characterized in that: The keyword processing of the search text using the trained keyword rewriting model to obtain the search keyword corresponding to the search text and the search response type corresponding to the search keyword include: The trained keyword rewriting model is used to rewrite the search text into a search keyword that matches the search input requirement of the search engine, and the search response type corresponding to the search keyword is predicted.

3. The content search method according to claim 1, characterized in that: The step of searching for search results matching the search text through a search engine based on the search keyword includes: Based on the search keyword, searching for search content matching the search text through a search engine; Extracting a summary of the search content to generate summary information corresponding to the search content; The summary information and the search content are used as search results matching the search text.

4. The content search method according to any one of claims 1 to 3, characterized in that: Before the keyword processing of the search text using the trained keyword rewriting model, the method further includes: Obtaining a training sample corresponding to a preset keyword rewriting model, wherein the training sample includes a search text sample, an expected search keyword corresponding to the search text sample, and an expected search response type; Using the preset keyword rewriting model to extract keywords and predict search response types for the search text sample, to obtain predicted search keywords and predicted search response types corresponding to the search text sample; Calculating loss information corresponding to the preset keyword rewriting model according to the expected search keyword and the predicted search keyword, as well as the expected search response type and the predicted search response type; The preset keyword rewriting model is converged based on the loss information to obtain a trained keyword rewriting model.

5. The content search method according to claim 4, characterized in that: The step of obtaining a training sample corresponding to a preset keyword rewriting model includes: Obtaining a training sample data set corresponding to a preset keyword rewriting model, and dividing the training sample data set into a simple training sample set and a difficult training sample set; Obtaining a first weight parameter corresponding to each training round of the preset keyword rewriting model for the simple training sample set, and a second weight parameter corresponding to each training round of the preset keyword rewriting model for the difficult training sample set; According to the first weight parameter and the second weight parameter, training samples of the preset keyword rewriting model in each training round are screened out from the simple training sample set and the difficult training sample set.

6. The content search method according to claim 5, characterized in that: The first weight parameter decreases with the number of training iterations of the preset keyword rewriting model, and the second weight parameter increases with the number of training iterations of the preset keyword rewriting model.

7. The content search method according to claim 4, characterized in that: The calculating the loss information corresponding to the preset keyword rewriting model according to the expected search keyword and the predicted search keyword, and the expected search response type and the predicted search response type includes: Calculating keyword prediction loss information corresponding to the preset keyword rewriting model according to the expected search keyword and the predicted search keyword; Calculating response type prediction loss information corresponding to the preset keyword rewriting model according to the expected search response type and the predicted search response type; The keyword prediction loss information and the response type prediction loss information are fused to obtain the loss information corresponding to the preset keyword rewriting model.

8. A content search device, characterized in that: include: An acquisition unit, used to acquire the search text; A processing unit, configured to perform keyword processing on the search text using a trained keyword rewriting model to obtain a search keyword corresponding to the search text and a search response type corresponding to the search keyword; A search unit, configured to search for search results matching the search text through a search engine based on the search keyword when the search response type is a search engine response type; a generating unit, configured to generate, when the search response type is a dialogue model response type, reply information matching the search text as a search result based on the search keyword by means of a dialogue generation model; A sending unit is used to send the search result to a sender corresponding to the search text.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a plurality of instructions, and the instructions are suitable for being loaded by a processor to execute the steps in the content search method according to any one of claims 1 to 7.

10. A computer device, characterized in that: The method comprises a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the content search method according to any one of claims 1 to 7 when executing the computer program.

Images