Data processing method, electronic equipment and computer storage medium

Abstract

The embodiment of the invention provides a data processing method, electronic equipment and a computer storage medium. The data processing method comprises the steps of obtaining target query data; performing data matching in an off-line index database, and judging whether target off-line reply data corresponding to the target query data exists in the off-line index database or not; the off-line index database comprises reply data corresponding to different historical query data; if the target offline reply data exists, determining the target offline reply data as reply data corresponding to the target query data; and if the target offline reply data does not exist, generating reply data corresponding to the target query data by adopting a retrieval enhancement generation mode based on the webpage content library. Through the embodiment of the invention, the quality of the reply data can be improved, and the speed and efficiency of online transmission of the reply data can be improved, so that the satisfaction degree of a user is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a data processing method, an electronic device, and a computer storage medium. Background Art

[0002] The fast-paced life and work environment has made people pay more and more attention to time efficiency, and they hope to get intuitive answers to questions when searching. Generative question-answering technology based on large language models can achieve "one search, one answer", significantly improve the efficiency of users in obtaining information, and help to achieve a personalized search experience.

[0003] However, obtaining question-answering results through the generative question-answering technology provided in the related art usually has the problem of low user satisfaction. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a data processing solution to at least partially solve the above-mentioned problem.

[0005] According to a first aspect of an embodiment of the present application, a data processing method is provided, including: Get target query data; Perform data matching in an offline index database to determine whether there is target offline reply data corresponding to the target query data in the offline index database; the offline index database contains reply data corresponding to different historical query data; If target offline reply data exists, determining the target offline reply data as reply data corresponding to the target query data; If the target offline reply data does not exist, the reply data corresponding to the target query data is generated by adopting a retrieval enhanced generation method based on the webpage content library.

[0006] According to a second aspect of an embodiment of the present application, another data processing method is provided, which is applied to a client device having a browser installed thereon, and the method includes: In response to a data input operation in a browser page, determining target query data; The target query data is sent to the server device, so that when the server device determines that there is target offline reply data corresponding to the target query data in the offline index database, the target offline reply data is determined as the reply data corresponding to the target query data; when it is determined that there is no target offline reply data in the offline index database, the reply data corresponding to the target query data is generated by using a retrieval enhanced generation method based on the webpage content library; Receive the response data returned by the server device and display the response data on the browser page.

[0007] According to the third aspect of an embodiment of the present application, there is provided an electronic device, comprising: a processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; the memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method described in the first aspect or the second aspect.

[0008] According to a fourth aspect of an embodiment of the present application, a computer storage medium is provided, on which a computer program is stored. When the program is executed by a processor, the method described in the first aspect or the second aspect is implemented.

[0009] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising computer instructions, wherein the computer instructions instruct a computing device to perform operations corresponding to the method described in the first aspect or the second aspect.

[0010] According to the data processing solution provided in the embodiment of the present application, after the target query data is acquired, data matching is first performed in the offline index database. When target offline reply data corresponding to the target query data exists in the offline index database, the target offline reply data in the offline index database is determined as the reply data corresponding to the above target query data; when the target offline reply data corresponding to the target query data does not exist in the offline index database, the reply data corresponding to the target query data is generated in real time based on the web page content library by adopting the retrieval enhanced generation method.

[0011] In the embodiment of the present application, a combination of offline and online methods is used to obtain the reply data corresponding to the target query data, and the reply data generation operation is efficiently implemented. Through pre-retrieval and generation in offline scenarios, the efficiency of reply data acquisition can be improved, and when the corresponding target offline reply data does not exist in the offline index database, it is filled in by real-time generation, thereby ensuring the relevance and accuracy of the reply data. Therefore, the embodiment of the present application can improve the quality of the reply data, and can also improve the speed and efficiency of the online transmission of the reply data, thereby improving user satisfaction. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the embodiments of the present application. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.

[0013] Figure 1 is a flowchart of a data processing method according to an embodiment of the present application; Figure 2 A schematic diagram of an example scenario according to an embodiment of the present application; Figure 3 is a schematic diagram of another scenario example according to an embodiment of the present application; Figure 4 is a schematic diagram of a prompt information generation process according to an embodiment of the present application; Figure 5 A schematic diagram of an exemplary system applicable to the data processing method of an embodiment of the present application; Figure 6 is a flowchart of another data processing method according to an embodiment of the present application; Figure 7 A schematic diagram of a reply display page according to an embodiment of the present application; Figure 8 is a structural block diagram of a data processing device according to an embodiment of the present application; Fig. 9 is a structural block diagram of another data processing device according to an embodiment of the present application; Fig.10 It is a schematic diagram of the structure of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to enable those skilled in the art to better understand the technical solutions in the embodiments of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the embodiments of the present application should fall within the scope of protection of the embodiments of the present application.

[0015] The specific implementation of the embodiment of the present application is further explained below in conjunction with the accompanying drawings of the embodiment of the present application.

[0016] Reference Figure 1 , Figure 1 The data processing method provided in the embodiment of the present application may include the following steps: Step 102, obtaining target query data.

[0017] Specifically, the target query data can be question data, that is, data to be obtained through the solution provided in the embodiment of the present application. In the embodiment of the present application, there is no limitation on the specific content of the target query data, as well as the amount of data. Exemplarily, the target query data can be a popular science question such as "What is the blue sky?", or a commodity shopping question such as "What are the mobile phone models under 1,000 yuan?", or a daily life question such as "Which place is suitable for traveling in December?", etc.

[0018] Step 104 , performing data matching in the offline index database to determine whether there is target offline reply data corresponding to the target query data in the offline index database.

[0019] If the target offline reply data exists, step 106 is executed; otherwise, if the target offline reply data does not exist, step 108 is executed.

[0020] The offline index database contains reply data corresponding to different historical query data.

[0021] Specifically, the offline index database can be a database constructed in advance based on historical query data and reply data corresponding to the historical query data. In an embodiment of the present application, an offline index database can be constructed in advance based on the historical query data given by the user, and the reply data corresponding to the historical query data. Afterwards, when the user's real-time target query data is received, in order to save retrieval time, data matching can be performed in the offline index database first. If the match is successful, the reply data that successfully matches is determined as the target offline reply data corresponding to the target historical query data; conversely, if the match fails, it indicates that the offline index database does not have the target offline reply data corresponding to the target historical query data. At this time, the generation method is used to obtain the reply data corresponding to the target query data.

[0022] In the embodiment of the present application, when matching data in the offline index database, the specific content of the matching algorithm used is not limited, and any suitable algorithm can be customized according to the actual situation. For example, the target query data can be directly matched with each historical query data stored in the offline index database (such as semantic matching, keyword matching, etc.); or index data can be first constructed based on each historical query data, and then the target query data can be matched with the above index data (such as semantic matching, keyword matching, etc.).

[0023] Step 106: determine the target offline reply data as the reply data corresponding to the target query data.

[0024] Specifically, when there is target offline reply data corresponding to the target query data in the offline index database, the target offline reply data can be determined as the reply data corresponding to the currently received target query data, and then the target offline reply data can be displayed to the user as an answer.

[0025] Step 108, generating response data corresponding to the target query data by adopting a search enhancement generation method based on the webpage content library.

[0026] Specifically, if it is determined through the above step 104 that there is no target offline reply data corresponding to the target query data in the offline index database, the reply data corresponding to the target query data can be obtained by real-time generation. In specific implementation, in order to improve the accuracy of the reply data finally generated and improve the generation quality, a search-enhanced generation method can be adopted to first perform content retrieval for the target query data in the pre-built webpage content library, thereby obtaining rich reference materials, so that the generative language model can generate more appropriate and accurate reply data based on the above reference materials in the subsequent process.

[0027] In addition, in order to further improve the generation quality, in the embodiment of the present application, in the process of constructing the web page content library, the web page content can also be supplied in a diversified manner, so that the source and format of the content data in the web page content library are richer. Specifically: the source of the content data in the web page content library can include not only web pages, but also some specified applications (that is, the content data in the web page content library is the resource data of the specified application); the format of the content data in the web page content library can include one or more of the following: table data, cache data, video data, picture data, text data. Furthermore, in terms of text data, it can also include: text data in word format, text data in PDF format, and so on. Through the above-mentioned diversified web page content supply, the embodiment of the present application can provide users with more comprehensive and in-depth response data to meet the diversified information needs of users.

[0028] In the embodiment of the present application, a combination of offline and online methods is used to obtain the reply data corresponding to the target query data, and the reply data generation operation is efficiently implemented. Through pre-retrieval and generation in offline scenarios, the efficiency of reply data acquisition can be improved, and when the corresponding target offline reply data does not exist in the offline index database, it is filled in by real-time generation, thereby ensuring the relevance and accuracy of the reply data. Therefore, the embodiment of the present application can improve the quality of the reply data, and can also improve the speed and efficiency of the online transmission of the reply data, thereby improving user satisfaction.

[0029] Optionally, in some of the embodiments, the data processing method may further include: Obtain historical query data and reply data corresponding to the historical query data; the reply data is obtained after retrieval enhancement generation processing is performed on the historical query data based on the web page content library; and construct an offline index database based on the historical query data and the reply data corresponding to the historical query data.

[0030] Specifically, the reply data corresponding to the historical query data in the above embodiment can be the reply data for the historical query data generated by using a retrieval enhancement generation method based on a pre-built web content library. Compared with other methods for generating reply data (such as manual generation, direct generation using a large language model, etc.), the above-mentioned retrieval-enhanced reply data generation method, by introducing an external knowledge base (i.e., a web content library), not only relies on the knowledge inside the model when generating answers, but more importantly, also refers to external data sources. Therefore, it can significantly reduce the problem of inaccurate or false reply data generated by a large language model, that is, it can effectively improve the accuracy of reply data.

[0031] Optionally, in some of the embodiments, after building the offline index database, the data processing method may further include: For the historical query data, perform retrieval enhancement generation processing based on the updated web page content library to obtain updated response data of the historical query data; The updated reply data is used to replace the reply data corresponding to the historical query data in the offline index database, so as to update the offline index database.

[0032] Specifically, when performing retrieval enhancement generation processing on a specific query data, it is usually performed based on the current web page content library. However, as time goes by, the content in the web page content library is usually updated. Taking the above problems into consideration, in the above embodiments of the present application, after completing the construction of the offline index database, the updated web page content library can be used again to perform retrieval enhancement generation processing on the historical query data in the offline index database, thereby generating updated reply data that has a higher degree of match with the historical query data to replace the original reply data. Through the above update operation, the accuracy of the reply data contained in the offline index database can be effectively improved, thereby improving the user experience.

[0033] Optionally, in some of the embodiments, the process of obtaining the historical query data and the reply data corresponding to the historical query data may specifically include: According to the preset historical data crawling rules, historical query data and response data corresponding to the historical query data are regularly obtained; Correspondingly, the process of constructing an offline index database based on the historical query data and the reply data corresponding to the historical query data may specifically include: Based on the historical query data and the reply data corresponding to the historical query data, an offline index database is constructed and updated.

[0034] Specifically, as time goes by, the question data people ask may change, and even for the same question, as the web page content library is constantly updated, the corresponding reply data may also change. Taking the above into account, in the above embodiments of the present application, historical query data and corresponding reply data can be regularly obtained according to certain data crawling rules, so as to continuously update the offline index database. Through the above method, the accuracy of the reply data contained in the offline index database can be effectively improved, thereby improving the user experience.

[0035] The preset historical data capture rules can be comprehensively considered based on factors such as computing resources and actual application scenarios. For example, the historical data capture rules can be to capture historical data once every fixed period; the historical data capture rules can also be to capture historical data continuously in a streaming manner, and so on.

[0036] Taking streaming continuous crawling as an example, historical query data can be continuously crawled, and based on the crawled historical query data, retrieval enhancement generation processing can be performed to build and continuously update the offline index database, that is, to continuously refresh and save the data.

[0037] Furthermore, after obtaining the historical query data, the query frequency information of each historical query data can be counted; based on the query frequency information, the historical query data is screened to obtain the target historical data; for each target historical data, retrieval enhancement generation processing is used to generate reply data corresponding to each target historical data; each target historical data and the reply data corresponding to each target historical data are used to obtain an index database.

[0038] In the above process, after capturing the historical query data, the historical query data is first screened according to the frequency information of the historical query data in the historical query scene, that is, the above query frequency information, so that the data with higher frequency of occurrence is used as the target historical data, so as to generate and update the index database based on the target historical data and the corresponding reply data. Through the above screening process, the data that finally exists in the index database can be the "hot" data that is more commonly used in the query scene, rather than the "unpopular" data with lower query frequency. In this way, after the target query data is obtained, the possibility of finding the target offline reply data corresponding to the target query data in the index database will be increased, that is: for the target query data, there is a greater probability that the corresponding generated reply data can be found from the index database, which helps to improve the efficiency of obtaining the reply data. Furthermore, since the reply data is generated offline in advance, and there is no need to perform the reply data generation operation in real time, the above method can further reduce the consumption of computing resources.

[0039] Optionally, in some of the examples, the response data corresponding to the target query data is generated by adopting a retrieval enhancement generation method based on the webpage content library, which can be specifically implemented as follows: For target query data, content retrieval is performed in a pre-built web content library to obtain multiple content retrieval results; Determine the webpage source information corresponding to each content search result, and screen the search results among the multiple content search results according to the webpage source information to obtain screened search results; Based on the filtered retrieval results, the response data corresponding to the target query data is generated through the generative language model.

[0040] Specifically, the pre-built web page content library may include web page content and web page source information of each web page content, wherein the web page source information is used to indicate which specific web page site the corresponding web page content comes from.

[0041] After obtaining multiple content search results, each content search result can be screened according to the source of the web page, so as to obtain search results with higher relevance to the target query data and higher content quality.

[0042] In the embodiment of the present application, there is no limitation on the specific screening strategy adopted when screening the search results according to the web page source information, and it can be customized according to the actual situation. For example, a whitelist can be set for the web page source information that may be involved, and when the source of the content search result matches the web page source information in the whitelist, the content search result is retained, otherwise, the content search result is deleted; an abnormal list can also be set for the web page source information that may be involved, and when the source of the content search result matches the web page source information in the abnormal list, the content search result is deleted, otherwise, the content search result is retained, and so on.

[0043] As for the method of using a generative language model to generate response data, before the model performs the generation operation, prompt information for guiding the model to perform the generation operation can be generated based on the filtered retrieval results and the obtained target query data. The above prompt information can improve the accuracy of the model's generation of response data and ensure that the generated response data has a high correlation with the target query data and the filtered retrieval results.

[0044] The specific implementation process can be: based on the filtered search results and the target query data, generate prompt information; use the prompt information as the model input of the generative language model, and generate reply data for replying to the target query data through the generative language model. Among them, the process of generating prompt information based on the filtered search results and the target query data can be implemented as: splicing the filtered search results and the target query data to obtain prompt information; it can also be implemented as: data fusion (such as keyword extraction, semantic analysis, etc.) of the filtered search results and the target query data to obtain prompt information, and so on.

[0045] According to the above-mentioned embodiment of the present application, for the target query data, after obtaining multiple content retrieval results through content retrieval, the web page source information corresponding to each content retrieval result is determined, and then the retrieval results are filtered in the above-mentioned multiple content retrieval results based on the web page source information to obtain the filtered retrieval results; then, based on the above-mentioned filtered retrieval results, the response data corresponding to the target query data is generated through the generative language model.

[0046] In the above embodiment of the present application, after obtaining the content retrieval results corresponding to the target query data through the content retrieval technology, the reply data generation operation is not performed based on the content retrieval results, but from the perspective of the source of the results, the retrieved content retrieval results are first screened, so that the results with low relevance to the target query data are removed from the content retrieval results, and then the reply data is generated according to the screened retrieval results. The above process can effectively reduce the noise contained in the input data packet input to the generative language model, thereby improving the semantic relevance and accuracy of the output results of the generative language model, improving the quality of the final reply data, and thus improving the user's search experience.

[0047] See also Figure 2 , Figure 2 This is a schematic diagram of a scenario of an embodiment of the present application. Figure 2 The schematic diagram shown briefly describes the process of obtaining the response data corresponding to the target query data in a real-time generation manner provided by the embodiment of the present application by taking a specific scenario as an example: When a user wants to get an answer to a query question, he can enter the search query term in the browser page, which is the target query data in the embodiment of the present application. Afterwards, when it is determined that there is no target offline reply data corresponding to the target query data in the offline index database, a real-time retrieval enhancement process can be executed to finally output the generated question-and-answer content and display it to the user online, which is Figure 2 Online exposure in.

[0048] The above search enhancement process may include the following three steps: Step 1: Request a search engine. Specifically: for the search query word input by the user, the search engine can perform a content search in a pre-built web page content library to obtain multiple content search results.

[0049] Step 2: Retrieve relevant web page content. Specifically: determine the web page source information corresponding to each content search result, and filter the search results among multiple content search results according to the web page source information to obtain the filtered search results, i.e. Figure 2 Related web content in .

[0050] Step 3: Assemble prompt information and call the model. Specifically: Based on the relevant web page content obtained in step 2 and the search query term input by the user, assemble and generate prompt information for guiding the model to perform the generation operation, and input the prompt information into the generative language model to perform the question and answer content generation operation through the model.

[0051] Optionally, in some of the embodiments, the process of filtering the search results among multiple content search results according to the webpage source information to obtain the filtered search results may specifically include: Determine the content quality level of each content search result according to the web page source information corresponding to each content search result and the predetermined correspondence between the web page source and the content quality level; According to the content quality level of each content search result, the search results are screened from the multiple content search results to obtain the screened search results.

[0052] Specifically, the webpage source information may be the webpage site information to which the content search result belongs. The content quality detection may be performed in advance on the historical contents of different webpage sites respectively, so as to determine the content quality level of each webpage site, wherein the content quality level represents the quality of the content search result in the webpage site; or the corresponding content quality level may be set for each webpage site information respectively based on experience.

[0053] In the above embodiment, different web page sources are divided into content quality levels in advance, and the correspondence between web page sources and content quality levels is stored. When each content search result is retrieved, the above correspondence can be queried based on the web page source information of each content search result, and then the content quality level of each content search result can be determined. After that, the content search results with better quality can be retained based on the content quality level of each content search result. The above process performs quality grading and quality screening of content search results from the perspective of web page sources, so that the final screened search results are high-quality content. The high-quality screened search results are used as reference materials to generate reply data, which can effectively improve the accuracy of the reply data.

[0054] Optionally, in some of the embodiments, the process of screening the retrieval results from the multiple content retrieval results according to the content quality level of each content retrieval result to obtain the screened retrieval results may include: Determine the title part and the text part in each content search result; For each content retrieval result, the target query data, title part and text part are used to calculate the relevance, and the semantic relevance between each content retrieval result and the target query data is obtained; the semantic relevance represents the relevance degree between the content retrieval result and the target query data in the semantic dimension; According to the content quality level of each content retrieval result and the semantic relevance between each content retrieval result and the target query data, the retrieval results are screened from the multiple content retrieval results to obtain the screened retrieval results.

[0055] Specifically, in the embodiments of the present application, there is no limitation on the specific calculation method used when calculating the semantic relevance between the content retrieval results and the target query data, and the setting can be customized according to the actual situation. For example, the semantic similarity between the target query data, the title part and the body part can be calculated, and then similarity fusion processing is performed based on the calculated semantic similarity between the two parts, and the fusion result is used to measure the semantic relevance between the content retrieval results and the target query data; or the target query data, the title part and the body part can be respectively input into a pre-trained relevance evaluation model by deep learning, and the semantic relevance between the content retrieval results and the target query data is output through the model, and so on.

[0056] In the above embodiment of the present application, when screening the search results, the two dimensions of the quality of the content search results and the semantic relevance are taken into consideration at the same time. In this way, the final screened search results not only have higher content quality, but also have a higher semantic relevance to the target query data, thereby ensuring that the final response results can have higher accuracy and fit, thereby improving the user's search experience.

[0057] Furthermore, in the above-mentioned embodiment of the present application, when calculating the semantic relevance, each content retrieval result is first split into fine-grained parts to obtain the title part and the text part of the content retrieval result, and then the target query data is used to calculate the relevance with the fine-grained title part and the text part. The above-mentioned semantic relevance calculation method performs a fine-grained split on the content retrieval result, and uses the split results to participate in the relevance calculation. Therefore, the calculation results can be made more accurate, further improving the relevance between the reference materials and the target query data, and ultimately improving the accuracy of the reply data.

[0058] Optionally, in some of the embodiments, the process of performing content retrieval in a pre-built webpage content library to obtain a plurality of content retrieval results may include: Perform exclusive content search in a pre-built target exclusive content library corresponding to the target query data to obtain exclusive search results corresponding to the target query data; perform general content search in a pre-built general content library to obtain general search results corresponding to the target query data; Based on the general search results and the exclusive search results, multiple content search results are obtained.

[0059] Specifically, for a specific target query data, the results obtained usually come from some general question-and-answer sites. However, for some highly professional target query data, such as "If the limit is infinite, does the limit exist", if the reference material still comes from a general question-and-answer site, users may question the credibility and professionalism of the answer, thus affecting their perception.

[0060] In order to solve the above problems, the above embodiments of the present application set up two retrieval recall links in parallel during the content retrieval process: one is a general recall link based on the general content library, and the other is an exclusive recall link based on the exclusive content library. When the target query data is obtained, on the one hand, appropriate reference content is selected from the general content library through the general recall link, and on the other hand, appropriate reference content is selected from the exclusive content library through the exclusive recall link, and then the final content retrieval result is obtained based on the recall results of the two links. The above recall process can obtain more professional content retrieval results, effectively increasing the proportion of professional content in the final reference materials, and therefore helping to improve the professionalism and credibility of the response data.

[0061] See also Figure 3 , Figure 3 FIG. 1 is a schematic diagram of another example scenario according to an embodiment of the present application. Figure 3 The recall strategies of the above two retrieval recall links are briefly described as follows: After obtaining the target query data, that is, Figure 3 After the question-answer query in , the reference materials can be recalled through the exclusive recall link and the general recall link respectively. Specifically: in the exclusive content recall module, the exclusive recall link can be used to recall the exclusive content from the exclusive content library, that is, the professional reference content corresponding to the question-answer query; the general recall link can be used to recall the general content from the general content library, that is, the general reference content corresponding to the question-answer query; then, the reference material selection module can be used to select the final reference material from the above exclusive content and general content.

[0062] After obtaining the reference material, the question-answer query and the reference material can be further combined to generate prompt information to guide the generative language model to generate response data corresponding to the question-answer query.

[0063] Optionally, in some of the embodiments, the process of performing content retrieval in a pre-built target-specific content library corresponding to the target query data to obtain a specific retrieval result corresponding to the target query data may include: Classify the target query data into categories to obtain target data categories corresponding to the target query data; Determine the target exclusive content library corresponding to the target query data according to the correspondence between the pre-built exclusive data categories and the exclusive content library; For the target query data, an exclusive content search is performed in the target exclusive content library to obtain an exclusive search result corresponding to the target query data.

[0064] Specifically, when obtaining exclusive search results, it is possible to preset and limit a plurality of exclusive data categories with high professional requirements, and construct corresponding exclusive content libraries for the exclusive data categories respectively, thereby forming a corresponding relationship between the exclusive data categories and the exclusive content libraries. Exemplarily, multiple exclusive data categories such as "education", "government affairs", "finance", "law", and "digital" can be set respectively, and the exclusive content libraries corresponding to the above categories can be collected and constructed respectively. Further, the professional reply webpage sites corresponding to each exclusive data category can be counted, so as to obtain the exclusive content libraries corresponding to each exclusive data category according to the statistical results. Exemplarily, for the exclusive data category of "law", the professional webpage sites corresponding to this category include "website 1", "website 2", "website 3", etc., then the exclusive content library corresponding to the exclusive data category of "law" can be constructed based on the webpage contents in the above-mentioned "website 1", "website 2", "website 3", etc.

[0065] The above embodiment of the present application, by setting different exclusive data categories and constructing exclusive content libraries for each exclusive data category in advance, can quickly and accurately determine the exclusive content library corresponding to the target query data, and then retrieve the corresponding exclusive search results in the above exclusive content library. This can effectively solve the problem of insufficient professionalism of reference materials in the generative question-answering process, improve the quality and credibility of the response data generated by the generative language model, and thus enhance the user experience.

[0066] Furthermore, the process of constructing the corresponding relationship between the exclusive content category and the exclusive content library may specifically include: Acquire multiple historical query data within a preset historical time period; perform keyword filtering on the above historical query data to obtain filtered query data; cluster the multiple filtered query data to obtain multiple exclusive content categories; determine the corresponding exclusive content library for each exclusive content category, and then establish a corresponding relationship between the exclusive content category and the exclusive content library.

[0067] Furthermore, for each exclusive content category, when determining the corresponding exclusive content library, it can be specifically implemented as follows: for a web site to be classified, web page content can be extracted from the web site, and then semantic analysis can be performed on the extracted web page content. If the analysis result shows that the extracted web page content is related to a target exclusive content category in a semantic dimension (such as the semantic similarity is greater than a preset threshold, or the extracted web page content contains keywords in the exclusive content category, etc.), then the web page content under the web site can be included in the exclusive content library corresponding to the target exclusive content category.

[0068] Optionally, in some of the embodiments, multiple content search results are obtained based on the general search results and the specific search results, including: Determine whether the exclusive search results meet the preset result screening conditions; If the conditions are met, multiple content search results are obtained from the exclusive search results; if the conditions are not met, the general search results and the exclusive search results are integrated to obtain multiple content search results.

[0069] Specifically, exemplarily, the result screening condition may be: whether the number of exclusive search results is greater than a preset number; or, the result screening condition may also be: among the exclusive search results, the number of relevant exclusive search results whose relevance to the target query data is greater than a preset relevance threshold is greater than a preset number; or, the result screening condition may also be: among the exclusive search results, there are relevant exclusive search results whose relevance to the target query data is greater than a preset relevance threshold, and so on.

[0070] When the exclusive search results meet the above-mentioned preset result screening conditions, it means that when the exclusive search results are used as reference materials, the generative language model can generate higher quality reply data. At this time, there is no need to refer to the general search results, but the content retrieval results can be selected from the exclusive search results as the final reference materials. On the contrary, if the exclusive search results cannot meet the above-mentioned preset result screening conditions, it means that there are not enough exclusive search results, or the correlation between the current exclusive search results and the target query data is too low. In this case, the general search results and the exclusive search results can be integrated to improve the quality of the final reference materials as much as possible, thereby improving the quality of the final generated reply data.

[0071] Optionally, in some of the embodiments, based on the filtered retrieval results, generating reply data corresponding to the target query data by a generative language model includes: Extracting content from the filtered search results to obtain title content and text content in the filtered search results; Using target query data, title content in the filtered search results, and text content in the filtered search results, to generate prompt information with a preset data format; The prompt information is input into the pre-trained generative language model, and the generative language model outputs the response data corresponding to the target query data.

[0072] Specifically, in the embodiment of the present application, the arrangement of the target query data, title content and text content in the prompt information, that is, the specific data format of the prompt information is not limited and can be customized according to actual conditions.

[0073] See also Figure 4 , Figure 4 It is a schematic diagram of a prompt information generation process according to an embodiment of the present application.

[0074] After obtaining the question-answer query, that is, after obtaining the target query data, the first N web page contents matching the question-answer query can be obtained through the web page service; then the web page contents can be filtered and screened according to the web page source information of each web page content, so as to obtain the filtered web page contents, as shown in the example below: Figure 4 As shown, three filtered web page contents are obtained; content extraction is performed from the filtered web page contents to obtain the title content and text content of each filtered web page content.

[0075] Afterwards, you can generate Figure 4 The prompt information with the preset data format shown, Figure 4 The prompt information is shown in an exemplary manner as consisting of three parts. The specific format of the prompt information includes: a fixed prompt word part, a filtered web page content part (in this part, for each filtered web page content, the title and text in the web page content are displayed in detail), and a fill prompt word part ( Figure 4 "Generating a summary for the question 'Why is the sky blue'" in .

[0076] In the embodiment of the present application, the specific content of the fixed prompt word part is not limited and can be customized. For example, it can be "You are an intelligent question-answering system. Please generate a short article based on the following reference articles retrieved from the Internet to answer the questions raised by the user", or, in more detail, it can also be "You are an intelligent question-answering system with strong reading comprehension ability. You can generate a comprehensive, professional, authoritative and well-structured short article based on the following reference articles retrieved from the Internet to answer the questions raised by the user", etc.

[0077] See also Figure 5 , Figure 5 This is a schematic diagram of an exemplary system for the data processing method of the embodiment of the present application. For ease of understanding, first combine Figure 5 The application scenarios of the data processing method provided in the embodiments of the present application are explained.

[0078] like Figure 5 As shown, the system 500 may include a server 502, a communication network 504 and / or one or more user devices (ie, client devices) 506. Figure 5 The example in FIG. 1 is a plurality of user devices.

[0079] The server 502 may be any appropriate device deployed with a generative service, including but not limited to a server cluster, a computing cloud server cluster, etc. In some embodiments, the server 502 may perform any appropriate function. For example, in some embodiments, the server 502 may be used to: perform data matching in an offline index database, determine whether there is target offline reply data corresponding to the target query data in the offline index database; if there is target offline reply data, determine the target offline reply data as the reply data corresponding to the target query data; if there is no target offline reply data, generate the reply data corresponding to the target query data by using a retrieval enhanced generation method based on the webpage content library; send the reply data to the corresponding user device 506 for the user to view.

[0080] In some embodiments, the communication network 504 can be any suitable combination of one or more wired and / or wireless networks. For example, the communication network 504 can include any one or more of the following: the Internet, an intranet, a wide area network (WAN), a local area network (LAN), a wireless network, a digital subscriber line (DSL) network, a frame relay network, an asynchronous transfer mode (ATM) network, a virtual private network (VPN) and / or any other suitable communication network. The user device 506 can be connected to the communication network 504 via one or more communication links (e.g., communication link 512), and the communication network 504 can be linked to the service end 502 via one or more communication links (e.g., communication link 514). The communication link can be any communication link suitable for transmitting data between the user device 306 and the service end 502, such as a network link, a dial-up link, a wireless link, a hard-wired link, any other suitable communication link or any suitable combination of such links.

[0081] User equipment 506 may include any one or more user equipment suitable for data processing. In some embodiments, user equipment 506 may include any suitable type of equipment. For example, in some embodiments, user equipment 506 may include a mobile device, a tablet computer, a laptop computer, a desktop computer, and / or any other suitable type of user equipment.

[0082] Reference Figure 6 , Figure 6 is a flowchart of another data processing method according to an embodiment of the present application. The data processing method provided in the embodiment of the present application can be Figure 5 The user device 506 (client device) with a browser installed in the system shown is executed. Specifically, the data processing method provided in this embodiment includes the following steps: Step 602, in response to a data input operation in a browser page, determining target query data.

[0083] Specifically, when a user wants to perform a question-and-answer search, a data input operation may be performed in a browser page, thereby inputting target query data into a client device.

[0084] The specific form of the data input operation is not limited in the embodiments of the present application. For example, the target query data in text format may be input to the client device by typing on a keyboard, or the data in audio format may be input to the client device by voice, so that the client device converts the audio data into text to obtain the target query data, etc.

[0085] Step 604, sending the target query data to the server device, so that when the server device determines that there is target offline reply data corresponding to the target query data in the offline index database, the target offline reply data is determined as the reply data corresponding to the target query data; when it is determined that there is no target offline reply data in the offline index database, the reply data corresponding to the target query data is generated based on the web page content library by adopting a retrieval enhanced generation method.

[0086] Step 606, receiving the reply data returned by the server device, and displaying the reply data in the browser page.

[0087] According to the data processing scheme provided in the embodiment of the present application, for the target query data, after obtaining multiple content retrieval results through content retrieval, the web page source information corresponding to each content retrieval result is determined, and then the retrieval results are filtered in the above multiple content retrieval results according to the web page source information to obtain the filtered retrieval results; based on the above filtered retrieval results, the response data corresponding to the target query data is generated through the generative language model.

[0088] In the above-mentioned embodiment of the present application, a combination of offline and online methods is adopted to obtain the reply data corresponding to the target query data, and the reply data generation operation is efficiently realized. Through pre-retrieval and generation in offline scenarios, the efficiency of reply data acquisition can be improved, and when the corresponding target offline reply data does not exist in the offline index database, it is filled in by real-time generation, thereby ensuring the relevance and accuracy of the reply data. Therefore, the embodiment of the present application can improve the quality of the reply data, and can also improve the speed and efficiency of the online transmission of the reply data, thereby improving user satisfaction.

[0089] Optionally, in some of the embodiments, the process of displaying the reply data in the browser page may specifically include: Create a reply display page; the reply display page includes: reply data display area and source display area; Receive the reply data returned by the server device, as well as the reference source information of the reply data; Load the reference source information of the reply data in the source display area of ​​the reply display page; The reply data is loaded in the reply data display area in the reply display page, and a reference source identifier is added to the loaded reply data; the reference source identifier is identification information generated for the reference source information.

[0090] Specifically, the solution provided by the embodiment of the present application sets up a reply data display area and a source display area in the browser page. In the process of data display, not only the reply data is displayed, but also the reference source information of the reply data and the identifier of the reference source are displayed in a coordinated manner. On the one hand, the reference source information is displayed in the source display area. On the other hand, more importantly, the identifier of the reference source is added to the end of the sentence of the displayed reply data, that is, in the generated reply data, the cited reference materials are clearly marked to ensure that the source of the reply data is clear and traceable. See Figure 7 , Figure 7 Schematic diagram of a reply display page according to an embodiment of the present application. For the target query data "lose a little thought" input by the user, the corresponding reply display page may include a reply data display area and a source display area; wherein the reply data display area displays the reply data, and the source display area displays the reference source. Further, from Figure 7It can be seen that a reference source identifier is also added at the end of the statement of the reply data still being displayed. Specifically, for the first sentence in the reply data, "Losing a bit is pronounced 'Shi'.", a reference source identifier [1] is added at the end of this sentence, indicating that this sentence is sourced from the first reference material in the reference source list. Similarly, a reference source identifier [2] is added at the end of the second sentence in the reply data, indicating that this sentence is sourced from the second reference material in the reference source list.

[0091] The solution of the collaborative display of the reference source information and the reference source identifier information provided in the above embodiments of the present application can achieve the following effects: Since users generally tend to trust information supported by reliable sources, displaying the reference source can increase the credibility of the reply data for users; in addition, displaying the reference source can enable users to understand how the data and information behind the reply data are obtained, which increases the transparency of the generation of the reply data. More importantly, when the reply data contains a large number of sentences and the sentences are not sourced from the same reference source, the way of collaborative display of the reference source information and the reference source identifier in the embodiments of the present application can enable users to intuitively and conveniently obtain the accurate reference sources of each sentence in the reply statement, further enhancing the user's search experience.

[0092] Optionally, in some embodiments, the above data processing method may further include: In response to a trigger operation on the target reference source information loaded in the source display area, display the target reference web page content corresponding to the target reference source information.

[0093] In other embodiments, the above data processing method may further include: In response to a trigger operation on the target reference source identifier added to the reply data in the reply data display area, determine the target reference source information according to the target reference source identifier; Display the target reference web page content corresponding to the target reference source information.

[0094] Specifically, when the user performs a trigger operation on a certain reference source information (hereinafter referred to as the target reference source information for the convenience of subsequent description) or a certain reference source identifier (hereinafter referred to as the target reference source identifier for the convenience of subsequent description) displayed on the browser page, the target reference web page content corresponding to the target reference source information is displayed to the user. With the help of the displayed target reference web page content, users can better understand the context of the reply data and enhance the overall user experience; moreover, since the target reference web page content is displayed, users can trace back to the original materials from which the reply data is extracted and verify the authenticity and accuracy of the reply data by themselves, that is, it is convenient to verify and check the reply data.

[0095] Optionally, in some of these embodiments, the reply display page may further include: a recommended information display area; Display the response data in the browser page, which can also include: Extract semantic information from the reply data and determine target service information related to the semantics of the reply data; Calling the service interface corresponding to the target service information, and receiving the service recommendation information matching the reply data through the service interface; The service recommendation information is displayed in the recommendation information display area.

[0096] Specifically, in the above-mentioned embodiment of the present application, a plurality of different display areas are set in the reply display page. In addition to the reply data display area for displaying the reply data and the source display area for displaying the reference source information, a recommendation information display area for displaying the service recommendation information is also set. After the associated target service information is determined from the reply data through semantic extraction, the matching service recommendation information is displayed to the user through the service interface call. Exemplarily, taking the target query data as "What are the mobile phone models within one thousand yuan", after obtaining the corresponding reply data, if it is determined through semantic extraction that the reply data contains certain mobile phone model information, the e-commerce service interface can be called to receive the purchase link and other service recommendation information of the above-mentioned mobile phone model through the e-commerce service interface, and the link is displayed in the recommendation information display area.

[0097] In the above embodiment, by dividing the reply display page into multiple modules, different types of information are displayed in different display module areas, thereby further enriching the user's search experience.

[0098] Optionally, in some of the embodiments, after sending the target query data to the server device, the data processing method may further include: The generation progress information is displayed on the browser page, and the generation progress information includes one of the following information: retrieval status information, sorting status information, generation status information, and result display information.

[0099] Specifically, it usually takes a certain amount of computing time from the time the client device obtains the target query data input by the user to the time the generated reply data is displayed to the user. In the above embodiment of the present application, after the client device sends the target query data to the server device, it displays the generation progress information to the user. Specifically: the retrieval status information indicates that content retrieval is currently in progress; the sorting status information indicates that the content retrieval results are currently being sorted to generate prompt information; the generation status information indicates that reply data is currently being generated based on the prompt information; the result display information indicates that the reply data generation has been completed and the generated reply data has been displayed in the interface.

[0100] The above information is naturally transitioned as the reply data is generated, which not only optimizes the user's interactive experience, but also invisibly reduces the user's sense of waiting, demonstrating a high degree of professionalism and humanized considerations.

[0101] Figure 8 1 is a structural block diagram of a data processing device according to an embodiment of the present application. The data processing device provided by the embodiment of the present application includes: The data acquisition module 802 is used to acquire target query data; The reply generation module 804 is used to match data in the offline index database and determine whether there is target offline reply data corresponding to the target query data in the offline index database; the offline index database contains reply data corresponding to different historical query data; if there is target offline reply data, the target offline reply data is determined as the reply data corresponding to the target query data; if there is no target offline reply data, the reply data corresponding to the target query data is generated based on the web page content library by adopting a retrieval enhanced generation method.

[0102] Optionally, in some of the embodiments, the data processing device further comprises: A construction module is used to obtain historical query data and reply data corresponding to the historical query data; the reply data is obtained after retrieval enhancement generation processing is performed on the historical query data based on the web page content library; based on the historical query data and the reply data corresponding to the historical query data, an offline index database is constructed.

[0103] Optionally, in some of the embodiments, after building the offline index database, the construction module is also used to: perform retrieval enhancement generation processing on the historical query data based on the updated web page content library to obtain updated reply data of the historical query data; and use the updated reply data to replace the reply data corresponding to the historical query data in the offline index database to update the offline index database.

[0104] Optionally, in some of the embodiments, the construction module, when executing the step of acquiring historical query data and reply data corresponding to the historical query data, is specifically used to: regularly acquire historical query data and reply data corresponding to the historical query data according to a preset historical data crawling rule; The construction module, when executing the step of building an offline index database based on historical query data and reply data corresponding to the historical query data, is specifically used to: build and update the offline index database based on the historical query data and reply data corresponding to the historical query data.

[0105] Optionally, in some embodiments, the reply generation module 804, when executing the step of generating reply data corresponding to the target query data by using the retrieval enhancement generation method based on the webpage content library, is specifically used to: For target query data, content retrieval is performed in a pre-built web content library to obtain multiple content retrieval results; Determine the webpage source information corresponding to each content search result, and screen the search results among the multiple content search results according to the webpage source information to obtain screened search results; Based on the filtered retrieval results, the response data corresponding to the target query data is generated through the generative language model.

[0106] Optionally, in some embodiments, the reply generation module 804, when performing the step of filtering the search results among the multiple content search results according to the webpage source information to obtain the filtered search results, is specifically used to: Determine the content quality level of each content search result according to the web page source information corresponding to each content search result and the predetermined correspondence between the web page source and the content quality level; According to the content quality level of each content search result, the search results are screened from the multiple content search results to obtain the screened search results.

[0107] Optionally, in some embodiments, the reply generation module 804, when performing the step of screening the retrieval results from the multiple content retrieval results according to the content quality level of each content retrieval result to obtain the screened retrieval results, is specifically configured to: Determine the title part and the text part in each content search result; For each content retrieval result, the target query data, title part and text part are used to calculate the relevance, and the semantic relevance between each content retrieval result and the target query data is obtained; the semantic relevance represents the relevance degree between the content retrieval result and the target query data in the semantic dimension; According to the content quality level of each content retrieval result and the semantic relevance between each content retrieval result and the target query data, the retrieval results are screened from the multiple content retrieval results to obtain the screened retrieval results.

[0108] Optionally, in some embodiments, the reply generation module 804, when performing the step of performing content retrieval in a pre-built webpage content library to obtain a plurality of content retrieval results, is specifically configured to: Perform exclusive content search in a pre-built target exclusive content library corresponding to the target query data to obtain exclusive search results corresponding to the target query data; perform general content search in a pre-built general content library to obtain general search results corresponding to the target query data; Based on the general search results and the exclusive search results, multiple content search results are obtained.

[0109] Optionally, in some embodiments, the reply generation module 804, when performing the step of performing content retrieval in a pre-built target-specific content library corresponding to the target query data to obtain a specific retrieval result corresponding to the target query data, is specifically configured to: Classify the target query data into categories to obtain target data categories corresponding to the target query data; Determine the target exclusive content library corresponding to the target query data according to the correspondence between the pre-built exclusive data categories and the exclusive content library; For the target query data, an exclusive content search is performed in the target exclusive content library to obtain an exclusive search result corresponding to the target query data.

[0110] Optionally, in some of the embodiments, the reply generation module 804, when executing the step of obtaining a plurality of content search results according to the general search results and the specific search results, is specifically configured to: Determine whether the exclusive search results meet the preset result screening conditions; If the conditions are met, multiple content search results are obtained from the exclusive search results; if the conditions are not met, the general search results and the exclusive search results are integrated to obtain multiple content search results.

[0111] Optionally, in some embodiments, the reply generation module 804, when executing the step of generating reply data corresponding to the target query data by using a generative language model based on the filtered retrieval results, is specifically configured to: Extracting content from the filtered search results to obtain title content and text content in the filtered search results; Using target query data, title content in the filtered search results, and text content in the filtered search results, to generate prompt information with a preset data format; The prompt information is input into the pre-trained generative language model, and the generative language model outputs the response data corresponding to the target query data.

[0112] The data processing device of this embodiment is used to implement the corresponding data processing method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here. In addition, the functional implementation of each module in the data processing device of this embodiment can refer to the description of the corresponding part in the above-mentioned method embodiment, which will not be described in detail here.

[0113] Fig. 9 1 is a structural block diagram of another data processing device according to an embodiment of the present application. The data processing device provided in the embodiment of the present application is applied to a client device with a browser installed, and the data processing device includes: The data determination module 902 is used to determine the target query data in response to the data input operation in the browser page; The data sending module 904 is used to send the target query data to the server device, so that when the server device determines that there is target offline reply data corresponding to the target query data in the offline index database, the target offline reply data is determined as the reply data corresponding to the target query data; when it is determined that there is no target offline reply data in the offline index database, the reply data corresponding to the target query data is generated by using a retrieval enhancement generation method based on the webpage content library; The data display module 906 is used to receive the reply data returned by the server device and display the reply data in the browser page.

[0114] Optionally, in some embodiments, the data display module 906 is specifically configured to: Create a reply display page; the reply display page includes: reply data display area and source display area; Receive the reply data returned by the server device, as well as the reference source information of the reply data; Load the reference source information of the reply data in the source display area of ​​the reply display page; The reply data is loaded in the reply data display area in the reply display page, and a reference source identifier is added to the loaded reply data; the reference source identifier is identification information generated for the reference source information.

[0115] Optionally, in some of the embodiments, the data processing device further comprises: The first display module is used to display target reference webpage content corresponding to the target reference source information in response to a triggering operation on the target reference source information loaded in the source display area.

[0116] Optionally, in some of the embodiments, the data processing device further comprises: A second display module is used to respond to a trigger operation of a target reference source identifier added to the reply data in the reply data display area, and determine the target reference source information according to the target reference source identifier; Display the target reference web page content corresponding to the target reference source information.

[0117] Optionally, in some of these embodiments, the reply display page further includes: a recommended information display area; Correspondingly, the data display module 906 is further used for: Perform semantic extraction on the reply data to determine target service information semantically related to the reply data; call a service interface corresponding to the target service information to receive service recommendation information matching the reply data through the service interface; and display the service recommendation information in a recommendation information display area.

[0118] Optionally, in some of the embodiments, the data processing device further includes: The progress display module is used to display the generation progress information in the browser page after sending the target query data to the server device. The generation progress information includes one of the following information: retrieval status information, sorting status information, generation status information, and result display information.

[0119] The data processing device of this embodiment is used to implement the corresponding data processing method in the above-mentioned embodiment, and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here. In addition, the functional implementation of each module in the data processing device of this embodiment can refer to the description of the corresponding part in the above-mentioned method embodiment, which will not be described in detail here.

[0120] Reference Fig.10 , shows a schematic diagram of the structure of an electronic device according to an embodiment of the present application. The specific embodiment of the present application does not limit the specific implementation of the electronic device.

[0121] like Fig.10 As shown, the electronic device may include: a processor (processor) 1002 , a communication interface (Communications Interface) 1004 , a memory (memory) 1006 , and a communication bus 1008 .

[0122] in: The processor 1002 , the communication interface 1004 , and the memory 1006 communicate with each other via a communication bus 1008 .

[0123] The communication interface 1004 is used to communicate with other electronic devices or servers.

[0124] The processor 1002 is used to execute the program 1010, and specifically can execute the relevant steps in the above method embodiment.

[0125] Specifically, the program 1010 may include program codes, which include computer operation instructions.

[0126] The processor 1002 may be a CPU, or an application specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the smart device may be processors of the same type, such as one or more CPUs; or processors of different types, such as one or more CPUs and one or more ASICs.

[0127] The memory 1006 is used to store the program 1010. The memory 1006 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), such as at least one disk memory.

[0128] The program 1010 may include multiple computer instructions. Specifically, the program 1010 may enable the processor 1002 to execute operations corresponding to the method described in any of the aforementioned method embodiments through the multiple computer instructions.

[0129] The specific implementation of each step in program 1010 can refer to the corresponding description of the corresponding steps and units in the above method embodiment, and has corresponding beneficial effects, which will not be repeated here. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working process of the above-described devices and modules can refer to the corresponding process description in the above method embodiment, which will not be repeated here.

[0130] The present application also provides a computer storage medium on which a computer program is stored, and when the program is executed by a processor, the method described in any of the above-mentioned multiple method embodiments is implemented. The computer storage medium includes, but is not limited to: a compact disc read-only memory (CD-ROM), a random access memory (RAM), a floppy disk, a hard disk, or a magneto-optical disk, etc.

[0131] An embodiment of the present application also provides a computer program product, including computer instructions, which instruct a computing device to execute operations corresponding to any one of the above-mentioned multiple method embodiments.

[0132] In addition, it should be noted that the user-related information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to sample data used to train the model, data used for analysis, stored data, displayed data, etc.) involved in the embodiments of the present application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0133] It should be pointed out that, according to the needs of implementation, the various components / steps described in the embodiments of the present application can be split into more components / steps, or two or more components / steps or partial operations of components / steps can be combined into new components / steps to achieve the purpose of the embodiments of the present application.

[0134] The above-mentioned method according to the embodiment of the present application can be implemented in hardware, firmware, or implemented as software or computer code that can be stored in a recording medium (such as a CD-ROM, RAM, floppy disk, hard disk or magneto-optical disk), or implemented as a computer code originally stored in a remote recording medium or a non-temporary machine-readable medium downloaded through a network and to be stored in a local recording medium, so that the method described herein can be stored in such software processing on a recording medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware (such as an application-specific integrated circuit (ASIC) or a field programmable gate array (FPGA)). It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component (e.g., a random access memory (RAM), a read-only memory (ROM), a flash memory, etc.) that can store or receive software or computer code, and when the software or computer code is accessed and executed by a computer, a processor, or hardware, the method described herein is implemented. In addition, when a general-purpose computer accesses the code for implementing the method shown herein, the execution of the code converts the general-purpose computer into a dedicated computer for executing the method shown herein.

[0135] Those of ordinary skill in the art will appreciate that the units and method steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the embodiments of the present application.

[0136] The above implementation methods are only used to illustrate the embodiments of the present application, and are not limitations on the embodiments of the present application. Ordinary technicians in the relevant technical field can make various changes and modifications without departing from the spirit and scope of the embodiments of the present application. Therefore, all equivalent technical solutions also belong to the scope of the embodiments of the present application. The scope of patent protection of the embodiments of the present application should be limited by the claims.

Claims

1. A data processing method, comprising: Get target query data; Perform data matching in an offline index database to determine whether there is target offline reply data corresponding to the target query data in the offline index database; the offline index database contains reply data corresponding to different historical query data; If target offline reply data exists, determining the target offline reply data as reply data corresponding to the target query data; If the target offline reply data does not exist, the reply data corresponding to the target query data is generated by adopting a retrieval enhanced generation method based on the webpage content library.

2. The method according to claim 1, wherein: The method further comprises: Acquire historical query data and reply data corresponding to the historical query data; the reply data is obtained after performing retrieval enhancement generation processing on the historical query data based on the webpage content library; An offline index database is constructed based on the historical query data and the reply data corresponding to the historical query data.

3. The method according to claim 2, wherein: After constructing the offline index database, the method further includes: For the historical query data, performing retrieval enhancement generation processing based on the updated webpage content library to obtain updated reply data of the historical query data; The updated reply data is used to replace the reply data corresponding to the historical query data in the offline index database, so as to update the offline index database.

4. The method according to claim 2, wherein: The acquiring of historical query data and reply data corresponding to the historical query data includes: According to the preset historical data crawling rules, historical query data and response data corresponding to the historical query data are regularly obtained; The step of constructing an offline index database based on the historical query data and the reply data corresponding to the historical query data includes: An offline index database is constructed and updated based on the historical query data and the reply data corresponding to the historical query data.

5. The method according to claim 1, wherein: The method of generating response data corresponding to the target query data by using a search enhancement generation method based on the webpage content library includes: For the target query data, content retrieval is performed in a pre-built webpage content library to obtain multiple content retrieval results; Determine web page source information corresponding to each content search result, and screen the search results among the multiple content search results according to the web page source information to obtain screened search results; Based on the filtered retrieval results, response data corresponding to the target query data is generated through a generative language model.

6. The method according to claim 5, wherein: The filtering of the search results among the multiple content search results according to the webpage source information to obtain the filtered search results includes: Determine the content quality level of each content search result according to the web page source information corresponding to each content search result and the predetermined correspondence between the web page source and the content quality level; According to the content quality level of each content search result, the search results are screened from the multiple content search results to obtain the screened search results.

7. The method according to claim 6, wherein: The method of screening the retrieval results from the plurality of content retrieval results according to the content quality level of each content retrieval result to obtain the screened retrieval results includes: Determine the title part and the text part in each content search result; For each content retrieval result, the target query data, the title part and the body part are used to perform relevance calculation to obtain the semantic relevance between each content retrieval result and the target query data; the semantic relevance represents the relevance degree between the content retrieval result and the target query data in the semantic dimension; According to the content quality level of each content retrieval result and the semantic relevance between each content retrieval result and the target query data, retrieval results are screened from the multiple content retrieval results to obtain screened retrieval results.

8. The method according to any one of claims 5 to 7, wherein: The content search is performed in the pre-built webpage content library to obtain multiple content search results, including: Perform exclusive content search in a pre-built target exclusive content library corresponding to the target query data to obtain exclusive search results corresponding to the target query data; perform general content search in a pre-built general content library to obtain general search results corresponding to the target query data; A plurality of content search results are obtained according to the general search results and the exclusive search results.

9. The method according to claim 8, wherein: The performing content retrieval in a pre-built target-specific content library corresponding to the target query data to obtain a specific retrieval result corresponding to the target query data includes: Classify the target query data into categories to obtain target data categories corresponding to the target query data; Determine the target exclusive content library corresponding to the target query data according to the pre-constructed correspondence between the exclusive data category and the exclusive content library; For the target query data, an exclusive content search is performed in the target exclusive content library to obtain an exclusive search result corresponding to the target query data.

10. The method according to claim 8, wherein: The obtaining of a plurality of content search results according to the general search results and the exclusive search results includes: Determine whether the exclusive search result meets the preset result screening condition; If the conditions are met, multiple content search results are obtained from the exclusive search results; if the conditions are not met, the general search results and the exclusive search results are merged to obtain multiple content search results.

11. The method according to any one of claims 5 to 7, wherein: The step of generating response data corresponding to the target query data by using a generative language model based on the filtered search results includes: Extracting content from the filtered search results to obtain title content in the filtered search results and text content in the filtered search results; Generate prompt information in a preset data format using the target query data, the title content in the filtered search results, and the text content in the filtered search results; The prompt information is input into a pre-trained generative language model, and the generative language model is used to output response data corresponding to the target query data.

12. A data processing method, applied to a client device having a browser installed thereon, the method comprising: In response to a data input operation in a browser page, determining target query data; Sending the target query data to the server device, so that when the server device determines that there is target offline reply data corresponding to the target query data in the offline index database, the target offline reply data is determined as the reply data corresponding to the target query data; When it is determined that the target offline reply data does not exist in the offline index database, the reply data corresponding to the target query data is generated by using a retrieval enhancement generation method based on the webpage content library; Receive the response data returned by the server device and display the response data on the browser page.

13. The method according to claim 12, wherein: The displaying of the reply data in the browser page includes: Creating a reply display page; the reply display page includes: a reply data display area and a source display area; Receive the reply data returned by the server device, as well as reference source information of the reply data; Loading the reference source information of the reply data in the source display area of ​​the reply display page; The reply data is loaded in the reply data display area in the reply display page, and a reference source identifier is added to the loaded reply data; the reference source identifier is identifier information generated for the reference source information.

14. The method according to claim 13, wherein: The method further comprises: In response to a triggering operation on the target reference source information loaded in the source display area, target reference web page content corresponding to the target reference source information is displayed.

15. The method according to claim 13, wherein: The method further comprises: In response to a triggering operation of adding a target reference source identifier for the reply data in the reply data display area, determining target reference source information according to the target reference source identifier; Display the target reference web page content corresponding to the target reference source information.

16. The method according to any one of claims 13 to 15, wherein: The reply display page also includes: a recommended information display area; The displaying of the reply data in the browser page further includes: Performing semantic extraction on the reply data to determine target service information semantically related to the reply data; Calling a service interface corresponding to the target service information, and receiving service recommendation information matching the reply data through the service interface; The service recommendation information is displayed in the recommendation information display area.

17. The method according to any one of claims 12 to 15, wherein: After sending the target query data to the server device, the method further includes: The generation progress information is displayed on the browser page, and the generation progress information includes one of the following information: retrieval status information, sorting status information, generation status information, and result display information.

18. An electronic device, comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform an operation corresponding to the method according to any one of claims 1-17.

19. A computer storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 17 is implemented.

20. A computer program product, comprising computer instructions, wherein the computer instructions instruct a computing device to execute operations corresponding to the method according to any one of claims 1 to 17.

Images