A data query method and device, computer equipment and a storage medium
By combining generative models and search engines in the data query process, the problem of incomplete or insufficient query results in existing technologies is solved, and more accurate and relevant query results are generated.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAOHONGSHU TECH CO LTD
- Filing Date
- 2023-07-19
- Publication Date
- 2026-07-24
AI Technical Summary
Existing technologies have problems in data querying, such as failing to retrieve the results needed by the user or retrieving too few results, resulting in ineffective queries.
The data to be queried is input into a pre-trained generative model, post-processed, and then input into a search engine. The generated data and search data are then fused together to generate query results.
It improves the effectiveness of data queries, ensuring that the generated query results are more accurate and relevant.
Smart Images
Figure CN117725110B_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of image processing technology, and in particular to a data query method, apparatus, computer equipment, and storage medium. Background Technology
[0002] Currently, data retrieval primarily relies on search engines. Users input query data (such as keywords or text) and the search engine's database searches for documents matching that query. However, this method requires retrieving results from existing data in the database, which may result in few or no results, or fail to retrieve the desired information. Therefore, finding more efficient data retrieval methods has become a key research focus. Summary of the Invention
[0003] This application provides a data query method, apparatus, computer device, and storage medium, which can effectively generate query result data corresponding to the data to be queried, thereby improving the effectiveness of data query.
[0004] In a first aspect, embodiments of this application provide a data query method, including:
[0005] Obtain the data to be queried and input the data to be queried into a pre-trained generative model to obtain the first generated data;
[0006] The first generated data is post-processed according to preset post-processing rules to obtain the second generated data;
[0007] Inputting the data to be queried into a preset search engine yields one or more search results corresponding to the data to be queried.
[0008] The second generated data and the one or more search data are fused together to obtain query result data corresponding to the data to be queried.
[0009] Secondly, embodiments of this application provide a data query device, including:
[0010] The acquisition unit is used to acquire the data to be queried and input the data to be queried into a pre-trained generative model to obtain the first generated data;
[0011] The processing unit is used to post-process the first generated data according to a preset post-processing rule to obtain the second generated data;
[0012] The search unit is used to input the data to be queried into a preset search engine to obtain one or more search data corresponding to the data to be queried.
[0013] The query unit is used to perform fusion processing on the second generated data and the one or more search data to obtain query result data corresponding to the data to be queried.
[0014] Thirdly, embodiments of this application provide a computer device, the computer device including: a processor and a memory, the processor being configured to execute the method described in the first aspect above.
[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium storing program instructions that, when executed, implement the method described in the first aspect above.
[0016] This application embodiment obtains the data to be queried and inputs it into a pre-trained generative model to obtain first generated data, which can be generated through the model. The first generated data is then post-processed according to preset post-processing rules to obtain second generated data, which helps improve the accuracy and relevance of the generated data. The data to be queried is then input into a preset search engine to obtain one or more search data corresponding to the data to be queried. The second generated data and one or more search data are then fused to obtain query result data corresponding to the data to be queried, which can effectively generate query result data corresponding to the data to be queried and improve the effectiveness of data query. Attached Figure Description
[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0018] Figure 1 This is a flowchart illustrating a data query method provided in an embodiment of this application;
[0019] Figure 2 This is a flowchart illustrating another data query method provided in an embodiment of this application;
[0020] Figure 3 This is a schematic diagram of the structure of a data query device provided in an embodiment of this application;
[0021] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation
[0022] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.
[0023] This application proposes a data query scheme. The scheme involves acquiring the data to be queried and inputting it into a pre-trained generative model to obtain first generated data. The first generated data is then post-processed according to preset post-processing rules to obtain second generated data. The data to be queried is then input into a preset search engine to obtain one or more search data corresponding to the data to be queried. Finally, the second generated data and the one or more search data are fused to obtain query result data corresponding to the data to be queried. This method eliminates the need for a database to generate query result data corresponding to the data to be queried, thus improving the effectiveness of data querying.
[0024] The data query method provided in this application can be applied to any scenario that requires data querying.
[0025] The data query method provided in this application embodiment can be applied to a data query device, which can be set in a computer device. In some embodiments, the computer device may include, but is not limited to, smart terminal devices such as smartphones, tablets, laptops, desktop computers, in-vehicle smart terminals, and smartwatches.
[0026] The data query method provided in the embodiments of this application will be illustrated below with reference to the accompanying drawings.
[0027] Please see details. Figure 1 , Figure 1 This is a flowchart illustrating a data query method provided in an embodiment of this application. The data query method in this embodiment can be executed by a data query device, which can be installed in a computer device.
[0028] S101: Obtain the data to be queried and input the data to be queried into the pre-trained generative model to obtain the first generated data.
[0029] In this embodiment of the application, a computer device can acquire data to be queried and input the data to be queried into a pre-trained generative model to obtain first generated data. The data to be queried may include, but is not limited to, one or more of characters, letters, and numbers. In one example, the data to be queried may be a query word or query text.
[0030] In one embodiment, before inputting the query data into a pre-trained generative model to obtain the first generated data, the computer device may acquire a first sample dataset, which includes multiple first sample query data, and input the first sample dataset into a preset deep learning model for training to obtain a first generative model; then acquire a second sample dataset and input the second sample dataset into the first generative model for training to obtain a second generative model.
[0031] Furthermore, the preset deep learning model may include, but is not limited to, Transformer, GPT, etc. The computer device can input multiple first-sample query data from the first-sample dataset into the preset deep learning model to obtain first-sample generated data. A first loss function value is determined based on the first-sample generated data. The model parameters of the deep learning model are adjusted based on the first loss function value. Multiple first-sample query data from the first-sample dataset are then input into the deep learning model. When the retrained first loss function value is less than a first function threshold, the first generation model is determined. Each first-sample query data in the first-sample dataset may include any one or more characters such as text, letters, and numbers.
[0032] In one implementation, when acquiring a second sample dataset, the computer device may acquire multiple second sample query data sets, input each second sample query data set into the first generation model to obtain second sample generated data corresponding to each second sample query data set; calculate the association score of each second sample generated data set, and select one or more second sample generated data sets in descending order of the association score; determine the binary dataset composed of the selected one or more second sample generated data sets and the second sample query data sets corresponding to each second sample generated data set as the second sample dataset. In some embodiments, the second sample query data sets may be the same as the first sample query data sets; in other embodiments, the second sample query data sets may be different from the first sample query data sets.
[0033] In one implementation, when calculating the association score of each second sample generated data, the computer device can calculate the content quality score, relevance score, and click-through rate of each second sample generated data; and perform a fusion process on the content quality score, relevance score, and click-through rate of each second sample generated data to obtain the association score of each second sample generated data.
[0034] Optionally, the computer device can calculate the content quality score, relevance score, and click-through rate of each second sample generated data. When fusing the content quality score, relevance score, and click-through rate of each second sample generated data, a weighted average can be applied to obtain the association score of each second sample generated data.
[0035] In one example, assuming a second sample generated data has a content quality score of a, a relevance score of b, and a click-through rate of c, the relevance score d of this second sample generated data can be calculated as shown in the following formula (1):
[0036]
[0037] Optionally, the content quality score of each second sample generated data is used to indicate the content quality of each second sample generated data, such as the writing quality of text. The computer device can calculate the content quality score of each second sample generated data according to the preset calculation rules, wherein the higher the score, the better the content quality of the second sample generated data.
[0038] Optionally, when calculating the relevance score of each second sample generated data, the computer device can use a cosine similarity algorithm or the like to calculate the similarity between each second sample generated data and the corresponding second sample query data. The higher the similarity, the higher the relevance between each second sample generated data and the corresponding second sample query data.
[0039] Optionally, when calculating the click-through rate of each second sample generated data, the computer device can obtain the number of clicks on each second sample generated data and the total number of clicks on all second sample generated data, and determine the ratio of the number of clicks on each second sample generated data to the total number of clicks on all second sample generated data as the click-through rate of each second sample generated data. In one example, assuming that second sample generated data x is clicked m times and second sample generated data y is clicked f times, then the click-through rate of x is as shown in formula (2) and the click-through rate of y is as shown in formula (3).
[0040]
[0041]
[0042] In one implementation, when the computer device selects one or more second sample generated data in descending order of the association score, it can select the top K second sample generated data in descending order of the association score, and further determine that the top K second sample generated data and the second sample query data that generated the K second sample generated data form a binary dataset, and determine the binary dataset as the second sample dataset, where K is an integer greater than 0.
[0043] In one example, for the second sample query word q1, the second sample generated data corresponding to q1 obtained by the first generation model is d1; for the second sample query word q2, the second sample generated data corresponding to q2 obtained by the first generation model is d2; and for the second sample query word q3, the second sample generated data corresponding to q3 obtained by the first generation model is d3. If the association scores are ranked from high to low as q1, q3, q2, then the binary dataset {(q1,d1),(q3,d3),(q2,d2)} can be determined.
[0044] Calculating the association score of each second sample generated data helps to improve the association of the second sample dataset, which in turn helps to further improve the relevance of the data generated by the trained second generative model.
[0045] In one implementation, when the computer device inputs the second sample dataset into the first generative model for training to obtain a second generative model, it can input the second sample query data from each pair of data in the second sample dataset into the first generative model for training, so that when the third sample generated data generated by the first generative model fits the second sample generated data in the pair of data, the second generative model is determined to be obtained.
[0046] Optionally, the computer device can input the second sample query data from each pair of data in the second sample dataset into the first generative model to obtain the third sample generated data, determine the second loss function value based on the third sample generated data, adjust the model parameters of the first generative model based on the second loss function value, and input the second sample query data from each pair of data in the second sample dataset into the first generative model after adjusting the model parameters for retraining. When the second loss function value obtained after retraining is less than the second function threshold, it is determined that the third sample generated data generated by the first generative model fits the second sample generated data in the pair of data.
[0047] Furthermore, when determining the second loss function value based on the third sample generated data, the computer device can compare the third sample generated data with the corresponding second sample generated data in the second sample dataset, and determine the second loss function value based on the comparison result. Optionally, when comparing the third sample generated data with the corresponding second sample generated data in the second sample dataset, the computer device can use a preset similarity algorithm to calculate the similarity between the third sample generated data and the corresponding second sample generated data in the second sample dataset, and determine the second loss function value based on the similarity. The preset similarity algorithm may include, but is not limited to, the cosine similarity algorithm.
[0048] By retraining the first generative model to fit the data generated by the first generative model to obtain the second generative model, the accuracy and relevance of the data generated by the second generative model can be improved.
[0049] Optionally, after the computer device inputs the second sample query data from each pair of data in the second sample dataset into the first generative model for training, and the third sample generated data generated by the first generative model fits the second sample generated data in the pair of data, and after determining that the second generative model has been obtained, it can further collect a third sample dataset. This third sample dataset includes a pair of data, each pair containing multiple pairs. Each pair includes a third sample query data and a fourth sample generated data. The fourth sample generated data is the optimal sample generated data corresponding to the third sample query data, selected by algorithm or manually. The third sample dataset is then used as input data for the second generative model, so that the fifth sample generated data generated by the second generative model fits the optimal sample generated data corresponding to the third sample query data in the third sample dataset.
[0050] By retraining the second generative model to fit the data generated by the second generative model, it is helpful to further improve the accuracy and relevance of the data generated by the second generative model.
[0051] In one embodiment, when the computer device inputs the query data into a pre-trained generative model to obtain first generated data, it can input the query data into the first generative model to obtain third generated data; and input the third generated data into a second generative model to obtain the first generated data.
[0052] S102: Post-process the first generated data according to the preset post-processing rules to obtain the second generated data.
[0053] In this embodiment of the application, the computer device can perform post-processing on the first generated data according to preset post-processing rules to obtain the second generated data.
[0054] In one embodiment, when a computer device performs post-processing on the first generated data according to preset post-processing rules to obtain second generated data, it may, if it detects that the first generated data satisfies the privacy conditions set in the preset post-processing rules, execute the step of inputting the data to be queried into a pre-trained generative model to obtain the first generated data, until the regenerated first generated data no longer satisfies the privacy conditions, at which point the second generated data is determined to be obtained. In some embodiments, the privacy conditions may include, but are not limited to, privacy data, including but not limited to usernames, addresses, telephone numbers, ID card numbers, etc. Optionally, the computer device may pre-train a discriminative model (such as the BERT model) to identify whether privacy data exists in the first generated data.
[0055] This approach can effectively address the issue of privacy leaks, help improve the security of private data, and enhance the user experience.
[0056] In one optional implementation, when a computer device detects that the first generated data meets the sensitive conditions set in the preset post-processing rules, it can add a prompt tag to the first generated data. This prompt tag indicates that the first generated data includes sensitive data, and the first generated data with the added prompt tag is determined to be the second generated data. In some embodiments, the sensitive data can be custom data, including but not limited to data related to financial management, healthcare, and laws and regulations.
[0057] This approach avoids generating sensitive data and improves the user experience.
[0058] S103: Input the data to be queried into a preset search engine to obtain one or more search data corresponding to the data to be queried.
[0059] In this embodiment, the computer device can input the data to be queried into a preset search engine to obtain one or more search results corresponding to the data to be queried. The preset search engine can be an online search engine, which will not be specifically described here.
[0060] S104: Merge the second generated data and the one or more search data to obtain query result data corresponding to the data to be queried.
[0061] In this embodiment of the application, the computer device can perform fusion processing on the second generated data and the one or more search data to obtain query result data corresponding to the data to be queried.
[0062] In one embodiment, when a computer device performs fusion processing on the second generated data and the one or more search data to obtain query result data corresponding to the data to be queried, it can obtain the relevance between the second generated data and each search data, and sort the one or more search data according to the relevance between the second generated data and each search data; and select one or more search data and the second generated data for text fusion processing in descending order of sorting to obtain query result data corresponding to the data to be queried.
[0063] In one optional implementation, when the computer device obtains the relevance between the second generated data and each search data, it can use a preset relevance algorithm to calculate the relevance between the second generated data and each search data. The preset relevance algorithm may include, but is not limited to, the cosine similarity algorithm.
[0064] In one optional implementation, when the computer device selects one or more search data and performs text fusion processing with the second generated data in descending order of sorting to obtain query result data corresponding to the data to be queried, it can select the top N search data in descending order of sorting and add the N search data to the second generated data to obtain query result data, where N is an integer greater than 0.
[0065] In one example, suppose the computer device selects the top 4 search texts in descending order of sorting, namely search text 1, search text 2, search text 3, and search text 4, and adds these 4 search texts to the second generated text S to obtain query result data, which includes search text 1, search text 2, search text 3, search text 4, and the second generated text S.
[0066] By combining search data obtained from search engines with data generated by models to determine the query results, the effectiveness of data retrieval can be improved.
[0067] This application embodiment obtains the data to be queried and inputs it into a pre-trained generative model to obtain first generated data, which can be generated through the model. The first generated data is then post-processed according to preset post-processing rules to obtain second generated data, which helps improve the accuracy and relevance of the generated data. The data to be queried is then input into a preset search engine to obtain one or more search data corresponding to the data to be queried. The second generated data and one or more search data are then fused to obtain query result data corresponding to the data to be queried, which can effectively generate query result data corresponding to the data to be queried and improve the effectiveness of data query.
[0068] Please see details. Figure 2 , Figure 2 This is a flowchart illustrating another data query method provided in this application embodiment. The data query method of this application embodiment can be executed by a data query device, which is located in a computer device, as explained above. Specifically, the method of this application embodiment includes the following steps.
[0069] S201: Retrieve the data to be queried.
[0070] S202: Input the data to be queried into the preset search engine to obtain the search dataset.
[0071] The search dataset includes multiple search data, such as document1, document2, ..., documentn, which may contain n documents.
[0072] S203: Check whether the data to be queried matches the data in the specified database. If the detection result is a match, proceed to step S204; if the detection result is a mismatch, proceed to step S205.
[0073] The specified database stores multiple pairs of data consisting of data to be queried and generated data. The generated data in the pairs of data stored in the specified database is obtained in advance by querying the data to be queried through the second generation model. The specified database stores the pairs of data and the correspondence between the data to be queried and the generated data in the pairs of data.
[0074] S204: Retrieve generated data corresponding to the data to be queried from the specified database.
[0075] When a computer device retrieves generated data corresponding to the data to be queried from the specified database, it can query the generated data corresponding to the current data to be queried from the specified database according to the correspondence between the data to be queried and the generated data in the specified database.
[0076] S205: Input the data to be queried into the second pre-trained generative model to obtain the generated data corresponding to the data to be queried, and store the correspondence between the generated data and the data to be queried in the specified database.
[0077] By storing the correspondence between the data to be queried and the generated data obtained using the model in a specified database, it is helpful to increase the data in the specified database.
[0078] S206: The generated data is merged with the various search data in the search dataset to obtain the query result data.
[0079] This application embodiment queries the data to be queried by combining a preset search engine and a trained second generative model, which can generate query result data that is not in the database and corresponds to the data to be queried, thereby improving the effectiveness and accuracy of data query.
[0080] Please see Figure 3 , Figure 3 This is a schematic diagram of the structure of a data query device provided in an embodiment of this application. Specifically, the device is disposed in a computer device and includes: an acquisition unit 301, a processing unit 302, a search unit 303, and a query unit 304;
[0081] The acquisition unit 301 is used to acquire the data to be queried and input the data to be queried into the pre-trained generative model to obtain the first generated data;
[0082] Processing unit 302 is used to post-process the first generated data according to preset post-processing rules to obtain the second generated data;
[0083] Search unit 303 is used to input the data to be queried into a preset search engine to obtain one or more search data corresponding to the data to be queried;
[0084] The query unit 304 is used to perform fusion processing on the second generated data and the one or more search data to obtain query result data corresponding to the data to be queried.
[0085] Furthermore, before the acquisition unit 301 inputs the data to be queried into the pre-trained generative model to obtain the first generated data, it is also used for:
[0086] Obtain a first sample dataset, which includes multiple first sample query data, and input the first sample dataset into a preset deep learning model for training to obtain a first generative model;
[0087] Obtain a second sample dataset and input it into the first generative model for training to obtain a second generative model.
[0088] Furthermore, when the acquisition unit 301 acquires the second sample dataset, it is specifically used for:
[0089] Multiple second sample query data are obtained, and each second sample query data is input into the first generation model to obtain the second sample generated data corresponding to each second sample query data.
[0090] Calculate the association score for each second sample generated data, and select one or more second sample generated data in descending order of the association score;
[0091] The dataset consisting of one or more selected second sample generated data and the corresponding second sample query data is defined as the second sample dataset.
[0092] Furthermore, when the acquisition unit 301 calculates the association score of each second sample generated data, it is specifically used for:
[0093] Calculate the content quality score, relevance score, and click-through rate of each of the second sample generated data;
[0094] The content quality score, relevance score, and click-through rate of each second sample generated data are fused to obtain the association score of each second sample generated data.
[0095] Further, when the acquisition unit 301 inputs the data to be queried into the pre-trained generative model to obtain the first generated data, it is specifically used for:
[0096] The data to be queried is input into the first generation model to obtain the third generation data;
[0097] The third generated data is input into the second generated model to obtain the first generated data.
[0098] Furthermore, when the processing unit 302 performs post-processing on the first generated data according to preset post-processing rules to obtain the second generated data, it is specifically used for:
[0099] If the first generated data is detected to meet the privacy conditions set in the preset post-processing rules, the step of inputting the data to be queried into the pre-trained generative model to obtain the first generated data is executed, until the regenerated first generated data no longer meets the privacy conditions, at which point the second generated data is determined to be obtained; or,
[0100] If the first generated data is detected to meet the sensitive conditions set in the preset post-processing rules, a prompt label is added to the first generated data. The prompt label is used to indicate that the first generated data includes sensitive data, and the first generated data with the added prompt label is determined to be the second generated data.
[0101] Furthermore, when the query unit 304 fuses the second generated data and the one or more search data to obtain query result data corresponding to the data to be queried, it is specifically used for:
[0102] Obtain the relevance between the second generated data and each search data, and sort the one or more search data according to the relevance between the second generated data and each search data;
[0103] According to the sorting order from high to low, one or more search data are selected and combined with the second generated data for text fusion processing to obtain query result data corresponding to the data to be queried.
[0104] This application embodiment obtains the data to be queried and inputs it into a pre-trained generative model to obtain first generated data, which can be generated through the model. The first generated data is then post-processed according to preset post-processing rules to obtain second generated data, which helps improve the accuracy and relevance of the generated data. The data to be queried is then input into a preset search engine to obtain one or more search data corresponding to the data to be queried. The second generated data and one or more search data are then fused to obtain query result data corresponding to the data to be queried, which can effectively generate query result data corresponding to the data to be queried and improve the effectiveness of data query.
[0105] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Specifically, the computer device includes: a memory 401 and a processor 402.
[0106] In one embodiment, the computer device further includes a data interface 403 for transmitting data information between the computer device and other devices.
[0107] The memory 401 may include volatile memory; the memory 401 may also include non-volatile memory; the memory 401 may also include a combination of the above types of memory. The processor 402 may be a central processing unit (CPU). The processor 402 may further include hardware chips. The hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The PLDs may be complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or any combination thereof.
[0108] The memory 401 is used to store programs, and the processor 402 can call the programs stored in the memory 401 to perform the following steps:
[0109] Obtain the data to be queried and input the data to be queried into a pre-trained generative model to obtain the first generated data;
[0110] The first generated data is post-processed according to preset post-processing rules to obtain the second generated data;
[0111] Inputting the data to be queried into a preset search engine yields one or more search results corresponding to the data to be queried.
[0112] The second generated data and the one or more search data are fused together to obtain query result data corresponding to the data to be queried.
[0113] Furthermore, before the processor 402 inputs the query data into the pre-trained generative model to obtain the first generated data, it is also used for:
[0114] Obtain a first sample dataset, which includes multiple first sample query data, and input the first sample dataset into a preset deep learning model for training to obtain a first generative model;
[0115] Obtain a second sample dataset and input it into the first generative model for training to obtain a second generative model.
[0116] Furthermore, when the processor 402 acquires the second sample dataset, it is specifically used for:
[0117] Multiple second sample query data are obtained, and each second sample query data is input into the first generation model to obtain the second sample generated data corresponding to each second sample query data.
[0118] Calculate the association score for each second sample generated data, and select one or more second sample generated data in descending order of the association score;
[0119] The dataset consisting of one or more selected second sample generated data and the corresponding second sample query data is defined as the second sample dataset.
[0120] Furthermore, when the processor 402 calculates the association score of each second sample generated data, it is specifically used for:
[0121] Calculate the content quality score, relevance score, and click-through rate of each of the second sample generated data;
[0122] The content quality score, relevance score, and click-through rate of each second sample generated data are fused to obtain the association score of each second sample generated data.
[0123] Furthermore, when the processor 402 inputs the query data into the pre-trained generative model to obtain the first generated data, it is specifically used for:
[0124] The data to be queried is input into the first generation model to obtain the third generation data;
[0125] The third generated data is input into the second generated model to obtain the first generated data.
[0126] Furthermore, when the processor 402 performs post-processing on the first generated data according to preset post-processing rules to obtain the second generated data, it is specifically used for:
[0127] If the first generated data is detected to meet the privacy conditions set in the preset post-processing rules, the step of inputting the data to be queried into the pre-trained generative model to obtain the first generated data is executed, until the regenerated first generated data no longer meets the privacy conditions, at which point the second generated data is determined to be obtained; or,
[0128] If the first generated data is detected to meet the sensitive conditions set in the preset post-processing rules, a prompt label is added to the first generated data. The prompt label is used to indicate that the first generated data includes sensitive data, and the first generated data with the added prompt label is determined to be the second generated data.
[0129] Further, when the processor 402 fuses the second generated data and the one or more search data to obtain query result data corresponding to the data to be queried, it is specifically used for:
[0130] Obtain the relevance between the second generated data and each search data, and sort the one or more search data according to the relevance between the second generated data and each search data;
[0131] According to the sorting order from high to low, one or more search data are selected and combined with the second generated data for text fusion processing to obtain query result data corresponding to the data to be queried.
[0132] This application embodiment obtains the data to be queried and inputs it into a pre-trained generative model to obtain first generated data, which can be generated through the model. The first generated data is then post-processed according to preset post-processing rules to obtain second generated data, which helps improve the accuracy and relevance of the generated data. The data to be queried is then input into a preset search engine to obtain one or more search data corresponding to the data to be queried. The second generated data and one or more search data are then fused to obtain query result data corresponding to the data to be queried, which can effectively generate query result data corresponding to the data to be queried and improve the effectiveness of data query.
[0133] Embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements this application. Figure 1 or Figure 2 The method described in the corresponding embodiment can also be implemented. Figure 3 The apparatus described in the embodiments corresponding to this application will not be repeated here.
[0134] The computer-readable storage medium can be an internal storage unit of the device described in any of the foregoing embodiments, such as the device's hard drive or memory. The computer-readable storage medium can also be an external storage device of the device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the device. Further, the computer-readable storage medium may include both internal and external storage units of the device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.
[0135] Embodiments of this application also provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various embodiments described above.
[0136] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0137] The above-disclosed embodiments are merely some of the embodiments of this application, and should not be construed as limiting the scope of this application. Those skilled in the art can understand that implementing all or part of the above embodiments and making equivalent changes in accordance with the claims of this application still fall within the scope of this invention.
Claims
1. A data query method, characterized in that, include: Retrieve the data to be queried; Detect whether the data to be queried matches the data in the specified database; If the detection result is a match, the generated data corresponding to the data to be queried is obtained from the specified database as the first generated data. The specified database stores multiple pairs of data consisting of the data to be queried and the generated data. The generated data in the pairs of data stored in the specified database is obtained in advance by querying the data to be queried through the generation model. The specified database stores the pairs of data and the correspondence between the data to be queried and the generated data in the pairs of data. If the detection result is a mismatch, the data to be queried is input into the pre-trained generative model to obtain the generated data corresponding to the data to be queried as the first generated data, and the correspondence between the generated data and the data to be queried is stored in the specified database. The first generated data is post-processed according to preset post-processing rules to obtain the second generated data; Inputting the data to be queried into a preset search engine yields one or more search results corresponding to the data to be queried. The second generated data and the one or more search data are fused together to obtain query result data corresponding to the data to be queried.
2. The method according to claim 1, characterized in that, Before inputting the query data into the pre-trained generative model to obtain the generated data corresponding to the query data as the first generated data, the method further includes: Obtain a first sample dataset, which includes multiple first sample query data, and input the first sample dataset into a preset deep learning model for training to obtain a first generative model; Obtain a second sample dataset and input it into the first generative model for training to obtain a second generative model.
3. The method according to claim 2, characterized in that, The process of obtaining the second sample dataset includes: Multiple second sample query data are obtained, and each second sample query data is input into the first generation model to obtain the second sample generated data corresponding to each second sample query data. Calculate the association score for each second sample generated data, and select one or more second sample generated data in descending order of the association score; The dataset consisting of one or more selected second sample generated data and the corresponding second sample query data is defined as the second sample dataset.
4. The method according to claim 3, characterized in that, The calculation of the association score for each second sample generated data includes: Calculate the content quality score, relevance score, and click-through rate of each of the second sample generated data; The content quality score, relevance score, and click-through rate of each second sample generated data are fused to obtain the association score of each second sample generated data.
5. The method according to claim 2, characterized in that, The step of inputting the query data into a pre-trained generative model to obtain generated data corresponding to the query data as the first generated data includes: The data to be queried is input into the first generation model to obtain the third generation data; The third generated data is input into the second generated model to obtain the first generated data.
6. The method according to claim 1, characterized in that, The step of post-processing the first generated data according to preset post-processing rules to obtain the second generated data includes: If the first generated data is detected to meet the privacy conditions set in the preset post-processing rules, the step of inputting the data to be queried into the pre-trained generative model to obtain the first generated data is executed, until the regenerated first generated data no longer meets the privacy conditions, at which point the second generated data is determined to be obtained; or, If the first generated data is detected to meet the sensitive conditions set in the preset post-processing rules, a prompt label is added to the first generated data. The prompt label is used to indicate that the first generated data includes sensitive data, and the first generated data with the added prompt label is determined to be the second generated data.
7. The method according to claim 1, characterized in that, The process of fusing the second generated data and the one or more search data to obtain query result data corresponding to the data to be queried includes: Obtain the relevance between the second generated data and each search data, and sort the one or more search data according to the relevance between the second generated data and each search data; According to the sorting order from high to low, one or more search data are selected and combined with the second generated data for text fusion processing to obtain query result data corresponding to the data to be queried.
8. A data query device, characterized in that, include: The retrieval unit is used to retrieve the data to be queried. The acquisition unit is further configured to detect whether the data to be queried matches data in a specified database; If the detection result is a match, the generated data corresponding to the data to be queried is retrieved from the designated database as the first generated data. The designated database stores multiple pairs of data consisting of the data to be queried and the generated data. The generated data in the pairs of data stored in the designated database is obtained in advance by querying the data to be queried using a generative model. The designated database stores the pairs of data and the correspondence between the data to be queried and the generated data within the pairs of data. If the detection result is a mismatch, the data to be queried is input into a pre-trained generative model to obtain the generated data corresponding to the data to be queried, which is then used as the first generated data. The correspondence between the generated data and the data to be queried is stored in the designated database. The processing unit is used to post-process the first generated data according to a preset post-processing rule to obtain the second generated data; The search unit is used to input the data to be queried into a preset search engine to obtain one or more search data corresponding to the data to be queried. The query unit is used to perform fusion processing on the second generated data and the one or more search data to obtain query result data corresponding to the data to be queried.
9. A computer device, characterized in that, The device includes a processor and a memory interconnected thereto, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to perform the method as described in any one of claims 1-7.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that, when executed, implement the method as described in any one of claims 1-7.