A data rewriting method and device, computer equipment and storage medium

Abstract

Embodiments of the present application disclose a data rewriting method and device, computer equipment and a storage medium. The method comprises: obtaining a sample data set, the sample data set comprising a plurality of sample query data, and determining a sample data feature vector corresponding to each sample query data in the plurality of sample query data; inputting the sample data feature vector corresponding to each sample query data into a preset neural network model for training to obtain a first data rewriting model; using a preset contrast learning algorithm and each sample query data to optimize the first data rewriting model to obtain a second data rewriting model; obtaining to-be-processed data, and inputting the to-be-processed data into the second data rewriting model to generate target rewriting data corresponding to the to-be-processed data. In this way, more accurate target rewriting data can be generated, the target rewriting data is used as query data for data query, and the retrieval efficiency and accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of data processing technology, and in particular to a data rewriting method, apparatus, computer equipment, and storage medium. Background Technology

[0002] In the field of information retrieval, query rewriting is an important issue. The purpose of query rewriting is to transform the user-input query (i.e., the data to be queried) into a more specific and precise query, thereby obtaining better search results. Traditional query rewriting methods are mostly based on rules and statistical models, and the results are not ideal. Summary of the Invention

[0003] This application provides a data rewriting method, apparatus, computer device, and storage medium that can generate more accurate query data, thereby improving retrieval efficiency and accuracy.

[0004] In a first aspect, embodiments of this application provide a data rewriting method, including:

[0005] Obtain a sample dataset, which includes multiple sample query data, and determine the sample data feature vector corresponding to each sample query data in the multiple sample query data;

[0006] The sample data feature vectors corresponding to the data to be queried for each sample are input into a preset neural network model for training to obtain the first data rewriting model.

[0007] The first data rewriting model is optimized using a preset contrastive learning algorithm and the query data of each sample to obtain a second data rewriting model.

[0008] Obtain the data to be processed and input the data to be processed into the second data rewriting model to generate target rewritten data corresponding to the data to be processed.

[0009] Secondly, embodiments of this application provide a data rewriting apparatus, including:

[0010] An acquisition unit is used to acquire a sample dataset, which includes multiple sample query data, and to determine the sample data feature vector corresponding to each sample query data in the multiple sample query data.

[0011] The training unit is used to input the sample data feature vectors corresponding to the query data of each sample into a preset neural network model for training, so as to obtain the first data rewriting model.

[0012] An optimization unit is used to optimize the first data rewriting model using a preset contrastive learning algorithm and the sample query data to obtain a second data rewriting model.

[0013] The generation unit is used to acquire the data to be processed and input the data to be processed into the second data rewriting model to generate target rewritten data corresponding to the data to be processed.

[0014] Thirdly, embodiments of this application provide a computer device, the computer device including: a processor and a memory, the processor being configured to execute the method described in the first aspect above.

[0015] Fourthly, embodiments of this application also provide a computer-readable storage medium storing program instructions that, when executed, implement the method described in the first aspect above.

[0016] This application embodiment can acquire a sample dataset, which includes multiple sample query data, and determine the sample data feature vector corresponding to each sample query data. The sample data feature vectors corresponding to each sample query data are input into a preset neural network model for training to obtain a first data rewriting model. A preset contrastive learning algorithm and the sample query data are used to optimize the first data rewriting model to obtain a second data rewriting model. Data to be processed is acquired and input into the second data rewriting model to generate target rewritten data corresponding to the data to be processed. This method can generate more accurate target rewritten data, which can then be used as the query data, helping to improve retrieval efficiency and accuracy. Attached Figure Description

[0017] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0018] Figure 1 This is a flowchart illustrating a data rewriting method provided in an embodiment of this application;

[0019] Figure 2 This is a flowchart illustrating another data rewriting method provided in an embodiment of this application;

[0020] Figure 3 This is a schematic diagram of the structure of a data rewriting device provided in an embodiment of this application;

[0021] Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0023] This application proposes a data rewriting method applicable to various retrieval scenarios. It involves acquiring a sample dataset comprising multiple sample query data and determining the feature vectors corresponding to each sample query data. These feature vectors are then input into a pre-defined neural network model for training, resulting in a first data rewriting model. This first model is then optimized using a pre-defined contrastive learning algorithm and the sample query data, yielding a second data rewriting model. Finally, the data to be processed is acquired and input into the second data rewriting model to generate target rewritten data corresponding to the data to be processed. This method generates more accurate target rewritten data, which can then be used as new retrieval data, thus improving retrieval efficiency and accuracy.

[0024] The data rewriting method provided in this application embodiment can be applied to a data rewriting device, which can be installed in a computer device. In some embodiments, the computer device may include, but is not limited to, smart terminal devices such as smartphones, tablets, laptops, desktop computers, in-vehicle smart terminals, and smartwatches.

[0025] The data rewriting method provided in the embodiments of this application will be illustrated below with reference to the accompanying drawings.

[0026] Please see details. Figure 1 , Figure 1 This is a flowchart illustrating a data rewriting method provided in an embodiment of this application. The data rewriting method in this embodiment can be executed by a data rewriting device, which can be located in a computer device.

[0027] S101: Obtain the sample dataset, which includes multiple sample query data, and determine the sample data feature vector corresponding to each sample query data in the multiple sample query data.

[0028] In an embodiment of the present application, a computer device may obtain a sample data set, which includes multiple sample data to be queried, and determine sample data feature vectors corresponding to each of the multiple sample data to be queried. Among them, the sample data to be queried may include, but is not limited to, text data composed of any one or more characters such as text, letters, numbers, etc.

[0029] In one embodiment, when the computer device determines the sample data feature vectors corresponding to each of the multiple sample data to be queried, it may perform feature vector representation (i.e., embedding representation) on each of the sample data to be queried to convert the sample data to be queried into the corresponding sample data feature vectors.

[0030] In one embodiment, when the computer device determines the sample data feature vectors corresponding to each of the multiple sample data to be queried, it may obtain one or more words corresponding to each of the sample data to be queried, convert each word into the corresponding word vector; splice the word vectors corresponding to each of the sample data to be queried to obtain a sequence vector, and determine the sequence vector as the sample data feature vector corresponding to each of the sample data to be queried.

[0031] For example, assume that a sample data to be queried is "steamed bun making method", then the computer device may obtain the 4 words "bao", "zi", "zuo", and "fa" corresponding to the sample data to be queried, convert the word "bao" into the corresponding bao word vector, the word "zi" into the corresponding zi word vector, the word "zuo" into the corresponding zuo word vector, and the word "fa" into the corresponding fa word vector; splice the bao word vector, zi word vector, zuo word vector, and fa word vector to obtain a sequence vector, and determine the sequence vector as the sample data feature vector corresponding to the sample data to be queried "steamed bun making method".

[0032] In this way, the word order information of the input sample data to be queried can be retained, which is beneficial for the subsequent model to generate more reasonable outputs (i.e., target rewritten data).

[0033] S102: Input the sample data feature vectors corresponding to each of the sample data to be queried into a preset neural network model for training to obtain a first data rewriting model.

[0034] In an embodiment of the present application, the computer device may input the sample data feature vectors corresponding to each of the sample data to be queried into a preset neural network model for training to obtain a first data rewriting model.

[0035] In some embodiments, the pre-set neural network model can be a Generative Pre-trained Transformer (GPT) model. The GPT model is a pre-trained language generation model that learns language representations through pre-training and can be used to generate natural language text. The GPT model has wide applications in tasks such as natural language generation, text classification, and language understanding. This GPT model is a neural network model based on a Transformer structure. This model only contains a decoder; both encoding and decoding in GPT are implemented through the decoder. The first part of the decoder is used to encode the sample query data, and then the encoded data is used to generate the output text.

[0036] In one embodiment, when the computer device inputs the sample data feature vectors corresponding to each sample query data into a preset neural network model for training to obtain a first data rewriting model, it can input the sample data feature vectors corresponding to each sample query data into the preset neural network model to generate the current character corresponding to each sample query data; when generating the current character, it obtains the vectors of all characters generated before the current character, and inputs the vectors of all characters generated before the current character into the preset neural network model for training to obtain the first data rewriting model.

[0037] In one implementation, when a computer device inputs the feature vectors of sample data corresponding to each sample of query data into a preset neural network model to generate the current character corresponding to each sample of query data, it can input the feature vector of sample data corresponding to one sample of query data into the preset neural network model. When generating the second character corresponding to the sample of query data, it obtains the vector of the first character generated before the second character and inputs the vector of the first character into the preset neural network model to generate the third character. It further obtains the vectors of the first character and the second character generated before the third character and inputs the vectors of the first character and the second character into the preset neural network model to generate the fourth character. This process is repeated to input the vectors of each character before the current character into the preset neural network model for training, thereby obtaining the first data rewriting model.

[0038] In one implementation, when a computer device inputs the vectors of all characters generated before the current character into a preset neural network model, it can concatenate the vectors of all characters generated before the current character to obtain a concatenated vector. This concatenated vector is then input into the preset neural network model for training, resulting in a first data rewriting model. In one example, assuming that the current character is preceded by a first character and a second character, the computer device can concatenate the vectors of the first and second characters, input the concatenated vector into the preset neural network model, and generate a fourth character.

[0039] In one embodiment, when the computer device inputs the sample data feature vectors corresponding to each sample query data into a preset neural network model for training to obtain a first data rewriting model, it can input the sample data feature vectors corresponding to each sample query data into the preset neural network model to generate the current character corresponding to each sample query data; when generating the current character, it obtains the vectors of the current character and all characters generated before the current character, and inputs the vectors of the current character and all characters generated before the current character into the preset neural network model for training to obtain the first data rewriting model.

[0040] In one implementation, a computer device can input the feature vector of sample data corresponding to a sample query data into a preset neural network model to generate the first character corresponding to the sample query data. The vector of the first character is then input into the preset neural network model to generate the second character. The vector of the second character is then obtained, and the vectors of the first and second characters are input into the preset neural network model to generate the third character. This process is repeated, with the vectors of the current character and all characters preceding the current character being input into the preset neural network model for training, to obtain the first data rewriting model.

[0041] In one embodiment, when the computer device inputs the sample data feature vectors corresponding to each sample query data into a preset neural network model for training to obtain a first data rewriting model, it can input the sample data feature vectors corresponding to each sample query data into the preset neural network model to generate the current character corresponding to each sample query data; when generating the current character, it obtains the vectors of all characters generated before the current character, and inputs the vectors of all characters generated before the current character and the sample data feature vectors corresponding to each sample query data into the preset neural network model for training to obtain the first data rewriting model.

[0042] In one implementation, a computer device can input the feature vector of sample data corresponding to a sample of query data into a preset neural network model to generate the first character corresponding to the sample of query data. The vector of the first character and the feature vector of the sample of query data corresponding to the sample of query data are then input into the preset neural network model to generate the second character. The vector of the second character is then obtained, and the vectors of the first character, the second character, and the feature vector of the sample of query data corresponding to the sample of query data are input into the preset neural network model to generate the third character. This process is repeated, with the vector of the current character, the vectors of all characters preceding the current character, and the feature vector of the sample of query data corresponding to the sample of query data being input into the preset neural network model for training, to obtain a first data rewriting model.

[0043] Training the first data rewriting model in this way helps improve its reliability and accuracy.

[0044] S103: Optimize the first data rewriting model using a preset contrastive learning algorithm and the query data of each sample to obtain the second data rewriting model.

[0045] In this embodiment of the application, the computer device can use a preset contrastive learning algorithm and the sample data to be queried to optimize the first data rewriting model and obtain a second data rewriting model.

[0046] In one embodiment, when a computer device optimizes a first data rewriting model using a preset contrastive learning algorithm and the sample query data to obtain a second data rewriting model, it can input the sample data feature vectors corresponding to the sample query data into the first data rewriting model to generate text data corresponding to the sample data feature vectors; compare the text data corresponding to the sample data feature vectors with the sample query data using the preset contrastive learning algorithm to obtain a third comparison result; and optimize the first data rewriting model based on the third comparison result to obtain the second data rewriting model.

[0047] In one embodiment, when the computer device optimizes the first data rewriting model based on the third comparison result to obtain the second data rewriting model, it can determine the second loss function value based on the third comparison result. If the second loss function value does not meet the second function threshold, the model parameters of the preset neural network model are adjusted based on the third comparison result. The sample data feature vectors corresponding to the data to be queried for each sample are input into the first data rewriting model after the model parameters are adjusted for retraining. If the second loss function value obtained after retraining meets the second function threshold, the second data rewriting model is determined to be obtained.

[0048] Furthermore, the computer device can use negative sampling within a batch to select negative samples, and use the query data to be queried input into the model as positive samples.

[0049] This application embodiment optimizes the first data rewriting model by employing a contrastive learning method to make the generated target rewritten data query closer to the good case and farther from the bad case.

[0050] The embodiments of this application combine GPT with contrastive learning to optimize data rewriting, resulting in a significant improvement in the effectiveness of data rewriting and improvements in metrics such as precision, recall, and F1 score.

[0051] S104: Obtain the data to be processed and input the data to be processed into the second data rewriting model to generate target rewritten data corresponding to the data to be processed.

[0052] In this embodiment of the application, a computer device can acquire data to be processed and input the data to be processed into the second data rewriting model to generate target rewritten data corresponding to the data to be processed.

[0053] This application embodiment can acquire a sample dataset, which includes multiple sample query data and query result data corresponding to each sample query data. It also determines the sample data feature vector corresponding to each sample query data. The sample data feature vectors corresponding to each sample query data are input into a preset neural network model for training to obtain a first data rewriting model. A preset contrastive learning algorithm and the sample query data are used to optimize the first data rewriting model to obtain a second data rewriting model. The data to be processed is acquired and input into the second data rewriting model to generate target rewritten data corresponding to the data to be processed. This method can generate more accurate target rewritten data, which can then be used as new retrieval data, helping to improve retrieval efficiency and accuracy.

[0054] Please see details. Figure 2 , Figure 2 This is a flowchart illustrating another data rewriting method provided in an embodiment of this application. The data rewriting method of this application embodiment can be executed by a data rewriting device, wherein the data rewriting device is disposed in a computer device, and the specific explanation of the computer device is as described above. Specifically, the method of this application embodiment includes the following steps.

[0055] S201: Obtain a sample dataset, which includes multiple sample query data, and determine the sample data feature vector corresponding to each sample query data in the multiple sample query data.

[0056] S202: Input the feature vector of the sample data corresponding to each sample to be queried into the preset neural network model to generate the current word corresponding to each sample to be queried.

[0057] In this embodiment of the application, the computer device can input the sample data feature vector corresponding to each sample data to be queried into a preset neural network model to generate the current word corresponding to each sample data to be queried.

[0058] S203: When generating the current character, obtain the vectors of all characters generated before the current character, and input the vectors of all characters generated before the current character into a preset neural network model for training to obtain the first data rewriting model.

[0059] In this embodiment of the application, the computer device can obtain the vectors of all characters generated before the current character when generating the current character, and input the vectors of all characters generated before the current character into a preset neural network model for training to obtain a first data rewriting model.

[0060] In one embodiment, the computer device can input the vectors of all characters generated before the current character into a preset neural network model to obtain the target character after the current character, until all characters corresponding to the sample data feature vectors of each sample query data are generated; the generated characters are compared with each sample query data to obtain a first comparison result, and the preset neural network model is trained based on the first comparison result to obtain a first data rewriting model.

[0061] For example, assume that the sample data feature vector corresponding to the sample query data "method of making steamed buns" is input into a preset neural network model to generate the current character "bao" corresponding to the sample query data "method of making steamed buns"; in the case of generating the current character "bao", since there are no other characters before the character "bao", continue to generate the character "zi" as the current character, and then obtain the vectors of all the characters "bao" generated before the current character "zi". Input the vectors of all the characters "bao" generated before the current character into the preset neural network model to obtain the target character "pi" after the current character. Then, input the vectors of all the characters "bao zi" before "pi" into the preset neural network model to obtain the target character "zuo" after the current character. Next, input the vectors of all the characters "bao zi pi" before "zuo" into the preset neural network model to obtain the target character "fa" after the current character, and generate all the characters "bao zi pi fa" corresponding to the sample data feature vector of the sample query data "method of making steamed buns"; compare all the generated characters "bao zi pi fa" with the sample query data "method of making steamed buns" to obtain a first comparison result, and train the preset neural network model according to the first comparison result to obtain a first data rewriting model.

[0062] In one embodiment, the computer device can input the vector of the current character and the vectors of all the characters generated before the current character into the preset neural network model to obtain the target character after the current character until all the characters corresponding to the sample data feature vector of each sample query data are generated; compare all the generated characters with each sample query data to obtain a first comparison result, and train the preset neural network model according to the first comparison result to obtain a first data rewriting model.

[0063] For example, assume that the sample data feature vector corresponding to the sample query data "steamed bun recipe" is input into a preset neural network model to generate the current character "bao" corresponding to the sample query data "steamed bun recipe"; in the case of generating the current character "bao", the vector of the character "bao" is input into the preset neural network model to generate the character "zi" as the current character, and then the vector of the current character "zi" and the vectors of all the characters "bao" generated before "zi" are obtained. The vector of the current character "zi" and the vectors of all the characters "bao" generated before "zi" are input into the preset neural network model to obtain the target character "pi" after the current character. Then, the vector of "pi" and the vectors of all the characters "bao zi" before "pi" are input into the preset neural network model to obtain the target character "zuo" after the current character. Then, the vector of "zuo" and the vectors of all the characters "bao zi pi" before "zuo" are input into the preset neural network model to obtain the target character "fa" after the current character, and all the characters "bao zi pi zuo fa" corresponding to the sample data feature vector of the sample query data "steamed bun recipe" are generated; the generated all characters "bao zi pi zuo fa" are compared with the sample query data "steamed bun recipe" to obtain a first comparison result, and the preset neural network model is trained according to the first comparison result to obtain a first data rewriting model.

[0064] In one embodiment, the computer device can input the vector of the current character, the vectors of all the characters generated before the current character, and the sample data feature vector of the sample query data into a preset neural network model to obtain the target character after the current character until all the characters corresponding to the sample data feature vector of each sample query data are generated; the generated all characters are compared with each sample query data to obtain a first comparison result, and the preset neural network model is trained according to the first comparison result to obtain a first data rewriting model.

[0065] For example, assume that the sample data feature vector corresponding to the sample query data "steamed bun recipe" is input into a preset neural network model to generate the current character "bao" corresponding to the sample query data "steamed bun recipe"; in the case of generating the current character "bao", the vector of the character "bao" and the sample data feature vector corresponding to "steamed bun recipe" are input into the preset neural network model to generate the character "zi" as the current character, and then the vector of the current character "zi" and the vectors of all the characters "bao" generated before "zi" are obtained. The vector of the current character "zi", the vectors of all the characters "bao" generated before "zi", and the sample data feature vector corresponding to "steamed bun recipe" are input into the preset neural network model to obtain the target character "pi" after the current character. Then, the vector of "pi", the vectors of all the characters "bao zi" before "pi", and the sample data feature vector corresponding to "steamed bun recipe" are input into the preset neural network model to obtain the target character "xian" after the current character. Then, the vector of "xian", the vector of "bao zi pi", and the sample data feature vector corresponding to "steamed bun recipe" are input into the preset neural network model to obtain the target character "zuo" after the current character. Further, the vector of "zuo", the vectors of all the characters "bao zi pi xian" before "zuo", and the sample data feature vector corresponding to "steamed bun recipe" are input into the preset neural network model to obtain the target character "fa" after the current character. The generated characters "bao zi pi xian zuo fa" are compared with the sample query data "steamed bun recipe" to obtain a first comparison result, and the preset neural network model is trained according to the first comparison result to obtain a first data rewriting model.

[0066] In this way, it helps to train a more reliable and effective first data rewriting model.

[0067] In one embodiment, when the computer device trains the preset neural network model according to the first comparison result to obtain the first data rewriting model, it can determine a third loss function value according to the first comparison result; in the case where the third loss function value does not meet the third function threshold, the model parameters of the preset neural network model are adjusted according to the first comparison result; the sample data feature vectors corresponding to each sample query data are input into the neural network model with adjusted model parameters for retraining, and when the third loss function value obtained from the retraining meets the function threshold, the first data rewriting model is determined.

[0068] In one implementation, when the computer device inputs the sample data feature vectors corresponding to each sample of query data into a preset neural network model for training to obtain a first data rewriting model, it can input the sample data feature vectors corresponding to each sample of query data into the preset neural network model to generate each character corresponding to each sample of query data; compare the generated characters corresponding to each sample of query data with each sample of query data to obtain a second comparison result, and determine a first loss function value based on the second comparison result; if the first loss function value does not meet the first function threshold, adjust the model parameters of the preset neural network model according to the second comparison result; input the sample data feature vectors corresponding to each sample of query data into the neural network model after adjusting the model parameters for retraining, and if the first loss function value obtained after retraining meets the first function threshold, determine that the first data rewriting model is obtained.

[0069] This approach can improve the reliability of the first data rewriting model, ensuring the relevance of the generated data to the sample query data, which is beneficial for generating more specific and accurate data.

[0070] S204: Optimize the first data rewriting model using a preset contrastive learning algorithm and the data to be queried for each sample to obtain the second data rewriting model.

[0071] S205: Obtain the data to be processed and input the data to be processed into the second data rewriting model to generate target rewritten data corresponding to the data to be processed.

[0072] This application embodiment can acquire a sample dataset, which includes multiple sample query data and query result data corresponding to each sample query data. It also determines the sample data feature vector corresponding to each sample query data. The sample data feature vectors corresponding to each sample query data are input into a preset neural network model to generate the current character corresponding to each sample query data. When the current character is generated, the vectors of all characters generated before the current character are obtained and input into the preset neural network model for training, resulting in a first data rewriting model. The first data rewriting model is then optimized using a preset contrastive learning algorithm and the sample query data to obtain a second data rewriting model. The data to be processed is acquired and input into the second data rewriting model to generate target rewritten data corresponding to the data to be processed. This method improves the reliability of the first data rewriting model, ensuring the relevance of the generated data to the sample query data. It facilitates the generation of more specific and accurate target rewritten data, enabling the target rewritten data to be used as new retrieval data, thus improving retrieval efficiency and accuracy.

[0073] Please see Figure 3 , Figure 3 This is a schematic diagram of a data rewriting device provided in an embodiment of this application. Specifically, the device is disposed in a computer device and includes: an acquisition unit 301, a training unit 302, an optimization unit 303, and a generation unit 304;

[0074] The acquisition unit 301 is used to acquire a sample dataset, which includes multiple sample query data, and to determine the sample data feature vector corresponding to each sample query data in the multiple sample query data.

[0075] Training unit 302 is used to input the sample data feature vectors corresponding to the sample data to be queried into a preset neural network model for training to obtain a first data rewriting model.

[0076] The optimization unit 303 is used to optimize the first data rewriting model using a preset contrastive learning algorithm and the query data of each sample to obtain a second data rewriting model.

[0077] The generation unit 304 is used to acquire the data to be processed and input the data to be processed into the second data rewriting model to generate target rewritten data corresponding to the data to be processed.

[0078] Furthermore, when the acquisition unit 301 determines the sample data feature vector corresponding to each sample data to be queried in the plurality of sample data to be queried, it is specifically used for:

[0079] Obtain one or more characters corresponding to the data to be queried in each sample, and convert each character into a corresponding character vector;

[0080] The word vectors corresponding to the query data of each sample are concatenated to obtain a sequence vector, and the sequence vector is determined to be the sample data feature vector corresponding to the query data of each sample.

[0081] Further, when the training unit 302 inputs the sample data feature vectors corresponding to each sample data to be queried into a preset neural network model for training, and obtains the first data rewriting model, it is specifically used for:

[0082] Input the feature vector of the sample data corresponding to each sample data to be queried into a preset neural network model to generate the current character corresponding to each sample data to be queried.

[0083] When generating the current character, obtain the vectors of all characters generated before the current character, and input the vectors of all characters generated before the current character into the preset neural network model for training to obtain the first data rewriting model.

[0084] Further, when the training unit 302 inputs the vectors of all characters generated before the current character into the preset neural network model for training to obtain the first data rewriting model, it is specifically used for:

[0085] The vectors of all characters generated before the current character are input into the preset neural network model to obtain the target character after the current character, until all characters corresponding to the sample data feature vectors of each sample query data are generated;

[0086] The generated characters are compared with the sample data to be queried to obtain a first comparison result, and the preset neural network model is trained based on the first comparison result to obtain the first data rewriting model.

[0087] Further, when the training unit 302 inputs the sample data feature vectors corresponding to each sample data to be queried into a preset neural network model for training, and obtains the first data rewriting model, it is specifically used for:

[0088] The feature vectors of the sample data corresponding to each sample data to be queried are input into a preset neural network model to generate each character corresponding to each sample data to be queried.

[0089] The characters corresponding to the generated sample query data are compared with the sample query data to obtain a second comparison result, and the first loss function value is determined based on the second comparison result.

[0090] If the first loss function value does not meet the first function threshold, the model parameters of the preset neural network model are adjusted according to the second comparison result;

[0091] The sample data feature vectors corresponding to the data to be queried for each sample are input into the neural network model after adjusting the model parameters for retraining. If the first loss function value obtained by retraining satisfies the first function threshold, the first data rewriting model is determined.

[0092] Furthermore, when the optimization unit 303 optimizes the first data rewriting model using a preset contrastive learning algorithm and the sample query data to obtain the second data rewriting model, it is specifically used for:

[0093] Input the sample data feature vectors corresponding to the query data of each sample into the first data rewriting model to generate text data corresponding to each sample data feature vector.

[0094] A third comparison result is obtained by comparing the text data corresponding to the feature vectors of each sample data with the query data of each sample data using a preset contrastive learning algorithm.

[0095] The first data rewriting model is optimized based on the third comparison result to obtain the second data rewriting model.

[0096] Furthermore, when the optimization unit 303 optimizes the first data rewriting model based on the third comparison result to obtain the second data rewriting model, it is specifically used for:

[0097] The second loss function value is determined based on the third comparison result. If the second loss function value does not meet the second function threshold, the model parameters of the preset neural network model are adjusted based on the third comparison result.

[0098] The sample data feature vectors corresponding to the query data of each sample are input into the first data rewriting model after adjusting the model parameters for retraining. If the second loss function value obtained by retraining satisfies the second function threshold, the second data rewriting model is determined.

[0099] This application embodiment can acquire a sample dataset, which includes multiple sample query data and query result data corresponding to each sample query data. It also determines the sample data feature vector corresponding to each sample query data. The sample data feature vectors corresponding to each sample query data are input into a preset neural network model for training to obtain a first data rewriting model. A preset contrastive learning algorithm and the sample query data are used to optimize the first data rewriting model to obtain a second data rewriting model. The data to be processed is acquired and input into the second data rewriting model to generate target rewritten data corresponding to the data to be processed. This method can generate more accurate target rewritten data, which can then be used as new retrieval data, helping to improve retrieval efficiency and accuracy.

[0100] Please see Figure 4 , Figure 4 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Specifically, the computer device includes: a memory 401 and a processor 402.

[0101] In one embodiment, the computer device further includes a data interface 403 for transmitting data information between the computer device and other devices.

[0102] The memory 401 may include volatile memory; the memory 401 may also include non-volatile memory; the memory 401 may also include a combination of the above types of memory. The processor 402 may be a central processing unit (CPU). The processor 402 may further include hardware chips. The hardware chips may be application-specific integrated circuits (ASICs), programmable logic devices (PLDs), or combinations thereof. The PLDs may be complex programmable logic devices (CPLDs), field-programmable gate arrays (FPGAs), or any combination thereof.

[0103] The memory 401 is used to store programs, and the processor 402 can call the programs stored in the memory 401 to perform the following steps:

[0104] Obtain a sample dataset, which includes multiple sample query data, and determine the sample data feature vector corresponding to each sample query data in the multiple sample query data;

[0105] The sample data feature vectors corresponding to the data to be queried for each sample are input into a preset neural network model for training to obtain the first data rewriting model.

[0106] The first data rewriting model is optimized using a preset contrastive learning algorithm and the query data of each sample to obtain a second data rewriting model.

[0107] Obtain the data to be processed and input the data to be processed into the second data rewriting model to generate target rewritten data corresponding to the data to be processed.

[0108] Furthermore, when the processor 402 determines the sample data feature vector corresponding to each sample data to be queried in the plurality of sample data to be queried, it is specifically used for:

[0109] Obtain one or more characters corresponding to the data to be queried in each sample, and convert each character into a corresponding character vector;

[0110] The word vectors corresponding to the query data of each sample are concatenated to obtain a sequence vector, and the sequence vector is determined to be the sample data feature vector corresponding to the query data of each sample.

[0111] Further, when the processor 402 inputs the sample data feature vectors corresponding to each sample query data into a preset neural network model for training to obtain the first data rewriting model, it is specifically used for:

[0112] Input the feature vector of the sample data corresponding to each sample data to be queried into a preset neural network model to generate the current character corresponding to each sample data to be queried.

[0113] When generating the current character, obtain the vectors of all characters generated before the current character, and input the vectors of all characters generated before the current character into the preset neural network model for training to obtain the first data rewriting model.

[0114] Further, when the processor 402 inputs the vectors of all characters generated before the current character into the preset neural network model for training to obtain the first data rewriting model, it is specifically used for:

[0115] The vectors of all characters generated before the current character are input into the preset neural network model to obtain the target character after the current character, until all characters corresponding to the sample data feature vectors of each sample query data are generated;

[0116] The generated characters are compared with the sample data to be queried to obtain a first comparison result, and the preset neural network model is trained based on the first comparison result to obtain the first data rewriting model.

[0117] Further, when the processor 402 inputs the sample data feature vectors corresponding to each sample query data into a preset neural network model for training to obtain the first data rewriting model, it is specifically used for:

[0118] The feature vectors of the sample data corresponding to each sample data to be queried are input into a preset neural network model to generate each character corresponding to each sample data to be queried.

[0119] The characters corresponding to the generated sample query data are compared with the sample query data to obtain a second comparison result, and the first loss function value is determined based on the second comparison result.

[0120] If the first loss function value does not meet the first function threshold, the model parameters of the preset neural network model are adjusted according to the second comparison result;

[0121] The sample data feature vectors corresponding to the data to be queried for each sample are input into the neural network model after adjusting the model parameters for retraining. If the first loss function value obtained by retraining satisfies the first function threshold, the first data rewriting model is determined.

[0122] Furthermore, when the processor 402 optimizes the first data rewriting model using a preset contrastive learning algorithm and the sample query data to obtain the second data rewriting model, it is specifically used for:

[0123] Input the sample data feature vectors corresponding to the query data of each sample into the first data rewriting model to generate text data corresponding to each sample data feature vector.

[0124] A third comparison result is obtained by comparing the text data corresponding to the feature vectors of each sample data with the query data of each sample data using a preset contrastive learning algorithm.

[0125] The first data rewriting model is optimized based on the third comparison result to obtain the second data rewriting model.

[0126] Furthermore, when the processor 402 optimizes the first data rewriting model based on the third comparison result to obtain the second data rewriting model, it specifically performs the following operations:

[0127] The second loss function value is determined based on the third comparison result. If the second loss function value does not meet the second function threshold, the model parameters of the preset neural network model are adjusted based on the third comparison result.

[0128] The sample data feature vectors corresponding to the query data of each sample are input into the first data rewriting model after adjusting the model parameters for retraining. If the second loss function value obtained by retraining satisfies the second function threshold, the second data rewriting model is determined.

[0129] This application embodiment can acquire a sample dataset, which includes multiple sample query data and query result data corresponding to each sample query data. It also determines the sample data feature vector corresponding to each sample query data. The sample data feature vectors corresponding to each sample query data are input into a preset neural network model for training to obtain a first data rewriting model. A preset contrastive learning algorithm and the sample query data are used to optimize the first data rewriting model to obtain a second data rewriting model. The data to be processed is acquired and input into the second data rewriting model to generate target rewritten data corresponding to the data to be processed. This method can generate more accurate target rewritten data, which can then be used as new retrieval data, helping to improve retrieval efficiency and accuracy.

[0130] Embodiments of this application also provide a computer-readable storage medium storing a computer program that, when executed by a processor, implements this application. Figure 1 or Figure 2 The method described in the corresponding embodiment can also be implemented. Figure 3 The apparatus described in the embodiments corresponding to this application will not be repeated here.

[0131] The computer-readable storage medium can be an internal storage unit of the device described in any of the foregoing embodiments, such as the device's hard drive or memory. The computer-readable storage medium can also be an external storage device of the device, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the device. Further, the computer-readable storage medium may include both internal and external storage units of the device. The computer-readable storage medium is used to store the computer program and other programs and data required by the computer device. The computer-readable storage medium can also be used to temporarily store data that has been output or will be output.

[0132] Embodiments of this application also provide a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the methods provided in the various embodiments described above.

[0133] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.

[0134] The above-disclosed embodiments are merely some of the embodiments of this application, and should not be construed as limiting the scope of this application. Those skilled in the art can understand that implementing all or part of the above embodiments and making equivalent changes in accordance with the claims of this application still fall within the scope of this invention.

Claims

1. A data rewriting method, characterized in that, include: Obtain a sample dataset, which includes multiple sample query data, and determine the sample data feature vector corresponding to each sample query data in the multiple sample query data; The first data rewriting model is obtained by inputting the sample data feature vectors corresponding to each sample data to be queried into a preset neural network model for training. The model includes: inputting the sample data feature vectors corresponding to each sample data to be queried into a preset neural network model to generate the current character corresponding to each sample data to be queried; and, when generating the current character, obtaining the vectors of all characters generated before the current character, and inputting the vectors of all characters generated before the current character into the preset neural network model for training to obtain the first data rewriting model. The step of inputting the vectors of all characters generated before the current character into the preset neural network model for training to obtain the first data rewriting model includes: inputting the vectors of all characters generated before the current character into the preset neural network model to obtain the target character after the current character, until all characters corresponding to the sample data feature vectors corresponding to each sample query data are generated; comparing the generated all characters with each sample query data to obtain a first comparison result, and training the preset neural network model based on the first comparison result to obtain the first data rewriting model; The first data rewriting model is optimized using a preset contrastive learning algorithm and the query data of each sample to obtain a second data rewriting model. This includes: inputting the sample data feature vectors corresponding to the query data of each sample into the first data rewriting model to generate text data corresponding to each sample data feature vector; comparing the text data corresponding to each sample data feature vector with the query data of each sample using the preset contrastive learning algorithm to obtain a third comparison result; and optimizing the first data rewriting model based on the third comparison result to obtain the second data rewriting model. Obtain the data to be processed and input the data to be processed into the second data rewriting model to generate target rewritten data corresponding to the data to be processed.

2. The method according to claim 1, characterized in that, Determining the feature vector of each sample data to be queried in the plurality of sample data to be queried includes: Obtain one or more characters corresponding to the query data of each sample, and convert each character into a corresponding character vector; The word vectors corresponding to the query data of each sample are concatenated to obtain a sequence vector, and the sequence vector is determined to be the sample data feature vector corresponding to the query data of each sample.

3. The method according to claim 2, characterized in that, The step of inputting the feature vectors of the sample data corresponding to each sample data to be queried into a preset neural network model for training to obtain a first data rewriting model includes: The feature vectors of the sample data corresponding to each sample data to be queried are input into a preset neural network model to generate each character corresponding to each sample data to be queried. The characters corresponding to the generated sample query data are compared with the sample query data to obtain a second comparison result, and the first loss function value is determined based on the second comparison result. If the first loss function value does not meet the first function threshold, the model parameters of the preset neural network model are adjusted according to the second comparison result; The sample data feature vectors corresponding to the data to be queried for each sample are input into the neural network model after adjusting the model parameters for retraining. If the first loss function value obtained by retraining satisfies the first function threshold, the first data rewriting model is determined.

4. The method according to claim 3, characterized in that, The step of optimizing the first data rewriting model based on the third comparison result to obtain the second data rewriting model includes: The second loss function value is determined based on the third comparison result. If the second loss function value does not meet the second function threshold, the model parameters of the preset neural network model are adjusted based on the third comparison result. The sample data feature vectors corresponding to the query data of each sample are input into the first data rewriting model after adjusting the model parameters for retraining. If the second loss function value obtained by retraining satisfies the second function threshold, the second data rewriting model is determined.

5. A data rewriting apparatus, the apparatus being used to perform the method as described in any one of claims 1-4, characterized in that, include: An acquisition unit is used to acquire a sample dataset, which includes multiple sample query data, and to determine the sample data feature vector corresponding to each sample query data in the multiple sample query data. The training unit is used to input the sample data feature vectors corresponding to the query data of each sample into a preset neural network model for training, so as to obtain the first data rewriting model. An optimization unit is used to optimize the first data rewriting model using a preset contrastive learning algorithm and the sample query data to obtain a second data rewriting model. The generation unit is used to acquire the data to be processed and input the data to be processed into the second data rewriting model to generate target rewritten data corresponding to the data to be processed.

6. A computer device, characterized in that, The device includes a processor and a memory interconnected thereto, wherein the memory is used to store a computer program, the computer program including program instructions, and the processor is configured to invoke the program instructions to perform the method as described in any one of claims 1-4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores program instructions that, when executed, implement the method as described in any one of claims 1-4.

Images