Data correction method and device, equipment, medium and program product

Abstract

The invention provides a data correction method and device, equipment, a medium and a program product, and the method comprises the steps: determining a target error type based on to-be-corrected data; determining a target correction rule based on the to-be-corrected data and the target error type; based on the target correction rule, correcting the to-be-corrected data to obtain corrected data; wherein the to-be-corrected data and the corrected data are expressed in a preset data exchange format; the target error type is related to the writing rule of the preset data exchange format. According to the method and the device, the accuracy of correcting the data for data exchange can be improved.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a data correction method, device, equipment, medium and program product. Background Art

[0002] In today's era of rapid digitalization and informatization, the rapid spread and mass generation of information has become the norm. Whether it is written communication, data processing or program operation, the existence of errors may lead to serious consequences. In particular, errors in the data used for data exchange will lead to serious consequences. For example, when a large language model returns JSON (JavaScript Object Notation, a lightweight data exchange format) data with an abnormal format, the business side often has no choice but to use backup data or perform unlimited business retries.

[0003] Therefore, error correction of data used for data exchange is critical to ensuring the accuracy, completeness and availability of information. Summary of the invention

[0004] The present application provides a data correction method, apparatus, device, medium and program product for improving the accuracy of correcting data used for data exchange.

[0005] According to a first aspect of an embodiment of the present application, a data correction method is provided, comprising:

[0006] Determine the target error type based on the data to be corrected;

[0007] Determining a target correction rule based on the data to be corrected and the target error type;

[0008] Based on the target correction rule, the data to be corrected is corrected to obtain corrected data;

[0009] The data to be corrected and the corrected data are represented in a preset data exchange format; and the target error type is related to a writing rule of the preset data exchange format.

[0010] Optionally, determining a target error type based on the data to be corrected includes:

[0011] Matching the data to be corrected with the sample error features in the sample feature library to obtain matching target sample error features and features to be corrected; wherein the features to be corrected are extracted from the data to be corrected;

[0012] Reading the error type corresponding to the target sample error feature from the sample feature library;

[0013] Determine the error type corresponding to the target sample error feature as the target error type corresponding to the feature to be corrected;

[0014] Among them, the sample feature library includes sample error features, error types, and a mapping relationship between sample error features and error types; the sample error features are represented in the preset data exchange format; and the error type is determined based on the writing rules of the preset data exchange format.

[0015] Optionally, the determining a target correction rule based on the data to be corrected and the target error type includes:

[0016] Completing the feature to be corrected in the data to be corrected to obtain a completed feature;

[0017] Acquire each correction rule corresponding to the target error type, and each sample error feature corresponding to each correction rule;

[0018] Based on the completed features and the sample error features corresponding to each correction rule, a target correction rule is determined from the correction rules.

[0019] Optionally, the step of completing the feature to be corrected in the data to be corrected to obtain the completed feature includes:

[0020] Obtaining original text content corresponding to the data to be corrected;

[0021] Obtain a target completion prompt word template corresponding to the target error type;

[0022] Generate a target completion prompt word based on the original text content, the to-be-corrected feature in the to-be-corrected data, and the target completion prompt word template;

[0023] Inputting the target completion prompt word into the large model to obtain the completed features;

[0024] The target completion prompt word is used to instruct the large model to complete the feature to be corrected in the data to be corrected based on the original text content and the target error type to obtain the completed feature.

[0025] Optionally, the correction rule is obtained by combining various basic editing operations.

[0026] Optionally, the correcting the data to be corrected based on the target correction rule to obtain corrected data includes:

[0027] Based on the target correction rule, the completed feature is corrected to obtain a corrected feature;

[0028] Based on the data to be corrected, the features to be corrected in the data to be corrected, and the corrected features, the data to be corrected is corrected to obtain corrected data.

[0029] Optionally, the method further comprises:

[0030] Obtaining user feedback on the corrected data;

[0031] Determine the user modified data based on the user feedback;

[0032] Based on the data to be corrected and the data modified by the user, training a data correction model to obtain a trained data correction model;

[0033] The data correction model is used to correct input data to obtain output data.

[0034] Optionally, the training of a data correction model based on the data to be corrected and the user-modified data to obtain a trained data correction model includes:

[0035] Based on the data to be corrected, the corrected data, the user-modified data, and the degree of matching between the corrected data and the user-modified data, a data correction model is trained to obtain a trained data correction model.

[0036] According to a second aspect of an embodiment of the present application, there is provided a data correction device, comprising:

[0037] A first processing unit, configured to determine a target error type based on the data to be corrected;

[0038] A second processing unit, configured to determine a target correction rule based on the data to be corrected and the target error type;

[0039] A correction unit, configured to correct the data to be corrected based on the target correction rule to obtain corrected data;

[0040] The data to be corrected and the corrected data are represented in a preset data exchange format.

[0041] According to a third aspect of an embodiment of the present application, there is provided an electronic device, including a memory and a processor;

[0042] The memory is connected to the processor and is used to store programs;

[0043] The processor is used to implement the data correction method as described in the first aspect by running the program in the memory.

[0044] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the data correction method as described in the first aspect is implemented.

[0045] According to a fifth aspect of an embodiment of the present application, a computer program product is provided, comprising computer program instructions, which, when executed by a processor, cause the processor to execute the data correction method as described in the first aspect.

[0046] In the present application, the data to be corrected and the corrected data are represented in a preset data exchange format. The data to be corrected and the corrected data are data used for data exchange. Based on the data to be corrected, the target error type is determined. The target error type is related to the writing rules of the preset data exchange format. In the process of correcting the data to be corrected, the writing rules of the preset data exchange format are fully considered, and then the target error type is determined, which can more accurately locate the target error type and improve the accuracy of locating the target error type. Based on the data to be corrected and the target error type, the target correction rule is determined. In the process of determining the correction rule, not only the target error type but also the text content of the data to be corrected itself is considered, which can improve the accuracy of determining the target correction rule. Moreover, on the basis of improving the accuracy of locating the target error type, the accuracy of determining the target correction rule can be further improved. Based on the target correction rule, the data to be corrected is corrected to obtain the corrected data. On the basis of improving the accuracy of determining the target correction rule, the accuracy of the corrected data can be further improved, thereby improving the accuracy of correcting the data used for data exchange. BRIEF DESCRIPTION OF THE DRAWINGS

[0047] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0048] Figure 1 A schematic diagram of a data correction method provided in an embodiment of the present application;

[0049] Figure 2 A schematic diagram of a process of step 101 provided in an embodiment of the present application;

[0050] Figure 3 A schematic diagram of a process of step 102 provided in an embodiment of the present application;

[0051] Figure 4A schematic diagram of a process of step 301 provided in an embodiment of the present application;

[0052] Figure 5 A schematic diagram of a process of step 103 provided in an embodiment of the present application;

[0053] Figure 6 A schematic diagram of a data correction method provided in an embodiment of the present application;

[0054] Figure 7 A schematic diagram of the structure of a data correction device provided in an embodiment of the present application;

[0055] Figure 8 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0056] Currently, errors in the data used for data exchange can lead to serious consequences. For example, when a large language model returns JSON data with an abnormal format, the business side often has no choice but to use backup data or perform unlimited business retries.

[0057] Therefore, error correction of data used for data exchange is crucial to ensure the accuracy, integrity and availability of information. Automatic error correction of data used for data exchange can not only improve work efficiency and reduce the cost of manual error correction, but also enhance the quality and reliability of information processing, providing strong support for the development of various fields.

[0058] At present, the existing solutions for error correction of data used for data exchange are mainly the following:

[0059] Solution 1

[0060] Based on the language understanding ability of the Large Language Model (LLM) pre-trained on massive text, error detection and correction are performed on the input data for data exchange. This technology performs well in processing complex language structures and semantic understanding, but may require a lot of computing resources.

[0061] Solution 2

[0062] Language knowledge is constructed into a knowledge graph structure, and the relationships and information in the knowledge graph are used to assist in judging and correcting errors in data used for data exchange. This technology can improve the accuracy of correction by utilizing rich knowledge, but the construction and maintenance of the knowledge graph is relatively complex.

[0063] Option 3

[0064] By defining a series of text editing operations (such as replacement, deletion, insertion, etc.), the erroneous data used for data exchange is gradually corrected. This technology is intuitive and easy to understand, but its ability to handle complex errors may be relatively limited.

[0065] The existing technology may still fail to deserialize after correction, for example, if it contains Chinese symbols, multiple separators, etc.

[0066] In order to improve the accuracy of correcting data used for data exchange, the present application provides a data correction method, device, equipment, medium and program product.

[0067] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0068] Exemplary Implementation Environment

[0069] The data correction method according to the embodiment of the present application can be executed by an electronic device such as a terminal device or a server. The terminal device can be a user device, a mobile device, a computing device, a wearable device, etc. The server can be an independent physical server, a server cluster composed of multiple physical servers, or a cloud server capable of cloud computing. The method can be implemented by a processor calling a computer-readable program instruction stored in a memory.

[0070] Exemplary Methods

[0071] See also Figure 1 In an exemplary embodiment, a data correction method is provided. Figure 1 As shown, the process of the data correction method mainly includes:

[0072] Step 101: determine the target error type based on the data to be corrected.

[0073] The data to be corrected is represented in a preset data exchange format. The target error type is related to the writing rule of the preset data exchange format.

[0074] In an exemplary embodiment, the preset data exchange format may be JSON, which is a lightweight data exchange format. The preset data exchange format may also be other formats, and the present application does not limit this. The following embodiments are explained using JSON as an example of the preset data exchange format.

[0075] In an exemplary embodiment, taking the preset data exchange format as JSON as an example, the writing rules of JSON may include the correct spelling of JSON, the sentence structure of JSON, and the semantic specification of JSON.

[0076] The correct spelling of JSON means that when writing JSON data, you need to make sure that all keys are enclosed in double quotes ("), and values ​​can be strings (enclosed in double quotes), numbers, Boolean values ​​(true or false), null, objects (enclosed in curly braces {}), or arrays (enclosed in square brackets []). An object is an unordered collection of key-value pairs, each key-value pair consists of a key and a value, and the key and value are separated by a colon ":", and the key-value pairs are separated by a comma ",". An array is an ordered collection of values, and the values ​​are separated by a comma ",".

[0077] The sentence structure of JSON is mainly composed of key-value pairs, which can be nested to form more complex objects or arrays.

[0078] The semantic specifications of JSON refer to key naming specifications (key names are usually named using lowercase letters and underscores (_) and avoid using uppercase letters, spaces or special characters), consistent data types (in the same array, the value type of all elements should be consistent), avoiding redundant data, using null to represent empty values, following consistent indentation and format, etc.

[0079] In some embodiments, Figure 2 As shown, step 101 includes:

[0080] Step 201 : Match the data to be corrected with the sample error features in the sample feature library to obtain matching target sample error features and features to be corrected.

[0081] The features to be corrected are extracted from the data to be corrected.

[0082] Step 202: read the error type corresponding to the target sample error feature from the sample feature library.

[0083] Step 203: determine the error type corresponding to the target sample error feature as the target error type corresponding to the feature to be corrected.

[0084] Among them, the sample feature library includes sample error features, error types, and the mapping relationship between sample error features and error types; the sample error features are represented in a preset data exchange format; the error type is determined based on the writing rules of the preset data exchange format.

[0085] In an exemplary embodiment, error types may include missing commas, unmatched brackets, use of single quotes instead of double quotes, missing key, unclear key intent, missing value, unclear value intent, etc. Error types may also include other types, which are not listed here one by one.

[0086] In an exemplary embodiment, the sample error feature may not be the entire sample error data. The entire sample error data may contain some correct content and some incorrect content. The sample error feature may be the incorrect content extracted from the sample error data. The sample error feature may also be the entire sample error data, and this application is not limited to this.

[0087] In an exemplary embodiment, the sample feature library includes sample error features, error types, and a mapping relationship between sample error features and error types. One error type may correspond to one sample error feature, and one error type may also correspond to at least two sample error features, and the present application does not limit this. For example, 1. Error type: missing comma, the corresponding sample error feature is ```json{"name":"Xiao Ming""age":18}```. 2. Error type: unmatched brackets, the corresponding sample error feature is ```json"name":"Xiao Ming""age":18}```. 3. Error type: use single quotes instead of double quotes, the corresponding sample error feature is ```json'name":"Xiao Ming""age":18}.

[0088] In an exemplary embodiment, steps 201 to 203 may be implemented by an error type recognition model, where the data to be corrected is input into the error type recognition model to obtain features to be corrected and target error types corresponding to the features to be corrected; wherein the features to be corrected are extracted from the data to be corrected; the error type recognition model is trained based on sample error features in a sample feature library and error types corresponding to the sample error features; the sample feature library includes sample error features, error types, and a mapping relationship between sample error features and error types; the sample error features are represented in a preset data exchange format; and the error type is determined based on the writing rules of the preset data exchange format.

[0089] The error type recognition model relies on accurate label classification of the sample feature library to make it easier to remember previous inputs, so that the error type recognition model can better support feature matching, precise search and intelligent recommendation capabilities.

[0090] In an exemplary embodiment, step 201 may include: matching the data to be corrected with the sample error features in the sample feature library based on a regular expression; extracting the features to be corrected in the data to be corrected that successfully match the sample error features; and determining the matched target sample error features and features to be corrected. Step 201 may also be implemented in other ways, and this application is not limited thereto.

[0091] The data to be corrected is matched with the sample error features in the sample feature library to obtain the matching target sample error features and features to be corrected, the error type corresponding to the target sample error features is read from the sample feature library, and the error type corresponding to the target sample error features is determined as the target error type corresponding to the features to be corrected. The sample error features are represented in a preset data exchange format, and the error type is determined based on the writing rules of the preset data exchange format. A broad knowledge graph is established through the sample feature library to analyze the data to be corrected, accurately obtain the features to be corrected of the related structural anomalies, and obtain the matching target sample error features and features to be corrected. By matching between the features, the error type can be more accurately located, thereby improving the accuracy of the target error type corresponding to the features to be corrected, and improving the accuracy of correcting the data used for data exchange.

[0092] In other embodiments, step 101 includes: generating an error type identification prompt word based on the data to be corrected; inputting the error type identification prompt word into the large model to obtain a target error type; wherein the error type identification prompt word is used to instruct the large model to identify the target error type corresponding to the data to be corrected.

[0093] Step 102: Determine a target correction rule based on the data to be corrected and the target error type.

[0094] In some embodiments, Figure 3 As shown, step 102 includes:

[0095] Step 301, completing the features to be corrected in the data to be corrected to obtain the completed features.

[0096] In some embodiments, Figure 4 As shown, step 301 includes:

[0097] Step 401, obtaining the original text content corresponding to the data to be corrected.

[0098] In an exemplary embodiment, the original text content corresponding to the data to be corrected may refer to the original context content corresponding to the data to be corrected. A specified number of relevant context documents or even the entire document collection may be included to obtain a more comprehensive context background. When expanding the context information, attention is paid to diversity and representativeness, and JSON texts covering different topics, styles, and fields are collected, so that the large model can be exposed to rich language expressions and semantic contexts, thereby obtaining more reasonable content.

[0099] In an exemplary embodiment, an exemplary data to be corrected is as follows:

[0100]

[0101] The original text content corresponding to this exemplary data to be corrected is as follows:

[0102] #Role

[0103] ##personal information

[0104] Your name is j_mid, female, 16 years old, birthday is July 1st. . . .

[0105] ##character

[0106] You are enthusiastic, cheerful and optimistic.

[0107] ## Family Information

[0108] You were born in Hefei, Anhui. Your father is a teacher and your mother is a surgeon.

[0109] #Skill

[0110] You are a professional psychological counselor who can understand user needs, respond based on user questions, recommend relevant user intent information, and reply to users in a lively and playful style.

[0111] #limit

[0112] - The reply should not exceed 50 words;

[0113] -Every time you recommend relevant intent information to a user, make sure it is consistent with the user's contextual semantic content.

[0114] Step 402: Obtain a target completion prompt word template corresponding to the target error type.

[0115] In an exemplary embodiment, the target error type may include key missing, key unclear intent, value missing, value unclear intent, etc. Key missing corresponds to a target completion prompt word template, key unclear intent corresponds to a target completion prompt word template, value missing corresponds to a target completion prompt word template, and value unclear intent corresponds to a target completion prompt word template.

[0116] Step 403: Generate a target completion prompt word based on the original text content, the features to be corrected in the data to be corrected, and the target completion prompt word template.

[0117] For example, an exemplary data to be corrected is as follows:

[0118]

[0119] One of the features to be corrected in the data to be corrected is "intent": "xxx, and the target error type corresponding to the feature to be corrected is unclear value intent. Based on the original text content, the feature to be corrected "intent": "xxx, and the target completion prompt word template corresponding to the unclear value intent, a target completion prompt word is generated, and the target completion prompt word is input into the large model to obtain the completed features corresponding to "intent": "xxx, so as to obtain a more accurate semantic expression.

[0120] One of the features to be corrected in the data to be corrected is "xx:", and the target error type corresponding to the feature to be corrected is unclear key intent. Based on the original text content, the feature to be corrected "xx:", and the target completion prompt word template corresponding to the unclear key intent, the target completion prompt word is generated, and the target completion prompt word is input into the large model to obtain the completed feature corresponding to "xx:" and obtain a more accurate complete key.

[0121] Step 404: input the target completion prompt word into the large model to obtain the completed features.

[0122] Among them, the target completion prompt word is used to instruct the large model to complete the features to be corrected in the data to be corrected based on the original text content and the target error type to obtain the completed features.

[0123] The original text content corresponding to the data to be corrected is obtained, and the target completion prompt word template corresponding to the target error type is obtained. Based on the original text content, the features to be corrected in the data to be corrected, and the target completion prompt word template, the target completion prompt word is generated, and the target completion prompt word is input into the large model to obtain the completed features. The corresponding completion prompt word template is designed for each error type, so that the features to be corrected in the data to be corrected can be more accurately completed based on the target error type, and the accuracy of the completed features can be improved. Moreover, in the process of completing the features to be corrected in the data to be corrected, the original text content corresponding to the data to be corrected is considered, which can improve the accuracy of the completed features. The large model is used to complete the features to be corrected in the data to be corrected based on the original text content and the target error type, and the completed features are obtained, so that the language understanding ability of the large model is fully utilized, so that the corrected data obtained based on the completed features is more in line with the user's expected content, and because the language understanding ability of the large model is fully utilized, the corrected data is prevented from being too fixed and rigid.

[0124] In other embodiments, step 301 includes: searching for target text content corresponding to the feature to be corrected in the original text content corresponding to the data to be corrected; and completing the feature to be corrected based on the target text content to obtain the completed feature.

[0125] Step 302: Acquire each correction rule corresponding to the target error type and the sample error features corresponding to each correction rule.

[0126] In an exemplary embodiment, one error type may correspond to one or more correction rules.

[0127] In an exemplary embodiment, one error type corresponds to multiple sample error features, and these sample error features may be divided into multiple categories, each of which corresponds to a correction rule. The correction rule is not only related to the error type, but also to the text content in the specific sample error feature.

[0128] In some embodiments, the correction rule is obtained by combining various basic editing operations.

[0129] In an exemplary embodiment, basic editing operations may include insert operations, delete operations, replace operations, etc. As needed, basic editing operations may also include other operations, which are not listed here one by one, and this application does not limit this.

[0130] Insertion operations are often used to supplement missing characters or words. For example, for the text "I go to school", if it is found that its complete expression should be "I go to school to study", it is necessary to insert the two words "study" after "school".

[0131] The deletion operation is mainly used to remove redundant or incorrect characters and words. For example, in the sentence "This problem is very re-repeated", the redundant "re" character should be deleted.

[0132] The replacement operation is used to correct incorrect characters or words. For instance, in the sentence "He has already left", the character "以" is replaced with "已" to make the text correct.

[0133] Each basic editing operation should have the ability to be arranged, that is, it is possible to select some basic editing operations from each basic editing operation, sort these selected basic editing operations in a certain order to form a correction rule. Therefore, multiple correction rules can be generated based on each basic editing operation.

[0134] Each basic editing operation can be arranged, plugged in, and mixed, thereby generating multiple correction rules, making the correction process more flexible and customizable.

[0135] For example, the correction rule corresponding to sample error feature 1 is deletion - replacement - duplicate removal, and the correction rule corresponding to sample error feature 2 is replacement - duplicate removal - correction.

[0136] Step 303: Based on the complemented feature and the sample error features corresponding to each correction rule, determine the target correction rule from each correction rule.

[0137] In an exemplary embodiment, step 303 may include: determining the semantic similarity between the complemented feature and the sample error features corresponding to each correction rule; based on the semantic similarity, determining the target correction rule from each correction rule. The correction rule corresponding to the sample error feature that is semantically similar to the complemented feature can be determined as the target correction rule, improving the accuracy of the obtained target correction rule and thus the accuracy of the corrected data.

[0138] In an exemplary embodiment, steps 302, 303, and 103 may be implemented through a text editing model. The complemented feature, the target error type, and the data to be corrected are input into the text editing model to obtain the corrected data; wherein, the text editing model is trained based on the error type, the correction rule corresponding to the error type, and the sample error feature corresponding to the correction rule.

[0139] The features to be corrected in the data to be corrected are completed to obtain the completed features, and the accuracy of the completed features is improved, thereby improving the accuracy of the correction. The correction rules corresponding to the target error type and the sample error features corresponding to each correction rule are obtained. Based on the completed features and the sample error features corresponding to each correction rule, the target correction rule is determined from each correction rule. The target correction rule is not only related to the error type, but also to the sample error features. It is flexibly applied according to the error type and specific circumstances to improve the accuracy of the target correction rule. Moreover, the target correction rule is determined based on the completed features. On the basis of improving the accuracy of the completed features, the accuracy of the target correction rule is further improved, thereby improving the correction accuracy.

[0140] In other embodiments, step 102 includes: obtaining each correction rule corresponding to the target error type and the sample error features corresponding to each correction rule; determining the target correction rule from the correction rules based on the features to be corrected in the data to be corrected and the sample error features corresponding to each correction rule.

[0141] Step 103, based on the target correction rule, correct the data to be corrected to obtain corrected data.

[0142] The corrected data is represented in a preset data exchange format.

[0143] In some embodiments, Figure 5 As shown, step 103 includes:

[0144] Step 501, based on the target correction rule, correct the completed feature to obtain the corrected feature.

[0145] Step 502, based on the data to be corrected, the features to be corrected in the data to be corrected, and the corrected features, correct the data to be corrected to obtain corrected data.

[0146] In an exemplary embodiment, step 502 may include: based on the data to be corrected, the features to be corrected in the data to be corrected, and the corrected features, correcting the data to be corrected by using a large model to obtain the corrected data.

[0147] In an exemplary embodiment, step 502 may also include: based on the data to be corrected, replacing the features to be corrected in the data to be corrected with the corrected features to obtain the corrected data.

[0148] Based on the target correction rule, the completed features are corrected to obtain the corrected features, and based on the data to be corrected, the features to be corrected in the data to be corrected, and the corrected features, the data to be corrected is corrected to obtain the corrected data. When correcting the data to be corrected, the syntax and semantics of the original data to be corrected are fully considered to ensure that the corrected data is accurate in logic and expression.

[0149] In other embodiments, step 103 includes: generating correction prompt words based on the data to be corrected and the target correction rules; inputting the correction prompt words into the large model to obtain the corrected data; wherein the correction prompt words are used to instruct the large model to correct the data to be corrected based on the target correction rules to obtain the corrected data.

[0150] In some embodiments, Figure 6 As shown, the data correction method also includes:

[0151] Step 601, obtaining user feedback on the corrected data.

[0152] In an exemplary embodiment, the user feedback may include a modification operation performed by the user on the modified data; the user feedback may also include user modified data obtained after the user modifies the modified data.

[0153] For example, the data to be corrected is "I am going to Gongyuan today", and the corrected data is "I am going to Gongyuan today". The "yuan" in "I am going to Gongyuan today" is mistakenly corrected to "yuan", and the user changes the "yuan" in "I am going to Gongyuan today" to "yuan". The user feedback can be the modification operation of changing the "yuan" in "I am going to Gongyuan today" to "yuan", and the user feedback can also be the data after the user's modification "I am going to the park today".

[0154] Step 602: Determine the user modified data based on the user feedback.

[0155] Step 603: Train the data correction model based on the data to be corrected and the data modified by the user to obtain a trained data correction model.

[0156] Among them, the data correction model is used to correct the input data to obtain output data.

[0157] In an exemplary embodiment, the data correction model is specifically used to determine the error type corresponding to the input data based on the input data; determine the correction rule corresponding to the input data based on the input data and the error type corresponding to the input data; and correct the input data based on the correction rule corresponding to the input data to obtain output data.

[0158] In the exemplary embodiment, in order to effectively utilize user feedback, a complete data collection and processing mechanism is established. First, it is necessary to ensure that the user's modification operations can be accurately captured, and the user's modified data is standardized and labeled. Then, these labeled data are integrated into the original training data to retrain the data correction model.

[0159] Obtain user feedback on the corrected data, determine the user's modified data based on the user feedback, train the data correction model based on the data to be corrected and the user's modified data, and obtain the trained data correction model. The data correction model can learn the user's specific language habits and correct usage in a specific context. In addition, user feedback can also supplement the data correction model's knowledge deficiencies in certain rare or specific areas, further improving the data correction model's ability to understand and correct diverse language expressions.

[0160] In some embodiments, step 603 includes: training the data correction model based on the data to be corrected, the corrected data, the user-modified data, and the degree of matching between the corrected data and the user-modified data to obtain a trained data correction model.

[0161] In an exemplary embodiment, the matching degree between the corrected data and the user's modified data is used as a reward signal. If the matching degree between the corrected data and the user's modified data is greater than a preset value, that is, the corrected data meets the user's expectations, a positive reward is given; otherwise, a negative reward is given. Through interaction with the environment, the optimal strategy is learned according to the reward signal, and reinforcement learning of data correction is achieved.

[0162] By continuously receiving such reward feedback, the data correction model can learn how to adjust the error correction strategy to increase the probability of generating more accurate correction results. For example, the data correction model may learn to prefer a certain error correction method in certain specific contexts.

[0163] In addition, reinforcement learning can also help the data correction model explore different error correction possibilities, avoid falling into the local optimal solution, and thus find a better error correction strategy. This enables the data correction model to continuously adapt to new language changes and user needs, and continuously improve the error correction effect.

[0164] In other embodiments, step 603 includes: directly training the data correction model based on the data to be corrected and the data modified by the user to obtain a trained data correction model.

[0165] In summary, in the present application, the data to be corrected and the corrected data are represented in a preset data exchange format. The data to be corrected and the corrected data are data used for data exchange. Based on the data to be corrected, the target error type is determined. The target error type is related to the writing rules of the preset data exchange format. In the process of correcting the data to be corrected, the writing rules of the preset data exchange format are fully considered, and then the target error type is determined, which can more accurately locate the target error type and improve the accuracy of locating the target error type. Based on the data to be corrected and the target error type, the target correction rule is determined. In the process of determining the correction rule, not only the target error type but also the text content of the data to be corrected itself is considered, which can improve the accuracy of determining the target correction rule. Moreover, on the basis of improving the accuracy of locating the target error type, the accuracy of determining the target correction rule can be further improved. Based on the target correction rule, the data to be corrected is corrected to obtain the corrected data. On the basis of improving the accuracy of determining the target correction rule, the accuracy of the corrected data can be further improved, thereby improving the accuracy of correcting the data used for data exchange.

[0166] Exemplary Devices

[0167] Accordingly, the embodiment of the present application also provides a data correction device, such as Figure 7 As shown, the data correction device comprises:

[0168] The first processing unit 701 is used to determine a target error type based on the data to be corrected;

[0169] A second processing unit 702 is used to determine a target correction rule based on the data to be corrected and the target error type;

[0170] A correction unit 703, configured to correct the data to be corrected based on the target correction rule to obtain corrected data;

[0171] The data to be corrected and the corrected data are represented in a preset data exchange format.

[0172] Optionally, the first processing unit 701 is specifically configured to:

[0173] Matching the data to be corrected with the sample error features in the sample feature library to obtain matching target sample error features and features to be corrected; wherein the features to be corrected are extracted from the data to be corrected;

[0174] Reading the error type corresponding to the target sample error feature from the sample feature library;

[0175] Determine the error type corresponding to the target sample error feature as the target error type corresponding to the feature to be corrected;

[0176] Among them, the sample feature library includes sample error features, error types, and a mapping relationship between sample error features and error types; the sample error features are represented in the preset data exchange format; and the error type is determined based on the writing rules of the preset data exchange format.

[0177] Optionally, the second processing unit 702 includes:

[0178] A completion subunit, used for completing the feature to be corrected in the data to be corrected to obtain a completed feature;

[0179] An acquisition subunit, used to acquire each correction rule corresponding to the target error type, and a sample error feature corresponding to each correction rule;

[0180] The processing subunit is used to determine a target correction rule from each correction rule based on the completed feature and the sample error features corresponding to each correction rule.

[0181] Optionally, a completion subunit is specifically used for:

[0182] Obtaining original text content corresponding to the data to be corrected;

[0183] Obtain a target completion prompt word template corresponding to the target error type;

[0184] Generate a target completion prompt word based on the original text content, the to-be-corrected feature in the to-be-corrected data, and the target completion prompt word template;

[0185] Inputting the target completion prompt word into the large model to obtain the completed features;

[0186] The target completion prompt word is used to instruct the large model to complete the feature to be corrected in the data to be corrected based on the original text content and the target error type to obtain the completed feature.

[0187] Optionally, the correction rule is obtained by combining various basic editing operations.

[0188] Optionally, the correction unit 703 is specifically configured to:

[0189] Based on the target correction rule, the completed feature is corrected to obtain a corrected feature;

[0190] Based on the data to be corrected, the features to be corrected in the data to be corrected, and the corrected features, the data to be corrected is corrected to obtain corrected data.

[0191] Optionally, the data correction device further includes:

[0192] An acquisition unit, used for acquiring user feedback on the corrected data;

[0193] A third processing unit, configured to determine user modified data based on the user feedback;

[0194] A training unit, used for training a data correction model based on the data to be corrected and the data modified by the user to obtain a trained data correction model;

[0195] The data correction model is used to correct input data to obtain output data.

[0196] Optionally, the training unit is specifically configured to:

[0197] Based on the data to be corrected, the corrected data, the user-modified data, and the degree of matching between the corrected data and the user-modified data, a data correction model is trained to obtain a trained data correction model.

[0198] The data correction device provided in this embodiment belongs to the same application concept as the data correction method provided in the above embodiments of this application, can execute the data correction method provided in any of the above embodiments of this application, and has the corresponding functional modules and beneficial effects of executing the data correction method. For technical details not fully described in this embodiment, please refer to the specific processing content of the data correction method provided in the above embodiments of this application, and will not be repeated here.

[0199] The functions implemented by the above-mentioned first processing unit 701, the second processing unit 702 and the correction unit 703 may be implemented by the same or different processors respectively, which is not limited in the embodiment of the present application.

[0200] It should be understood that the units in the above devices can be implemented in the form of a processor calling software. For example, the device includes a processor, the processor is connected to a memory, and instructions are stored in the memory. The processor calls the instructions stored in the memory to implement any of the above methods or realize the functions of each unit of the device, wherein the processor can be a general-purpose processor, such as a CPU or a microprocessor, etc., and the memory can be a memory in the device or a memory outside the device. Alternatively, the units in the device can be implemented in the form of hardware circuits, and the functions of some or all units can be realized by designing the hardware circuits. The hardware circuit can be understood as one or more processors; for example, in one implementation, the hardware circuit is an ASIC, and the functions of some or all of the above units are realized by designing the logical relationship of the components in the circuit; for another example, in another implementation, the hardware circuit can be implemented by PLD, taking FPGA as an example, which can include a large number of logic gate circuits, and the connection relationship between the logic gate circuits is configured by the configuration file, so as to realize the functions of some or all of the above units. All units of the above devices can be implemented in the form of a processor calling software, or in the form of hardware circuits, or in part by a processor calling software, and the remaining part is implemented in the form of hardware circuits.

[0201] In an embodiment of the present application, a processor is a circuit with the ability to process signals. In one implementation, the processor may be a circuit with the ability to read and run instructions, such as a CPU, a microprocessor, a GPU, or a DSP; in another implementation, the processor may implement certain functions through the logical relationship of a hardware circuit, and the logical relationship of the hardware circuit is fixed or reconfigurable, such as a hardware circuit implemented by an ASIC or PLD, such as an FPGA. In a reconfigurable hardware circuit, the process of the processor loading a configuration document to implement the hardware circuit configuration can be understood as the process of the processor loading instructions to implement the functions of some or all of the above units. In addition, it can also be a hardware circuit designed for artificial intelligence, which can be understood as an ASIC, such as an NPU, TPU, DPU, etc.

[0202] It can be seen that each unit in the above device can be one or more processors (or processing circuits) configured to implement the above method, such as: CPU, GPU, NPU, TPU, DPU, microprocessor, DSP, ASIC, FPGA, or a combination of at least two of these processor forms.

[0203] In addition, all or part of the units in the above device can be integrated together, or can be implemented independently. In one implementation, these units are integrated together and implemented in the form of a SOC. The SOC may include at least one processor for implementing any of the above methods or implementing the functions of each unit of the device. The type of the at least one processor may be different, for example, including a CPU and an FPGA, a CPU and an artificial intelligence processor, a CPU and a GPU, etc.

[0204] Exemplary Electronic Devices

[0205] An embodiment of the present application provides an electronic device, see Figure 8 As shown, the device includes:

[0206] Memory 200 and processor 210;

[0207] The memory 200 is connected to the processor 210 and is used to store programs;

[0208] The processor 210 is used to implement the data correction method disclosed in any of the above embodiments by running the program stored in the memory 200.

[0209] Specifically, the electronic device may further include: a bus, a communication interface 220 , an input device 230 and an output device 240 .

[0210] The processor 210, the memory 200, the communication interface 220, the input device 230 and the output device 240 are connected to each other via a bus.

[0211] A bus may include a pathway that transfers information between components of a computer system.

[0212] The processor 210 may be a general-purpose processor, such as a general-purpose central processing unit (CPU), a microprocessor, etc., or an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the program of the scheme of the present invention. It may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0213] The processor 210 may include a main processor, and may also include a baseband chip, a modem, and the like.

[0214] The memory 200 stores a program for executing the technical solution of the present invention, and may also store an operating system and other key services. Specifically, the program may include a program code, and the program code includes a computer operation instruction. More specifically, the memory 200 may include a read-only memory (ROM), other types of static storage devices that can store static information and instructions, a random access memory (RAM), other types of dynamic storage devices that can store information and instructions, a disk storage, a flash, and the like.

[0215] The input device 230 may include a device for receiving data and information input by a user, such as a keyboard, a mouse, a camera, a scanner, a light pen, a voice input device, a touch screen, a pedometer, or a gravity sensor.

[0216] Output device 240 may include devices that allow information to be output to a user, such as a display screen, printer, speaker, etc.

[0217] The communication interface 220 may include any device such as a transceiver to communicate with other devices or communication networks, such as Ethernet, a radio access network (RAN), a wireless local area network (WLAN), etc.

[0218] The processor 210 executes the program stored in the memory 200 and calls other devices, which can be used to implement each step of any data correction method provided in the above embodiments of the present application.

[0219] Exemplary computer program products and storage media

[0220] In addition to the above-mentioned methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions, which, when executed by a processor, enable the processor to execute the steps of the data correction method according to various embodiments of the present application described in any of the above-mentioned embodiments of this specification.

[0221] The computer program product may be written in any combination of one or more programming languages ​​to write program codes for performing the operations of the embodiments of the present application, including object-oriented programming languages, such as Java, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0222] In addition, the embodiment of the present application may also be a storage medium on which a computer program is stored. The computer program is executed by a processor to execute the steps of the data correction method according to various embodiments of the present application described in any of the above embodiments of this specification, and specifically the following steps may be implemented:

[0223] Step 101, determining a target error type based on the data to be corrected;

[0224] Step 102, determining a target correction rule based on the data to be corrected and the target error type;

[0225] Step 103, based on the target correction rule, correct the data to be corrected to obtain corrected data;

[0226] The data to be corrected and the corrected data are represented in a preset data exchange format; the target error type is related to the writing rules of the preset data exchange format.

[0227] For the aforementioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the order of the actions described, because according to the present application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0228] It should be noted that each embodiment in this specification is described in a progressive manner, and each embodiment focuses on the differences from other embodiments, and the same or similar parts between the embodiments can be referred to each other. For the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0229] The steps in the methods of each embodiment of the present application can be adjusted in sequence, combined and deleted according to actual needs, and the technical features recorded in each embodiment can be replaced or combined.

[0230] The modules and sub-modules in the devices and terminals in the various embodiments of the present application can be combined, divided and deleted according to actual needs.

[0231] In the several embodiments provided in the present application, it should be understood that the disclosed terminals, devices and methods can be implemented in other ways. For example, the terminal embodiments described above are only schematic, for example, the division of modules or submodules is only a logical function division, and there may be other division methods in actual implementation, for example, multiple submodules or modules can be combined or integrated into another module, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or modules, which can be electrical, mechanical or other forms.

[0232] The modules or submodules described as separate components may or may not be physically separated, and the components of the modules or submodules may or may not be physical modules or submodules, that is, they may be located in one place, or they may be distributed on multiple network modules or submodules. Some or all of the modules or submodules may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0233] In addition, each functional module or submodule in each embodiment of the present application may be integrated into one processing module, or each module or submodule may exist physically separately, or two or more modules or submodules may be integrated into one module. The above-mentioned integrated modules or submodules may be implemented in the form of hardware or in the form of software functional modules or submodules.

[0234] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professionals and technicians may use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0235] The steps of the method or algorithm described in conjunction with the embodiments disclosed herein may be implemented directly by hardware, software units executed by a processor, or a combination of the two. The software units may be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the art.

[0236] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0237] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present application. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present application. Therefore, the present application will not be limited to the embodiments shown herein, but will conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A data correction method, characterized in that: include: Determine the target error type based on the data to be corrected; Determining a target correction rule based on the data to be corrected and the target error type; Based on the target correction rule, the data to be corrected is corrected to obtain corrected data; Wherein, the data to be corrected and the corrected data are represented in a preset data exchange format; The target error type is related to the writing rule of the preset data exchange format.

2. The data correction method according to claim 1, characterized in that: The step of determining the target error type based on the data to be corrected includes: Matching the data to be corrected with the sample error features in the sample feature library to obtain matching target sample error features and features to be corrected; wherein the features to be corrected are extracted from the data to be corrected; Reading the error type corresponding to the target sample error feature from the sample feature library; Determine the error type corresponding to the target sample error feature as the target error type corresponding to the feature to be corrected; Among them, the sample feature library includes sample error features, error types, and a mapping relationship between sample error features and error types; the sample error features are represented in the preset data exchange format; and the error type is determined based on the writing rules of the preset data exchange format.

3. The data correction method according to claim 2, characterized in that: The step of determining a target correction rule based on the data to be corrected and the target error type includes: Completing the feature to be corrected in the data to be corrected to obtain a completed feature; Acquire each correction rule corresponding to the target error type, and each sample error feature corresponding to each correction rule; Based on the completed features and the sample error features corresponding to each correction rule, a target correction rule is determined from the correction rules.

4. The data correction method according to claim 3, characterized in that: The step of completing the feature to be corrected in the data to be corrected to obtain the completed feature includes: Obtaining original text content corresponding to the data to be corrected; Obtain a target completion prompt word template corresponding to the target error type; Generate a target completion prompt word based on the original text content, the to-be-corrected feature in the to-be-corrected data, and the target completion prompt word template; Inputting the target completion prompt word into the large model to obtain the completed features; The target completion prompt word is used to instruct the large model to complete the feature to be corrected in the data to be corrected based on the original text content and the target error type to obtain the completed feature.

5. The data correction method according to claim 3, characterized in that: The correction rule is obtained by combining various basic editing operations.

6. The data correction method according to claim 3, characterized in that: The step of correcting the data to be corrected based on the target correction rule to obtain corrected data includes: Based on the target correction rule, the completed feature is corrected to obtain a corrected feature; Based on the data to be corrected, the features to be corrected in the data to be corrected, and the corrected features, the data to be corrected is corrected to obtain corrected data.

7. The data correction method according to claim 1, characterized in that: The method further comprises: Obtaining user feedback on the corrected data; Determine the user modified data based on the user feedback; Based on the data to be corrected and the data modified by the user, training a data correction model to obtain a trained data correction model; The data correction model is used to correct input data to obtain output data.

8. The data correction method according to claim 7, characterized in that: The step of training a data correction model based on the data to be corrected and the user-modified data to obtain a trained data correction model includes: Based on the data to be corrected, the corrected data, the user-modified data, and the degree of matching between the corrected data and the user-modified data, a data correction model is trained to obtain a trained data correction model.

9. A data correction device, characterized in that: include: A first processing unit, configured to determine a target error type based on the data to be corrected; A second processing unit, configured to determine a target correction rule based on the data to be corrected and the target error type; A correction unit, configured to correct the data to be corrected based on the target correction rule to obtain corrected data; The data to be corrected and the corrected data are represented in a preset data exchange format.

10. An electronic device, characterized in that: including memory and processor; The memory is connected to the processor and is used to store programs; The processor is used to implement the data correction method according to any one of claims 1 to 8 by running the program in the memory.

11. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the data correction method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The method comprises computer program instructions, which, when executed by a processor, enable the processor to perform the data correction method according to any one of claims 1 to 8.

Images