Text correction method and device, electronic equipment, storage medium and program product

Abstract

The invention relates to the technical field of text correction, and particularly provides a text correction method and device, electronic equipment, a storage medium and a program product. The method comprises the following steps: respectively carrying out mask processing on a plurality of target objects of specified types contained in a to-be-processed text to obtain a target text; according to the mask position in the target text, performing fragment interception on the target file to obtain a plurality of text fragments containing masks; according to a vector conversion model, performing vector conversion on each text fragment to obtain a text vector corresponding to each text fragment; performing unsupervised clustering processing on each text vector to obtain at least one text vector cluster and a reference text vector corresponding to the text vector cluster; and according to the reference object corresponding to each reference text vector, correcting the target object corresponding to the text vector contained in the corresponding text vector cluster. Therefore, the problem that error correction of the text is difficult is solved.

Description

Technical Field

[0001] The present disclosure relates to the technical field of text correction, and in particular to a method, device, electronic device, storage medium and program product for text correction. Background Art

[0002] In some data sorting scenarios, some content in the text is often prone to errors. For example, when using image recognition technologies such as optical character recognition (OCR) to perform text recognition on text images, specific entities in the image recognition text are usually inaccurately recognized due to their sparse distribution. Among them, entities mainly include names of people, places, institutions, and proper nouns. Therefore, how to accurately correct text errors is a problem that needs to be solved. Summary of the invention

[0003] To solve the above problems, embodiments of the present disclosure provide a method, device, electronic device, storage medium and program product for text correction.

[0004] In one aspect, an embodiment of the present disclosure provides a method for text correction, including:

[0005] Masking is performed on multiple target objects of specified types contained in the text to be processed to obtain the target text;

[0006] According to the mask position in the target text, the target file is segmented to obtain multiple text segments containing the mask;

[0007] According to the vector conversion model, each text segment is converted into a vector to obtain the text vector corresponding to each text segment;

[0008] Performing unsupervised clustering processing on each text vector to obtain at least one text vector cluster and its corresponding reference text vectors;

[0009] According to the reference object corresponding to each reference text vector, the target object corresponding to the text vector contained in the corresponding text vector cluster is corrected.

[0010] In one embodiment, before performing mask processing on multiple target objects of specified types contained in the text to be processed, the method further includes:

[0011] Perform text recognition on the image to be processed to obtain the text to be processed;

[0012] Identifies target objects of a specified type within the target text.

[0013] In one embodiment, before performing vector conversion on each text segment according to the vector conversion model to obtain the text vectors corresponding to each text segment, the method further includes:

[0014] For multiple initial samples in the target task training data, target object recognition is performed respectively to obtain the target objects of the specified type contained in each initial sample;

[0015] Perform mask processing on each target object in each initial sample respectively to obtain multiple target samples;

[0016] According to each mask position in each target sample, construct a plurality of text construction data corresponding to each target sample;

[0017] According to the text construction data corresponding to each target sample, the pre-trained model is fine-tuned to obtain a trained vector conversion model; the pre-trained model is a model for vector conversion trained based on general data.

[0018] In one implementation, according to each mask position in each target sample, a plurality of text construction data corresponding to each target sample is constructed, including:

[0019] According to the mask position in each target sample, each target sample is segmented to obtain multiple text segments corresponding to each target sample;

[0020] For each text segment in each target sample, corresponding text construction data is generated according to the text segment, the masked target object in the text segment, and the negative sample label; the negative sample label is different from the masked target object in the text segment.

[0021] In one implementation, according to the reference object corresponding to each reference text vector, the target object corresponding to the text vector contained in the corresponding text vector cluster is corrected, including:

[0022] For each target object corresponding to each text vector cluster, perform the following steps:

[0023] Determine the similarity between the reference object corresponding to the text vector cluster and the target object;

[0024] If it is determined that the similarity meets the text correction condition, the target object is corrected to the reference object corresponding to the text vector cluster.

[0025] In one implementation, determining the similarity between a reference object corresponding to a text vector cluster and a target object includes:

[0026] Determine the number of characters that differ between the reference object corresponding to the text vector cluster and the target object; the number is negatively correlated with the similarity;

[0027] Determine if the similarity meets the text correction criteria, including:

[0028] If the quantity is not higher than the quantity threshold, it is determined that the text correction condition is met.

[0029] In one aspect, an embodiment of the present disclosure provides a text correction device, including:

[0030] A masking unit, used for performing masking processing on a plurality of target objects of specified types contained in the to-be-processed text respectively, to obtain a target text;

[0031] A cutting unit, used for cutting the target file into segments according to the mask position in the target text, and obtaining a plurality of text segments containing the mask;

[0032] A conversion unit, used to perform vector conversion on each text segment according to a vector conversion model to obtain text vectors corresponding to each text segment; the vector conversion model is obtained by fine-tuning a pre-trained model;

[0033] A clustering unit, used for performing unsupervised clustering processing on each text vector to obtain at least one text vector cluster and its corresponding reference text vectors;

[0034] The correction unit is used to correct the target object corresponding to the text vector contained in the corresponding text vector cluster according to the reference object corresponding to each reference text vector.

[0035] In one implementation, the mask unit is further used for:

[0036] Perform text recognition on the image to be processed to obtain the text to be processed;

[0037] Identifies target objects of a specified type within the target text.

[0038] In one implementation, the conversion unit is further configured to:

[0039] For multiple initial samples in the target task training data, target object recognition is performed respectively to obtain the target objects of the specified type contained in each initial sample;

[0040] Perform mask processing on each target object in each initial sample respectively to obtain multiple target samples;

[0041] According to each mask position in each target sample, construct a plurality of text construction data corresponding to each target sample;

[0042] According to the text construction data corresponding to each target sample, the pre-trained model is fine-tuned to obtain a trained vector conversion model; the pre-trained model is a model for vector conversion trained based on general data.

[0043] In one implementation, the conversion unit is further configured to:

[0044] According to the mask position in each target sample, each target sample is segmented to obtain multiple text segments corresponding to each target sample;

[0045] For each text segment in each target sample, corresponding text construction data is generated according to the text segment, the masked target object in the text segment, and the negative sample label; the negative sample label is different from the masked target object in the text segment.

[0046] In one implementation, the correction unit is used to:

[0047] For each target object corresponding to each text vector cluster, perform the following steps:

[0048] Determine the similarity between the reference object corresponding to the text vector cluster and the target object;

[0049] If it is determined that the similarity meets the text correction condition, the target object is corrected to the reference object corresponding to the text vector cluster.

[0050] In one implementation, the correction unit is used to:

[0051] Determine the number of characters that differ between the reference object corresponding to the text vector cluster and the target object; the number is negatively correlated with the similarity;

[0052] The calibration unit is used to:

[0053] If the quantity is not higher than the quantity threshold, it is determined that the text correction condition is met.

[0054] In one aspect, an electronic device is provided in an embodiment of the present disclosure, including:

[0055] Processor; and

[0056] A memory stores computer instructions, wherein the computer instructions are used to cause a processor to execute the steps of the method provided in any of the various optional implementations of text correction described above.

[0057] On the one hand, an embodiment of the present disclosure provides a computer-readable storage medium storing computer instructions for causing a computer to execute the steps of a method provided in any of the various optional implementations of text correction described above.

[0058] On the one hand, a computer program product is provided in an embodiment of the present disclosure, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device executes the steps of the method provided in any of the various optional implementations of any of the above-mentioned text corrections.

[0059] The method for text correction in the disclosed embodiment includes performing mask processing on multiple target objects of specified types contained in the processed text to obtain the target text; according to the mask position in the target text, the target file is segmented to obtain multiple text segments containing the mask; according to the vector conversion model, each text segment is vector converted to obtain the text vector corresponding to each text segment; each text vector is unsupervised clustered to obtain at least one text vector cluster and its corresponding reference text vector; according to the reference object corresponding to each reference text vector, the target object corresponding to the text vector contained in the corresponding text vector cluster is corrected. In this way, the target object in the text can be corrected by combining the vector conversion model and the unsupervised clustering algorithm, solving the problem that the text is difficult to correct. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] Figure 1 It is a flowchart of a method for training a vector conversion model in an embodiment of the present disclosure.

[0061] Figure 2 It is a flowchart of a method for text correction in an embodiment of the present disclosure.

[0062] Figure 3 It is a flow chart of a method for target object correction in an embodiment of the present disclosure.

[0063] Figure 4 It is a structural block diagram of a text correction device in an embodiment of the present disclosure.

[0064] Figure 5 It is a structural schematic diagram of an electronic device in an embodiment of the present disclosure. DETAILED DESCRIPTION

[0065] The technical solution of the present disclosure will be described clearly and completely below in conjunction with the accompanying drawings. Obviously, the described implementation is a part of the implementation of the present disclosure, rather than all the implementations. Based on the implementation in the present disclosure, all other implementations obtained by ordinary technicians in the field without creative work are within the scope of protection of the present disclosure. In addition, the technical features involved in the different implementations of the present disclosure described below can be combined with each other as long as they do not conflict with each other.

[0066] In practical applications, text images are usually recognized through OCR for data collation. However, since the recognition process of OCR and other technologies is usually limited to the correction of single-line text content, it is difficult to combine the context and take the whole text into consideration, so some specific content often has the problem of inaccurate recognition due to sparse distribution and other reasons.

[0067] Based on the defects of the above-mentioned related technologies, the embodiments of the present disclosure provide a method, device, electronic device, storage medium and program product for text correction, aiming to solve the problem of difficult to accurately correct text errors.

[0068] A method for text correction is provided in an embodiment of the present disclosure. The method can be applied to electronic devices. The present disclosure does not limit the type of electronic device. The method can be any type of device suitable for implementation, such as a terminal device and a server, etc. The present disclosure will not elaborate on this.

[0069] In the embodiment of the present application, model training is performed first, and then text correction is performed using the trained vector conversion model.

[0070] See also Figure 1 FIG. 1 is a flow chart of a method for training a vector conversion model in an embodiment of the present disclosure. Figure 1 The method is described below. The specific implementation process of the method is as follows:

[0071] Step 101: performing target object recognition on multiple initial samples in the target task training data respectively to obtain target objects of specified types contained in each initial sample.

[0072] In one implementation, target task training data including multiple initial samples is obtained, and target object recognition is performed on each initial sample using a target object recognition model or the like to obtain target objects of a specified type contained in each initial sample.

[0073] For example, the initial sample can be an open source electronic book, etc. The target object of a specified type can be set according to the actual application scenario. The target object can be an entity in the text, for example, the target object can be a name of a person or a work name, etc. The target object recognition model can be a named entity recognition (NER) model. In actual applications, the target object recognition model can be selected according to the actual application scenario, and there is no limitation here.

[0074] In practical applications, the specified type and target task training data can be selected specifically according to the actual target task scenario, and no restrictions are imposed here.

[0075] Step 102: Perform mask processing on each target object in each initial sample to obtain multiple target samples.

[0076] For example, the recognized names of people and works can be masked (MASK), that is, their positions in the text are marked with [MASK].

[0077] Step 103: construct multiple text construction data corresponding to each target sample according to each mask position in each target sample.

[0078] In one implementation, when executing step 103, the following steps may be used:

[0079] S1031: Segment each target sample according to the mask position in each target sample to obtain a plurality of text segments corresponding to each target sample.

[0080] In the embodiment of the present disclosure, only a mask position in a target sample is taken as an example to illustrate the text segment extraction. A text segment of a specified length and centered at a mask position in the target sample is extracted from the target sample.

[0081] For example, from the target sample, a text segment with a length of 512 words (ie, a specified length) and a [MASK] at its center is extracted.

[0082] Each text segment contains at least one mask position corresponding to a target object. The lengths of different text segments can be the same or different.

[0083] In practical applications, the specified length can be set according to the actual application scenario and is not limited here. The specified length can be determined according to the input length of the vector conversion model.

[0084] S1032: For each text segment in each target sample, generate corresponding text construction data according to the text segment, the masked target object in the text segment, and the negative sample label; the negative sample label is different from the masked target object in the text segment.

[0085] Optionally, the data format of the text construction data can be expressed as: {"query":str, "pos":List[str], "neg":List[str]}.

[0086] Among them, "query":str represents the text fragment parameter: text fragment, "pos":List[str] represents the positive sample label variable: positive sample label, and "neg":List[str] represents the negative sample label variable: negative sample label.

[0087] There may be one or more positive sample labels and negative sample labels, which are not limited here. The positive sample label is the masked target object in the text segment.

[0088] Step 104: construct data according to the text corresponding to each target sample, fine-tune the pre-trained model, and obtain a trained vector conversion model; the pre-trained model is a model for vector conversion that is trained based on general data.

[0089] In one implementation, the contrast loss can be determined by contrasting the loss function, the positive sample labels, the negative sample labels in the text construction data, and their corresponding model output results, and the model parameters are adjusted according to the contrast loss until a trained vector conversion model is obtained.

[0090] The pre-trained model can be a trained bge-large-zh 1.5 model. The bge-large-zh 1.5 model is a Chinese text embedding model that aims to convert Chinese text into a high-dimensional vector representation. The text vector obtained by text conversion can capture the semantic information in the text, so that semantically similar texts are closer in the vector space, which is convenient for similarity search and clustering tasks. In addition to using the bge-large-zh 1.5 model, other pre-trained models can also be considered, such as Bidirectional Encoder Representations from Transformers (BERT) and Generative Pretrained Transformer (GPT). These models are widely used in the field of natural language processing and can capture the deep semantic information of the text. They are suitable for the recognition of names and work titles in OCR. In practical applications, the pre-trained model can be selected according to the actual application scenario, and there is no restriction here.

[0091] Fine-tuning refers to the process of combining transfer learning and continuing to train the model for a specific task based on the pre-trained model. The purpose of this is to use the knowledge learned by the pre-trained model on large-scale data (i.e. general data) while adjusting the model to adapt to new tasks or data sets. Fine-tuning is usually performed on small-scale and task-specific data, so that the model can learn the characteristics and rules of a specific task.

[0092] One of the main advantages of transfer learning is that it can reduce the need for training data and improve training efficiency. In traditional machine learning, models require a large amount of labeled data to train. However, in transfer learning, the pre-trained model has already learned the basic features, so less data is needed to train a specific task. This is very beneficial for situations where models need to be deployed quickly or training resources are limited.

[0093] In the disclosed embodiment, the model training process includes selecting a pre-trained model, pre-processing training data, and fine-tuning the model, and then an optimized vector conversion model can be obtained. The bge-large-zh 1.5 model and the like are used as pre-trained models. The pre-trained model can convert Chinese text into a high-dimensional vector representation, capture the semantic information in the text, and thus can be combined with context information to achieve the proximity of semantically similar texts in the vector space. The pre-trained model is fine-tuned to obtain a trained vector conversion model, so that the vector conversion model further optimizes the representation ability of the context information of the entire text recognition target object on the basis of maintaining the original pre-training knowledge, improves the accuracy of semantic embedding, enables the model to adapt to new tasks or data sets, and improves the performance of the model on specific tasks.

[0094] In this way, after the model training is completed, the vector conversion in this article can be performed through the trained vector conversion model.

[0095] See also Figure 2 FIG. 1 is a flowchart of a text correction method in an embodiment of the present disclosure. The specific implementation process of the method is as follows:

[0096] Step 201: Masking is performed on multiple target objects of specified types contained in the text to be processed to obtain the target text.

[0097] Optionally, the text to be processed may be image-recognized text obtained by recognizing an image using OCR technology, or may be text of other types of books and the like.

[0098] In one implementation, the implementation process of step 201 further includes:

[0099] Perform text recognition on the image to be processed to obtain the text to be processed; determine the target object of the specified type in the target text through the target object recognition model.

[0100] For text recognition and target object recognition, please refer to the above step 101.

[0101] Step 202: According to the mask position in the target text, the target file is segmented to obtain multiple text segments containing the mask.

[0102] Among them, step 202 can be performed based on a principle similar to the text segment interception in step 103, which will not be described in detail here.

[0103] Step 203: According to the vector conversion model, vector conversion is performed on each text segment to obtain a text vector corresponding to each text segment.

[0104] In one implementation, each text segment is input into a vector conversion model to obtain a text vector (embedding) corresponding to each text segment.

[0105] Among them, the vector conversion model can start from the entire text to be processed (such as the whole book), make full use of the contextual information of the target object (such as the entity) in the entire text to be processed, and characterize the masked target object, so as to ensure that the vector cosine similarity of different text fragments containing the same target object is higher.

[0106] Step 204: Perform unsupervised clustering processing on each text vector to obtain at least one text vector cluster and its corresponding reference text vectors.

[0107] In one implementation, each text vector is subjected to unsupervised clustering extraction by an unsupervised clustering algorithm to obtain one or more text vector clusters, and a representative reference text vector in each text vector cluster can be identified.

[0108] Among them, the text vector cluster includes multiple text vectors. Optionally, the unsupervised clustering algorithm can adopt the Affinity Propagation algorithm. The Affinity Propagation algorithm is a clustering algorithm based on graph theory, which aims to identify clusters and exemplars in the data, that is, the text vector clusters and their corresponding reference text vectors in the embodiment of the present disclosure. Unlike traditional clustering algorithms, Affinity Propagation does not require the prior specification of the number of clusters, nor does it require random initialization of cluster centers. Instead, it obtains the final clustering result by calculating the similarity between data points (that is, the text vectors in the embodiment of the present disclosure).

[0109] Optionally, the unsupervised clustering algorithm can also use a density-based clustering algorithm (Density-Based Spatial Clustering of Applications with Noise, DBSCAN), or a K-means algorithm (K-means). Different unsupervised clustering algorithms have their own advantages when processing large-scale data sets. For example, K-means works better when processing spherical data sets, while DBSCAN is suitable for processing data sets with different densities. In practical applications, the unsupervised clustering algorithm can be set according to the actual application scenario, and there is no restriction here.

[0110] In this way, unsupervised clustering algorithms such as the Affinity Propagation algorithm can automatically select the number of clusters from the data. There is no need to specify the number of clusters in advance or to randomly initialize the cluster centers. Instead, the final clustering result is obtained by calculating the similarity between data points, which improves the accuracy and efficiency of clustering.

[0111] Step 205: Correct the target object corresponding to the text vector contained in the corresponding text vector cluster according to the reference object corresponding to each reference text vector.

[0112] In one implementation, when executing step 205, the following steps are performed for each target object corresponding to each text vector cluster:

[0113] S2051: Determine the similarity between the reference object corresponding to the text vector cluster and the target object.

[0114] S2052: If it is determined that the similarity meets the text correction condition, the target object is corrected to a reference object corresponding to the text vector cluster.

[0115] Combine the following Figure 3 , for an example of the method of target object correction, see Figure 3 FIG. 1 is a flow chart of a method for target object correction, wherein the specific implementation process of the method includes:

[0116] Step 301: Determine the number of characters that are different between the reference object corresponding to the text vector cluster and the target object; the number is negatively correlated with the similarity.

[0117] Step 302: If the quantity is not higher than the quantity threshold, it is determined that the text correction condition is met.

[0118] For example, the quantity threshold is set to 1, the reference object is the work title "Remembrance of Things Past", and the work title corresponding to a text vector in the text vector cluster is "Remembrance of Things Past". If the number of different characters between the two is 1 and is not higher than the quantity threshold 1, it is determined that the text correction conditions are met.

[0119] Optionally, the characters may also be words, that is, whether the text correction condition is met may be determined based on the number of words of the difference.

[0120] After clustering, the reference object (e.g., name of a person or work title) corresponding to the representative point (i.e., reference object) in each class (i.e., text vector cluster) is used as the standard answer. If the number of characters between other target objects in the class and the standard answer does not exceed a quantity threshold gamma, it will be corrected to the standard answer. By introducing an unsupervised clustering algorithm, the number of clusters can be automatically selected from the data, thereby more accurately correcting target object errors (e.g., name errors and work title errors) in the text to be processed (e.g., OCR recognized text).

[0121] For example, the phrase "Spring River Moon Night" appears 7 times in a book, 5 of which are correct and 2 are wrong, namely: Spring River Moon Night, the river reflects the moonlight, and the fragrance of flowers spreads far away with the wind. In the beautiful scenery of Spring River Moon Night, we strolled along the river and felt the tranquility of nature. Spring River Moon Night is an eternal theme in the poet's pen and an immortal scene on the painter's canvas. That night, the moonlight on the Spring River Moon Night was like water and the flowers were like brocade, making people linger. Spring River Moon Night is a poem that has been passed down through the ages and a touching painting. In the romantic atmosphere of Spring River Moon Night, lovers made an eternal vow. Spring River Moon Night is not only a display of the beauty of nature, but also a yearning for a better life in people's hearts. The book can then be subjected to target object recognition, masking, text fragment extraction, text vector conversion, clustering, and text correction based on the clustering results. It can be determined that "Spring Flowers and Moon Night" and "Spring River Transformed into Moon Night" are only one character different from "Spring River Flowers and Moon Night", so both "Spring Flowers and Moon Night" and "Spring River Transformed into Moon Night" can be corrected to "Spring River Flowers and Moon Night".

[0122] One application scenario of the disclosed embodiment is to perform text correction on text recognized based on OCR technology, which can be applied to fields such as electronic book digitization and document management. It can optimize OCR recognition results, ensure the consistency of target object recognition, and maintain data integrity and accuracy. It is particularly suitable for fields such as law, finance, and medicine that have extremely high requirements for data accuracy. In actual applications, it can also be applied to other text correction scenarios, which will not be elaborated here.

[0123] In the disclosed embodiment, the target object in the text to be processed is masked, and the masked target text is vector-converted through a vector conversion model, so that the semantic information of each text segment in the target text can be accurately captured through comparative learning of context information, and the distance between semantically similar text segments in the vector space can be made closer, so as to facilitate tasks such as similarity search and clustering. In addition, an unsupervised clustering algorithm is combined to cluster and intercept the text vectors of multiple text segments of the target text output by the vector conversion model, thereby improving the accuracy and efficiency of clustering, and the correction of the target object through the representative points of the cluster can reduce the inaccuracy and inconsistency of the target object recognition and improve the efficiency of data sorting.

[0124] Furthermore, it combines transfer learning and context contrast training technology, that is, fine-tuning the pre-trained model with specific target task data, so that based on the knowledge of the pre-trained model, the model can be optimized for specific target tasks, so that the trained vector conversion model can quickly adapt to new target tasks, improve the accuracy of semantic embedding, and reduce the consumption of training data and other resources. It is particularly suitable for situations where data labeling costs are high or data is scarce, because it can further optimize the representation ability of the context information of the target object in the entire text on the basis of maintaining the original pre-training knowledge, reduce the demand for training data, and improve training efficiency. It is particularly suitable for scenarios with limited resources and scarce data, and has broad application prospects and significant practical value.

[0125] Furthermore, in the OCR text recognition and error correction scenarios, the target object recognition model is used to identify the target object, and masking, unsupervised clustering and error correction are performed in sequence, which can optimize the recognition results of the OCR text, improve the automation level of OCR text recognition and error correction, improve efficiency, reduce manual intervention, and reduce the error rate.

[0126] Based on the same inventive concept, a device for text correction is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the above-mentioned device and equipment is similar to that of a method for text correction, the implementation of the above-mentioned device can refer to the implementation of the method, and the repeated parts will not be repeated. The device can be applied to electronic devices. The present disclosure does not limit the type of electronic devices. It can be any type of device suitable for implementation, such as terminal devices and servers, etc. The present disclosure will not repeat them. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, it is formed by the processor of the electronic device in which it is located reading the corresponding computer program instructions in the non-volatile memory into the internal memory for execution.

[0127] See also Figure 4FIG. 1 is a block diagram of a text correction device according to an embodiment of the present disclosure. In some embodiments, the text correction device according to an example of the present disclosure includes:

[0128] The masking unit 401 is used to perform masking processing on multiple target objects of specified types contained in the text to be processed respectively to obtain the target text;

[0129] The interception unit 402 is used to intercept the target file according to the mask position in the target text, and obtain multiple text segments containing the mask;

[0130] A conversion unit 403 is used to perform vector conversion on each text segment according to a vector conversion model to obtain a text vector corresponding to each text segment; the vector conversion model is obtained by fine-tuning a pre-trained model;

[0131] A clustering unit 404 is used to perform unsupervised clustering processing on each text vector to obtain at least one text vector cluster and its corresponding reference text vectors;

[0132] The correction unit 405 is used to correct the target object corresponding to the text vector included in the corresponding text vector cluster according to the reference object corresponding to each reference text vector.

[0133] In one implementation, the mask unit 401 is further configured to:

[0134] Perform text recognition on the image to be processed to obtain the text to be processed;

[0135] Identifies target objects of a specified type within the target text.

[0136] In one implementation, the conversion unit 403 is further configured to:

[0137] For multiple initial samples in the target task training data, target object recognition is performed respectively to obtain the target objects of the specified type contained in each initial sample;

[0138] Perform mask processing on each target object in each initial sample respectively to obtain multiple target samples;

[0139] According to each mask position in each target sample, construct a plurality of text construction data corresponding to each target sample;

[0140] According to the text construction data corresponding to each target sample, the pre-trained model is fine-tuned to obtain a trained vector conversion model; the pre-trained model is a model for vector conversion trained based on general data.

[0141] In one implementation, the conversion unit 403 is further configured to:

[0142] According to the mask position in each target sample, each target sample is segmented to obtain multiple text segments corresponding to each target sample;

[0143] For each text segment in each target sample, corresponding text construction data is generated according to the text segment, the masked target object in the text segment, and the negative sample label; the negative sample label is different from the masked target object in the text segment.

[0144] In one implementation, the correction unit 405 is used to:

[0145] For each target object corresponding to each text vector cluster, perform the following steps:

[0146] Determine the similarity between the reference object corresponding to the text vector cluster and the target object;

[0147] If it is determined that the similarity meets the text correction condition, the target object is corrected to the reference object corresponding to the text vector cluster.

[0148] In one implementation, the correction unit 405 is used to:

[0149] Determine the number of characters that differ between the reference object corresponding to the text vector cluster and the target object; the number is negatively correlated with the similarity;

[0150] The correction unit 405 is used for:

[0151] If the quantity is not higher than the quantity threshold, it is determined that the text correction condition is met.

[0152] The method for text correction in the disclosed embodiment includes performing mask processing on multiple target objects of specified types contained in the processed text to obtain the target text; according to the mask position in the target text, the target file is segmented to obtain multiple text segments containing the mask; according to the vector conversion model, each text segment is vector converted to obtain the text vector corresponding to each text segment; each text vector is unsupervised clustered to obtain at least one text vector cluster and its corresponding reference text vector; according to the reference object corresponding to each reference text vector, the target object corresponding to the text vector contained in the corresponding text vector cluster is corrected. In this way, the target object in the text can be corrected by combining the vector conversion model and the unsupervised clustering algorithm, solving the problem that the text is difficult to correct.

[0153] The text correction method in the embodiment of the present disclosure includes:

[0154] In an embodiment of the present disclosure, an electronic device is further provided, including:

[0155] Processor; and

[0156] The memory stores computer instructions, where the computer instructions are used to enable the processor to execute the method of any of the above embodiments.

[0157] In an embodiment of the present disclosure, a computer-readable storage medium is provided, which stores computer instructions, and the computer instructions are used to enable a computer to execute a method in any of the above-mentioned embodiments.

[0158] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code runs in a processor of an electronic device, the processor in the electronic device implements any of the above-mentioned embodiments of the method.

[0159] Figure 5 FIG. 5 shows a schematic diagram of the structure of an electronic device 5000. Figure 5 As shown, the electronic device 5000 includes: a processor 5010 and a memory 5020 , and optionally, may also include a power supply 5030 , a display unit 5040 , and an input unit 5050 .

[0160] The processor 5010 is the control center of the electronic device 5000. It connects various components using various interfaces and lines, and performs various functions of the electronic device 5000 by running or executing software programs and / or data stored in the memory 5020, thereby monitoring the electronic device 5000 as a whole.

[0161] In the embodiment of the present disclosure, the processor 5010 executes the various steps in the above embodiment when calling the computer program stored in the memory 5020.

[0162] Optionally, the processor 5010 may include one or more processing units; preferably, the processor 5010 may integrate an application processor and a modem processor, wherein the application processor mainly processes an operating system, a user interface, and applications, and the modem processor mainly processes wireless communications. It is understandable that the modem processor may not be integrated into the processor 5010. In some embodiments, the processor and the memory may be implemented on a single chip, and in some embodiments, they may also be implemented separately on separate chips.

[0163] The memory 5020 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, various applications, etc.; the data storage area may store data created according to the use of the electronic device 5000, etc. In addition, the memory 5020 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices, etc.

[0164] The electronic device 5000 also includes a power source 5030 (such as a battery) for supplying power to various components. The power source can be logically connected to the processor 5010 through a power management system, so that the power management system can manage functions such as charging, discharging, and power consumption.

[0165] The display unit 5040 may be used to display information input by the user or information provided to the user and various menus of the electronic device 5000, and in the embodiment of the present disclosure, is mainly used to display the display interface of each application in the electronic device 5000 and objects such as text and pictures displayed in the display interface. The display unit 5040 may include a display panel 5041. The display panel 5041 may be configured in the form of a liquid crystal display (LCD), an organic light-emitting diode (OLED), etc.

[0166] The input unit 5050 may be used to receive information such as numbers or characters input by the user. The input unit 5050 may include a touch panel 5051 and other input devices 5052. The touch panel 5051, also known as a touch screen, may collect touch operations of the user on or near it (such as operations of the user using a finger, a stylus, or any other suitable object or accessory on or near the touch panel 5051).

[0167] Specifically, the touch panel 5051 can detect the user's touch operation, detect the signal brought by the touch operation, convert these signals into touch point coordinates, send them to the processor 5010, and receive and execute the command sent by the processor 5010. In addition, the touch panel 5051 can be implemented in various types such as resistive, capacitive, infrared, and surface acoustic wave. Other input devices 5052 can include but are not limited to one or more of a physical keyboard, function keys (such as volume control keys, power on / off keys, etc.), a trackball, a mouse, a joystick, etc.

[0168] Of course, the touch panel 5051 can cover the display panel 5041. When the touch panel 5051 detects a touch operation on or near it, it is transmitted to the processor 5010 to determine the type of touch event. Then the processor 5010 provides corresponding visual output on the display panel 5041 according to the type of touch event. Figure 5 In the embodiment, the touch panel 5051 and the display panel 5041 are used as two independent components to realize the input and output functions of the electronic device 5000, but in some embodiments, the touch panel 5051 and the display panel 5041 can be integrated to realize the input and output functions of the electronic device 5000.

[0169] The electronic device 5000 may also include one or more sensors, such as a pressure sensor, a gravity acceleration sensor, a proximity light sensor, etc. Of course, according to the needs of specific applications, the electronic device 5000 may also include other components such as a camera. Since these components are not the key components used in the embodiments of the present disclosure, Figure 5 It is not shown and will not be described in detail.

[0170] Those skilled in the art will understand that Figure 5 The electronic device is merely an example and does not limit the electronic device, and may include more or less components than those shown in the figure, or may combine certain components, or may include different components.

[0171] For the convenience of description, the above parts are divided into modules (or units) according to their functions and described separately. Of course, when implementing the present disclosure, the functions of each module (or unit) can be implemented in the same or multiple software or hardware.

Claims

1. A method for text correction, characterized in that: The method comprises: Masking is performed on multiple target objects of specified types contained in the text to be processed to obtain the target text; According to the mask position in the target text, the target file is segmented to obtain multiple text segments containing the mask; According to the vector conversion model, each text segment is converted into a vector to obtain the text vector corresponding to each text segment; Performing unsupervised clustering processing on each text vector to obtain at least one text vector cluster and its corresponding reference text vectors; According to the reference object corresponding to each reference text vector, the target object corresponding to the text vector contained in the corresponding text vector cluster is corrected.

2. The method according to claim 1, characterized in that Before respectively performing mask processing on a plurality of target objects of a specified type contained in the to-be-processed text, the method further comprises: Perform text recognition on the image to be processed to obtain the text to be processed; A target object of the specified type in the target text is determined.

3. The method according to claim 1 or 2, characterized in that: Before performing vector conversion on each text segment according to the vector conversion model to obtain text vectors corresponding to each text segment, the method further includes: Recognize the target objects for each of the multiple initial samples in the target task training data to obtain the target objects of the specified type contained in each of the initial samples; Perform mask processing on each target object in each initial sample respectively to obtain multiple target samples; According to each mask position in each target sample, construct a plurality of text construction data corresponding to each target sample; According to the text construction data corresponding to each target sample, the pre-trained model is fine-tuned to obtain the trained vector conversion model; the pre-trained model is a model for vector conversion trained based on general data.

4. The method according to claim 3, characterized in that The step of constructing a plurality of text construction data corresponding to each target sample according to each mask position in each target sample includes: According to the mask position in each target sample, each target sample is segmented to obtain multiple text segments corresponding to each target sample; For each text segment in each target sample, corresponding text construction data is generated according to the text segment, the masked target object in the text segment, and the negative sample label; the negative sample label is different from the masked target object in the text segment.

5. The method according to claim 1 or 2, characterized in that: The method of correcting the target object corresponding to the text vector contained in the corresponding text vector cluster according to the reference object corresponding to each reference text vector comprises: For each target object corresponding to each text vector cluster, perform the following steps: Determine the similarity between the reference object corresponding to the text vector cluster and the target object; If it is determined that the similarity meets the text correction condition, the target object is corrected to be a reference object corresponding to the text vector cluster.

6. The method according to claim 5, characterized in that The determining the similarity between the reference object corresponding to the text vector cluster and the target object includes: Determine the number of characters that are different between the reference object corresponding to the text vector cluster and the target object; the number is negatively correlated with the similarity; The determining that the similarity meets the text correction condition includes: If the quantity is not higher than the quantity threshold, it is determined that the text correction condition is met.

7. A text correction device, characterized in that: The device comprises: A masking unit, used for performing masking processing on a plurality of target objects of specified types contained in the to-be-processed text respectively, to obtain a target text; A cutting unit, used for cutting the target file into segments according to the mask position in the target text, to obtain a plurality of text segments containing the mask; A conversion unit, used to perform vector conversion on each text segment according to a vector conversion model to obtain text vectors corresponding to each text segment; the vector conversion model is obtained by fine-tuning a pre-trained model; A clustering unit, used for performing unsupervised clustering processing on each text vector to obtain at least one text vector cluster and its corresponding reference text vectors; The correction unit is used to correct the target object corresponding to the text vector contained in the corresponding text vector cluster according to the reference object corresponding to each reference text vector.

8. An electronic device, characterized in that: include: processor; as well as A memory storing computer instructions, wherein the computer instructions are used to enable the processor to execute the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: Computer instructions are stored, and the computer instructions are used to make a computer execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer-readable code, or a non-volatile computer-readable storage medium carrying the computer-readable code. When the computer-readable code is executed in a processor of an electronic device, the processor in the electronic device performs the method according to any one of claims 1 to 6.

Images