Text error correction method and system fusing domain knowledge graph and deep learning model
By integrating domain knowledge graphs and deep learning models, the problems of character distortion and misjudgment of professional terms caused by scanning paper documents in government documents have been solved. This has enabled accurate correction of physical character distortion and semantic anomalies, improving the accuracy and standardization of digital processing of government documents.
Patent Information
- Application Number
- CN202511871798.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-12-12
- Publication Date
- 2026-03-20
AI Technical Summary
Existing text correction schemes based on bidirectional encoders are ineffective in handling character deformation caused by scanning paper documents. They struggle to capture abnormal character structures resulting from physical deformation and are prone to misjudging technical terms and inaccurate standardization in government documents.
By integrating domain knowledge graphs and deep learning models, a knowledge graph for the government domain is established by acquiring target scan images and historical official document data. Character deformation features are extracted using graph convolutional networks and multi-scale feature fusion technology. Combined with deep learning models, multiple rounds of iterative optimization are performed to generate error-corrected text that meets the specifications.
It achieves precise correction of physical deformation of characters in government documents and accurate recognition of professional terms, ensuring that the output text fully meets the requirements of government documents in terms of semantic accuracy and format standardization, thereby improving the accuracy and reliability of digital processing.
Smart Images
Figure CN121708613A_ABST
Abstract
Description
Technical Field
[0001] This application relates to the field of computer information processing technology, and in particular to a text correction method and system that integrates domain knowledge graphs and deep learning models. Background Technology
[0002] In the process of digitizing government documents, paper documents often have physical defects such as paper wrinkles or printing offsets when scanned. The resulting digitized text is often accompanied by problems such as character deformation and blurred strokes. These defects not only directly affect the accuracy of optical character recognition, but also cause continuous interference to subsequent text semantic understanding. Therefore, it is necessary to develop a comprehensive error correction technology that can handle both physical deformation and semantic errors at the same time.
[0003] There is currently a text correction scheme based on bidirectional encoder representation. This scheme uses a pre-trained language model to detect semantic errors in the text and makes full use of the context to identify and correct errors. This scheme has shown strong semantic understanding ability in general text correction tasks, thus effectively handling vocabulary usage errors and grammatical structure problems.
[0004] However, this scheme has obvious limitations when processing text from scanned documents. For example, its text representation method is not sensitive to character morphological features, making it difficult to effectively capture abnormal character structures caused by physical deformation. At the same time, the training corpus of this model has relatively limited knowledge of professional government terminology and normative expressions. When faced with specific scenarios containing professional terminology and fixed sentence patterns in official documents, it is easy to misjudge professional terminology and misjudge normative expressions. Summary of the Invention
[0005] This application provides a text correction method and system that integrates domain knowledge graphs and deep learning models to solve the problems of low accuracy and poor standardization in the digital processing of government documents in the prior art.
[0006] To address the aforementioned technical problems, in a first aspect, this application provides a text correction method that integrates domain knowledge graphs and deep learning models, comprising:
[0007] Acquire the target scan image and historical document data;
[0008] Based on the aforementioned historical official document data, a knowledge graph for the government affairs field will be established;
[0009] Image processing is performed on the target scan image to obtain character change information, and topology correction data is generated based on the character change information;
[0010] The topology correction data is fused with the target scan image at multiple scales to generate character structure information. The character structure information is then associated with the government affairs knowledge graph through a graph convolutional network to generate error location information.
[0011] Based on the error location information, a correction text sequence is generated using a deep learning model. The correction text sequence is then iteratively optimized through a discriminator network to generate the final text that meets the preset specification requirements.
[0012] Optionally, the step of multi-scale fusion of the topology correction data and the target scanned image to generate character structure information includes:
[0013] Position correction parameters and contour repair information are extracted from the topology correction data;
[0014] At the pixel scale, the position correction parameters are corrected point by point with the original pixels of the target scanned image;
[0015] At the feature scale, the target scan image is convolved to obtain a convolutional feature map, and the contour repair information is channel-concatenated with the convolutional feature map;
[0016] Based on the point-by-point correction results and channel cascading results, character structure information is obtained by fusing them through a multi-scale feature pyramid.
[0017] Optionally, the character structure information is obtained by fusing the point-by-point correction results and channel concatenation results through a multi-scale feature pyramid, including:
[0018] The point-by-point correction result is used as the first feature map, and the channel cascade result is used as the second feature map;
[0019] The second feature map is dimensionality reduced to obtain the third feature map;
[0020] The first feature map, the second feature map, and the third feature map are fused in multiple levels using a feature pyramid network. During the fusion process, the third feature map is upsampled and then superimposed with the second feature map to obtain the first fused feature map. The first fused feature map is then upsampled and then added element-wise with the first feature map to obtain the second fused feature map.
[0021] Channel dimension compression is performed on the second fused feature map to generate character structure information.
[0022] Optionally, the step of generating an error-corrected text sequence based on the error location information and combining it with a deep learning model, and then performing multiple rounds of iterative optimization on the error-corrected text sequence through a discriminator network to generate final text that meets preset specification requirements, includes:
[0023] The error type and error location information in the error location information are combined and encoded into a location marker sequence;
[0024] The location tag sequence and the government affairs domain knowledge graph are input into a deep learning model. The encoder of the deep learning model performs semantic enhancement processing on the entity embedding vector of the government affairs domain knowledge graph based on the location tag sequence and combined with a multi-head attention mechanism to obtain the enhanced text representation.
[0025] The deep learning model's decoder generates multiple alternative complete texts based on the enhanced text representation, forming a sequence of error-corrected texts.
[0026] Using a discriminator network, calculate the consistency score and compliance score between each candidate complete text in the error-correcting text sequence and the specification requirements;
[0027] Based on the consistency score and the conformity score, the candidate complete texts in the error-correcting text sequence are prioritized and sorted using a reordering algorithm.
[0028] The highest-priority candidate complete text in the priority ranking results is subjected to multiple rounds of local optimization. In each round of optimization, the optimization strategy is adjusted according to the feedback of the discriminator network until the preset number of iterations is reached. The final optimized complete text is then output as the final text that meets the specification requirements.
[0029] Optionally, the step of associating the character structure information with the government affairs domain knowledge graph through a graph convolutional network to generate error location information includes:
[0030] Based on the character structure information, a character structure graph is constructed, and a bidirectional message passing mechanism is established between the character structure graph and the target scanned image through a graph convolutional network.
[0031] Using a two-way message passing mechanism, the association strength value between the character nodes in the character structure graph and the entity nodes in the government affairs domain knowledge graph is calculated;
[0032] Based on the association strength value, inconsistencies between the region corresponding to the character node and the region corresponding to the entity node are detected, and error location information is generated based on the inconsistencies.
[0033] Optionally, establishing a knowledge graph in the government affairs domain based on the historical official document data includes:
[0034] Extract normative information and professional terminology information from the historical official document data;
[0035] Calculate the semantic similarity between the normative information and the technical terminology information to establish a hierarchical association.
[0036] The hierarchical relationships are expanded using a relation propagation algorithm to form semantic connections, and an entity relationship network is formed based on these semantic connections.
[0037] The entity relationship network is structured using graph structure optimization methods to form a knowledge graph for the government affairs domain.
[0038] Optionally, the step of performing image processing on the target scanned image to obtain character change information, and generating topology correction data based on the character change information, includes:
[0039] The target scan image is scanned line by line to obtain grayscale distribution data;
[0040] Based on the grayscale distribution data, the phase distortion distribution of each character region is determined, and character change information is generated based on the phase distortion distribution.
[0041] Based on the character change information, a morphological reconstruction algorithm is used to analyze the geometric deformation features of the character outline;
[0042] Based on the geometric deformation features, a deformation mapping relationship of the character surface is constructed using a surface fitting method, and topology correction data is generated according to the deformation mapping relationship.
[0043] Secondly, this application provides a text correction system that integrates domain knowledge graphs and deep learning models, comprising:
[0044] The acquisition module is used to acquire the target scanned image and historical document data, wherein the target scanned image contains character distortion areas caused by paper wrinkles and printing offset;
[0045] A module is established to build a knowledge graph in the government affairs field based on the historical official document data;
[0046] The generation module is used to perform image processing on the target scan image to obtain character change information, and generate topology correction data based on the character change information;
[0047] The fusion module is used to perform multi-scale fusion of the topology correction data and the target scan image to generate character structure information, and to associate the character structure information with the government affairs domain knowledge graph through a graph convolutional network to generate error location information;
[0048] The optimization module is used to generate an error-correcting text sequence based on the error location information and in conjunction with a deep learning model, and to perform multiple rounds of iterative optimization on the error-correcting text sequence through a discriminator network to generate the final text that meets the preset specification requirements.
[0049] Thirdly, this application provides an electronic device, comprising:
[0050] Memory, used to store computer programs;
[0051] A processor, configured to implement the steps of the text correction method for integrating domain knowledge graphs and deep learning models as described in the first aspect above, when executing the computer program.
[0052] Fourthly, this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement the steps of the text correction method for integrating domain knowledge graphs and deep learning models as described in the first aspect above.
[0053] This application provides a text correction method that integrates a domain knowledge graph and a deep learning model. The method includes: acquiring a target scanned image and historical official document data, wherein the target scanned image contains character distortion regions caused by paper wrinkles and printing offsets; establishing a government affairs domain knowledge graph based on the historical official document data; performing image processing on the target scanned image to obtain character change information, and generating topology correction data based on the character change information; fusing the topology correction data and the target scanned image at multiple scales to generate character structure information; associating the character structure information with the government affairs domain knowledge graph through a graph convolutional network to generate error location information; generating a corrected text sequence based on the error location information and combined with a deep learning model; and performing multiple rounds of iterative optimization on the corrected text sequence through a discriminator network to generate final text that meets preset specification requirements.
[0054] The technical solution provided in this application has the following beneficial effects:
[0055] This application provides an image to be corrected containing physical defects and constructs foundational materials based on domain knowledge. It also quantifies the degree and distribution of character deformation caused by physical defects, thus providing parameter basis for correcting character geometric deformation and laying the foundation for subsequent image restoration work. Based on the above parameters and knowledge, combined with the features of the original image and deformation correction parameters, enhanced features that simultaneously preserve detailed texture and overall structure can be formed to achieve cross-validation of character physical form and domain semantics, thereby accurately locating glyph errors and semantic anomalies. Furthermore, error location information is used to guide the text generation process, ultimately ensuring that the output text fully meets the requirements of government documents in terms of semantic accuracy and format standardization.
[0056] Furthermore, this application effectively embeds geometric deformation correction information into image features through a multi-level feature fusion mechanism, ensuring that the generated character structure information can not only restore the original shape of the deformed area at the pixel level, but also maintain the semantic structure integrity of the character itself at the feature level. This provides a feature representation with both detail accuracy and semantic fidelity for subsequent error localization work.
[0057] These or other aspects of this application will become more apparent in the following description of the embodiments. Attached Figure Description
[0058] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0059] Figure 1 A flowchart illustrating a text correction method that integrates a domain knowledge graph and a deep learning model, provided as an embodiment of this application;
[0060] Figure 2 A schematic diagram illustrating a specific implementation of a text correction method that integrates domain knowledge graphs and deep learning models, provided in an embodiment of this application;
[0061] Figure 3 This is a schematic diagram of the structure of a text correction system that integrates a domain knowledge graph and a deep learning model, as provided in an embodiment of this application. Detailed Implementation
[0062] Analysis of the background technology reveals that existing text correction schemes based on pre-trained language models face a dual challenge when dealing with scanned government documents. On the one hand, their text representation mechanisms are not sensitive to character morphological features, making it difficult to effectively capture abnormal character structures caused by paper wrinkles or printing offsets. On the other hand, the model has limited knowledge of professional government terminology and standardized expressions, which can easily lead to misjudgments of professional terms and inaccurate standardized expressions in specific scenarios involving scheme terminology and fixed sentence patterns in official documents.
[0063] To address the aforementioned challenges, this application proposes a text correction method that integrates domain knowledge graphs and deep learning models. This method constructs a government domain knowledge graph to provide necessary semantic support for standardized expressions and professional terms. Simultaneously, it utilizes topology correction data and multi-scale feature fusion technology to extract character deformation features, and then leverages graph convolutional networks to achieve cross-validation between character physical form and domain semantics. This multimodal collaborative working mechanism can accurately identify character shape errors caused by physical defects and effectively ensure the accuracy of professional terms and standardized expressions in official documents, thereby achieving a unification of physical deformation correction and semantic standardization correction in the context of digital processing of government documents.
[0064] To enable those skilled in the art to better understand the present application, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are merely some embodiments of the present application, and not all embodiments. Based on the embodiments in this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.
[0065] The core of this application is to provide a text error correction method that integrates domain knowledge graphs and deep learning models. A flowchart illustrating one specific implementation is shown below. Figure 1 As shown, the method includes:
[0066] Step 101: Obtain the target scan image and historical document data.
[0067] In step 101, the target scan image, also known as the original scan image, refers to the digital image of a paper document obtained by a scanning device. The target scan image contains areas where character strokes are distorted or blurred due to paper wrinkles or printing offsets. Historical document data refers to the collection of standardized electronic documents accumulated in the past, which serves as the data foundation for constructing a knowledge graph.
[0068] For example, in the image obtained by scanning a document titled "Notice on the Advancement of Work in Area A" from a certain department, the characters for "management" in the third line were broken due to paper wrinkles. To address this issue, 500 similar notices from the past five years were retrieved from the historical database, and the scanned image resolution was set to 600 dpi. These historical documents were then parsed using UTF-8 encoding and stored in a temporary database.
[0069] Step 102: Based on the historical document data, establish a knowledge graph for the government affairs field.
[0070] In step 102, the government domain knowledge graph is a semantic network that includes normative expression entities and specialized terminology entities and their relationships, where nodes represent scheme terms or standard sentence structures, and edges represent semantic logical relationships between terms.
[0071] In this embodiment, high-frequency scheme terms and fixed phrases are extracted from historical official document data as entities. Then, the hierarchical relationship between entities is constructed through semantic similarity calculation. The discrete entities are organized into a network knowledge base with multi-level indexes using a graph structure optimization method, and finally a government affairs domain knowledge graph containing terminology definitions, expression norms and logical constraints is formed.
[0072] For example, 86 core terms such as "urban management" and "municipal facilities" and 12 standard sentence patterns such as "Notice on Further Strengthening the Management of xxx" were extracted from the aforementioned 500 documents as basic entities. Then, the relationship between terms was established by calculating the cosine similarity of word vectors. For example, the similarity between "urban management" and "municipal management" reached 0.91. Finally, a knowledge graph of the government domain containing a three-layer semantic structure was constructed.
[0073] Step 103: Perform image processing on the target scan image to obtain character change information, and generate topology correction data based on the character change information.
[0074] In step 103, the character change information is matrix data describing the degree of deformation of the character region, which includes phase gradient information; the topology correction data is a set of parameters used to correct the geometric deformation of the character, which includes position correction parameters and contour repair information.
[0075] In this embodiment, after the scanned image is grayscaled, the phase distortion distribution of each character region is calculated using geometric distortion analysis technology to generate character deformation information representing the changes. Based on this data, a morphological reconstruction algorithm is further used to analyze the deformation characteristics of the character contour. Finally, a deformation mapping relationship is established using a surface fitting method to generate corresponding topology correction data.
[0076] For example, calculate the phase distortion of the character region of the character "pipe" in the image in step 101 through the polynomial fitting formula , where represents the phase distortion function, is the number of basis functions, is the basis function, is the fitting coefficient, the weight coefficient corresponding to the i-th basis function; based on this, calculate the character change information representing the deformation degree of the character, and generate topological correction data containing the coordinates of 15 key points accordingly.
[0077] Step 104: Perform multi-scale fusion on the topological correction data and the target scanned image to generate character structure information, and associate the character structure information with the government affairs domain knowledge graph through a graph convolutional network to generate error location information.
[0078] In step 104, the character structure information is enhanced features obtained through multi-scale fusion, and simultaneously includes the detailed texture and overall layout features of the character;
[0079] The error location information refers to the digital description of the accurately located error type and coordinate position after analysis by the graph convolutional network, where the error type includes abnormal glyph structure and semantic logic error, and the coordinate position identifies the physical position of the specific error point in the document in the form of row and column numbers.
[0080] In the embodiment of the present application, perform point-by-point superposition on the position correction parameters in the topological correction data and the pixels of the original image, and at the same time perform a cascading operation on the contour repair information and the channels of the convolutional feature map, and then fuse these multi-scale features through a feature pyramid network to generate character structure information; then use the graph convolutional network to calculate the association strength between the character nodes and the knowledge graph entities, and finally generate the corresponding error location information according to the area where the association strength is abnormal.
[0081] For example, after fusing the topological correction data of the character "pipe" with the original image, process it through a 3-layer feature pyramid network to obtain 256-dimensional character structure information; when these information are analyzed by the graph convolutional network, it is found that the association strength between this character and the "management" node in the knowledge graph is 0.31, lower than the normal threshold of 0.7, and then generate error location information located at row 3 and column 5 based on this abnormal result.
[0082] Step 105: Based on the error location information, combine it with a deep learning model to generate a corrected text sequence, and perform multiple rounds of iterative optimization on the corrected text sequence through a discriminator network to generate the final text that meets the preset specification requirements.
[0083] In step 105, the error correction text sequence is a set of multiple alternative corrected texts generated by a deep learning model, and the deep learning model may include a sequence-to-sequence model;
[0084] The specific content of the preset specification requirements includes the standardized expressions of the normative expressions and special terms constructed based on historical government official documents, the logical constraint conditions of the entity relationships defined in the government affairs domain knowledge graph, and the semantic normativity features represented by the embedded vectors of the domain knowledge graph.
[0085] In the embodiments of the present application, the error location information is encoded into a position marker sequence and then input into the sequence-to-sequence model, and at the same time, the embedded vectors of the knowledge graph are combined to generate multiple candidate texts; then, the discriminator network calculates the consistency scores of each candidate text in the context and its compliance scores with the official document specifications, and after re-ranking and multiple rounds of optimization steps, the final text that meets the requirements of the government official document specifications is finally output.
[0086] For example, for the error of the character "管", the model generated 5 candidate words such as "管理" and "管控". After calculation, the discriminator obtained that the consistency score of "管理" in the context is 0.92 and its structural compliance score is 0.88. After 3 rounds of optimization, the final output is the document title "Notice on Further Strengthening the Management of Field A" that meets the specifications, and this result fully meets the requirements of the government official document specifications.
[0087] This method solves the recognition errors caused by glyph distortion through the collaborative work of the knowledge graph and the deep learning model, and at the same time ensures the normativity of professional terms and official document formats, thereby improving the accuracy and reliability of digital official document processing.
[0088] To solve the recognition accuracy problem caused by character deformation in the scanned images of government official documents, in some embodiments, step 104: The multi-scale fusion of the topological correction data and the target scanned image to generate character structure information, such as Figure 2 shown, includes:
[0089] Step 201: Extract the position correction parameters and contour repair information from the topological correction data.
[0090] In step 201, the position correction parameters are a set of data describing the coordinate offsets that each pixel point needs to be adjusted, and the contour repair information is a multi-dimensional numerical sequence representing the correction direction and size of the character contour shape.
[0091] Step 202: At the pixel scale, perform point-by-point correction of the position correction parameters and the original pixels of the target scanned image.
[0092] In step 202, the point-by-point correction result is a preliminary corrected image generated by superimposing the position correction parameters one-to-one with the pixels in the original image.
[0093] In this embodiment, the horizontal and vertical offsets in the position correction parameters are added to the original pixel coordinates of the corresponding positions to directly correct the pixel position deviation caused by physical deformation, and finally form an intermediate image with accurate geometric position.
[0094] Step 203: At the feature scale, perform convolution processing on the target scan image to obtain a convolution feature map, and concatenate the contour repair information with the convolution feature map.
[0095] In step 203, the convolutional feature map is the feature representation obtained after performing a convolution operation on the original image, and the channel concatenation result is an enhanced feature formed by splicing the contour repair information and the convolutional feature map in the channel dimension.
[0096] In this embodiment, the original scanned image is first processed by multi-layer convolution to extract its texture features. Then, the contour restoration information is copied and expanded to the same spatial size as the convolution feature map. Finally, the texture features and the expanded contour restoration information are spliced together in the channel dimension to form a composite feature containing detail restoration information.
[0097] Step 204: Based on the point-by-point correction results and channel cascade results, the character structure information is obtained by fusing them through a multi-scale feature pyramid.
[0098] In step 204, the multi-scale feature pyramid is a hierarchical structure containing feature maps of different resolutions.
[0099] In this embodiment, the result of point-by-point correction is used as the low-level high-resolution feature, and the result of channel concatenation is used as the middle-level semantic feature. The top-level global feature is obtained by downsampling the middle-level feature. Then, the top-level feature is upsampled and superimposed with the middle-level feature in a top-down manner, and then upsampled again and concatenated with the low-level feature. Finally, the character structure information that integrates multi-scale information is obtained through convolution compression operation.
[0100] Here is a specific example:
[0101] Following the previous example, position correction parameters and contour repair information are first extracted from the generated topology correction data containing the coordinates of 15 key points. The position correction parameters record the coordinate offset of each pixel in the form of a two-dimensional array. For example, the offset of key point [125, 368] is [0.5, -0.3]. The contour repair information is a 32-dimensional numerical sequence [0.38, 0.25, ..., 0.40].
[0102] In the pixel-scale processing stage, the position correction parameters are used to perform point-by-point correction on the original scanned image at 600 dpi. Specifically, the pixel with coordinates [125, 368] is corrected to the new position [125.5, 367.7] according to its offset [0.5, -0.3]. The calculation process can adopt the following coordinate transformation formula , where represents the new coordinate vector, represents the original coordinate vector, represents the offset vector, and the coordinate unit is pixel;
[0103] In the feature-scale processing stage, the original image is subjected to convolution processing with a 3×3 convolution kernel to obtain a convolution feature map with 64 channels. At the same time, the 32-dimensional contour repair information is replicated and extended to a feature map with 32 channels, and then channel concatenation is performed with the convolution feature map to form an enhanced feature map with 96 channels;
[0104] Finally, based on the point-by-point correction result and the channel concatenation result, fusion is performed through a feature pyramid consisting of 3 layers. Specifically, the point-by-point correction result is used as the bottom layer feature of 512×512 pixels, the 96-channel feature map is downsampled to the middle layer feature of 256×256 pixels and the top layer feature of 128×128 pixels, and the feature pyramid network is used for top-down fusion. Among them, the top layer feature is upsampled by 2 times and then added element by element to the middle layer feature, and the fusion result is upsampled by 2 times and then channel concatenated with the bottom layer feature;
[0105] Finally, the number of channels is compressed by 1×1 convolution to generate 256-dimensional character structure information, which completely retains the detailed texture at the stroke connection and the overall glyph structure features of the corrected character "pipe".
[0106] In the embodiment of the present application, through the multi-scale fusion mechanism, the geometric correction information and the original image features are organically combined, so that the character structure information not only corrects the position deviation caused by physical deformation, but also enhances the semantic feature representation of the character, providing a high-quality feature basis for subsequent error localization.
[0107] To further improve the quality of the character structure information, in some embodiments, step 20④: The fusion through the multi-scale feature pyramid based on the point-by-point correction result and the channel concatenation result to obtain the character structure information includes:
[0108] Step 30①: Use the point-by-point correction result as the first feature map and the channel concatenation result as the second feature map.
[0109] In step 301, the first feature map refers to the image features containing precise location information obtained after point-by-point correction, and the second feature map refers to the enhanced feature representation containing character component outline features formed after channel cascading.
[0110] In this embodiment, the feature data with precise geometric position generated by point-by-point correction is used as the basic feature layer, while the composite features containing contour repair information after channel cascading are used as the intermediate feature layer, thereby establishing the required feature foundation for subsequent multi-scale analysis.
[0111] Step 302: Perform dimensionality reduction on the second feature map to obtain the third feature map.
[0112] In step 302, the third feature map refers to the feature representation containing high-level semantic information obtained by dimensionality reduction of the second feature map.
[0113] In this embodiment, convolution operation is used to compress the spatial dimension of the second feature map, reducing its data volume while retaining important features, thereby extracting abstract features that can characterize the overall structure and semantic content of the character.
[0114] Step 303: The first feature map, the second feature map, and the third feature map are fused at multiple levels using a feature pyramid network. During the fusion process, the third feature map is upsampled and then superimposed with the second feature map to obtain a first fused feature map. The first fused feature map is upsampled and then added element-wise with the first feature map to obtain a second fused feature map.
[0115] In step 303, the first fused feature map refers to the intermediate result after the third feature map and the second feature map are fused, and the second fused feature map refers to the final feature representation after the first fused feature map and the first feature map are further fused.
[0116] In this embodiment, the spatial resolution of the third feature map is restored by upsampling, and then it is superimposed with the second feature map to achieve complementarity between high-level semantics and mid-level features. Then, the obtained fusion result is upsampled again and added element by element with the first feature map to complete the full-scale feature integration from semantics to details.
[0117] Step 304: Compress the second fused feature map by channel dimension to generate character structure information.
[0118] In this embodiment of the application, the number of channels of the second fused feature map is reduced by using a 1×1 convolution kernel, thereby reducing its data dimension while retaining the multi-scale fused features, and thus generating a compact feature representation that contains both detailed texture and semantic information.
[0119] In this embodiment, features at different levels of abstraction are organically integrated through a multi-scale feature pyramid fusion mechanism, so that the generated character structure information simultaneously possesses accurate geometric position features, clear component outline features, and high-level semantic features, thereby providing a comprehensive and reliable feature foundation for subsequent error detection.
[0120] To further improve the accuracy and standardization of text correction for government documents, in some embodiments, step 105: based on the error location information, combined with a deep learning model, generating an error-corrected text sequence, and performing multiple rounds of iterative optimization on the error-corrected text sequence through a discriminator network to generate final text that meets preset standard requirements, includes:
[0121] Step 401: Combine and encode the error type and error location information in the error location information into a location marker sequence.
[0122] In step 401, the position mark sequence refers to a digital sequence formed by combining the error type code and the error position code in a predetermined format.
[0123] In this embodiment of the application, the error type numerical code and the row and column position numerical code in the error location information are combined into a sequence in the order of type first and position second, thereby providing structured error description information for subsequent deep learning models.
[0124] Step 402: Input the location tag sequence and the government domain knowledge graph into the deep learning model. Through the encoder of the deep learning model, based on the location tag sequence, and combined with a multi-head attention mechanism, perform semantic enhancement processing on the entity embedding vector of the government domain knowledge graph to obtain the enhanced text representation.
[0125] In step 402, the entity embedding vector of the government domain knowledge graph refers to the low-dimensional dense numerical vector that maps the preset normative expression entities and entity nodes in the government domain knowledge graph to graph embedding technology. These vectors are obtained by feature learning of entity nodes and their relationships through graph neural networks. Their source and associated objects are entity nodes in the government domain knowledge graph containing preset normative expressions and special terminology entities established through historical official document data.
[0126] Enhanced text representation refers to the text feature representation that incorporates domain knowledge after semantic enhancement using a knowledge graph.
[0127] In this embodiment, after the location tag sequence is input into the encoder, the entity embedding vectors in the government domain knowledge graph are fused with text features through a multi-head attention mechanism so that the text representation can simultaneously contain the original semantics and domain knowledge features.
[0128] Step 403: Using the decoder of the deep learning model, based on the enhanced text representation, generate multiple alternative complete texts to form an error-corrected text sequence.
[0129] In this embodiment of the application, a number of candidate texts are generated autoregressively by a decoder based on the enhanced text representation, wherein each candidate text is a possible correction result of the original erroneous text.
[0130] Step 404: Using a discriminator network, calculate the consistency score and compliance score between each candidate complete text in the error-corrected text sequence and the specification requirements.
[0131] In step 404, the discriminator network is an independent network module separate from the aforementioned sequence-to-sequence deep learning model. Together, they constitute a complete text correction system, in which the sequence-to-sequence model is responsible for generating candidate texts, and the discriminator network is responsible for evaluating and filtering the candidate texts.
[0132] Consistency score refers to the degree of matching between candidate text and entity semantics in knowledge graph, while conformity score refers to the degree of conformity between candidate text and official document format specifications.
[0133] In the embodiments of this application, the discriminator network calculates the semantic similarity between each candidate text and the knowledge graph entity vector, as well as the structural similarity between the candidate text sentence structure and the official document template, thereby generating an evaluation score in two dimensions.
[0134] Specific implementation process: The discriminator network performs calculations through a dual-channel evaluation mechanism. In the semantic consistency evaluation channel, candidate texts are matched with entities in the government domain knowledge graph, and the cosine similarity of their vector representations is calculated. The calculation formula is as follows: ,in For cosine similarity, and These represent the first and second lines of the candidate text vectors, respectively. The component and the first component of the knowledge graph entity vector The text is divided into several components; in the structural conformity assessment channel, the candidate text is compared with the standard template of government documents to analyze its sentence structure, paragraph format and the standardization of terminology.
[0135] For example, when evaluating the candidate text "strengthen urban management", the discriminator network calculates its similarity to entities such as "urban management" and "municipal management" in the knowledge graph through the semantic consistency channel, thus obtaining a consistency score of 0.92. At the same time, it analyzes its structural features with official document notices through the structural conformity channel, thus obtaining a conformity score of 0.88.
[0136] Step 405: Based on the consistency score and the conformity score, prioritize each candidate complete text in the error-correcting text sequence using a reordering algorithm.
[0137] In this embodiment, the consistency score and conformity score output by the discriminator network are weighted and summed according to preset weights. Then, the candidate texts in the error correction text sequence are reordered from high to low according to the total score obtained by the summation. This reordering step is not performed by the discriminator network, but is a subsequent independent processing step. This step receives the consistency score and conformity score output by the discriminator network as its input data and rearranges the candidate texts through an independent sorting algorithm.
[0138] Step 406: Perform multiple rounds of local optimization on the highest priority candidate complete text in the priority ranking results. In each round of optimization, adjust the optimization strategy according to the feedback of the discriminator network until the preset number of iterations is reached. The final optimized complete text is then output as the final text that meets the specification requirements.
[0139] In this embodiment, the candidate text ranked first is modified locally, such as by word replacement and sentence structure adjustment. After each round of modification, its score is re-evaluated by a discriminator network. The subsequent optimization direction is then dynamically adjusted based on the score changes until the preset number of iterations is reached, at which point the final text is output.
[0140] In the embodiments of this application, the text generation enhanced by knowledge graph and the multi-round discrimination optimization mechanism ensure that the output text accurately corrects the original errors.
[0141] To further improve the accuracy of error location, in some embodiments, step 104: associating the character structure information with the government affairs domain knowledge graph through a graph convolutional network to generate error location information includes:
[0142] Step 501: Based on the character structure information, construct a character structure graph, and establish a bidirectional message passing mechanism between the character structure graph and the target scanned image through a graph convolutional network.
[0143] In step 501, the character structure graph is a graph structure composed of characters as nodes and spatial relationships between characters as edges. The bidirectional message passing mechanism is a computational method in graph convolutional networks to realize feature interaction between nodes.
[0144] In this embodiment, character structure information is mapped to feature vectors of nodes in the graph, and edge connections are established based on the relative positions of characters in the document. Then, bidirectional feature flow between character nodes and knowledge graph entity nodes is achieved through graph convolutional layers.
[0145] Step 502: Calculate the association strength value between the character nodes in the character structure graph and the entity nodes in the government affairs domain knowledge graph using a two-way message passing mechanism.
[0146] In step 502, the association strength value is a numerical value that characterizes the degree of semantic relevance between character nodes and knowledge entity nodes.
[0147] In this embodiment of the application, during the bidirectional message passing process, the similarity between character node features and knowledge entity node features is calculated by attention weight, and then a quantified association strength value is generated based on the relative distance in the feature space.
[0148] Step 503: Based on the association strength value, detect the inconsistency area between the region corresponding to the character node and the region corresponding to the entity node, and generate error location information based on the inconsistency area.
[0149] In step 503, the inconsistent region refers to the character region where the character morphological features do not match the semantic features of the knowledge entity.
[0150] In this embodiment, the association strength value is compared with a preset threshold to identify character nodes with low association strength. Then, based on the deviation between the character position and association strength corresponding to these nodes, error location information containing error type and location information is generated.
[0151] In this embodiment of the application, cross-modal association analysis of graph convolutional networks is used to achieve deep fusion detection of character physical form and domain semantics, which can accurately locate semantic inconsistency problems caused by deformation and provide a reliable basis for subsequent error correction.
[0152] To construct a high-quality domain knowledge base to support text error correction, in some embodiments, step 102: establishing a government affairs domain knowledge graph based on the historical official document data includes:
[0153] Step 601: Extract normative information and professional terminology information from the historical official document data.
[0154] In step 601, normative information refers to the standard sentence structure that appears repeatedly in government documents, and professional terminology information refers to professional vocabulary and fixed expressions unique to the government field.
[0155] In this embodiment of the application, fixed sentence patterns and professional terms are identified from historical official document data through text pattern matching and part-of-speech analysis. Sentence structures with complete semantics are used as normative information, while professional terms expressing specific concepts are used as professional terminology information.
[0156] Step 602: Calculate the semantic similarity between the normative information and the technical terminology information to establish a hierarchical association.
[0157] In step 602, semantic similarity is a numerical value that measures the degree of semantic association between different text units, and hierarchical association is a tree-like structure relationship established based on semantic similarity.
[0158] In this embodiment, text units are converted into vector representations using a word vector model, and the cosine similarity between different vectors is calculated to obtain semantic similarity values. Then, a hierarchical association structure from general to specific is constructed based on the magnitude of the similarity values.
[0159] Step 603: Expand the hierarchical association relationship using the relationship propagation algorithm to form a semantic connection relationship, and form an entity relationship network based on the semantic connection relationship.
[0160] In step 603, semantic connection relationship refers to indirect semantic association established through intermediate entities. Entity relationship network is a mesh structure composed of entity nodes and semantic relationship edges.
[0161] In this embodiment of the application, based on the established hierarchical relationship, the indirect relationship between entities is discovered through the relationship propagation algorithm, and the direct and indirect relationships are organized together into a network structure with multi-hop connection capability.
[0162] Step 604: The entity relationship network is structured using a graph structure optimization method to form a knowledge graph for the government affairs domain.
[0163] In this embodiment, community detection and redundant edge elimination are performed on the entity relationship network, densely connected regions are merged into semantic communities, and duplicate and secondary connections are removed, thereby forming a well-structured knowledge graph for the government domain.
[0164] In this embodiment of the application, a domain knowledge graph with complete semantic relationships and hierarchical structure is constructed through systematic knowledge extraction and structured processing, thereby providing a reliable semantic judgment basis for text correction.
[0165] To accurately characterize and correct character deformation, in some embodiments, step 103: performing image processing on the target scanned image to obtain character change information, and generating topology correction data based on the character change information, includes:
[0166] Step 701: Scan the target image line by line to obtain grayscale distribution data.
[0167] In step 701, grayscale distribution data refers to matrix data of the brightness values of each pixel in the scanned image.
[0168] In this embodiment, a linear optical sensor is used to perform photoelectric conversion line by line on the scanned image, thereby recording the grayscale value of each pixel position and forming complete image brightness distribution information.
[0169] Step 702: Based on the grayscale distribution data, determine the phase distortion distribution of each character region, and generate character change information based on the phase distortion distribution.
[0170] In step 702, the character region is an image region containing independent text symbols obtained by performing text line detection and character segmentation on the scanned image of government documents. It can also refer to the local image range where a single character or punctuation mark in the scanned image is deformed due to physical defects.
[0171] Phase distortion distribution is spatial data that describes the degree of light wavefront distortion in a character region.
[0172] In this embodiment, the phase gradient of each character region is first calculated based on grayscale distribution data using wavefront sensing technology, and then the phase gradient information is integrated into a distortion field description covering the entire character region.
[0173] Step 703: Based on the character change information, use a morphological reconstruction algorithm to analyze the geometric deformation features of the character outline.
[0174] In step 703, geometric deformation features refer to the changes in geometric properties of the character outline during the deformation process.
[0175] In this embodiment, the ideal outline of the character is first reconstructed through morphological operations based on the character change information. Then, the geometric feature changes of the character during the deformation process are extracted by comparing the difference between the actual outline and the ideal outline.
[0176] Step 704: Based on the geometric deformation features, construct the deformation mapping relationship of the character surface through the surface fitting method, and generate topology correction data according to the deformation mapping relationship.
[0177] In step 704, the deformation mapping relationship describes the coordinate transformation relationship of the character surface from the distorted state to the standard state.
[0178] In this embodiment, a mapping function from the distorted character surface to the standard character surface is first established based on the geometric deformation characteristics through surface fitting. Then, topology correction data containing position correction and shape repair parameters is generated according to the mapping function.
[0179] In this embodiment, a complete processing chain from optical distortion detection to geometric deformation correction is established to accurately quantify and repair the physical deformation of characters, thereby providing an accurate geometric correction basis for subsequent multi-scale feature fusion.
[0180] Figure 3 This application provides a schematic diagram of the structure of a text correction system that integrates domain knowledge graphs and deep learning models, as shown in the embodiments of this application. Figure 3 As shown, the detailed implementation section describes:
[0181] The acquisition module 31 is used to acquire the target scanned image and historical document data, wherein the target scanned image contains character distortion areas caused by paper wrinkles and printing offset.
[0182] Module 32 is used to establish a knowledge graph in the field of government affairs based on the historical official document data.
[0183] The generation module 33 is used to perform image processing on the target scan image to obtain character change information, and generate topology correction data based on the character change information.
[0184] The fusion module 34 is used to perform multi-scale fusion of the topology correction data and the target scan image to generate character structure information, and to associate the character structure information with the government affairs domain knowledge graph through a graph convolutional network to generate error location information.
[0185] The optimization module 35 is used to generate an error-correcting text sequence based on the error location information and in conjunction with a deep learning model, and to perform multiple rounds of iterative optimization on the error-correcting text sequence through a discriminator network to generate the final text that meets the preset specification requirements.
[0186] The text correction system that integrates domain knowledge graphs and deep learning models in this application is used to implement the aforementioned text correction method that integrates domain knowledge graphs and deep learning models. Therefore, the specific implementation of the text correction system that integrates domain knowledge graphs and deep learning models can be found in the embodiment section of the text correction method that integrates domain knowledge graphs and deep learning models above. The specific implementation can be referred to the description of the corresponding embodiments, which will not be repeated here.
[0187] This application also provides an electronic device, comprising: a memory for storing a computer program; and a processor for executing the computer program to implement the steps of the text correction method that integrates domain knowledge graphs and deep learning models as described above.
[0188] This application also provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of any of the above-described text correction methods that integrate domain knowledge graphs and deep learning models.
[0189] In one exemplary embodiment, the aforementioned computer-readable storage medium may include, but is not limited to, various media capable of storing computer programs, such as USB flash drives, read-only memory, random access memory, portable hard drives, magnetic disks, or optical disks.
[0190] Embodiments of this application also provide a computer program product, which includes a computer program that, when executed by a processor, implements the steps in any of the above embodiments of the text correction method that integrates domain knowledge graphs and deep learning models.
[0191] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.
[0192] The foregoing has provided a detailed description of a text correction method and system integrating domain knowledge graphs and deep learning models, as provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The descriptions of the embodiments above are merely for the purpose of helping to understand the method and its core ideas. It should be noted that those skilled in the art can make various improvements and modifications to this application without departing from its principles, and these improvements and modifications also fall within the protection scope of this application.
Claims
1. A text correction method integrating domain knowledge graphs and deep learning models, characterized in that, include: Acquire target scan images and historical document data; Based on the aforementioned historical official document data, a knowledge graph for the government affairs field will be established; Image processing is performed on the target scan image to obtain character change information, and topology correction data is generated based on the character change information; The topology correction data is fused with the target scan image at multiple scales to generate character structure information. The character structure information is then associated with the government affairs knowledge graph through a graph convolutional network to generate error location information. Based on the error location information, a correction text sequence is generated using a deep learning model. The correction text sequence is then iteratively optimized through a discriminator network to generate the final text that meets the preset specification requirements.
2. The text correction method integrating domain knowledge graphs and deep learning models according to claim 1, characterized in that, The step of fusing the topology correction data with the target scanned image at multiple scales to generate character structure information includes: Position correction parameters and contour repair information are extracted from the topology correction data; At the pixel scale, the position correction parameters are corrected point by point with the original pixels of the target scanned image; At the feature scale, the target scan image is convolved to obtain a convolutional feature map, and the contour repair information is channel-concatenated with the convolutional feature map; Based on the point-by-point correction results and channel cascading results, character structure information is obtained by fusing them through a multi-scale feature pyramid.
3. The text correction method integrating domain knowledge graphs and deep learning models according to claim 2, characterized in that, The character structure information is obtained by fusing the point-by-point correction results and channel concatenation results through a multi-scale feature pyramid, including: The point-by-point correction result is used as the first feature map, and the channel cascade result is used as the second feature map; The second feature map is dimensionality reduced to obtain the third feature map; The first feature map, the second feature map, and the third feature map are fused in multiple levels using a feature pyramid network. During the fusion process, the third feature map is upsampled and then superimposed with the second feature map to obtain the first fused feature map. The first fused feature map is then upsampled and then added element-wise with the first feature map to obtain the second fused feature map. Channel dimension compression is performed on the second fused feature map to generate character structure information.
4. The text correction method integrating domain knowledge graphs and deep learning models according to claim 1, characterized in that, Based on the error location information, and combined with a deep learning model, an error-corrected text sequence is generated. This sequence is then iteratively optimized multiple times using a discriminator network to generate final text that meets preset specifications. This process includes: The error type and error location information in the error location information are combined and encoded into a location marker sequence; The location tag sequence and the government affairs domain knowledge graph are input into a deep learning model. The encoder of the deep learning model performs semantic enhancement processing on the entity embedding vector of the government affairs domain knowledge graph based on the location tag sequence and combined with a multi-head attention mechanism to obtain the enhanced text representation. The deep learning model's decoder generates multiple alternative complete texts based on the enhanced text representation, forming a sequence of error-corrected texts. Using a discriminator network, calculate the consistency score and compliance score between each candidate complete text in the error-correcting text sequence and the specification requirements; Based on the consistency score and the conformity score, the candidate complete texts in the error-correcting text sequence are prioritized and sorted using a reordering algorithm. The highest-priority candidate complete text in the priority ranking results is subjected to multiple rounds of local optimization. In each round of optimization, the optimization strategy is adjusted according to the feedback of the discriminator network until the preset number of iterations is reached. The final optimized complete text is then output as the final text that meets the specification requirements.
5. The text correction method integrating domain knowledge graphs and deep learning models according to claim 1, characterized in that, The step of associating the character structure information with the government affairs domain knowledge graph through a graph convolutional network to generate error location information includes: Based on the character structure information, a character structure graph is constructed, and a bidirectional message passing mechanism is established between the character structure graph and the target scanned image through a graph convolutional network. Using a two-way message passing mechanism, the association strength value between the character nodes in the character structure graph and the entity nodes in the government affairs domain knowledge graph is calculated; Based on the association strength value, inconsistencies between the region corresponding to the character node and the region corresponding to the entity node are detected, and error location information is generated based on the inconsistencies.
6. The text correction method integrating domain knowledge graphs and deep learning models according to claim 1, characterized in that, The establishment of a knowledge graph in the government affairs field based on the historical official document data includes: Extract normative information and professional terminology information from the historical official document data; Calculate the semantic similarity between the normative information and the technical terminology information to establish a hierarchical association. The hierarchical relationships are expanded using a relation propagation algorithm to form semantic connections, and an entity relationship network is formed based on these semantic connections. The entity relationship network is structured using graph structure optimization methods to form a knowledge graph for the government affairs domain.
7. The text correction method integrating domain knowledge graphs and deep learning models according to claim 1, characterized in that, The step of performing image processing on the target scan image to obtain character change information and generating topology correction data based on the character change information includes: The target scan image is scanned line by line to obtain grayscale distribution data; Based on the grayscale distribution data, the phase distortion distribution of each character region is determined, and character change information is generated based on the phase distortion distribution. Based on the character change information, a morphological reconstruction algorithm is used to analyze the geometric deformation features of the character outline; Based on the geometric deformation features, a deformation mapping relationship of the character surface is constructed using a surface fitting method, and topology correction data is generated according to the deformation mapping relationship.
8. A text correction system integrating domain knowledge graphs and deep learning models, characterized in that, include: The acquisition module is used to acquire the target scan image and historical document data; A module is established to build a knowledge graph in the government affairs field based on the historical official document data; The generation module is used to perform image processing on the target scan image to obtain character change information, and generate topology correction data based on the character change information; The fusion module is used to perform multi-scale fusion of the topology correction data and the target scan image to generate character structure information, and to associate the character structure information with the government affairs domain knowledge graph through a graph convolutional network to generate error location information; The optimization module is used to generate an error-correcting text sequence based on the error location information and in conjunction with a deep learning model, and to perform multiple rounds of iterative optimization on the error-correcting text sequence through a discriminator network to generate the final text that meets the preset specification requirements.
9. An electronic device, characterized in that, include: Memory, used to store computer programs; A processor, configured to implement the steps of the text correction method for fusing domain knowledge graphs and deep learning models as described in any one of claims 1 to 7 when executing the computer program.
10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program that, when executed by a processor, enables the text correction method that integrates a domain knowledge graph and a deep learning model as described in any one of claims 1 to 7.