A detection-guided Chinese text correction method based on adaptive multi-modal fusion

Abstract

The present application relates to the technical field of natural language processing and deep learning, and particularly relates to a detection-guided Chinese text correction method based on adaptive multi-modal fusion; the method takes MacBERT as a semantic backbone, and introduces an input forgetting gate to filter bottom input noise; secondly, a fine-grained pinyin encoder decoupled from tone and a distribution-aligned enhanced glyph encoder are constructed; then, a cascaded gate residual fusion mechanism is designed, and multi-modal information is gradually injected in the order of pinyin and glyph to avoid feature conflicts; finally, a detection-guided representation enhancement strategy is proposed, which uses detection confidence probability to explicitly highlight potential error areas, and opens up the deep collaboration path between the two tasks. In the SIGHAN benchmark data set, the correction F1 value, correction recall rate and detection recall rate of the method reach 81.0%, 84.1% and 85.0% respectively, which are superior to the existing SOTA model, and have excellent correction effectiveness and robustness.

Description

Technical Field

[0001] This invention relates to the fields of natural language processing and deep learning technology, specifically to a Chinese text error correction method based on adaptive multimodal fusion and detection guidance. Background Technology

[0002] Chinese text correction is a crucial foundational task in the field of natural language processing, used to automatically detect and correct spelling errors in Chinese text. It plays a central role in search engines, optical character recognition post-processing, and automatic scoring systems. Numerous studies have confirmed that phonetic similarity and glyph similarity are the core factors leading to Chinese spelling errors.

[0003] Currently, there are numerous methods and systems for Chinese text error correction. Among them, a technical solution closely related to this invention is: Invention Patent Publication No.: CN120805897A, Title: A Two-Stage Chinese Spelling Error Correction Method and System Based on Pinyin Enhancement. This method proposes to integrate pinyin features and masking language strategies to construct a multi-dimensional feature framework that includes semantic, positional, and pinyin representations, thereby achieving accurate prediction and efficient correction of text errors. Although this method includes pinyin and semantic information, it ignores the glyph information of Chinese characters. Furthermore, the fusion of information from multiple different modalities often generates noise, which significantly impacts the performance of error correction.

[0004] Overall, existing multimodal spelling correction techniques still suffer from two inherent technical defects: First, the coarse-grained direct parallel fusion approach lacks a low-level filtering mechanism when integrating modalities, making it easy for heterogeneous modalities to conflict in the semantic space. This causes the original multimodal features to attenuate during forward propagation in deep networks, inevitably introducing a large amount of external feature noise. Second, the error detection task and the error correction task are loosely coupled for a long time, resulting in the error correction branch completely lacking precise prior knowledge of error location. When the model processes characters with complex dependencies, it is very easy to cause overcorrection by modifying the correct characters. Summary of the Invention

[0005] The purpose of this invention is to provide a detection-guided Chinese text error correction method based on adaptive multimodal fusion, so as to solve the technical problems in the prior art, such as noise introduced by heterogeneous conflicts of multimodal features, and inaccurate error localization and overcorrection caused by insufficient interaction between detection and error correction tasks.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A detection-guided Chinese text error correction method based on adaptive multimodal fusion includes the following steps:

[0008] Step 1: Obtain the input Chinese character sequence to be processed and map it to the initial embedding vector E.token After filtering the input noise signal following the initial embedding layer, the denoised embedded E filtered After adding absolute position encoding, it is fed into the backbone encoder to obtain a clean contextual semantic representation H. text ;

[0009] Step 2: Extract fine-grained pinyin features from the input Chinese character sequence. Decompose each Chinese character in the input sequence into three independent pinyin units: initial consonant, final vowel, and tone. Then, after dimensionality reduction, weight matrix projection, and activation processing, output the pinyin representation H. pinyin ;

[0010] Step 3: Extract the glyph structure features of the input Chinese character sequence. Input the grayscale image of the Chinese character to extract local strokes and structural features. After dimensionality reduction and linear projection processing, output the glyph features H. glyph ;

[0011] Step 4: Construct a cascaded gated residual fusion mechanism. Following a fixed temporal logic of injecting auxiliary information first (phonetic features then glyph features), the auxiliary information is fed into a context-aware gating network to calculate the fusion weight coefficient g. First, the phonetic features obtained in Step 2 are injected into the H obtained in Step 1. text To obtain a preliminary enhanced semantic representation, this preliminary representation is then used as a new input state to continue performing the same gating mechanism to fuse the glyph features obtained in step 3, and finally output the multimodal enhanced representation H. fused ;

[0012] Step 5: Execute the detection-guided representation enhancement strategy to enhance the multimodal representation H obtained in Step 4. fused Input an auxiliary detection head and calculate the binary classification detection confidence probability P for each character in the input sequence that is an error character. det ; to detect confidence probability P det As a soft attention mask, H enhances the multimodal representation. fused Adaptive amplitude scaling is performed, and the scaled features are fused back into the original multimodal augmented representation to obtain the augmented representation H. enhanced ;

[0013] Step 6: Convert the enhanced representation H obtained in Step 5 into a single representation. enhanced Input the final error-correcting classification head for prediction, and output the corrected text.

[0014] Furthermore,

[0015] Step 1: Obtain the input Chinese character sequence to be processed and map it to the initial embedding vector E. token After the initial embedding layer, a learnable input forget gate is introduced to adaptively filter the input noise signal. The core operation of the forget gate is defined as shown in Equation (1).

[0016] (1)

[0017] Among them, W f Let σ represent the learnable weight matrix in the network, and let σ represent the Sigmoid activation function. Represents the element-wise multiplication operation of tensors; the embedding E after forget gate denoising. filtered After adding absolute position encoding, it is fed into the backbone encoder to obtain a clean contextual semantic representation H. text ;

[0018] Furthermore,

[0019] Step 2: Extract fine-grained pinyin features from the input Chinese character sequence. A fine-grained pinyin encoder rigorously decomposes each character in the input sequence into three independent pinyin units: initial consonant, final vowel, and tone. These three pinyin units are then independently mapped into high-dimensional feature vectors v through a pre-defined embedding layer. ini v fin and v tone Then, dimension reduction is performed according to equation (2);

[0020] (2)

[0021] Among them, [v ini ; v fin ; v tone The expression represents the direct concatenation of three feature vectors along a specified dimension, followed by projection onto a weight matrix and activation processing, outputting a compact pinyin representation H rich in nonlinear combinational rules. pinyin ;

[0022] Furthermore,

[0023] Step 3: Extract the glyph structure features of the input Chinese character sequence, input the grayscale image of the Chinese character into the enhanced glyph encoder, extract local strokes and structural features, and compress the spatial dimension through adaptive average pooling; the model introduces a distribution alignment mechanism in the projection layer, as shown in Equation (3);

[0024] (3)

[0025] Among them, h img W represents the initial character glyph feature vector, which is the intermediate image feature representation of the Chinese character image after convolution extraction and flattening, before distribution alignment. p Represents the weight matrix, b p These represent bias vectors, which together with respect to h img Perform a linear projection operation to output glyph features H that are consistent with the semantic distribution of the text. glyph ;

[0026] Furthermore,

[0027] Step 4: Based on equation (4), construct a cascaded gated residual fusion mechanism, inject auxiliary information according to a fixed temporal logic of first pinyin features and then character shape features, for any specified auxiliary modal feature h aux and the semantic state H of the current stage curr This involves mapping auxiliary features to a high-dimensional semantic space, aligning the feature dimensions, and representing them as... ;

[0028] (4)

[0029] Among them, W proj and b proj The learnable parameters of the linear projection layer are represented by the projection matrix and the bias vector, respectively. Then, the semantic features and the aligned auxiliary features are concatenated and fed into the context-aware gated network and the fusion weight coefficient g is calculated according to Equation (5).

[0030] (5)

[0031] Among them, W gate and b gate The learnable parameters of the gated network are used to perform linear transformation on the concatenated joint features. Then, using the residual connection mechanism, the weighted and filtered auxiliary features are superimposed on the semantic stream according to Equation (6).

[0032] (6)

[0033] in, H represents the aligned auxiliary feature. curr Let represent the semantic state at the current stage, and g represent the fusion weight coefficient. According to the cascaded framework, the model first injects the pinyin features into the basic H... text To obtain a preliminary enhanced semantic representation, this preliminary representation is then used as a new input state to continue performing the same gating mechanism to fuse glyph features, and finally output the multimodal enhanced representation H. fused ;

[0034] Furthermore,

[0035] Step 5: Execute the detection-guided representation enhancement strategy to enhance the multimodal representation H. fused Input the auxiliary detection head and calculate the binary classification detection confidence probability of each character in the input sequence as an error character according to Equation (7);

[0036] (7)

[0037] in, and Both represent the learnable parameters of the classification layer in a linear network; with the detection confidence probability P det As a soft attention mask, H enhances the multimodal representation. fused Adaptive amplitude scaling is performed, and the scaled features are fused back into the original multimodal enhanced representation to explicitly highlight latent error features, resulting in the enhanced representation H shown in Equation (8). enhanced ;

[0038] (8)

[0039] Furthermore,

[0040] Step 6: Model joint training phase, the total loss function is defined as the weighted combination of error correction task and detection task as shown in equation (9);

[0041] (9)

[0042] Among them, L corr L represents the error correction auxiliary loss. det The detection auxiliary loss is represented by λ, which is a hyperparameter used to adjust the proportion of the detection task in the overall joint optimization. It controls the extent to which the supervision signal of the detection task assists the backbone network learning. The detection auxiliary loss L is calculated using binary classification focusing loss with class weight parameter α and hard sample focusing parameter γ according to equation (10). det ;

[0043] (10)

[0044] in, It is the model's predicted probability of the true class; the enhanced feature H enhanced The final error-correcting classification head is input for prediction, and the network weights are optimized and updated using a loss function to obtain the corrected text.

[0045] The design concept of this invention is as follows:

[0046] This method uses MacBERT as its semantic backbone and introduces an input forget gate to filter low-level input noise. Secondly, it constructs a fine-grained pinyin encoder decoupled from phonology and tone, and a distribution-aligned enhanced glyph encoder. Then, it designs a cascaded gated residual fusion mechanism, progressively injecting multimodal information in a logical order of pinyin first, then glyph, to avoid feature conflicts. Finally, it proposes a detection-guided representation enhancement strategy, explicitly highlighting potential error regions using detection confidence probabilities, thus establishing a deep collaborative path between the two tasks. On the SIGHAN benchmark dataset, this method achieves an error correction F1 score of 81.0%, an error correction recall of 84.1%, and a detection recall of 85.0%, outperforming existing state-of-the-art models and demonstrating superior error correction effectiveness and robustness.

[0047] Compared with the prior art, the beneficial effects of the present invention using the above-described technology are as follows:

[0048] 1) This invention constructs a fine-grained multimodal coding architecture and an input forget gate mechanism, which realizes adaptive intelligent filtering of multimodal input noise from the underlying network physical structure, significantly improving the purity of the original features and the robustness of the overall model;

[0049] 2) This invention designs a cascaded gated residual fusion mechanism that strictly follows the modal priority logic from shallow to deep. Through a phased and controllable feature injection pipeline, it successfully achieves hierarchical deep integration of heterogeneous modal information, avoiding the feature conflict and attenuation problems caused by traditional coarse-grained splicing.

[0050] 3) This invention proposes a detection-guided representation enhancement strategy, which innovatively utilizes the probability distribution of the detection stage to explicitly highlight potential error features, and combines it with an optimization function that dynamically focuses on difficult samples, thus completely opening up a deep bidirectional collaborative path between the detection and error correction tasks, effectively solving the pain point of overcorrection caused by blind error correction. Attached Figure Description

[0051] Figure 1 This is a model diagram of the method of the present invention;

[0052] Figure 2 This is a model diagram of the Pinyin encoder in the method of the present invention;

[0053] Figure 3 This is a model diagram of the character encoder in the method of the present invention. Detailed Implementation

[0054] The following examples illustrate in detail the specific implementation of the detection-guided Chinese text error correction method based on adaptive multimodal fusion of the present invention.

[0055] A detection-guided Chinese text error correction method based on adaptive multimodal fusion is proposed. Its core model framework includes a semantic encoder, a pinyin encoder, a glyph encoder, a cascaded gated residual fusion mechanism, and a detection-guided representation enhancement mechanism. The specific implementation steps are as follows:

[0056] Step 1: The model uses the pre-trained language model MacBERT as its underlying architecture. The model diagram of this invention is shown below. Figure 1 As shown; the input text sequence is mapped to the initial embedding E. token After that, a learnable input forget gate mechanism is directly configured to perform noise filtering. The core operation of the forget gate is defined as shown in Equation (1).

[0057] (1)

[0058] Among them, W f Let σ represent the learnable weight matrix in the network, and let σ represent the Sigmoid activation function. Represents the element-wise multiplication operation of tensors; the embedding E after forget gate denoising. filtered After adding absolute position encoding, it is fed into the backbone encoder to obtain a clean contextual semantic representation H. text ;

[0059] Step 2: Decouple each input Chinese character into three absolutely independent pinyin units: initial consonant, final vowel, and tone. These three pinyin units are then independently mapped into high-dimensional feature vectors v through a pre-defined embedding layer. ini v fin v tone A non-linear fusion module is used to reduce the dimensionality of these subsyllable features. In the specific implementation, the output dimension of the Pinyin encoder is set to 6.

[0060] (2)

[0061] Among them, [v ini ; v fin ; v tone The expression represents the direct concatenation of three feature vectors along a specified dimension, followed by projection onto a weight matrix and activation processing, outputting a compact pinyin representation H rich in nonlinear combinational rules. pinyin The model diagram of the Pinyin encoder of the present invention is shown below. Figure 2 As shown;

[0062] Step 3: Extract the glyph structure features of the input Chinese character sequence. In the specific implementation, a standardized grayscale image of Chinese characters of 32×32 pixels is used as the initial visual input, and the output dimension of the glyph encoder is set to 56. First, local stroke and structural features are extracted by stacking two layers of convolutional blocks, and the spatial dimension is compressed by adaptive average pooling. In order to solve the problem of inconsistent distribution between visual features and semantic features, the model introduces a distribution alignment mechanism in the projection layer, as shown in Equation (3).

[0063] (3)

[0064] Among them, h img W represents the initial character glyph feature vector, which is the intermediate image feature representation of the Chinese character image after convolution extraction and flattening, before distribution alignment. p Represents the weight matrix, b p These represent bias vectors, which together with respect to h img A linear projection operation is performed; then, the Tanh function is used to strictly constrain the numerical range of the glyph embedding to the interval [-1, 1], thereby aligning the visual features with the underlying semantic features and outputting the glyph feature H.glyph ;

[0065] Step 4: The model strictly follows the order of first pinyin features and then character shape features to inject auxiliary information. For the auxiliary modal features h representing the previously calculated pinyin or character shape features... aux and the semantic state H of the current stage curr The linear projection module maps the auxiliary features to a high-dimensional semantic space, aligning the feature dimensions and representing them as follows: ;

[0066] (4)

[0067] Among them, W proj and b proj The learnable parameters of the linear projection layer are represented by the projection matrix and the bias vector, respectively. Then, the semantic features are concatenated with the aligned auxiliary features and fed into the context-aware gated network and the fusion weight coefficient g is calculated according to Equation (5).

[0068] (5)

[0069] Among them, W gate and b gate The learnable parameters of the gated network are used to perform linear transformation on the concatenated joint features. Finally, the weighted and filtered auxiliary features are superimposed on the semantic stream using the residual connection mechanism according to Equation (6).

[0070] (6)

[0071] in, H represents the aligned auxiliary feature. curr This represents the semantic state at the current stage, and g represents the fusion weight coefficient. The model first injects the pinyin features into the basic H... text To obtain a preliminary enhanced semantic representation, this preliminary representation is then used as a new input state to continue performing the same gating mechanism to fuse glyph features, and finally output the multimodal enhanced representation H. fused ;

[0072] Step 5: Apply the multimodal enhancement characterization H obtained above fused The complete input is fed into an auxiliary detection network, which is a two-layer MLP that directly outputs a scalar detection confidence probability P for each input character as an error character. det As shown in equation (7);

[0073] (7)

[0074] in, and Both represent learnable parameters of the linear network classification layer; after obtaining the probability values, based on the probability vector P det Residual enhancement is performed on the multimodal features to highlight the error regions on the feature map, as shown in Equation (8);

[0075] (8)

[0076] In practice, when the probability of the model determining a character as incorrect approaches 1, the amplitude of the feature corresponding to that position is explicitly amplified; when the determination is correct, P... det When the value approaches 0, the feature retains its initial value, thereby reducing the risk of miscorrection of correct characters from the root cause.

[0077] Step 6: During the end-to-end training phase, the total loss function is defined as a weighted combination of the error correction task and the detection task, as shown in equation (9).

[0078] (9)

[0079] Among them, L corr L represents the error correction auxiliary loss. det Let λ represent the detection auxiliary loss. λ is a hyperparameter used to adjust the proportion of the detection task in the overall joint optimization. It controls the extent to which the supervision signal of the detection task assists the backbone network learning; the detection auxiliary loss L... det According to equation (10), a binary classification focusing loss with class weight parameter α and hard sample focusing parameter γ must be used:

[0080] (10)

[0081] in, This represents the model's predicted probability of the true class. In practice, γ=2.0 is set to reduce the weight of simple samples, and α=0.75 to adjust the weight imbalance between positive and negative samples. The enhanced features are input into the final error-correcting classification head for prediction, and this loss function is used to optimize and update the network weights. In the actual inference stage, the model directly searches for the highest confidence level in the full vocabulary based on the error-correcting classification head, thereby outputting the corrected target plain text Chinese character sequence.

[0082] In this embodiment, the AdamW optimizer is used with a learning rate of 5 × 10⁻⁵ and 20 training epochs. Table 1 shows the performance of this Chinese text correction method and the baseline model on the SIGHAN13, 14, and 15 test sets. The results show that the method used in the model outperforms existing state-of-the-art models on most key metrics, demonstrating the effectiveness of the proposed multimodal fusion mechanism and detection guidance strategy.

[0083] As shown in Table 1, this method achieved the best performance in sentence-level error correction F1 scores across all three datasets: on the most challenging SIGHAN15 dataset, it achieved an F1 score of 81.0%, surpassing the previous state-of-the-art models SCOPE and MDCSpell. This demonstrates that introducing fine-grained phonetic and morphological features through a cascaded gated residual fusion mechanism can significantly improve the model's semantic recovery capability in complex scenarios.

[0084] Notably, this method demonstrates particularly strong performance in recall. On SIGHAN15, the error correction recall reached 84.1%, significantly higher than SCOPE's 82.3% and ReaLiSe's 79.9%. This result strongly validates the effectiveness of the "detection-guided representation enhancement strategy" proposed in this paper—by explicitly "highlighting" potential error features, it successfully captures hidden errors ignored by traditional models, thereby significantly reducing the missed error rate.

[0085] Error detection is a prerequisite for error correction, and its accuracy directly affects the upper limit of error correction. As shown in Table 1, our method also maintains a very high recall rate in the detection task. On SIGHAN15, the detection recall rate reaches 85.2%, which is better than SCOPE and MDCSpell. This indicates that the introduction of the input forget gate and fine-grained pinyin features makes the model more sensitive to distinguishing characters with similar shapes and sounds. Although our method's error detection F1 is slightly lower than SCOPE, the input forget gate mechanism achieves a high recall rate of 85.2% through an aggressive strategy. This is due to the joint optimization of the detection-guided representation enhancement strategy and the error correction network, which effectively intercepts false positives in the initial detection during the decoding stage. This high recall ultimately translates into an error correction performance gain, enabling our method to surpass the baseline on sentence-level error correction F1.

[0086] Table 1. Performance of the method in error detection and correction.

[0087]

[0088] The examples described in this specification are merely enumerations of implementation forms of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A detection-guided Chinese text error correction method based on adaptive multimodal fusion, characterized in that, Includes the following steps: Step 1: Obtain the input Chinese character sequence to be processed and map it to the initial embedding vector E. token After filtering the input noise signal following the initial embedding layer, the denoised embedded E filtered After adding absolute position encoding, it is fed into the backbone encoder to obtain a clean contextual semantic representation H. text ; Step 2: Extract fine-grained pinyin features from the input Chinese character sequence. Decompose each Chinese character in the input sequence into three independent pinyin units: initial consonant, final vowel, and tone. Then, after dimensionality reduction, weight matrix projection, and activation processing, output the pinyin representation H. pinyin ; Step 3: Extract the glyph structure features of the input Chinese character sequence. Input the grayscale image of the Chinese character to extract local strokes and structural features. After dimensionality reduction and linear projection processing, output the glyph features H. glyph ; Step 4: Construct a cascaded gated residual fusion mechanism. Following a fixed temporal logic of injecting auxiliary information first (phonetic features then glyph features), the auxiliary information is fed into a context-aware gating network to calculate the fusion weight coefficient g. First, the phonetic features obtained in Step 2 are injected into the H obtained in Step 1. text To obtain a preliminary enhanced semantic representation, this preliminary representation is then used as a new input state to continue performing the same gating mechanism to fuse the glyph features obtained in step 3, and finally output the multimodal enhanced representation H. fused ; Step 5: Execute the detection-guided representation enhancement strategy to enhance the multimodal representation H obtained in Step 4. fused Input an auxiliary detection head and calculate the binary classification detection confidence probability P for each character in the input sequence that is an error character. det ; to detect confidence probability P det As a soft attention mask, H enhances the multimodal representation. fused Adaptive amplitude scaling is performed, and the scaled features are fused back into the original multimodal augmented representation to obtain the augmented representation H. enhanced ; Step 6: Convert the enhanced representation H obtained in Step 5 into a single representation. enhanced Input the final error-correcting classification head for prediction, and output the corrected text.

2. The detection-guided Chinese text error correction method based on adaptive multimodal fusion according to claim 1, characterized in that, In step 1, a learnable input forget gate is introduced after the initial embedding layer to adaptively filter the input noise signal. The operation definition of the forget gate is shown in equation (1). (1)； Among them, W f Let σ represent the learnable weight matrix in the network, and let σ represent the Sigmoid activation function. This represents the element-wise multiplication operation of a tensor.

3. The detection-guided Chinese text error correction method based on adaptive multimodal fusion according to claim 1, characterized in that, In step 2, the three independent pinyin units are each independently mapped into high-dimensional feature vectors v through a preset embedding layer. ini v fin and v tone Then, dimension reduction is performed according to equation (2); (2)； Among them, [v ini ; v fin ; v tone This indicates that the three feature vectors are directly concatenated along a specified dimension, and then subjected to projection and activation processing using a weight matrix.

4. The detection-guided Chinese text error correction method based on adaptive multimodal fusion according to claim 1, characterized in that, In step 3, the grayscale image of the Chinese character is input to the encoder to extract local strokes and structural features, and the spatial dimension is compressed by adaptive average pooling; then a distribution alignment mechanism is introduced in the projection layer, as shown in equation (3); (3)； Among them, h img W represents the initial character glyph feature vector, which is the intermediate image feature representation of the Chinese character image after convolution extraction and flattening, before distribution alignment. p Represents the weight matrix, b p These represent bias vectors, which together with respect to h img Perform a linear projection operation to output glyph features H that are consistent with the semantic distribution of the text. glyph .

5. The detection-guided Chinese text error correction method based on adaptive multimodal fusion according to claim 1, characterized in that, The specific steps for constructing the cascaded gated residual fusion mechanism in step 4 are as follows: For any given auxiliary modal feature h aux and the semantic state H of the current stage curr This involves mapping auxiliary features to a high-dimensional semantic space, aligning the feature dimensions, and representing them as... ; (4)； Among them, W proj and b proj The learnable parameters of the linear projection layer are represented by the projection matrix and the bias vector, respectively. Then, the semantic features and the aligned auxiliary features are concatenated and fed into the context-aware gated network and the fusion weight coefficient g is calculated according to Equation (5). (5)； Among them, W gate and b gate Represents the learnable parameters of the gated network; Next, using the residual connection mechanism, the weighted and filtered auxiliary features are superimposed onto the semantic stream according to Equation (6); (6)； in, H represents the aligned auxiliary feature. curr represents the semantic state of the current stage, and g represents the fusion weight coefficient.

6. The detection-guided Chinese text error correction method based on adaptive multimodal fusion according to claim 1, characterized in that, The specific steps of step 5 are as follows: Multimodal enhancement characterization H fused Input the auxiliary detection head, and calculate the binary classification detection confidence probability P of each character in the input sequence as an error character according to equation (7). det ; (7)； in, and Both represent the learnable parameters of the classification layer in a linear network; with the detection confidence probability P det As a soft attention mask, H enhances the multimodal representation. fused Adaptive amplitude scaling is performed, and the scaled features are fused back into the original multimodal enhanced representation to explicitly highlight latent error features, resulting in the enhanced representation H shown in Equation (8). enhanced ; (8)。 7. The detection-guided Chinese text error correction method based on adaptive multimodal fusion according to claim 1, characterized in that, In step 6, during the joint training phase, the total loss function is obtained by weighting the error correction task and the detection task. Backpropagation of the loss is then used to optimize and update the weights of the entire network, ultimately obtaining the corrected text.

Images