Optical character recognition method and device, storage medium and electronic equipment

Abstract

The embodiment of the invention discloses an optical character recognition method and device, a storage medium and electronic equipment, and the method comprises the steps: inputting a to-be-recognized character image into an optical character recognition model, and carrying out the character feature uncertainty estimation of the to-be-recognized character image through the optical character recognition model, generating a character feature mean value graph and a character feature variance graph, and performing character track initial recognition based on the character feature mean value graph and the character feature variance graph to obtain an initial recognition character track sequence and a decoded character track metadata packet sequence for the initial recognition character track sequence; on the basis of the decoded character trajectory metadata packet sequence, performing trajectory verification processing on the initial recognition character trajectory sequence to obtain a trajectory recognition error metadata mode and a character trajectory editing operation for the trajectory recognition error metadata mode; and track correction is performed on the preliminary recognition character track sequence based on the character track editing operation to generate a corrected text sequence, and the corrected text sequence for the to-be-recognized character image is output.

Description

Technical Field

[0001] This specification relates to the fields of computer vision and artificial intelligence technology, and in particular to an optical character recognition method, device, storage medium and electronic device. Background Technology

[0002] With the widespread use of mobile smart devices, the demand for photographing documents and invoices and recognizing text using portable devices such as smartphones is increasing. However, under less-than-ideal low-light conditions, such as at night or in shadowy indoor areas, the captured images often suffer from a series of degradation problems, including low signal-to-noise ratio, uneven brightness, decreased contrast, color distortion, and motion blur. These factors severely interfere with the performance of optical character recognition (OCR) systems, leading to a sharp decline in recognition accuracy. Summary of the Invention

[0003] This specification provides an optical character recognition method, apparatus, storage medium, and electronic device, the technical solutions of which are as follows: In a first aspect, embodiments of this specification provide an optical character recognition method, the method comprising: Obtain an image of the character to be recognized and input the image of the character to be recognized into an optical character recognition model; The optical character recognition model is used to estimate the uncertainty of character features in the image of the character to be recognized, and a character feature mean map and a character feature variance map are generated. The character feature variance map is used to characterize the degree of uncertainty of the feature estimation of the character feature mean map. The optical character recognition model performs initial character trajectory recognition based on the character feature mean map and character feature variance map to obtain an initial recognized character trajectory sequence and a decoded character trajectory metadata data sequence for the initial recognized character trajectory sequence; The optical character recognition model performs trajectory verification processing on the initially recognized character trajectory sequence based on the decoded character trajectory metadata data sequence to obtain a trajectory recognition error metadata pattern and a character trajectory editing operation for the trajectory recognition error metadata pattern. Based on the character trajectory editing operation, the initially recognized character trajectory sequence is corrected to generate a correction text sequence, and the correction text sequence for the character image to be recognized is output.

[0004] Secondly, embodiments of this specification provide an optical character recognition device, the device comprising: The image acquisition module is used to acquire an image of the character to be recognized and input the image of the character to be recognized into the optical character recognition model; The character recognition module is used to estimate the uncertainty of character features in the image of the character to be recognized through the optical character recognition model, and generate a character feature mean map and a character feature variance map. The character feature variance map is used to characterize the degree of uncertainty of the feature estimation of the character feature mean map. The character recognition module is used to perform initial character trajectory recognition based on the character feature mean map and character feature variance map using the optical character recognition model to obtain an initial recognized character trajectory sequence and a decoded character trajectory metadata data sequence for the initial recognized character trajectory sequence; The character recognition module is used to perform trajectory verification processing on the preliminary recognized character trajectory sequence based on the decoded character trajectory metadata data sequence using the optical character recognition model to obtain a trajectory recognition error metadata pattern and a character trajectory editing operation for the trajectory recognition error metadata pattern. Based on the character trajectory editing operation, the module performs trajectory correction on the preliminary recognized character trajectory sequence to generate a correction text sequence and outputs a correction text sequence for the character image to be recognized.

[0005] Thirdly, embodiments of this specification provide a computer storage medium storing a plurality of instructions adapted for loading by a processor and executing the above-described method steps.

[0006] Fourthly, embodiments of this specification provide an electronic device that may include: a processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the above-described method steps.

[0007] The beneficial effects of the technical solutions provided in some embodiments of this specification include at least the following: In one or more embodiments of this specification, the electronic device inputs an image of a character to be recognized into an optical character recognition model. The optical character recognition model performs uncertainty estimation of the character features of the image to be recognized, generating a mean map and a variance map of the character features. This fundamentally avoids the information bottleneck problem caused by generating deterministic intermediate images in traditional image enhancement methods, and completely quantifies and transmits the uncertainty information of the image. Furthermore, the initial character trajectory recognition based on the mean map and variance map of the character features can avoid unreliable visual regions, obtaining a preliminary character trajectory sequence and a decoded character trajectory metadata data sequence for the preliminary character trajectory sequence. This significantly improves the robustness of the initial recognition even in the face of severe noise and missing information. Furthermore, based on the decoded character trajectory metadata data sequence, trajectory verification processing is performed on the preliminary recognized character trajectory sequence to obtain trajectory recognition error metadata patterns and character trajectory editing operations for trajectory recognition error metadata patterns. Based on the character trajectory editing operations, trajectory correction is performed on the preliminary recognized character trajectory sequence to generate and output a corrected text sequence. By analyzing metadata containing rich information such as visual uncertainty and attention behavior, recognition errors caused by specific visual degradation can be accurately located and repaired. The entire optical character recognition process achieves high-precision text recognition by probabilistically modeling the image degradation process and using the generated uncertainty metadata to guide the subsequent recognition and correction process. Attached Figure Description

[0008] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this specification. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0009] Figure 1 This is a flowchart illustrating an optical character recognition method provided in an embodiment of this specification; Figure 2 This is a flowchart illustrating the internal processing of an optical character recognition model provided in the embodiments of this specification. Figure 3 This is a flowchart illustrating a character feature uncertainty estimation method provided in the embodiments of this specification; Figure 4 This is a schematic diagram of the internal flow of a feature uncertainty estimation module provided in the embodiments of this specification; Figure 5 This is a schematic diagram of a character trajectory initial recognition process provided in the embodiments of this specification; Figure 6This is a flowchart illustrating the calculation of an uncertainty-weighted attention mechanism for an uncertainty perception and recognition module, as provided in the embodiments of this specification. Figure 7 This is a schematic diagram of the structure of an optical character recognition device provided in the embodiments of this specification; Figure 8 This is a schematic diagram of the structure of an electronic device provided in the embodiments of this specification. Detailed Implementation

[0010] The technical solutions in the embodiments of this specification will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this specification, and not all embodiments. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0011] In the description of this specification, it should be understood that the terms "first," "second," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance. In the description of this specification, it should be noted that, unless otherwise expressly specified and limited, "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units is not limited to the listed steps or units, but may optionally include steps or units not listed, or may optionally include other steps or units inherent to these processes, methods, products, or devices. Those skilled in the art can understand the specific meaning of the above terms in this specification based on the specific circumstances. Furthermore, in the description of this specification, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, and B alone. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship.

[0012] In related technologies, the problem of interference with the performance of traditional optical character recognition (OCR) systems, leading to a sharp decline in recognition accuracy, can be addressed through two main strategies: 1. Two-stage serial approach: This method first uses a separate image enhancement module (such as denoising, brightness correction, super-resolution, etc.) to preprocess the low-light image, generating an image with better visual quality, and then inputs it into a standard OCR engine for recognition. The main drawbacks of this paradigm are: a) Error accumulation: The image enhancement module may introduce new artifacts or lose key text details, and these errors will be propagated and may be amplified by subsequent OCR modules; b) Inconsistent optimization objectives: There is a difference between the optimization objectives of the image enhancement module (such as peak signal-to-noise ratio PSNR or structural similarity SSIM) and the optimization objective of the OCR module (character recognition accuracy). Optimizing visual quality does not always lead to the best recognition results; c) Information bottleneck: The enhancement process is a lossy compression, and valuable information about the "uncertainty" in the original image (e.g., which areas are unrecoverable due to extreme darkness) is completely discarded when generating a deterministic output image.

[0013] 2. End-to-End Robustness Enhancement Methods: These methods attempt to directly improve the robustness of OCR models to low-light inputs by designing more powerful deep neural network architectures or training them on large datasets containing degraded images. While these methods avoid some of the problems of serial methods to a certain extent, their performance still encounters bottlenecks when faced with extremely degraded images or images with a large amount of missing information. Essentially, these models are still trying to solve a deterministic text output from an ill-posed problem. When the input information is insufficient to support a unique solution, the model is prone to producing "illusions" or errors based on prior data.

[0014] Furthermore, some methods cascade a separate language model after OCR for post-processing error correction. However, these plain text-based error correction models lack access to the original visual information, making it difficult to correct errors that are caused by specific visual confusions (such as 'c' and 'e', ​​or 'l' and 'i' having similar shapes when blurred), but which may be equally reasonable in language.

[0015] In summary, existing technologies have failed to fundamentally address the information uncertainty problem in low-light OCR, lacking a mechanism to quantify, transmit, and effectively utilize this uncertainty throughout the recognition process, thus limiting recognition accuracy. This specification aims to address the problem of low text recognition accuracy caused by the inability to effectively process and utilize the uncertainty introduced by image degradation when handling low-light photographed and scanned images. Specifically, this specification strives to overcome the error accumulation and information bottleneck defects of traditional serial methods, as well as the performance bottleneck of end-to-end models when information is severely lacking.

[0016] The present specification will now be described in detail with reference to specific embodiments.

[0017] In one embodiment, such as Figure 1 As shown, an optical character recognition method is proposed, which is an end-to-end deep learning text recognition method designed for images captured under low light and complex imaging conditions. This optical character recognition method estimates the uncertainty of character features by probabilistically modeling the image degradation process, and uses the generated uncertainty metadata to guide the subsequent recognition and correction process, thereby achieving high-precision text recognition.

[0018] This method can be implemented using a computer program and can run on optical character recognition devices based on the von Neumann architecture. The computer program can be integrated into applications or run as a standalone utility application. The optical character recognition device can be an electronic device, including but not limited to: personal computers, tablets, handheld devices, vehicle-mounted devices, server devices, computing devices, or other processing devices connected to a wireless modem. In different networks, terminal devices can be called by different names, such as: user equipment, access terminal, user unit, user station, mobile station, mobile station, remote station, remote terminal, mobile device, user terminal, terminal, wireless communication device, user agent or user equipment, cellular phone, cordless phone, device in 5G networks or future evolved networks, etc.

[0019] Specifically, the optical character recognition method includes: S102: Obtain the image of the character to be recognized, and input the image of the character to be recognized into the optical character recognition model; The image of the character to be recognized can come from various sources depending on the optical character recognition (OCR) scenario, including but not limited to digital photographs captured by digital cameras or mobile phone cameras, digital image files generated by scanning with a scanner, or existing image files (e.g., PNG, JPEG, BMP formats) loaded directly from storage devices. In the context of this invention, the image of the character to be recognized can be, in particular, an image acquired under non-ideal imaging conditions such as low light, uneven lighting, the presence of shadows, or blurriness.

[0020] Optical character recognition model: This is an end-to-end deep neural network model used in this specification. In some embodiments of this specification, the optical character recognition model structurally includes a feature uncertainty estimation module, an uncertainty perception and recognition module, and a recognition trajectory correction module, which can collaboratively complete the entire process from image uncertainty estimation to final text correction.

[0021] For example, a field service worker needs to use their work phone to photograph and digitize the serial number on a device's nameplate in a dimly lit underground parking garage. The worker opens a dedicated application on their phone and takes a picture of the nameplate. Due to insufficient ambient light, the resulting photo (the image of the character to be recognized) is generally low in brightness and may have motion blur due to slight hand tremors. After the worker confirms the use of the photo on the application interface, the application uses this photo file as the image of the character to be recognized and as input parameters to invoke the optical character recognition model integrated within the program. The data stream is initiated internally, and the pixel matrix is ​​transmitted to the model's feature uncertainty estimation module, thus starting subsequent steps such as S104 to identify the serial number on the nameplate.

[0022] S104: The optical character recognition model is used to estimate the uncertainty of character features in the image of the character to be recognized, and a character feature mean map and a character feature variance map are generated. The character feature variance map is used to characterize the uncertainty of the feature estimation of the character feature mean map. Character Feature Mean Map: This refers to a tensor generated by the optical character recognition model during the estimation of character feature uncertainty. It is spatially related to the input image, and each element represents the optimal point-by-point estimate of the features of a clear image under ideal lighting conditions. The character feature mean map can be viewed as the model's "best guess" for clear feature content.

[0023] Character Feature Variance Map: This refers to a tensor with the same dimension as the character feature mean map, generated during the uncertainty estimation of character features by the optical character recognition model. Each element quantifies the uncertainty or confidence level of the feature estimation at the corresponding position in the mean map. A higher variance value indicates a lower confidence level in the model's feature estimation at that location; conversely, a lower variance value indicates a higher confidence level.

[0024] Feature estimation uncertainty refers to a quantifiable measure of information loss in the image of the character to be identified due to signal degradation (e.g., noise, blurring, insufficient brightness). In this embodiment of the invention, this uncertainty is specifically numerically represented by the character feature variance map.

[0025] In some embodiments, the optical character recognition model includes a feature uncertainty estimation module, an uncertainty perception recognition module, and a recognition trajectory correction module; such as Figure 2 As shown, Figure 2This is a flowchart of the internal processing of an optical character recognition model; the "low-light input image" corresponds to the operation of "acquiring the image of the character to be recognized" in step S102, representing the original data sent to the optical character recognition model. The "feature uncertainty estimation module" executes step S104, which receives the "low-light input image" and performs "character feature uncertainty estimation". Its processing result is divided into two parallel output paths: "feature mean map" and "feature variance map", that is, "generating character feature mean map and character feature variance map" as described in step S104; then the "uncertainty perception recognition module" takes the "feature mean map" and "feature variance map" generated in the previous step as input and performs "initial character trajectory recognition". This module also produces two parallel outputs: preliminary recognition of character trajectory sequence ( Figure 2 The system takes the "preliminary recognition sequence" and the "decoded character trajectory metadata data sequence" generated in the previous step as input, and performs "trajectory verification processing" and "trajectory correction" to obtain the final corrected text sequence.

[0026] In one feasible implementation, the character feature uncertainty estimation in S104 is not a traditional deterministic image enhancement, but rather a probabilistic modeling of the image features of the character to be recognized. Uncertainty estimation treats the ideal, sharp features as an unobservable latent variable and, based on the conditional variational autoencoder architecture of the optical character recognition model, infers the posterior probability distribution of this latent variable under a given input image. This posterior probability distribution is further parameterized as a Gaussian distribution, and then a character feature mean map and a character feature variance map are generated based on this Gaussian distribution.

[0027] Furthermore, this is specifically executed through the feature uncertainty estimation module integrated within the optical character recognition model. Based on the inference network (which can be regarded as an encoder based on a conditional variational autoencoder architecture) in the feature uncertainty estimation module, the image of the character to be recognized from step S102 is taken as input. The inference network performs a forward propagation calculation on the input image. Its network structure directly outputs two independent tensors through pre-trained model. These two tensors are the parameters of the Gaussian distribution mentioned above: the character feature mean map and the character feature variance map. The generated character feature mean map and character feature variance map serve as a probabilistic feature representation. S106: The optical character recognition model performs initial character trajectory recognition based on the character feature mean map and character feature variance map to obtain an initial recognized character trajectory sequence and a decoded character trajectory metadata data sequence for the initial recognized character trajectory sequence; The initial character trajectory sequence refers to the initial text output generated during the initial character trajectory recognition process. This initial character trajectory sequence represents the model's preliminary judgment of the input image content and may contain recognition errors caused by factors such as image degradation in some cases.

[0028] The decoded character trajectory metadata data sequence refers to a structured set of metadata corresponding to the preliminary identified character trajectory sequence. Each metadata data packet in the sequence corresponds one-to-one with a character in the preliminary identified sequence and records detailed "trajectory" information of the character's generation process. In some embodiments, this information includes at least: the output probability of the character, the attention weight vector used during decoding, and the mean and variance of the attention-weighted average features. This sequence provides detailed procedural evidence to assist in subsequent trajectory verification processing.

[0029] In one feasible implementation, an uncertainty-aware recognition module of the optical character recognition model performs initial character trajectory recognition based on the character feature mean map and character feature variance map. The uncertainty-aware recognition module uses the character feature mean map and character feature variance map output by the aforementioned feature uncertainty estimation module as input. It employs an improved uncertainty-aware attention mechanism for sequence decoding. When calculating attention weights, this mechanism uses feature variance as a penalty term, thereby adaptively reducing attention to high-uncertainty visual regions. Simultaneously, when generating the context vector for decoding, the module performs differentiable random sampling from the feature distribution defined by the mean and variance. The module ultimately outputs a preliminarily recognized text sequence and a decoded trajectory metadata package generated for each character in the sequence, which includes, for example, information such as the probability of the character, the corresponding attention distribution, and the feature mean and variance statistics of the dependent visual region.

[0030] Specifically, the uncertainty-aware recognition module can employ a Transformer-based encoder-decoder architecture for autoregressive sequence recognition. Its core lies in utilizing uncertainty information. On one hand, based on an uncertainty-weighted attention mechanism, at each time step of decoding, when the decoder calculates attention energy to determine which region of the feature map to focus on, it introduces an uncertainty penalty term guided by the character feature variance map. This penalty term significantly reduces the model's attention to regions with high feature variance values ​​(i.e., high uncertainty regions), thereby guiding the model to rely on more reliable visual evidence. On the other hand, based on a sampling-based context vector generation mechanism, when calculating the context vector used to generate the current character, to ensure the decoder's direct robustness to feature uncertainty, this module does not directly use the mean feature as the "value" in the attention mechanism. Instead, it randomly samples from the feature distribution defined by the mean and variance maps, using the sampled features to calculate the context vector.

[0031] In each step of autoregressive decoding, while predicting the current character, the module also constructs a decoded character metadata data packet containing its complete process information. After the entire sequence is decoded, this step finally generates and outputs two parallel sequence data: the preliminary character trajectory sequence and the decoded character trajectory metadata data packet sequence, which are then passed to the next module of the model for execution of step S108.

[0032] S108: The optical character recognition model performs trajectory verification processing on the preliminary recognized character trajectory sequence based on the decoded character trajectory metadata data sequence to obtain a trajectory recognition error metadata pattern and a character trajectory editing operation for the trajectory recognition error metadata pattern. Based on the character trajectory editing operation, the preliminary recognized character trajectory sequence is corrected to generate a correction text sequence, and the correction text sequence for the character image to be recognized is output.

[0033] Trajectory verification processing: This refers to the process of performing posterior analysis and diagnosis. Unlike traditional proofreading that relies on single text information, this process uses an auditing network to perform in-depth analysis on the decoded character trajectory metadata data sequence containing multimodal information (such as recognition probability, attention distribution, visual uncertainty, etc.) in order to find potential evidence of recognition errors.

[0034] Trajectory recognition error metadata pattern: refers to a statistically significant, recurring temporal pattern in the decoded character trajectory metadata data packet sequence that can predict a specific type of recognition error. This pattern is multi-temporal and multimodal; for example, it could be a combination of "attention maps of two consecutive characters that highly overlap spatially, and both of them have high average visual uncertainty," which might correspond to a character merging error caused by visual adhesion.

[0035] Character trajectory editing operation: refers to a correction action predicted for each character position in the initial identification sequence. In some embodiments, the set of character trajectory editing operations may include: keeping the current character, deleting the current character, and replacing the current character with any other character in the vocabulary.

[0036] In some embodiments, a recognition trajectory correction module integrated through an optical character recognition model receives the decoded character trajectory metadata data sequence output by the aforementioned uncertainty-aware recognition module. A sequence analysis network (such as a Transformer encoder) processes this metadata sequence, learning to identify statistically significant temporal patterns that indicate specific types of recognition errors. Based on the analysis of the entire decoded trajectory, this module predicts an editing operation (such as keep, delete, or replace) for each character position in the initial recognition sequence. This correction decision process is based on the result of a joint audit of cross-modal (visual uncertainty, attentional behavior, sequence probability) information throughout the recognition process, rather than simple language model constraints.

[0037] The trajectory correction module can use a multimodal input encoding layer to convert each structured decoded character metadata data packet into a fixed-dimensional vector, thereby transforming the sequence of decoded character trajectory metadata data packets into an information-dense vector sequence H. DTP Vector sequence H DTP In the trajectory correction module, a sequence analysis network (auditing network) implemented as a Transformer encoder is used. This auditing network uses its multi-head self-attention mechanism to capture long-range dependencies in the sequence, verify the entire decoded trajectory, identify trajectory recognition error metadata patterns, and determine the context-aware diagnostic vector sequence H. audit For each character position t in the sequence, its corresponding diagnostic vector H audit,t Processed by a linear classifier and a Softmax layer, the classifier outputs a probability distribution across all possible character trajectory editing operations. For each character position in the initial identification sequence, especially misidentified character positions, the editing operation with the highest probability is selected as the final decision. By sequentially executing these editing operations, the initially identified sequence is corrected, generating the final corrected text sequence. In the embodiments of this specification, the image of the character to be recognized is input into the optical character recognition model. The optical character recognition model performs uncertainty estimation of the character features of the image to be recognized, generating a mean map and a variance map of the character features. This fundamentally avoids the information bottleneck problem caused by generating deterministic intermediate images in traditional image enhancement methods, and completely quantifies and transmits the uncertainty information of the image. Furthermore, the initial recognition of character trajectories based on the mean map and variance map of character features can avoid unreliable visual regions, and obtain a preliminary recognized character trajectory sequence and a decoded character trajectory metadata data sequence for the preliminary recognized character trajectory sequence. This significantly improves the robustness of the initial recognition in the face of severe noise and missing information. Furthermore, based on the decoded character trajectory metadata data sequence, trajectory verification processing is performed on the preliminary recognized character trajectory sequence to obtain trajectory recognition error metadata patterns and character trajectory editing operations for trajectory recognition error metadata patterns. Based on the character trajectory editing operations, trajectory correction is performed on the preliminary recognized character trajectory sequence to generate and output a corrected text sequence. By analyzing metadata containing rich information such as visual uncertainty and attention behavior, recognition errors caused by specific visual degradation can be accurately located and repaired. The entire optical character recognition process achieves high-precision text recognition by probabilistically modeling the image degradation process and using the generated uncertainty metadata to guide the subsequent recognition and correction process.

[0038] Optional, please see Figure 3 , Figure 3 This is a flowchart illustrating a character feature uncertainty estimation method proposed in this specification. The optical character recognition model includes a feature uncertainty estimation module, which specifically performs character feature uncertainty estimation on the image of the character to be recognized using the optical character recognition model, generating a character feature mean map and a character feature variance map, including: S202: The inference network of the feature uncertainty estimation module uses the ideal character image clarity features of the character image to be identified as latent variables; The inference network can be a trained deep convolutional neural network, also known as an encoder. It receives an image of a character to be recognized and approximates the posterior probability distribution of the ideal sharp features corresponding to that image.

[0039] Ideal character image sharpness features refer to the theoretically perfect representation of the character image to be recognized under ideal imaging conditions (e.g., sufficient lighting, no noise, no blur). Since actual acquired character images often suffer from various quality degradations, this ideal sharpness feature is physically unknown and cannot be directly obtained.

[0040] Latent variables refer to features of the image of the character to be recognized that are considered as hidden variables that need to be estimated through probabilistic inference. Due to degradation in the input image, sharp features are considered as hidden variables that need to be estimated through probabilistic inference.

[0041] To illustrate, when the image of the character to be recognized is input into the feature uncertainty estimation module, its internal inference network is activated first. Instead of directly attempting to convert the degraded input features into a deterministic "enhanced" feature, it sets a latent variable, formally defining the "ideal sharp feature of the character image" as a latent variable. The fundamental reason is that the mapping from a low-quality observed image to a high-quality sharp feature is "one-to-many," inherently uncertain. For example, a blurry pixel may have several plausible true sharp values.

[0042] By setting clear features as latent variables, a clear probabilistic inference objective is established for subsequent processing: instead of solving for a single deterministic solution, the inference network transforms into solving the posterior probability distribution of the latent variable under given observed image conditions. This enables the inference network to systematically handle and quantify uncertainty.

[0043] S204: Through the inference network, the posterior probability distribution of the latent variable under the character image to be identified is inferred to obtain the image posterior probability distribution parameters. Based on the image posterior probability distribution parameters, a character feature mean map representing the clarity features of the character image and a character feature variance map quantifying the uncertainty of the clarity features of the character image are generated.

[0044] Posterior probability distribution: refers to the conditional probability distribution of the latent variable (ideal clear features) after observing a specific image of the character to be identified. This distribution is an updated description of the probabilities of all possible values ​​of the latent variable after combining prior knowledge and observational data evidence.

[0045] Image posterior probability distribution parameters: These refer to a set of parameters that completely define the approximate posterior probability distribution. In this embodiment, since the approximate distribution is parameterized as a Gaussian distribution, its parameters are specifically the mean and variance of the Gaussian distribution. These two parameters are carried in the data structure by two tensors (i.e., the feature mean map and the feature variance map).

[0046] Indicative, inferring the network's reception of the image of the character to be recognized. lowThe inference network performs inference processing. Specifically, through its deep convolutional neural network structure, the inference network performs a nonlinear transformation on the input image, aiming to calculate the best Gaussian approximation of the true posterior probability distribution of the latent variables. After forward propagation calculation through the inference network, it directly generates two independent tensors, which are the posterior probability distribution parameters of the image.

[0047] Mean parameter: The tensor of the first output is defined as the mean parameter of the approximate Gaussian distribution, i.e., the character feature mean map.

[0048] Variance parameter: The tensor of the second output is defined as the log-variance parameter of the approximate Gaussian distribution, i.e., the characteristic log-variance plot.

[0049] Furthermore, based on the distribution parameters generated in the previous step, two feature maps are finally constructed and generated as outputs: the character feature mean map is directly composed of the mean parameters and is used to characterize the optimal estimate of the ideal clear features.

[0050] The character feature variance map is obtained by performing element-wise exponential operations on the feature log variance map, and is used to quantify the uncertainty of the optimal estimate.

[0051] In one specific implementation, the function of the feature uncertainty estimation module is to process the input low-light image. The processing aims to infer the sharp feature map corresponding to ideal illumination. posterior probability distribution The feature uncertainty estimation module adopts a conditional variational autoencoder (CVAE) structure, such as... Figure 4 As shown, Figure 4 This is a schematic diagram of the internal flow of a feature uncertainty estimation module, which includes an inference network. (Also known as encoder) and generator network (Also known as a decoder): Inference Networks By a deep convolutional neural network Implementation, for example, using a U-Net encoder portion with residual connections or a ResNet-50 as the backbone network. This network receives images such as low-light images awaiting character recognition. As input, the output consists of two entities with the same spatial dimension. and channel dimensions Tensor: Character Feature Mean Plot Character feature variance plot .

[0052] The inference network processing can be represented as:

[0053] Character Feature Mean Plot Character feature variance plot A Gaussian distribution with diagonal covariance was jointly parameterized, which served as a parameter for the posterior probability distribution of the image. Approximation: It can be represented as: ; This represents the inference processing steps of the inference network, which uses a variational distribution to approximate the posterior probability distribution parameters of the image. The variational distribution is parameterized as a Gaussian distribution with diagonal covariance. ; In this representation, the character feature mean map It is a point-by-point maximum a posteriori probability estimate of the ideal feature map, which intuitively represents the best guess of the image content.

[0054] and This measures the variance of the estimate at each feature point and each channel, precisely quantifying the uncertainty caused by input image quality degradation (such as noise and dark areas). For example, a completely dark, informationless region in the image corresponds to... The value will be significantly higher in clear areas, and conversely, it will be lower in clear areas.

[0055] In the embodiments described in this specification, the above-described method, through an optimized inference network, infers the posterior probability distribution of ideal, clear features as latent variables, thus transforming the traditional deterministic image processing paradigm into a probabilistic inference framework. This shift prevents the model from being forced to output a single, potentially erroneous, enhanced image. Instead, it generates a probabilistic feature representation containing an "optimal estimate" (feature mean map) and a "confidence assessment" (feature variance map). This fundamentally solves the "information bottleneck" problem in existing technologies because it accurately quantifies and fully preserves the key uncertainties introduced by image degradation, providing unprecedentedly rich input for subsequent recognition and correction modules to make more robust and well-founded decisions.

[0056] Furthermore, in some embodiments, such as Figure 4 As shown, the feature uncertainty estimation module includes an inference network and a generator network based on a conditional variational autoencoder structure. The model training process of the feature uncertainty estimation module may include the following methods: S302: During the model training process of the feature uncertainty estimation module, the mean map and variance map of character features generated by the inference network for the character image of the sample are obtained, and the noise tensor with the same feature dimension as the variance map of character features is sampled. During the model training process of the feature uncertainty estimation module, the forward propagation of the inference network for recognizing character images of samples can be referred to S202-S204, the only difference being the different image processing objects.

[0057] Inferring the mean and variance maps of character features output by the network in each round of forward propagation; noise tensor This refers to a tensor with the same feature dimensions as the character feature mean map and variance map, whose element values ​​are randomly sampled from a standard normal distribution (i.e., a distribution with a mean of 0 and a variance of 1). This tensor is used to introduce necessary randomness during reparameter sampling.

[0058] To illustrate, take a sample image from the training dataset to recognize characters. low The input is then fed into the inference network of the feature uncertainty estimation module. The inference network performs one forward propagation, generating the corresponding character feature mean map μ. F Character feature variance plot Σ F Simultaneously, a noise tensor with the same dimensions as the aforementioned feature map is generated. .

[0059] S304: Based on the noise tensor, the character feature mean map, and the character feature variance map, reparameter sampling is performed using the first calculation formula to obtain a sampled feature map; Reparameter sampling is a technique widely used in variational autoencoder models. Its purpose is to maintain the random sampling characteristics while ensuring that the sampling process itself is differentiable with respect to the distribution parameters (mean and variance in this case), thus allowing optimization algorithms (such as gradient descent) to train the entire model end-to-end. The first calculation formula is a specific implementation of this technique.

[0060] The first calculation formula satisfies the following formula:

[0061] in, The sampled feature map, For the noise tensor, This represents an element-wise multiplication operation, which makes the sampling process differentiable and allows the error gradient to backpropagate to the encoder parameters. The mean map of the character features, the This is the variance map of the character features.

[0062] Indicative of generative networks (Also known as the decoder) is used during the training phase to learn the degradation process of reconstructing the corresponding character image to be recognized from a clear feature map sample. To maintain gradient transferability during training, this module uses reparameterization techniques from the inference network. Sampling is performed using the first calculation formula in the defined distribution; S306: The sampled feature map is reconstructed and decoded using the generator network to obtain a reconstructed character image for the character image identified in the sample; Subsequently, the feature maps received by the decoder-generator network, whose structure is symmetrical with the encoder, are used for reconstruction decoding and output the reconstructed character image, which can be represented as:

[0063] Among them, the reconstructed character image is ; This section represents the sampled feature map. ; S308: The second calculation formula is used to determine the uncertainty estimation loss for the reconstructed character image and the sample recognized character image, and the model parameters of the feature uncertainty estimation module are adjusted based on the feature uncertainty estimation loss; During training, the objective of the feature uncertainty estimation module is to maximize the evidence lower bound (ELBO), and its corresponding loss function is... For the second calculation formula: The second calculation formula satisfies the following formula:

[0064] Among them, the To estimate the loss for the aforementioned feature uncertainty, the The reconstruction loss is determined based on the reconstructed character image and the sample recognized character image. The posterior probability distribution parameters of the image are... Identify character images for the samples. The weighting coefficients are... The preset image prior probability distribution parameters, the This is the KL divergence regularization operation.

[0065] The first term in the above equation is the input sample character image. With reconstructed character images The refactoring term can be represented as In practice, it is usually caused by reconstruction loss. Approximated by the negative value, such as the L1 norm loss. Or it can be combined with perceptual loss. The second term is the KL divergence regularization term, which constrains the approximate posterior. Priors approximating a standard normal distribution , It is its weighting coefficient, used to balance reconstruction fidelity and regularization strength.

[0066] After processing by this module, the original deterministic image input is transformed into a probabilistic feature representation. This representation not only contains the best estimate of the image content but also encapsulates the spatially variable uncertainty information that accompanies this estimate. This output serves as the input basis for the next module's robust recognition, realizing the transformation from image to probabilistic feature representation and providing crucial uncertainty information for subsequent processing.

[0067] In this specification, a feature uncertainty estimation module receives a low-light image as input. Unlike traditional methods for generating deterministic enhanced features, this module treats the ideal, sharp feature as a latent variable and, based on the principle of a conditional variational autoencoder (CVAE), infers the posterior probability distribution of this latent variable given the input image. This distribution is parameterized as a Gaussian distribution, so the module's output is a pair of tensors: a feature mean map and a feature variance map. The feature mean map represents the optimal estimate of the ideal feature, while the feature variance map quantifies the uncertainty or confidence of this estimate point-by-point and channel-by-channel.

[0068] Optional, please see Figure 5 , Figure 5 This is a schematic diagram of the initial character trajectory recognition process proposed in this specification. The optical character recognition model includes an uncertainty-aware recognition module, which is based on a Transformer encoder-decoder architecture, but its cross-attention mechanism is specially designed to utilize uncertainty information, thereby achieving an adaptive trade-off for the quality of visual evidence; The process of performing initial character trajectory recognition using the optical character recognition model based on the character feature mean map and character feature variance map to obtain a preliminary recognized character trajectory sequence and a decoded character trajectory metadata data sequence for the preliminary recognized character trajectory sequence can be performed in the following manner: S402: The encoder of the uncertainty perception and recognition module performs visual position encoding on the character feature mean map to obtain a context-aware feature sequence, and flattens the character feature variance map to obtain a squared difference feature sequence. The encoder of the uncertainty-aware recognition module can be an internal visual Transformer encoder. Its function is to receive the flattened mean map of character features and, through its self-attention mechanism, interact with all features in the sequence using global contextual information, ultimately outputting a context-aware feature sequence. The optical character recognition model includes an uncertainty-aware recognition module, which is a Transformer-based encoder-decoder architecture, but its cross-attention mechanism is specially designed to utilize uncertainty information, thereby achieving an adaptive trade-off for the quality of visual evidence. (See [link to relevant documentation]). Figure 6 , Figure 6 This is a computational flowchart of an uncertainty-weighted attention mechanism for an uncertainty perception and recognition module; Context-aware feature sequence: refers to the output sequence obtained after the character feature mean map is processed by the encoder. Each feature vector in this sequence not only contains local information of its corresponding image region, but also aggregates global contextual information from the entire image through a self-attention mechanism. The squared difference feature sequence refers to the feature sequence obtained after flattening the character feature variance map. This sequence corresponds one-to-one with the context-aware feature sequence in spatial location, where each vector quantifies the degree of uncertainty of the feature at the corresponding location.

[0069] like Figure 6 As shown, the mean value plot of character features (such as...) Figure 6 The feature mean map shown is flattened into a one-dimensional feature sequence, and then visual position encoding is added to it to preserve spatial location information, forming a feature sequence. This feature sequence is processed by a standard visual Transformer encoder, which uses its internal multi-head self-attention layer to model the global information of the features in the sequence to capture global contextual information, generating a context-aware feature sequence. ,in .

[0070] Character feature variance plot (like Figure 6 The image shown is a feature variance plot. The same flattening operation is performed on the squared difference feature sequence (e.g., ...). Figure 6 The variance feature sequence shown), and They correspond one-to-one in spatial location.

[0071] S404: The uncertainty perception and recognition module's decoder performs cross-attention processing based on the decoder's hidden state, the context-aware feature sequence, and the squared difference feature sequence to obtain an attention weight vector; For reference, please see below. Figure 6As shown, at any time step t during decoding, the decoder has an internal hidden state h. t (as a query), and receive the context-aware feature sequence H generated by S402. μ (as key) and the squared difference feature sequence H Σ The decoder calculates its hidden state h. t With context-aware feature sequence H μ The similarity is calculated for each feature vector. The key is how this similarity is calculated (i.e.,...). Figure 6 In the computation of attention energy shown, an uncertainty penalty term is introduced, consisting of the variance values ​​at corresponding positions in the squared difference feature sequence. This penalty term reduces the attention energy in regions with high feature variance values, thereby reducing the model's attention to unreliable visual regions. Finally, the attention energy at all positions is normalized using a softmax function (e.g., ...). Figure 6 The normalized attention weights shown in the figure can be used to obtain the final attention weight vector. Each element of this attention weight vector represents the degree of attention the decoder pays to each region of the image in the current step.

[0072] In some embodiments, the decoder of the uncertainty-aware recognition module performs cross-attention processing based on the decoder hidden state, the context-aware feature sequence, and the squared difference feature sequence to obtain an attention weight vector, which can be achieved in the following manner: The decoder of the uncertainty perception and recognition module uses the decoder hidden state as the query vector and the context-aware feature sequence as the key vector. Based on the decoder hidden state, the context-aware feature sequence, and the squared difference feature sequence, the visual position attention energy is calculated using a third calculation formula. The visual position attention energy of all sequence positions is normalized to obtain the attention weight vector. The third calculation formula satisfies the following formula:

[0073] Among them, the The visual position attention energy for the i-th sequence position These are the query vector projection matrix and the key vector projection matrix, respectively. It is the dimension of the key vector. and They are the first The feature mean and variance vector of each sequence position, the The operator represents the character trajectory by calculating the sum of the variances of all feature channels as a variance vector. This is the control coefficient for the intensity of uncertainty penalty.

[0074] To illustrate, the decoder is an autoregressive Transformer decoder. At each decoding time step... The decoder is based on its current hidden state. (Can be viewed as a query vector) and visual feature sequence Cross-attention calculation is performed using (the key vector Key). The key innovation of this process lies in the attention energy function in the third calculation formula. The definition, in the third calculation formula, introduces a factor derived from the variance map based on the standard scaled dot product attention. Uncertainty penalty item for guidance; in It is a learnable projection matrix. It is the dimension of the key vector. and These are the first two numbers in the visual sequence. The feature mean and variance vector at each position. The operator represents the character trajectory by calculating the sum of the variances of all feature channels as a variance vector. The uncertainty penalty strength control coefficient is either a learnable scalar parameter or a fixed hyperparameter used to control the strength of the uncertainty penalty. The underlying logic of this third calculation is that when the decoder is searching for relevant visual cues, it significantly reduces attention to regions with higher uncertainty (larger sum of variances) for two regions with similar content (measured by the dot product term). Then, the attention energy at all locations is calculated using the Softmax function. Normalization is performed to obtain the final uncertainty-weighted attention weight vector. , can be represented as follows:

[0075] S406: Random sampling is performed based on the context-aware feature sequence and the squared difference feature sequence to obtain a sampled feature distribution, and a context vector is obtained by weighting the attention weight vector and the sampled feature distribution; Random sampling refers to a stochastic process of drawing a single sample from a probability distribution. Specifically, it is the process of drawing a specific feature vector sample from a pointwise Gaussian distribution defined by a context-aware feature sequence (as the mean) and a squared difference feature sequence (as the variance). This process is achieved through reparameterization techniques to ensure the differentiability of the entire computational graph.

[0076] The sampling feature distribution refers to the specific feature vector or sequence of feature vectors obtained through the above-mentioned random sampling process. It can be regarded as a possible instance of visual features containing uncertain perturbations, rather than a fixed, deterministic mean estimate.

[0077] The context vector refers to the generalized visual feature vector generated by the attention mechanism at the current time step t of decoding, which is used to guide character prediction. It is the final result obtained by weighting and summing the (projected) sampled features using the attention weight vector.

[0078] In calculating the context vector used to generate characters To ensure the decoder is directly robust to feature uncertainties, this module uses context-aware feature sequences. Sum of squared difference characteristic sequences Instead of directly using the mean feature, random sampling is performed within the defined feature distribution to obtain the sampled features. As a value vector.

[0079] In some embodiments, the random sampling based on the context-aware feature sequence and the squared difference feature sequence is performed to obtain a sampled feature distribution, and a context vector is obtained by weighting the attention weight vector and the sampled feature distribution. This can be achieved using the following method: 1. Based on the standard normal distribution, sample the sequence position noise tensor for each sequence position. Then, based on the context-aware feature sequence, the squared difference feature sequence, and the sequence position noise tensor, perform random sampling using the fourth calculation formula to obtain the sampling feature distribution (e.g., ...). Figure 6 The sampling features shown in the figure); First, the generated context-aware feature sequence and squared difference feature sequence are treated together as parameters of a pointwise Gaussian distribution. That is, for each position i in the sequence, H... μ,i H is the mean of the characteristic distribution at that location. Σ,i It is its variance.

[0080] Then, random sampling is performed, randomly sampling the feature distribution at each position i to obtain a sampled feature vector H. sample,t,i The sampling process is defined by the following fourth calculation formula: The fourth calculation formula satisfies the following formula:

[0081] Among them, the The sampling feature distribution of the i-th sequence position at decoding time step t, The mean feature parameter of the i-th sequence position of the context-aware feature sequence at decoding time step t, wherein Let be the variance vector of the i-th sequence position of the squared difference feature sequence at decoding time step t. Let be the sequence position noise tensor of the i-th sequence position at decoding time step t. ; The key to this step is that for higher uncertainty (i.e., variance) (Larger) features, its sampling result H sample,t,i It will be at the mean H μ,i There is a larger range of fluctuations nearby.

[0082] 2. Based on the attention weight vector and the sampling feature distribution, the context vector is obtained by weighting using the fifth calculation formula (e.g., ...). Figure 6 The weighted computation context vector shown is shown. The fifth calculation formula satisfies the following formula:

[0083] Among them, the The context vector is N, where N is the total decoding time steps. The value projection matrix, the Let be the attention weight vector at the decoding time step t.

[0084] Using the calculated attention weight vector αt, project the value vector onto the matrix W. V The projected sampled feature sequences are weighted and summed to obtain the final context vector. .

[0085] S408: Based on the context vector, obtain the preliminary identified character trajectory sequence and the decoded character trajectory metadata data sequence for the preliminary identified character trajectory sequence.

[0086] Preliminary identification of character trajectory sequences (C) raw ): This refers to the initial text output generated by the uncertainty perception and recognition module, which is a sequence of recognized characters. This sequence is the object of processing by the subsequent correction module.

[0087] Decoded Character Trajectory Metadata Packet Sequence (DTPs): This refers to a structured set of metadata generated synchronously with the initially identified character trajectory sequence. Each metadata packet (DTP) in the sequence records the complete context information of the corresponding character's generation, which is the key basis for subsequent trajectory verification processing.

[0088] Indicatively, using the context vector, two parallel tasks are performed at the current decoding time step t: predicting the output character to generate a preliminary recognized character trajectory sequence, and constructing the metadata packet of the character to generate a decoded character trajectory metadata packet sequence.

[0089] Character prediction process: First, the context vector v generated in step S406 t With the decoder's hidden state h at the current time step t To combine. The hidden state h t It contains historical information about the already generated character sequences. This combined information (e.g., a fused state vector obtained through vector concatenation or element-wise addition) is input into a feedforward neural network. The last layer of the feedforward neural network is typically a softmax classifier, which outputs a probability distribution covering all candidate characters. The character with the highest probability in this probability distribution is selected as the recognition result c at the current time step t. t The character c t It was then added to the initial character trajectory sequence C that was being constructed. raw The end of.

[0090] Decoding trajectory metadata data packet construction process: After determining the output character 'c' at the current time step t It will also carry the output character 'c'. t Predicted character confidence probability (all output characters c) t The predicted character confidence probability constitutes, for example, Figure 6 The character probability distribution shown will be used to synchronously construct a corresponding decoded character metadata data packet (DTP). t Then, based on its composition, a decoded character trajectory metadata data sequence (such as...) is generated for the initially identified character trajectory sequence. Figure 6 The decoded estimated metadata packet is shown.

[0091] In some embodiments, the process of obtaining the preliminary identified character trajectory sequence based on the context vector and the decoded character trajectory metadata data sequence for the preliminary identified character trajectory sequence can be performed in the following manner: 1. The context vector and the decoder hidden state of the decoder are combined by the decoder of the uncertainty perception and recognition module to obtain the character fusion information state vector. Based on the character fusion information state vector, the character trajectory is predicted to obtain the preliminary recognized character corresponding to each decoding time step. Based on all the preliminary recognized characters, the preliminary recognized character trajectory sequence is obtained. 2. The character metadata data packet for each of the initially identified characters is determined by the decoder of the uncertainty perception and recognition module, and a sequence of decoded character trajectory metadata data packets is generated based on the character metadata data packets of all the initially identified characters; The character metadata data includes a pre-identified character, a predicted character confidence probability corresponding to the pre-identified character, an attention weight vector corresponding to the context vector, the mean of the average features corresponding to all the attention weight vectors, and the average variance of the average features corresponding to all the attention weight vectors.

[0092] Specifically, for each generated character Construct a decoded trajectory metadata packet (DTP) with the following data structure:

[0093] in, The character predicted at the current time step t; Predicting the confidence probability of a character can be understood as the model's initial output of the recognized character. The confidence probability; : is used to calculate the context vector The attention weight vector used during the process is essentially a record of the model's performance in recognizing the character 'c'. t At that time, which areas of the image were "looked at", and the degree of attention paid to each area; It is the average feature mean of the attention weight vector after attention weighting, representing the model's performance in recognition. What are the average "content" characteristics of the area of ​​interest at that time? It is the average feature variance of the attention weight vector after attention weighting, which is a crucial piece of information as it represents how high the average "uncertainty" is in the region the model is interested in.

[0094] The output of this module includes the initially identified sequences. and the corresponding meta-data packet sequence The latter provides detailed, multimodal procedural evidence for the audit correction of the next module.

[0095] In this specification, the uncertainty-aware recognition module takes the feature mean map and feature variance map output by the aforementioned feature uncertainty estimation module as input, and employs an improved uncertainty-aware attention mechanism for sequence decoding. When calculating attention weights, this mechanism uses the feature variance as a penalty term, thereby adaptively reducing attention to visual regions with high uncertainty. Simultaneously, when generating the context vector for decoding, the module performs differentiable random sampling from the feature distribution defined by the mean and variance. The module ultimately outputs a preliminarily recognized text sequence, and a decoded trajectory metadata package generated for each character in the sequence, containing the character's probability, the corresponding attention distribution, and the feature mean and variance statistics of the dependent visual region.

[0096] In one feasible implementation, the optical character recognition model includes a recognition trajectory correction module, which specifically performs trajectory verification processing on the preliminary recognized character trajectory sequence based on the decoded character trajectory metadata data sequence using the optical character recognition model to obtain a trajectory recognition error metadata pattern and a character trajectory editing operation for the trajectory recognition error metadata pattern. Alternatively, the following approach can be referenced: S502: The recognition trajectory correction module performs vector conversion on the decoded character trajectory metadata data packet sequence to obtain the decoded trajectory metadata sequence; The trajectory correction module receives the metadata data packet sequence (DTPs) output by UAR-Mod as its sole input. Its function is to perform posterior analysis of the entire decoding process and execute high-precision sequence correction. Unlike traditional correction models that rely on plain text information, this module makes more evidence-based correction decisions by "auditing" the decoding process.

[0097] Since each decoded trajectory metadata packet (DTP) contains various types and dimensions of data (characters, scalar probabilities, vectorized attention maps, etc.), the first step involves using a multimodal input coding layer to convert each structured DTP into a fixed-dimensional, information-dense vector representation hDTP,t. The multimodal input coding layer maps the heterogeneous data types in the DTP to a unified high-dimensional vector space.

[0098] in, It is a character embedding matrix. It is a series of linear projection matrices. This indicates vector concatenation. It is a scalar that can be directly projected through a linear layer.

[0099] This represents the probability value p(c) of the scalar. tIt is converted into a vector through a linear projection matrix.

[0100] This indicates that the mean and variance of the attention-weighted average features are converted into vectors through a linear projection matrix.

[0101] This represents the high-dimensional attention weight vector α t The attention weight vector is compressed into a fixed-length vector through pooling operations (such as average pooling). For higher dimensionality, a single pooling layer (such as average pooling) can be used. Alternatively, a small convolutional network can be used to compress it into a fixed-length vector. In this way, discrete characters, scalar probabilities, vectorized attention maps, and statistical features are uniformly encoded into a sequence of decoded trajectory metadata.

[0102] S504: The recognition trajectory correction module uses a multi-head attention mechanism to perform long-range error character trajectory recognition on the decoded trajectory metadata sequence to obtain the trajectory recognition error metadata pattern of the error-recognized character, and integrates the character context trajectory metadata of the error-recognized character based on the error metadata pattern to obtain a reference context-aware diagnostic vector for each error-recognized character. Auditing network identifies error patterns: encoded vector sequence H DTP It is input into a sequence analysis network Auditor ψ The network is preferably a multi-layer Transformer encoder, whose internal multi-head self-attention mechanism is particularly adept at capturing complex long-range dependencies in sequences. When processing this sequence, its core self-attention mechanism comes into play. For each character vector h in the sequence... DTP,t The self-attention mechanism compares the vector of the current character with the vectors of all other characters in the sequence (including itself), calculating the relevance or "attention score." This means that when analyzing whether the t-th character might be incorrect, the auditing network not only looks at the metadata of the t-th character itself, but also examines the metadata of all characters before and after it, and determines which other characters' "files" are most relevant to the current character. By weighted summing of the metadata of all other characters (the weights being the attention scores), the network generates a new, context-aware diagnostic vector H for the t-th character. audit,t This new vector not only contains its own original information, but also aggregates all relevant information from the entire sequence.

[0103] The auditing network outputs a context-aware diagnostic vector at each time step t. .

[0104] Learning Objectives: Auditor ψ After training, the network learns and identifies recurring metadata patterns associated with specific error types. This ability to "identify error metadata patterns" is learned during the training phase. The network's goal is to predict correct editing operations (such as keep, delete, and replace). To achieve this, the parameters of its self-attention mechanism must be adjusted through backpropagation to learn how to discover relevant information that predicts errors.

[0105] For example, character merging errors caused by visual adhesion (such as 'm' being misidentified as 'rn') may correspond to a specific pattern in its DTP sequence: that is, the attention maps αt of 'r' and 'n' have significant overlap, and the average uncertainty of their corresponding regions is high. All are too high. The auditing network has learned to identify this kind of error "fingerprint" that combines "time series + multimodal".

[0106] Specifically: when the network processes the metadata vector of the character 'r', its self-attention mechanism learns through training to pay close attention to the metadata vector of its immediate neighbor, the character 'n'.

[0107] The network will detect a recurring pattern: if the metadata of the character 'r' contains [attention map region A, high uncertainty], and the metadata of its adjacent character 'n' contains [attention map region B, high uncertainty], and [regions A and B have significant overlap in the original image], then these two characters are very likely a merging error.

[0108] This complex error association, which spans multiple time steps and integrates various data types (attention graphs, uncertainty values), is also known as the "error metadata pattern".

[0109] By stacking multiple layers of Transformer encoders, the network is able to capture increasingly complex and abstract error patterns.

[0110] Finally, the output diagnostic vector H audit,t It is essentially a highly condensed "diagnostic report" that instructs subsequent classifiers: "After a comprehensive review of the entire recognition process, it is determined that the t-th character has an xx% probability of being correct and a yy% probability of needing to be replaced with 'm'."

[0111] S508: The character trajectory editing operation is obtained by the recognition trajectory correction module performing editing operation prediction processing on the incorrectly recognized character based on the reference context-aware diagnostic vector.

[0112] Finally, the trajectory correction module operates on the (reference) context-aware diagnostic vector through a linear classifier and a softmax layer to predict an edit operation α. t The probability distribution.

[0113]

[0114] A set of editing operations This includes: keeping the current character (KEEP), deleting the current character (DELETE), and replacing the current character with any other character from the vocabulary (REPLACE_x). This is achieved through the initial identification sequence. By applying the most frequent editing operations one by one, especially performing corresponding character trajectory editing operations to correct misidentified characters, the final corrected text sequence can be generated. .

[0115] Optionally, the core of the Track Correction Module (RTC-Mod) is a sequence analysis network, which is functionally referred to as an auditor network. ψ ).

[0116] After the auditing network completes a deep analysis of the entire metadata sequence, it generates a highly condensed context-aware diagnostic vector for each character in the initial identification sequence. This vector contains a reference context-aware diagnostic vector for erroneous characters. This diagnostic vector is the direct basis for predicting editing operations. The next step is a standard classification task.

[0117] Input diagnostic vector: For the t-th character in the initial recognition sequence Craw, its corresponding diagnostic vector Haudit,t is fed into the prediction process. This vector can be understood as a comprehensive diagnostic report on whether the t-th character is "reliable" and "if unreliable, what kind of error it might be".

[0118] Processed using a linear classifier: Diagnostic vector H audit,t First, a linear classifier (usually a fully connected layer or linear layer in deep learning) is used. The role of the linear layer is to perform dimensionality reduction and feature transformation. It maps the high-dimensional, complex diagnostic vector to a new, lower-dimensional vector space. The dimension of this new vector is exactly equal to the total number of all possible editing operations.

[0119] For example, if the vocabulary has 1000 characters, then the total number of edit operations is 1 (KEEP) + 1 (DELETE) + 1000 (REPLACE_x) = 1002. The linear classifier will then convert the diagnostic vector into a vector of length 1002, called "logits". Each position in this vector corresponds to a specific edit operation, and its value represents the raw score by which the model considers the operation correct.

[0120] Using the Softmax function: The resulting "logic score" vector is then fed into a Softmax function, which transforms these raw scores, which can be of any value, into a normalized probability distribution.

[0121] Each value in the output vector can be directly interpreted as the probability of the corresponding editing operation. For example, the value in the output vector corresponding to the "KEEP" operation might be 0.01, the value corresponding to "DELETE" might be 0.05, and the value corresponding to "REPLACE_'a'" might be 0.9, which means that the model is 90% confident that the current character should be replaced with 'a'.

[0122] Finally, a decision is made and executed. For each position t, the model selects the edit operation with the highest probability value in the probability distribution output by Softmax as the final decision. During the execution phase, the system iterates through the initial recognition sequence C. raw At each position, the corresponding editing operation with the highest probability is performed to construct the final corrected text sequence.

[0123] This process ensures that the correction decision for each character is a quantitative and probabilistically optimal choice made after the uncertainty perception and recognition module has fully "audited" the contextual information of the entire recognition process.

[0124] In this specification, the recognition trajectory correction module receives the decoded character trajectory metadata data sequence output by the aforementioned uncertainty-aware recognition module. It employs a sequence analysis network (such as a Transformer encoder) to process this metadata sequence, learning to identify statistically significant temporal patterns that indicate specific types of recognition errors. Based on the analysis of the entire decoded trajectory, this module predicts an editing operation (such as keep, delete, or replace) for each character position in the initial recognition sequence. This correction decision process is based on the result of a joint audit of cross-modal (visual uncertainty, attentional behavior, sequence probability) information throughout the recognition process, rather than simple language model constraints.

[0125] In some embodiments, the optical character recognition model includes a feature uncertainty estimation module, an uncertainty-aware recognition module, and a recognition trajectory correction module. During the model training phase, end-to-end joint training is performed using a combined loss function to ensure that all modules work collaboratively to serve the final text recognition goal. Total loss function Defined as the weighted sum of the losses of the three main components:

[0126] in, It is the loss of the feature uncertainty estimation module, as described in S302-S308 above, which includes reconstruction loss and KL divergence term.

[0127] This is a preliminary character trajectory sequence output by the uncertainty perception and recognition module UAR-Mod. With real character trajectory label sequence The cross-entropy loss calculated between them is used to supervise the accuracy of the initial identification.

[0128] This is the sequence of edit operations for all character trajectory editing operations predicted by the RTC-Mod recognition trajectory correction module, and from... arrive The cross-entropy loss is calculated between preset real editing operation sequences (which can be pre-calculated using algorithms such as dynamic programming).

[0129] The real character trajectory label sequence and the preset real editing operation sequence are labeled in advance on the sample character recognition images using expert services during the model training phase; These are hyperparameters used to balance the importance of different tasks and can be dynamically adjusted during training.

[0130] During training, the gradient changes from... Starting from there, it propagates backward through the entire computation graph. This means that... The error signal not only optimizes the trajectory correction module itself but also passes through the uncertainty-aware recognition module, ultimately affecting the parameters of the feature uncertainty estimation module. This end-to-end gradient flow allows the feature uncertainty estimation module (FUE-Mod) to implicitly consider whether such uncertainty representation is beneficial to subsequent recognition and correction when learning how to estimate feature uncertainty. Similarly, the uncertainty-aware recognition module (UAR-Mod) learns to generate metadata data that is easier for the trajectory correction module (RTC-Mod) to "audit" and correct. This end-to-end joint optimization mechanism achieves end-to-end collaboration from pixel-level uncertainty quantization to feature-level uncertainty awareness and recognition, and then to process-level metadata auditing and correction, thereby achieving globally optimal recognition performance.

[0131] To better understand the solution described in this specification, the following examples illustrate the application of the optical character recognition model involved in the above-mentioned optical character recognition method: The optical character recognition model involved in optical character recognition methods can be applied to mobile nighttime document scanning and recognition scenarios: The example of mobile nighttime document scanning and recognition is intended to illustrate the specific implementation of the present invention in a smartphone application, and to address the pain point of users taking document photos and performing text recognition at night or in low-light conditions.

[0132] In this implementation scenario, the optical character recognition model can be instantiated as a lightweight architecture optimized for mobile devices. Specifically, the encoder of the Feature Uncertainty Estimation Module (FUE-Mod) Using MobileNetV3-Small as the backbone, its classification head is removed and followed by a decoder head consisting of transposed convolutions to output the feature mean map. Plot of log-variance Its spatial dimension is 1 / 8 of the input image, and it has 128 channels. The Uncertainty Awareness Recognition Module (UAR-Mod) adopts a micro Transformer architecture (4-layer encoder, 4-layer decoder, 4 heads per layer) to balance performance and computational cost. The audit network of the Recognition Trajectory Correction Module (RTC-Mod) A computationally more efficient bidirectional GRU network was chosen. The model was trained on a large-scale synthetic dataset generated by applying physically accurate low-light degradation models (including Poisson-Gaussian noise, non-uniform illumination fields, and motion blur) to sharp document images. The training process employed the AdamW optimizer and used a combined loss function for end-to-end optimization, specifically:

[0133] After training, the entire model undergoes INT8 dynamic range quantization using TensorFlow Lite to reduce its size and accelerate inference on mobile devices. In practical applications, when a user takes a photo of a contract with severely uneven lighting, FUE-Mod accurately outputs feature maps with high variance in shadow and blurred areas. Subsequently, when UAR-Mod recognizes text in these areas, its uncertainty-aware attention mechanism automatically reduces its focus on these unreliable features due to the high variance. Its third calculation formula is:

[0134] Even if the model initially misidentifies the ambiguous word "Signature" as "Signatuxe," the decoded trajectory metadata (DTP) it generates for 'x' records an extremely low recognition probability and an extremely high average visual uncertainty. Ultimately, after receiving the complete DTP sequence, RTC-Mod's auditing network identifies this "low confidence + high uncertainty" error pattern and, combined with the context, accurately predicts the edit operation to replace 'x' with 'r', thus outputting the correct text. The entire process achieves near real-time performance on mainstream mobile devices, demonstrating excellent robustness and accuracy.

[0135] The optical character recognition model involved in optical character recognition methods can be applied to the digitization and restoration of ancient books and documents: That is, digital scanning and character recognition are performed on ancient Chinese literature that has severely deteriorated (such as faded ink, broken strokes, and damaged paper) due to its long age and improper preservation. In this scenario, the model performance of the optical character recognition model is the primary consideration. Therefore, the optical character recognition model, the Pro-VPAN model, is instantiated as a powerful heavyweight network. The Feature Uncertainty Estimation Module (FUE-Mod) uses Swin Transformer-Base as the encoder-decoder architecture to handle the complex textures and damaged patterns in ancient books with its powerful multi-scale feature capture ability, and the number of output channels is 256. The core networks of the Uncertainty Awareness Recognition Module (UAR-Mod) and the Recognition Trajectory Correction Module (RTC-Mod) both adopt the standard, deeper and wider Transformer architecture (6 layers, 8 heads, 512 hidden dimensions) to make full use of the rich features extracted by FUE-Mod and analyze the long-distance text dependencies. In particular, a MERGE operation is added to the editing operation set of RTC-Mod, which is specifically used to repair the common error that a single character is misrecognized as multiple parts due to broken strokes. The training dataset of the model is constructed by cooperating with experts in historical document restoration to simulate the degradation effects (such as yellowed paper, mildew spots, ink corrosion, etc.) of high-definition ancient book images in a programmed and historically accurate manner. The training is carried out on a multi-GPU server cluster, and a loss function focusing on the correction performance is used:

[0136] The model is finally deployed on the server of the literature digitization center and provides services through an API. When processing a scanned copy of a Ming Dynasty engraved edition with severely broken strokes, a complete character of "門" may be misrecognized as two consecutive characters of "日" and "月" by UAR-Mod due to visual separation. However, the DTPs generated for these two misrecognized characters will contain key metadata: their attention maps are adjacent in the original feature space, and both have high visual uncertainty marks caused by broken strokes. When analyzing the entire DTP sequence, the self-attention mechanism of the Transformer audit network of RTC-Mod can capture this unique pattern of "spatial proximity + double high uncertainty" and trigger the MERGE operation. This operation will merge the visual evidence of these two characters, re-judge in combination with the context, and finally correct the sequence of "日" and "月" into the correct single character of "門". This fully demonstrates the unique ability of the present invention in processing extremely degraded images and repairing structural recognition errors, providing unprecedented accuracy and reliability for the digitization of precious historical documents.

[0137] The above description is merely an example of some implementation scenarios of the present invention and is not intended to limit the present invention in any way. Any person skilled in the art can make some modifications or alterations to the disclosed technical content to create equivalent implementation scenarios without departing from the scope of the present invention. Any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention, without departing from the scope of the present invention, shall still fall within the protection scope of the present invention.

[0138] Compared with related technologies, the solutions of one or more embodiments in this specification have the following significant advantages: 1. Fundamentally solves the information bottleneck problem: By introducing probabilistic feature representation, this invention quantifies the uncertainty information of the input image and completely transmits it to the downstream task, avoiding the information loss caused by generating deterministic intermediate representations in traditional enhancement methods.

[0139] 2. Significantly improved robustness of recognition: The attention mechanism of uncertainty perception enables the model to intelligently weigh the quality of visual evidence during decoding and automatically avoid unreliable image regions, thus making robust recognition decisions even in the face of severe noise and missing information. 3. Highly accurate diagnostic correction is achieved: The decoding trajectory auditing mechanism ensures that the correction process is no longer a blind text matching process. By analyzing metadata containing rich information such as visual uncertainty and attention behavior, the model can accurately locate and correct recognition errors caused by specific visual degradations, with an accuracy far exceeding that of independent language model post-processors. 4. End-to-end collaborative optimization is achieved: A unified framework and joint training mechanism ensure the consistency of objectives across the three logical stages of enhancement, recognition, and correction. Error signals from later modules can be backpropagated and guide earlier modules to learn and generate representations more beneficial to the final task, thereby achieving global optimum.

[0140] The following will combine Figure 7 This specification provides a detailed description of the optical character recognition device provided in the embodiments. It should be noted that... Figure 7 The optical character recognition device shown is used to perform the functions described in this specification. Figures 1-6 The methods shown in the embodiments are illustrated for ease of explanation, showing only the parts related to the embodiments of this specification. For specific technical details not disclosed, please refer to this specification. Figures 1-6 The example shown.

[0141] Please see Figure 7This diagram illustrates the structure of an optical character recognition device according to an embodiment of this specification. The optical character recognition device 1 can be implemented as all or part of a user terminal through software, hardware, or a combination of both. According to some embodiments, the optical character recognition device 1 includes an image acquisition module 11 and a character recognition module 12, specifically used for: Image acquisition module 11 is used to acquire an image of the character to be recognized and input the image of the character to be recognized into an optical character recognition model; The character recognition module 12 is used to estimate the uncertainty of character features in the image of the character to be recognized through the optical character recognition model, and generate a character feature mean map and a character feature variance map. The character feature variance map is used to characterize the uncertainty of the feature estimation of the character feature mean map. The character recognition module 12 is used to perform initial character trajectory recognition based on the character feature mean map and character feature variance map using the optical character recognition model to obtain an initial recognized character trajectory sequence and a decoded character trajectory metadata data sequence for the initial recognized character trajectory sequence; The character recognition module 12 is used to perform trajectory verification processing on the preliminary recognized character trajectory sequence based on the decoded character trajectory metadata data sequence through the optical character recognition model to obtain a trajectory recognition error metadata pattern and a character trajectory editing operation for the trajectory recognition error metadata pattern, perform trajectory correction on the preliminary recognized character trajectory sequence based on the character trajectory editing operation to generate a correction text sequence, and output the correction text sequence for the character image to be recognized.

[0142] Optionally, the optical character recognition model includes a feature uncertainty estimation module. The step of estimating the character feature uncertainty of the image to be recognized using the optical character recognition model to generate a character feature mean map and a character feature variance map includes: using the ideal character image clarity features of the image to be recognized as latent variables through the inference network of the feature uncertainty estimation module; using the inference network to infer the posterior probability distribution of the latent variables under the image to be recognized to obtain image posterior probability distribution parameters; and generating a character feature mean map characterizing the clarity features of the character image and a character feature variance map quantifying the uncertainty of the clarity features of the character image based on the image posterior probability distribution parameters.

[0143] Optionally, the feature uncertainty estimation module includes an inference network and a generator network based on a conditional variational autoencoder structure, and the method includes: During the model training process of the feature uncertainty estimation module, the mean map and variance map of character features generated by the inference network for the sample recognized character image are obtained, and a noise tensor with the same feature dimension as the character feature variance map is sampled; based on the noise tensor, the mean map of character features, and the variance map of character features, a first calculation formula is used to perform reparameter sampling to obtain a sampled feature map; the sampled feature map is reconstructed and decoded by the generation network to obtain a reconstructed character image for the sample recognized character image; a second calculation formula is used to determine the uncertainty estimation loss for the reconstructed character image and the sample recognized character image, and the model parameters of the feature uncertainty estimation module are adjusted based on the feature uncertainty estimation loss; The first calculation formula satisfies the following formula:

[0144] in, The sampled feature map, For the noise tensor, This represents an element-wise multiplication operation, the... The mean map of the character features, the This is the variance map of the character features.

[0145] The second calculation formula satisfies the following formula:

[0146] Among them, the To estimate the loss for the aforementioned feature uncertainty, the The reconstruction loss is determined based on the reconstructed character image and the sample recognized character image. The posterior probability distribution parameters of the image are... Identify character images for the samples. The weighting coefficients are... The preset image prior probability distribution parameters, the This is the KL divergence regularization operation.

[0147] Optionally, the optical character recognition model includes an uncertainty-aware recognition module. The step of obtaining a preliminary character trajectory sequence and a decoded character trajectory metadata data sequence based on the character feature mean map and character feature variance map through the optical character recognition model includes: using the encoder of the uncertainty-aware recognition module to perform visual position encoding on the character feature mean map to obtain a context-aware feature sequence, and flattening the character feature variance map to obtain a squared difference feature sequence; using the decoder of the uncertainty-aware recognition module to perform cross-attention processing based on the decoder hidden state, the context-aware feature sequence, and the squared difference feature sequence to obtain an attention weight vector; performing random sampling based on the context-aware feature sequence and the squared difference feature sequence to obtain a sampling feature distribution; weighting based on the attention weight vector and the sampling feature distribution to obtain a context vector; and obtaining the preliminary character trajectory sequence and the decoded character trajectory metadata data sequence based on the context vector.

[0148] Optionally, the step of obtaining an attention weight vector by cross-attention processing based on the decoder hidden state, the context-aware feature sequence, and the squared difference feature sequence through the decoder of the uncertainty perception and recognition module includes: using the decoder hidden state as the query vector and the context-aware feature sequence as the key vector, calculating visual position attention energy based on the decoder hidden state, the context-aware feature sequence, and the squared difference feature sequence using a third calculation formula, and normalizing the visual position attention energy of all sequence positions to obtain the attention weight vector; The third calculation formula satisfies the following formula:

[0149] Among them, the The visual position attention energy for the i-th sequence position These are the query vector projection matrix and the key vector projection matrix, respectively. It is the dimension of the key vector. and They are the first The feature mean and variance vector of each sequence position, the The operator represents the character trajectory by calculating the sum of the variances of all feature channels as a variance vector. This is the control coefficient for the intensity of uncertainty penalty.

[0150] Optionally, the step of randomly sampling the sampled feature distribution based on the context-aware feature sequence and the squared difference feature sequence, and then weighting the attention weight vector and the sampled feature distribution to obtain the context vector, includes: The sequence position noise tensor of each sequence position is sampled based on a standard normal distribution. The sampling feature distribution is obtained by random sampling based on the context-aware feature sequence, the squared difference feature sequence, and the sequence position noise tensor using a fourth calculation formula. The context vector is obtained by weighting the attention weight vector and the sampling feature distribution using the fifth calculation formula. The fourth calculation formula satisfies the following formula:

[0151] Among them, the The sampling feature distribution of the i-th sequence position at decoding time step t, The mean feature parameter of the i-th sequence position of the context-aware feature sequence at decoding time step t, wherein Let be the variance vector of the i-th sequence position of the squared difference feature sequence at decoding time step t. Let be the sequence position noise tensor of the i-th sequence position at decoding time step t; The fifth calculation formula satisfies the following formula:

[0152] Among them, the The context vector is N, where N is the total decoding time steps. The value projection matrix, the Let be the attention weight vector at the decoding time step t.

[0153] Optionally, obtaining the preliminary identified character trajectory sequence and the decoded character trajectory metadata data sequence based on the context vector includes: combining the context vector with the decoder hidden state of the decoder using the decoder of the uncertainty-aware identification module to obtain a character fusion information state vector; predicting the character trajectory based on the character fusion information state vector to obtain the preliminary identified character corresponding to each decoding time step; obtaining the preliminary identified character trajectory sequence based on all the preliminary identified characters; determining the character metadata data for each of the preliminary identified characters using the decoder of the uncertainty-aware identification module; and generating the decoded character trajectory metadata data sequence based on all the character metadata data of the preliminary identified characters; wherein the character metadata data includes the preliminary identified character, the predicted character confidence probability corresponding to the preliminary identified character, the attention weight vector corresponding to the context vector, the mean of the average features corresponding to all the attention weight vectors, and the mean variance of the average features corresponding to all the attention weight vectors.

[0154] Optionally, the optical character recognition model includes a recognition trajectory correction module. The step of performing trajectory verification processing on the preliminary recognized character trajectory sequence based on the decoded character trajectory metadata data sequence using the optical character recognition model to obtain a trajectory recognition error metadata pattern and a character trajectory editing operation based on the trajectory recognition error metadata pattern includes: performing vector transformation on the decoded character trajectory metadata data sequence using the recognition trajectory correction module to obtain a decoded trajectory metadata sequence; using a multi-head attention mechanism, the recognition trajectory correction module performs long-range error character trajectory recognition on the decoded trajectory metadata sequence to obtain a trajectory recognition error metadata pattern for the erroneous characters, and integrates character context trajectory metadata for the erroneous characters based on the error metadata pattern to obtain a reference context-aware diagnostic vector for each erroneous character; and performing editing operation prediction processing on the erroneous characters based on the reference context-aware diagnostic vector using the recognition trajectory correction module to obtain a character trajectory editing operation.

[0155] It should be noted that the optical character recognition device provided in the above embodiments is only illustrated by the division of the above functional modules when executing the optical character recognition method. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above. In addition, the optical character recognition device and the optical character recognition method embodiments provided in the above embodiments belong to the same concept, and the implementation process can be found in the method embodiments, which will not be repeated here. The sequence numbers of the embodiments in this specification are only for description and do not represent the superiority or inferiority of the embodiments.

[0156] In the embodiments of this specification, the electronic device inputs the image of the character to be recognized into the optical character recognition model. The optical character recognition model performs uncertainty estimation of the character features of the image to be recognized, generating a mean map and a variance map of the character features. This fundamentally avoids the information bottleneck problem caused by generating deterministic intermediate images in traditional image enhancement methods, and completely quantifies and transmits the uncertainty information of the image. Furthermore, the initial recognition of character trajectories based on the mean map and variance map of character features can avoid unreliable visual regions, and obtain a preliminary recognized character trajectory sequence and a decoded character trajectory metadata data sequence for the preliminary recognized character trajectory sequence. This significantly improves the robustness of the initial recognition in the face of severe noise and missing information. Furthermore, based on the decoded character trajectory metadata data sequence, trajectory verification processing is performed on the preliminary recognized character trajectory sequence to obtain trajectory recognition error metadata patterns and character trajectory editing operations for trajectory recognition error metadata patterns. Based on the character trajectory editing operations, trajectory correction is performed on the preliminary recognized character trajectory sequence to generate and output a corrected text sequence. By analyzing metadata containing rich information such as visual uncertainty and attention behavior, recognition errors caused by specific visual degradation can be accurately located and repaired. The entire optical character recognition process achieves high-precision text recognition by probabilistically modeling the image degradation process and using the generated uncertainty metadata to guide the subsequent recognition and correction process.

[0157] This specification also provides a computer storage medium that can store multiple instructions adapted to be loaded and executed by a processor as described above. Figures 1-6 The optical character recognition method of the illustrated embodiment can be found in the following document for a detailed execution process: Figures 1-6 Detailed descriptions of the illustrated embodiments are omitted here. This specification also provides a computer program product storing at least one instruction, which is loaded and executed by the processor as described above. Figures 1-6 The optical character recognition method of the illustrated embodiment can be found in the following document for a detailed execution process: Figures 1-6 The specific details of the illustrated embodiments will not be elaborated here.

[0158] Please refer to Figure 8 This is a structural block diagram of an electronic device provided in an embodiment of this specification. The electronic device in this specification may include one or more of the following components: a processor 1010, a memory 1020, an input device 1030, an output device 1040, and a bus 1050. The processor 1010, memory 1020, input device 1030, and output device 1040 may be connected to each other via the bus 1050.

[0159] Processor 1010 may include one or more processing cores. Processor 1010 connects to various parts of the electronic device using various interfaces and lines, and performs various functions and processes data by running or executing instructions, programs, code sets, or instruction sets stored in memory 1020, and by calling data stored in memory 1020. Optionally, processor 1010 may be implemented using at least one hardware form of digital signal processing (DSP), field-programmable gate array (FPGA), or programmable logic array (PLA). Processor 1010 may integrate one or more of a central processing unit (CPU), graphics processing unit (GPU), and modem. The CPU primarily handles the operating system, user interface, and applications; the GPU is responsible for rendering and drawing the displayed content; and the modem handles wireless communication. It is understood that the modem may also not be integrated into processor 1010 and may be implemented separately through a communication chip.

[0160] The memory 1020 may include random access memory (RAM) or read-only memory (ROM). Optionally, the memory 1020 may include non-transitory computer-readable storage medium. The memory 1020 may be used to store instructions, programs, code, code sets, or instruction sets.

[0161] The input device 1030 is used to receive input instructions or data, and includes, but is not limited to, a keyboard, mouse, camera, microphone, or touch device. The output device 1040 is used to output instructions or data, and includes, but is not limited to, a display device and a speaker. In this embodiment, the input device 1030 can be a temperature sensor for acquiring the operating temperature of the electronic device. The output device 1040 can be a speaker for outputting audio signals.

[0162] In addition, those skilled in the art will understand that the structure of the electronic device shown in the above figures does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements. For example, the electronic device may also include radio frequency circuits, input units, sensors, audio circuits, wireless fidelity (WIFI) modules, power supplies, Bluetooth modules, etc., which will not be described in detail here.

[0163] In the embodiments of this specification, the executing entity for each step can be the electronic device described above. Optionally, the executing entity for each step can be the operating system of the electronic device. The operating system can be Android, iOS, or other operating systems; this specification does not limit this.

[0164] exist Figure 8 In the electronic device, the processor 1010 can be used to call a program stored in the memory 1020 and execute it to implement the optical character recognition method as described in the various method embodiments of this specification.

[0165] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The program can be stored in a computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. The storage medium can be a magnetic disk, optical disk, read-only memory, or random access memory, etc.

[0166] The above-disclosed embodiments are merely preferred embodiments of this specification and should not be construed as limiting the scope of this specification. Therefore, any equivalent variations made in accordance with the claims of this specification shall still fall within the scope of this specification.

Claims

1. An optical character recognition method, characterized in that, The method includes: Obtain an image of the character to be recognized and input the image of the character to be recognized into an optical character recognition model; The optical character recognition model is used to estimate the uncertainty of character features in the image of the character to be recognized, and a character feature mean map and a character feature variance map are generated. The character feature variance map is used to characterize the degree of uncertainty of the feature estimation of the character feature mean map. The optical character recognition model performs initial character trajectory recognition based on the character feature mean map and character feature variance map to obtain an initial recognized character trajectory sequence and a decoded character trajectory metadata data sequence for the initial recognized character trajectory sequence; The optical character recognition model performs trajectory verification processing on the initially recognized character trajectory sequence based on the decoded character trajectory metadata data sequence to obtain a trajectory recognition error metadata pattern and a character trajectory editing operation for the trajectory recognition error metadata pattern. Based on the character trajectory editing operation, the initially recognized character trajectory sequence is corrected to generate a correction text sequence, and the correction text sequence for the character image to be recognized is output.

2. The method according to claim 1, characterized in that, The optical character recognition model includes a feature uncertainty estimation module. The step of estimating the uncertainty of character features in the image of the character to be recognized using the optical character recognition model, and generating a character feature mean map and a character feature variance map, includes: The inference network of the feature uncertainty estimation module uses the ideal character image clarity features of the character image to be identified as latent variables. The inference network is used to infer the posterior probability distribution of the latent variable under the character image to be identified to obtain the image posterior probability distribution parameters. Based on the image posterior probability distribution parameters, a character feature mean map representing the clarity features of the character image and a character feature variance map quantifying the uncertainty of the clarity features of the character image are generated.

3. The method according to claim 2, characterized in that, The feature uncertainty estimation module includes an inference network and a generator network based on a conditional variational autoencoder structure, and the method includes: During the model training process of the feature uncertainty estimation module, the mean map and variance map of character features generated by the inference network for the character image of the sample are obtained, and a noise tensor with the same feature dimension as the variance map of character features is sampled. Based on the noise tensor, the character feature mean map, and the character feature variance map, a first calculation formula is used to perform reparameter sampling to obtain a sampled feature map; The sampled feature map is reconstructed and decoded using the generator network to obtain a reconstructed character image for the character image identified in the sample. The second calculation formula is used to determine the uncertainty estimation loss for the reconstructed character image and the sample recognized character image, and the model parameters of the feature uncertainty estimation module are adjusted based on the feature uncertainty estimation loss. The first calculation formula satisfies the following formula: in, The sampled feature map, For the noise tensor, This represents an element-wise multiplication operation, the... The character feature mean map, the The character feature variance map; The second calculation formula satisfies the following formula: Among them, the To estimate the loss for the aforementioned feature uncertainty, the The reconstruction loss is determined based on the reconstructed character image and the sample recognized character image. The posterior probability distribution parameters of the image are... Identify character images for the samples. The weighting coefficients are... The preset image prior probability distribution parameters, the This is the KL divergence regularization operation.

4. The method according to claim 1, characterized in that, The optical character recognition model includes an uncertainty perception recognition module. The process of obtaining a preliminary character trajectory sequence and a decoded character trajectory metadata data sequence based on the optical character recognition model using the character feature mean map and character feature variance map includes: The encoder of the uncertainty perception and recognition module performs visual position encoding on the character feature mean map to obtain a context-aware feature sequence, and flattens the character feature variance map to obtain a squared difference feature sequence. The decoder of the uncertainty perception and recognition module obtains an attention weight vector by performing cross-attention processing based on the decoder hidden state, the context-aware feature sequence, and the squared difference feature sequence. A sampling feature distribution is obtained by random sampling based on the context-aware feature sequence and the squared difference feature sequence; A context vector is obtained by weighting the attention weight vector and the sampling feature distribution. Based on the context vector, a preliminary character trajectory sequence and a decoded character trajectory metadata data sequence for the preliminary character trajectory sequence are obtained.

5. The method according to claim 4, characterized in that, The decoder of the uncertainty-aware recognition module obtains an attention weight vector by performing cross-attention processing based on the decoder hidden state, the context-aware feature sequence, and the squared difference feature sequence, including: The decoder of the uncertainty perception and recognition module uses the decoder hidden state as the query vector and the context-aware feature sequence as the key vector. Based on the decoder hidden state, the context-aware feature sequence, and the squared difference feature sequence, the visual position attention energy is calculated using a third calculation formula. The visual position attention energy of all sequence positions is normalized to obtain the attention weight vector. The third calculation formula satisfies the following formula: Among them, the The visual position attention energy for the i-th sequence position These are the query vector projection matrix and the key vector projection matrix, respectively. It is the dimension of the key vector. and They are the first The feature mean and variance vector of each sequence position, the The operator represents the character trajectory by calculating the sum of the variances of all feature channels as a variance vector. This is the control coefficient for the intensity of uncertainty penalty.

6. The method according to claim 4, characterized in that, The process of randomly sampling based on the context-aware feature sequence and the squared difference feature sequence to obtain a sampling feature distribution, and then weighting the attention weight vector and the sampling feature distribution to obtain a context vector, includes: The sequence position noise tensor of each sequence position is sampled based on a standard normal distribution. The sampled feature distribution is obtained by random sampling based on the context-aware feature sequence, the squared difference feature sequence, and the sequence position noise tensor using a fourth calculation formula. The context vector is obtained by weighting the attention weight vector and the sampling feature distribution using the fifth calculation formula. The fourth calculation formula satisfies the following formula: Among them, the The sampling feature distribution of the i-th sequence position at decoding time step t, The mean feature parameter of the i-th sequence position of the context-aware feature sequence at decoding time step t, wherein Let be the variance vector of the i-th sequence position of the squared difference feature sequence at decoding time step t. Let be the sequence position noise tensor of the i-th sequence position at decoding time step t; The fifth calculation formula satisfies the following formula: Among them, the The context vector is N, where N is the total decoding time steps. The value projection matrix, the Let be the attention weight vector at the decoding time step t.

7. The method according to claim 4, characterized in that, The process of obtaining the preliminary identified character trajectory sequence and the decoded character trajectory metadata data sequence based on the context vector includes: The uncertainty perception and recognition module uses a decoder to combine the context vector with the decoder's hidden state to obtain a character fusion information state vector. Based on the character fusion information state vector, character trajectory prediction is performed to obtain the preliminary recognized character for each decoding time step. Based on all the preliminary recognized characters, a preliminary recognized character trajectory sequence is obtained. The decoder of the uncertainty perception and recognition module determines the character metadata data packet for each of the initially identified characters, and generates a sequence of decoded character trajectory metadata data packets based on the character metadata data packets of all the initially identified characters; The character metadata data includes a pre-identified character, a predicted character confidence probability corresponding to the pre-identified character, an attention weight vector corresponding to the context vector, the mean of the average features corresponding to all the attention weight vectors, and the average variance of the average features corresponding to all the attention weight vectors.

8. The method according to claim 1, characterized in that, The optical character recognition model includes a recognition trajectory correction module. The process of performing trajectory verification processing on the initially recognized character trajectory sequence based on the decoded character trajectory metadata data sequence using the optical character recognition model to obtain a trajectory recognition error metadata pattern and a character trajectory editing operation for the trajectory recognition error metadata pattern includes: The recognition trajectory correction module performs vector conversion on the decoded character trajectory metadata data packet sequence to obtain the decoded trajectory metadata sequence. The recognition trajectory correction module uses a multi-head attention mechanism to perform long-range error character trajectory recognition on the decoded trajectory metadata sequence to obtain the trajectory recognition error metadata pattern of the error character. Based on the error metadata pattern, the error character is integrated with the character context trajectory metadata to obtain a reference context-aware diagnostic vector for each error character. The character trajectory editing operation is obtained by the recognition trajectory correction module performing editing operation prediction processing on the incorrectly recognized character based on the reference context-aware diagnostic vector.

9. A computer storage medium, characterized in that, The computer storage medium stores a plurality of instructions adapted for loading by a processor and executing the steps of the method as described in any one of claims 1 to 8.

10. An electronic device, characterized in that, include: A processor and a memory; wherein the memory stores a computer program adapted to be loaded by the processor and to execute the steps of the method as described in any one of claims 1 to 8.

Images