An optical character absence detection method based on anomaly detection

Abstract

The application discloses an optical character missing detection method based on anomaly detection. The method comprises the following steps: pre-processing an input image to obtain a pre-processed image; inputting the pre-processed image into a target detection model to obtain position information and a category label of each character, and cutting out a single character image according to the position information; inputting the single character image and the corresponding category label into an anomaly detection model; the anomaly detection model comprises an encoder, a memory-enhanced bottleneck layer and a decoder connected in sequence; the memory-enhanced bottleneck layer comprises a memory matrix, the memory matrix is logically divided into a plurality of semantically independent feature subspaces, each feature subspace corresponds to a character category, and the like; and the application aims to solve the problems of cross-category semantic confusion and missing detection of slight missing caused by similar character forms in an industrial scene.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for detecting missing optical characters based on anomaly detection. Background Technology

[0002] Optical character recognition (OCR) technology has made significant progress over the past few decades and is widely used in document digitization, license plate recognition, and ticket processing. Modern OCR systems incorporate deep learning techniques, such as convolutional neural networks (CNNs), and under ideal conditions, can achieve near-human level accuracy. However, in practical industrial applications, character loss due to various factors remains a common problem, severely impacting the reliability and practicality of OCR systems.

[0003] On industrial production lines, characters printed on product surfaces may be partially missing due to printing equipment malfunctions, insufficient ink, uneven substrate surfaces, or environmental factors (such as dust or changes in lighting). Traditional detection methods mainly rely on regular template matching or simple pixel comparison. These methods have strict requirements on image quality, lighting conditions, and character position, and have poor adaptability. With the intelligent transformation of manufacturing, production lines are placing higher demands on real-time quality inspection, requiring automated solutions that can quickly and accurately identify missing characters. Furthermore, in critical sectors such as finance and healthcare, the integrity of document characters directly impacts business security, and traditional manual sampling methods are inefficient and prone to overlooking problems.

[0004] The main technical challenges currently facing character missing detection include: First, traditional threshold-based segmentation methods struggle with complex backgrounds and low-contrast images; when character printing quality is poor or background interference is severe, the false positive and false negative rates increase significantly. Second, template-matching methods require pre-creating high-precision templates for each font, size, and layout, lacking flexibility in dynamic production environments and unable to handle character deformation or rotation. Third, while existing deep learning OCR systems offer strong recognition capabilities, they are primarily optimized for recognizing complete characters, with limited ability to detect partially missing characters, and their high computational complexity makes them unsuitable for real-time industrial applications.

[0005] More critically, most existing methods require a large number of labeled anomalous samples (i.e., samples with various character missing cases) for training, while collecting a sufficient number and diversity of anomalous samples in actual production is extremely costly. This need for small-sample and zero-sample anomaly detection makes traditional supervised learning methods difficult to apply. However, existing unsupervised anomaly detection methods (such as autoencoders based on memory reconstruction) face the risk of cross-category semantic confusion when applied to character missing detection. Specifically, when the character '8' has partial stroke missing, the model may incorrectly match it to the normal memory prototype of the character '6', resulting in a smaller reconstruction error and missed detection. Specifically, during feature reconstruction, the model cannot distinguish characters with similar topological structures (e.g., the characters "8" and "6"). When the character "8" has partial stroke missing, its anomalous features are easily incorrectly matched by the model to the normal memory prototype of the character "6", resulting in reconstructed features that are very close to the normal distribution, ultimately causing the model to classify the actual defect as a normal structure, resulting in missed detection. Furthermore, character defects on production lines exhibit diverse forms, ranging from complete absence to partial stroke loss, and from ink diffusion to substrate contamination. Traditional methods struggle to establish a unified detection framework. Achieving accurate detection of various character loss scenarios without relying on a large number of abnormal samples and overcoming cross-category semantic confusion has become a major bottleneck restricting the industrial application of this technology. Summary of the Invention

[0006] In view of this, the purpose of this invention is to propose an optical character missing detection method based on anomaly detection, which aims to solve the problem of cross-category semantic confusion and missed detection of minor missing characters caused by similar character shapes in industrial scenarios.

[0007] According to one aspect of the present invention, an optical character missing detection method based on anomaly detection is provided, the method comprising: The input image is preprocessed to obtain a preprocessed image; The preprocessed image is input into the target detection model to obtain the position information and category label of each character, and a single character image is cropped according to the position information; The single character image and its corresponding category label are input into the anomaly detection model. The anomaly detection model includes an encoder, a memory-enhanced bottleneck layer, and a decoder connected in sequence. The memory-enhanced bottleneck layer contains a memory matrix, which is logically divided into multiple semantically independent feature subspaces, each corresponding to a character category. The encoder encodes the input character image and outputs the original features. The memory-enhanced bottleneck layer generates a category constraint mask based on the input category label, limiting the retrieval of prototype features only from the feature subspace corresponding to the category label in the memory matrix, and performs a weighted summation of the retrieved prototype features to output the reconstructed features. The original features and the reconstructed features are compared, and the cosine similarity between them is calculated. An anomaly detection result is generated based on the cosine similarity to indicate the location of missing characters.

[0008] In the above technical solution, a method for optical character missing detection based on semantic category constraints is proposed. Compared with existing unsupervised anomaly detection methods based on shared memory matrices (such as MemAE), this solution differs in the structure and addressing method of the memory matrix as follows: First, independent feature subspaces are partitioned by category. Existing methods mix and store normal prototypes from different categories in the same memory matrix, leading to overlapping feature spaces. This approach explicitly divides the memory matrix into multiple subspaces, each corresponding to a character category. This structure forces the model to access only the prototype of the current category during reconstruction, avoiding cross-category interference caused by overlapping feature spaces. For samples where the character "8" is missing some strokes but visually resembles "6", the model cannot retrieve the prototype from the "6" subspace, thus maintaining a significant difference between the original and reconstructed features and reducing the probability of missed detections.

[0009] Secondly, the hard addressing constraint driven by category labels. Existing methods rely on unsupervised similarity matching to automatically retrieve prototypes. When the incomplete character features have a higher similarity to the prototype of another category, mismatches are prone to occur. This solution uses the category labels output by the front-end object detection model to generate a mask, resetting the attention weights corresponding to subspaces outside the current category to invalid (e.g., negative infinity), forcing the model to address only within the specified subspace. This constraint is independent of feature similarity calculation, fundamentally blocking the possibility of cross-category retrieval, making the anomaly detection results more robust to partial missing character morphology.

[0010] Third, cosine similarity comparison between original and reconstructed features. Compared to directly comparing pixel differences between the input and reconstructed images, this scheme performs cosine similarity measurement at the feature level. Feature-level comparison has better invariance to low-level image changes such as illumination and contrast, and its computational cost is lower than pixel-level reconstruction, making it suitable for real-time industrial detection scenarios.

[0011] In summary, this scheme effectively suppresses the common cross-category semantic confusion problem in character missing detection by introducing semantic category constraints without requiring a large number of abnormal training samples. Furthermore, the combination of category masking and feature-level comparison improves the accuracy and stability of detection.

[0012] In some embodiments, the memory matrix is ​​logically divided into multiple semantically independent feature subspaces in the following ways: Based on the number N of character categories, the memory matrix will be... The system is divided into N feature subspaces, each with a dimension of D. Each subspace stores the prototype features of normal samples corresponding to the character category.

[0013] In the above technical solution, the partitioning method of the memory matrix is ​​further defined. Compared with the overall memory matrix (all category prototypes are stored in an unordered manner) used in the existing memory-enhanced autoencoder, the matrix structure defined in this solution, which is partitioned by row and indexed by category, has the following differences: Firstly, explicit row-level category mapping. Existing methods typically define only the total number of slots in the memory matrix. However, no fixed category semantics are assigned to the slots; the model freely matches them during addressing. This scheme maps each row of the matrix to a specific character category (e.g., row 1 corresponds to "0", row 2 to "1", ... row 36 to "Z"). This deterministic mapping allows category labels to be directly converted into row indices when the model receives the category labels. At that time, the search scope can be immediately narrowed down to the first... The subspace containing the row does not require additional similarity comparisons to guess the category. This eliminates addressing ambiguity at the data structure level and significantly reduces the probability of incorrect cross-category retrieval.

[0014] Secondly, subspace dimensions are stored independently and in isolation. Each subspace dimension... The prototype features of different categories are consistent and physically stored in non-overlapping matrix row intervals. This structure ensures that prototype updates for each category do not interfere with each other during training—when the model learns a normal sample of "8", only the first row is modified. The parameters within the row subspace remain unchanged, while the parameters in other subspaces (such as "6") remain unchanged. This isolation prevents cross-contamination of features from different classes within the memory matrix. Especially when the number of samples in some classes is unbalanced, the majority class does not squeeze the representation space of the minority class, thus maintaining consistent representativeness of the normal prototypes for all classes.

[0015] Thirdly, the scalability and computational efficiency of row-based partitioning. Matrix dimension. middle, Directly corresponds to the total number of character categories. For fixed feature dimensions, when adding new character categories (such as lowercase letters or special symbols), only the number of rows in the matrix needs to be increased, without changing the overall model architecture; simultaneously, the category constraint mask can be implemented through simple row indexing operations (transforming non-classified characters into their corresponding classes). The attention weights of the rows are reset to negative infinity, and the computational complexity of this operation is O(n). , don’t follow Linear growth is beneficial for rapid inference in real-time industrial monitoring.

[0016] In summary, the row-based memory matrix partitioning method defined in this scheme solidifies category semantics into the model parameter structure in the form of row indices, providing a low-overhead, highly deterministic, and easily extensible implementation foundation for the aforementioned category constraint addressing, and effectively supporting the solution of cross-category confusion problems in anomaly detection.

[0017] In some embodiments, the category constraint mask is generated as follows: When the input category label is L, in the memory matrix, the attention weight position corresponding to the Lth feature subspace is set to a queryable state, and the attention weight positions corresponding to all other non-L category feature subspaces are set to negative infinity, so that the model can only retrieve prototype features from the Lth feature subspace.

[0018] In the above technical solution, the specific generation method of the category constraint mask is further defined. Compared with the full-category soft attention (all memory slots participate in similarity calculation, and the weights decay naturally through softmax) used in existing unsupervised anomaly detection methods (such as MemAE), the mask generation method defined in this solution has the following differences: First, it uses hard masking rather than soft attenuation. In existing methods, even if the model wants to mask prototypes from other categories, it can only adjust naturally through similarity scores. If the features of the missing character unexpectedly have a high similarity to a prototype from another category, its attention weights may still be activated. This approach explicitly masks non-current category prototypes. The attention weights for each category are reset to negative infinity, and their output probability is strictly zero after softmax computation. This masking mechanism prevents mismatches for partially missing "8" characters, even though their encoded features have a cosine similarity of 0.9 to the prototype of "6". Because the mask has reset the weights corresponding to the "6" subspace to negative infinity, the model cannot access the prototype.

[0019] Secondly, label-based deterministic indexing eliminates the need for additional similarity comparisons. Existing methods require calculating the similarity between the query vector and all memory slots during the addressing process, resulting in a complexity of O(n log n). This solution utilizes category labels. Directly locate to the For each subspace, only the similarity of prototypes within that subspace needs to be calculated. On one hand, the computational cost is reduced from the entire matrix to a single subspace, compared to the total number of categories. Decoupling facilitates real-time industrial detection; on the other hand, the retrieval process is no longer affected by the similarity of other types of prototypes, avoiding the fluctuation of attention weights caused by similarity value fluctuations, and improving the stability of inference.

[0020] Third, the numerical implementation of the negative infinity mask is universal. This operation has standard implementations in different deep learning frameworks (such as adding a minimum value). (To the corresponding position). The implementation cost of this design is extremely low, it does not require modification of the underlying structure of the model, it is easy to quickly modify and deploy on existing MemAE class models, and it has good engineering practicality.

[0021] In summary, this mask generation method, by hard-shielding non-target subspaces and label-driven deterministic indexes, ensures the execution of category constraints from a mechanism perspective, and has clear advantages in improving detection reliability, reducing computational overhead, and simplifying engineering implementation.

[0022] In some embodiments, the training phase of the anomaly detection model includes: Only normal samples with complete character structures are used as training data; Freeze all parameters of the encoder and update only the parameters of the memory enhancement bottleneck layer and the decoder; End-to-end training is performed using a weighted joint loss of the reconstruction loss function and the memory entropy loss function.

[0023] The above technical solution defines the training strategy for the anomaly detection model, aiming to enable the model to learn the pure feature distribution of normal characters in each category without relying on anomalous samples, while ensuring the stable establishment of the category isolation properties of the memory matrix. Compared with the "encoder-decoder joint optimization" training paradigm commonly used in existing memory-enhanced autoencoders (MemAE) and their variants, the training phase of this solution has the following differences: First, the encoder parameters are frozen, and only the bottleneck layer and decoder are updated. Existing methods typically allow the encoder to continuously adjust its feature extraction method during training as the reconstruction target changes. This can lead to the encoder "masking" subtle anomalous features to cater to the reconstruction task, reducing the sensitivity of subsequent anomaly detection. This approach completely freezes the encoder parameters, maintaining a fixed feature mapping function. This constraint forces the memory-enhanced bottleneck layer and decoder to adapt to a constant encoded feature space, making it impossible to reduce training loss by adjusting the encoder. This means that the encoded features of any input sample (whether normal or anomalous) remain consistent before and after training, and the sensitivity of anomaly detection depends entirely on whether the bottleneck layer can retrieve a matching prototype from memory. When an anomalous sample is input during testing, its encoded features fall in the same space as the encoded features of normal samples of the same class during training. However, because the anomalous region cannot be well reconstructed from the normal prototype of the same class, the reconstruction error is significantly amplified.

[0024] Secondly, only normal samples are used, eliminating the need for anomaly labeling. This feature is a standard setting for unsupervised anomaly detection, but this approach combines it with a frozen encoder strategy, creating a synergistic effect: because the encoder is frozen, the model cannot "overfit" to normal sample variations (such as slight printing unevenness) in the training set by changing the feature extraction method, thus forcing the memory matrix to store more generalizable prototypes. This combination makes the model more tolerant of intra-class differences in normal samples (such as different fonts and stroke thicknesses) while maintaining sensitivity to missing regions, eliminating the need to retrain the encoder for each font or printing condition in actual production lines.

[0025] Third, weighted joint training of reconstruction loss and memory entropy loss. Reconstruction loss (such as region focus cosine loss) ensures that the reconstructed features output by the bottleneck layer are highly consistent with the original features of the encoder; memory entropy loss encourages sparsity of attention weights, avoiding the model averaging features from multiple different prototypes. Weighted joint training enables the bottleneck layer to maintain high-fidelity reconstruction while forming a sparse addressing pattern where "each input activates only a very small number of the most relevant memory slots". When combined with the frozen encoder, this sparsity further enhances the purity of the category subspace: normal samples from different categories will explicitly activate different memory slots within their respective subspaces without cross-category interference. This serves as a warm-up for the category constraint mask in the subsequent inference stage, enabling the model to have built-in category discrimination capabilities during the training phase.

[0026] In summary, this training phase establishes an anomaly detection model with constant encoding features, sparse addressing, and a pure category subspace by freezing the encoder, optimizing only the bottleneck layer and decoder, and combining reconstruction and entropy loss, without using anomalous samples.

[0027] In some embodiments, the reconstruction loss function is a region focus cosine loss function, used to measure the similarity between the output features of the encoder and the decoder; The memory entropy loss function is used to constrain the attention weights of the memory enhancement bottleneck layer so that they satisfy the sparse distribution characteristics.

[0028] In the above technical solution, the two loss functions used in the training phase of the anomaly detection model are further defined. Together, they constitute a weighted joint loss, guiding the model to learn high-fidelity reconstruction capabilities of normal characters and class-pure addressing patterns without relying on anomalous samples. Compared with the mean squared error (MSE) loss or standard cosine similarity loss commonly used in existing anomaly detection methods, the region focus cosine loss function defined in this solution has the following differences: First, MSE treats the reconstruction error of each pixel equally, while standard cosine similarity only provides a global or feature vector-wise similarity measure, both lacking the ability to focus on local anomalous regions. This solution introduces a focus mechanism, assigning higher weights to local regions with lower similarity (corresponding to potential anomalous locations), causing the loss function to focus more on regions difficult to reconstruct during training. For normal samples, this loss prompts the model to improve the reconstruction accuracy of all regions (especially detailed regions prone to subtle fluctuations); while for anomalous samples during testing, because the model has already imposed stronger constraints on low-similarity regions during training, the reconstruction error caused by anomalous regions will be more significant, thereby improving the sensitivity of anomaly detection. Furthermore, the "region" characteristic allows the loss function to preserve spatial location information, generating pixel-level anomaly heatmaps rather than a single global score.

[0029] Secondly, a memory entropy loss function is introduced to constrain the sparsity of attention weights. Without such constraints, the model may tend to average and weight query features with multiple memory prototypes of different categories, leading to semantic ambiguity in the reconstructed features and weakening the isolation effect of the category subspaces. The memory entropy loss, by minimizing the entropy value of the attention weight distribution, forces the model to activate only a very small number of the most relevant memory slots during addressing. This constraint strengthens the sparse addressing pattern of the memory matrix, ensuring that each input sample is mapped to the most representative prototype in its category subspace, rather than a mixture of multiple prototypes. When used in conjunction with the aforementioned category constraint mask, sparsity further guarantees that the reconstructed features originate from a single or a few similar prototypes, effectively avoiding feature confusion caused by cross-category averaging (e.g., reconstructing a missing "8" into a mixture of "6" and "8").

[0030] In summary, the two loss functions complement each other in weighted joint training: the region focus cosine loss improves reconstruction accuracy and amplifies anomaly differences, while the memory entropy loss maintains the determinism of addressing and the purity of categories.

[0031] In some embodiments, comparing the original features with the reconstructed features, calculating the cosine similarity between the two, and generating an anomaly detection result indicating the location of missing characters based on the cosine similarity, further includes: Extract the self-attention matrix of the last layer of the decoder and calculate the attention entropy at each pixel position; The output features of the encoder and decoder at multiple levels are extracted, and the cosine similarity of each level is calculated to generate a multi-scale anomaly heatmap. The attention entropy is used as a confidence weight and fused with the multi-scale anomaly heatmap to obtain a fused anomaly map.

[0032] In the above technical solution, after obtaining the cosine similarity between the original features and the reconstructed features, the specific generation method of the anomaly detection results is further defined. This process aims to utilize the model confidence information represented by attention entropy to suppress the interference of background noise on the anomaly heatmap and improve the accuracy of anomaly localization. Compared with existing anomaly detection methods based on reconstruction error that directly use single-layer or simple averaging multi-scale difference maps, the fusion method defined in this solution has the following differences: First, we introduce self-attention entropy as a confidence weight. Existing methods (such as MemAE and STPM) typically rely solely on the differences between encoder and decoder features (such as cosine similarity or Euclidean distance) to generate heatmaps, without considering the model's own confidence in these differences. Our approach extracts the self-attention matrix from the last layer of the decoder and calculates the attention entropy at each pixel location. Attention entropy reflects the concentration of the model's attention distribution when focusing on a pixel: low entropy indicates the model focuses on a few key locations (high certainty), while high entropy indicates scattered attention (high uncertainty). Even if background regions exhibit certain feature differences due to texture randomness, their attention entropy is often high (the model is not confident that the difference is meaningful), while true character-missing regions, due to the disruption of normal structure, cause the model's attention to focus on abnormal edges, resulting in lower entropy values. Therefore, using attention entropy as a confidence weight to modulate the heatmap can effectively suppress false detections caused by background texture while preserving or even enhancing the response to true defective regions.

[0033] Secondly, the choice of the computation location for the decoder's self-attention entropy. The self-attention matrix is ​​taken from the last layer of the decoder. The features of this layer have undergone memory correction and multiple transformations, and have already incorporated the semantic information after class constraint reconstruction. Compared to using the encoder or shallow attention, the attention of the last layer of the decoder focuses more on the global contextual relationships required for the final reconstruction, and its entropy value better reflects the model's uncertainty about the output. This design aligns the confidence weights and reconstruction errors at the semantic level, making the fusion more reasonable.

[0034] In summary, this scheme effectively distinguishes between pseudo-differences caused by background texture and true character missing differences by fusing attention entropy as a confidence weight with a multi-scale cosine similarity heatmap, while maintaining a high recall rate and significantly reducing the false detection rate.

[0035] In some embodiments, the fusion is specifically performed as follows: in, , The encoder and decoder are respectively the first Layer output features, For cosine similarity, This is the normalized attention entropy. For upsampling operation, The number of feature layers, This is the final anomaly graph.

[0036] In the above technical solution, the specific calculation method of the fused anomaly map is further defined; compared with the multi-scale fusion strategy of simple summation, maximum value or unweighted average used in existing anomaly detection methods, the fusion formula defined in this solution has the following differences: First, the multiplicative fusion coefficient Instead of additive bias, existing methods often use confidence levels when introducing confidence. The additive form's result is significantly affected by the absolute value. This formula will... As a multiplier factor, it directly scales the difference values. Attention entropy The lower the attention entropy (closer to 0), the more focused the model's attention is on the current pixel (high confidence), with the coefficient approaching 1 and the difference value hardly amplified. The higher the attention entropy (closer to 1), the more dispersed the model's attention (low confidence), with the coefficient approaching 2 and the difference value amplified to nearly twice its original value. The multiplicative structure ensures that the contribution weight of high-confidence regions to the final anomaly map is significantly higher than that of low-confidence regions, while additive bias cannot achieve this relative amplification effect.

[0037] Secondly, the method uses a layer-by-layer weighted upsampling and averaging approach, rather than merging and then upsampling. Some methods first upsample the difference maps of each layer to the same size and then calculate the average, but the weighting coefficients are only calculated for a certain layer or the last layer, and are directly applied to all layers. This formula applies weights to each layer... After calculating the difference value independently, multiply by the same factor. (This entropy comes from the last layer of the decoder), and then it is upsampled. Since the attention entropy reflects the global attention distribution of the final layer of the decoder, applying it uniformly to all layers is equivalent to modulating the difference map at each scale with a common confidence map. This maintains the adaptability of multi-scale information to defect size and ensures the consistency of confidence standards. Using different entropy values ​​for different layers would introduce inconsistencies in confidence between scales, potentially leading to excessive suppression of small-scale differences.

[0038] Thirdly, As the difference value, its range is fixed. Compared to mean squared error (whose value range depends on the feature magnitude), cosine similarity difference is scale-invariant, and its complement... The angular deviation was quantified. This difference value was multiplied by a multiplicative factor. After multiplication, it still remains Within the specified range, the values ​​are stable, requiring no additional normalization layer, which is beneficial for setting uniform thresholds across samples and categories. Simultaneously, the upsampling operation maintains spatial continuity, ultimately... It can directly output an anomaly heatmap with the same resolution as the input image, which facilitates subsequent pixel-by-pixel determination.

[0039] In summary, this fusion formula achieves confidence modulation of reconstruction differences through multiplicative weighting, single-entropy multi-scale sharing, and a fixed-range difference metric, thereby improving the robustness of anomaly localization with minimal computational overhead.

[0040] In some embodiments, the method further includes a post-processing step: Based on the F1 score maximization strategy, multiple candidate thresholds are traversed, and the precision and recall corresponding to each threshold are calculated. The threshold with the highest F1 score is selected as the optimal classification threshold. If there are no positive samples in the current sample, the maximum value in the fusion anomaly map is used as the optimal classification threshold. When the pixel value in the fused anomaly map is greater than or equal to the optimal threshold, it is determined to be an abnormal region; otherwise, it is determined to be a normal region, and a binary anomaly mask is generated.

[0041] The above technical solution further introduces a post-processing step. This step aims to transform continuously distributed anomaly scores into discrete pixel-level classification results. Compared with the fixed thresholds (such as the 95th percentile of the reconstruction error distribution of normal samples) or heuristic thresholds based on training set statistics commonly used in existing anomaly detection methods, the post-processing method defined in this solution has the following differences: Firstly, the threshold selection is adaptively based on maximizing the F1 score. Fixed thresholds or quantile methods rely on assumptions about the error distribution of normal samples, and the threshold may become ineffective when the distribution of anomalies in the test data changes. This approach utilizes positive and negative samples from the validation set (or the test set itself, if with a small number of labeled samples) to calculate precision and recall by traversing candidate thresholds, directly optimizing the F1 score. This strategy does not assume a specific form of the error distribution, but rather uses actual detection performance as the optimization objective, automatically adapting to the threshold requirements of different character datasets and different defect severity levels. For character missing detection tasks, the missing area may range from a tiny gap to the complete missing character. Maximizing the F1 score can balance the costs of false positives (classifying normal areas as defects) and false negatives (missing real defects), obtaining the optimal decision boundary that matches the current scenario.

[0042] Secondly, a fallback threshold is provided for cases without positive examples. Existing methods typically assume that the validation set must contain both positive and negative examples. However, in industrial online inspection, batches may consist entirely of qualified products (without any abnormal samples). In such cases, the threshold based on maximizing F1 cannot be calculated (recall cannot be defined). This solution limits the threshold to the maximum value in the fused anomaly graph if no positive examples exist in the current sample. This logic ensures that the algorithm can still run with batches of purely normal samples. By setting the threshold to the maximum value in the current anomaly graph, the binary mask for that batch is entirely zero (no anomaly region), avoiding system crashes due to a lack of positive examples or a large number of false positives caused by arbitrarily set thresholds.

[0043] Third, the directness and interpretability of generating binary masks through pixel-by-pixel comparison. The results of threshold segmentation are directly output as binary anomaly masks. These masks can be overlaid with the original image to display the contours of the abnormal regions, providing quantitative information for subsequent manual re-inspection or defect statistics. They also facilitate the direct calculation of pixel-level evaluation metrics (such as IoU) with the labeled ground truth.

[0044] In summary, this post-processing step solves the practicality problems in industrial scenarios, such as the difficulty in pre-fixing thresholds and the potential lack of abnormal samples in the training set, through F1-driven threshold adaptation and a fallback mechanism for no positive examples. At the same time, it outputs intuitive detection results in the form of a binary mask.

[0045] According to another aspect of the present invention, an optical character missing detection system based on anomaly detection is provided, the system comprising: The preprocessing module is used to preprocess the input image to obtain a preprocessed image; The detection and localization module is used to input the preprocessed image into the target detection model, obtain the position information and category label of each character, and crop out a single character image based on the position information; An anomaly detection module includes an encoder, a memory-enhanced bottleneck layer, and a decoder connected in sequence. The memory-enhanced bottleneck layer contains a memory matrix, which is logically divided into multiple semantically independent feature subspaces, each corresponding to a character category. The anomaly detection module receives a single character image and its corresponding category label. The encoder encodes the input character image and outputs the original features. The memory-enhanced bottleneck layer generates a category constraint mask based on the input category label, limiting the retrieval of prototype features only from the feature subspace corresponding to the category label in the memory matrix. The retrieved prototype features are then weighted and summed to output reconstructed features. An anomaly detection module is used to compare the original features with the reconstructed features, calculate the cosine similarity between the two, and generate anomaly detection results to indicate the location of missing characters based on the cosine similarity.

[0046] In order to better utilize the above method, this application proposes an optical character missing detection system based on anomaly detection. Each module corresponds to a step of the above method, and its specific principle has been described above and will not be repeated here.

[0047] According to another aspect of the present invention, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the above-described method.

[0048] In the above technical solution, to better operate and use the method, it is stored in a computer-readable storage medium and implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated upon here. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a flowchart illustrating an embodiment of an optical character missing detection method based on anomaly detection according to the present invention. Figure 2 This is a schematic diagram of the image preprocessing process of an embodiment of an optical character missing detection method based on anomaly detection according to the present invention; Figure 3 This is a schematic diagram of the anomaly detection model architecture of an embodiment of the optical character missing detection method based on anomaly detection according to the present invention; Figure 4 This is a schematic diagram of the encoder architecture of an embodiment of an optical character missing detection method based on anomaly detection according to the present invention; Figure 5 This is a schematic diagram of the memory enhancement bottleneck layer processing logic of an embodiment of an optical character missing detection method based on anomaly detection according to the present invention; Figure 6 This is a schematic diagram of the anomaly detection model during the training phase of an embodiment of an optical character missing detection method based on anomaly detection according to the present invention. Figure 7 This is a schematic diagram of an embodiment of an optical character missing detection device based on anomaly detection according to the present invention. Detailed Implementation

[0051] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be particularly noted that the following embodiments are for illustrative purposes only and do not limit the scope of the invention. Similarly, the following embodiments are only some, not all, embodiments of the present invention, and all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0052] Example 1 This embodiment of character missing detection first requires necessary preprocessing operations on the input image, including basic image processing steps such as grayscale conversion, contrast enhancement, image denoising, and image binarization, to ensure the image quality of subsequent processing. The preprocessed image is then fed into the character detection module, which uses an object detection algorithm to accurately obtain the position information and preliminary character category information of each character in the image. The output format includes character category, position coordinates, and size. Based on this positioning information, the system accurately crops each character region and performs standardization processing, including size normalization to a uniform size and grayscale value normalization, to ensure that all character images have a consistent format and size specifications. After character standardization, the system collects and filters normal samples as the sole dataset for model training. Normal samples refer to high-quality character images with complete structure, clear strokes, and no missing or deformed parts.

[0053] During the model training phase, the system uses only normal samples as training data and employs unsupervised learning based on semantic category constraints to construct the anomaly detection model. During training, the model not only learns the general feature distribution of normal characters but also utilizes the true category labels of the input characters to guide the addressing and updating of the memory matrix. This allows for the construction of an independent and pure prototype feature subspace for each character category (such as "0", "8", "A", etc.) in the memory space. This strategy avoids the difficulty of collecting a large number of anomaly samples and fundamentally solves the problem of missed detections caused by cross-category feature confusion.

[0054] In the model detection phase, the input image first undergoes the same preprocessing and character extraction steps as in the training phase. Then, each character region is fed into the classification branch of the anomaly detection model to obtain the predicted category information in real time. This predicted category information is used as a constraint to precisely lock the prototype feature subspace of the corresponding category in the memory matrix. The memory matrix is ​​reconstructed only based on the prototype features within this subspace. The system compares the input character image with the reconstruction results under category constraints, calculating the feature-level difference and the attention entropy of the self-attention layer, thereby generating a high-precision anomaly heatmap. By setting a judgment threshold, the system automatically identifies regions with large reconstruction errors and marks them as anomalous characters or anomalous stroke regions. The model further outputs an anomaly severity score for each character to quantify the severity of missing or damaged parts. When the detection results exceed the set threshold, the system can visually display the outline of the anomalous region on the image, generating a visualized detection image.

[0055] Specifically as follows: Please see Figure 1 A method for detecting missing optical characters based on anomaly detection, the method comprising: S1. Preprocess the input image to obtain a preprocessed image; S2. Input the preprocessed image into the target detection model to obtain the position information and category label of each character, and crop out a single character image based on the position information; For example, this example will illustrate steps S1 and S2 together.

[0056] Please see Figure 2 Image preprocessing is the foundation of the entire detection process, primarily responsible for optimizing the original input image to provide high-quality image data for subsequent character detection and anomaly analysis. First, grayscale conversion is performed, transforming the color image into a grayscale image, unifying color channels, and reducing computational complexity. Then, an adaptive histogram equalization algorithm is used to enhance image contrast, specifically optimized for scenes with uneven lighting or low contrast. In the denoising stage, a local mean denoising algorithm is used to effectively eliminate noise interference while preserving character edge details. Finally, an adaptive thresholding method is used for binarization.

[0057] The generated binarized image is input into the object detection algorithm for character localization. The object detection algorithm employs a deep learning detection model based on the YOLO architecture, which integrates character category recognition. The model input is the preprocessed binarized image, and the output is a set of character images cropped according to the detection location. The system simultaneously identifies the precise location information, size parameters, and semantic category labels of each character in the image through the object detection algorithm. Based on the detection results, the system precisely crops the original image, generating a standardized image subset containing independent character regions. This standardized subset carries the character category information assigned by the object detection model, serving as input data for the subsequent anomaly detection model to implement "semantic category constraint" addressing, ensuring that each character image possesses correct semantic information before being sent for anomaly detection.

[0058] S3. Input the single character image and its corresponding category label into the anomaly detection model. The anomaly detection model includes an encoder, a memory enhancement bottleneck layer, and a decoder connected in sequence. The memory enhancement bottleneck layer contains a memory matrix, which is logically divided into multiple semantically independent feature subspaces, each corresponding to a character category. The encoder encodes the input character image and outputs the original features. The memory enhancement bottleneck layer generates a category constraint mask based on the input category label, limiting the retrieval of prototype features only from the feature subspace corresponding to the category label in the memory matrix, and performs a weighted summation of the retrieved prototype features to output the reconstructed features. In this embodiment, the memory matrix is ​​logically divided into multiple semantically independent feature subspaces in the following ways: Based on the number N of character categories, the memory matrix will be... The system is divided into N feature subspaces, each with a dimension of D. Each subspace stores the prototype features of normal samples corresponding to the character category.

[0059] In this embodiment, the category constraint mask is generated as follows: When the input category label is L, in the memory matrix, the attention weight position corresponding to the Lth feature subspace is set to a queryable state, and the attention weight positions corresponding to all other non-L category feature subspaces are set to negative infinity, so that the model can only retrieve prototype features from the Lth feature subspace.

[0060] In this embodiment, the training phase of the anomaly detection model includes: Only normal samples with complete character structures are used as training data; Freeze all parameters of the encoder and update only the parameters of the memory enhancement bottleneck layer and the decoder; End-to-end training is performed using a weighted joint loss of the reconstruction loss function and the memory entropy loss function.

[0061] In this embodiment, the reconstruction loss function is the region focus cosine loss function, which is used to measure the similarity between the output features of the encoder and the decoder; The memory entropy loss function is used to constrain the attention weights of the memory enhancement bottleneck layer so that they satisfy the sparse distribution characteristics.

[0062] For example, the anomaly detection model comprises three parts: an encoder, a memory-enhanced bottleneck layer based on semantic category constraints, and a decoder. The encoder encodes the input image information, mapping the original image to a latent space to extract high-dimensional semantic features. The memory-enhanced bottleneck layer is the core for achieving semantic isolation and controlled reconstruction, used to apply category constraints and prototype matching to the encoded features. The decoder decodes and reconstructs the memory-corrected latent features, generating feature outputs consistent with the distribution of normal samples in the current category, used for identifying anomalous regions. During this process, the system can extract intermediate features at the encoder and decoder stages for similarity measurement, thereby achieving accurate identification and localization of anomalous regions.

[0063] The encoder model used in this example is as follows: Figure 3 As shown, the overall network is built based on a self-attention mechanism and adopts an end-to-end architecture consisting of an image patch embedding layer, a positional encoding module, a Transformer encoding layer, and an output layer. The model first receives an input image of size 3×128×128 and divides it into 8×8 fixed-size image patches, each patch being 3×16×16. Then, the positional encoding module maps all image patches to a unified feature dimension of 384, resulting in 64 image patch features, forming the final input sequence of 64×384. The core part is a 12-layer stacked Transformer encoder structure. Each encoder consists of a multi-head self-attention module and a feedforward fully connected network. The self-attention module has 12 attention heads used to capture global dependencies between image patches; the feedforward network includes two linear transformation layers and a GELU activation function to enhance the model's non-linear expressive power. Please refer to [link to relevant documentation]. Figure 4 Each Transformer encoder is equipped with residual connections and layer normalization structures to enhance gradient stability and accelerate model convergence. This example uses only the target layers of eight encoder blocks (layers 2 through 9) as output for similarity analysis and feature reconstruction, thereby achieving accurate detection and recognition of image anomalies.

[0064] The decoder used in this example consists of eight standard Transformer modules, maintaining the same overall structure as the encoder, but with a lightweight design in the attention structure; each self-attention module contains only six attention heads. The decoder primarily restores the semantically clean feature maps corrected by the memory-enhanced bottleneck layer to representations in the original space, generating reconstructed results for anomaly localization. This design significantly reduces computational complexity and memory usage while preserving the model's reconstruction capabilities, providing efficient and reliable feature restoration for anomaly detection tasks.

[0065] The bottleneck layer used in this example implements a memory addressing mechanism based on semantic category constraints. The core structure of the bottleneck layer is a learnable memory matrix. (in N This represents the total number of memory slots. D (As a feature dimension), this matrix is ​​logically divided according to character category into N Each feature subspace is semantically independent. See also... Figure 5 During operation, the bottleneck layer receives high-dimensional feature vectors from the encoder output. Z and the semantic category label of the current character. Label (True values ​​are used during training, and predicted values ​​are used during inference). Then, based on the category labels... Label The system generates a category constraint mask. This mask is in... Label The corresponding memory subspace location is placed on top. 0 In all other non Label Negative infinity is assigned to the subspace location of the category, thus forcibly isolating interference from cross-category features. The model is forced to rely solely on... Label The most matching normal prototypes are selected from the corresponding memory subspaces and weighted summed to obtain semantically pure reconstructed features. This mechanism ensures that even if the input contains anomalies, the decoder can only receive the most ideal normal features within that category for reconstruction, thus greatly enhancing the difference between the input and the reconstructed result in the abnormal region. The overall memory matrix and constraint mask are called attention weights.

[0066] Please see Figure 6 During the model training phase, the system sequentially feeds each input image into the encoder and decoder modules for feature extraction and reconstruction. Both the encoder and decoder consist of eight layers. TransformerModule Composition. After one forward propagation, the encoder and decoder each output eight sets of feature vectors. This example introduces a feature consistency constraint between the encoder and decoder outputs at the same level, measuring the accuracy of feature reconstruction by calculating the similarity loss between them. During training, only the model parameters of the decoder are updated, while the encoder parameters are kept frozen to ensure that the features extracted by the encoder are stable and reliable, and to guide the decoder to gradually learn how to recover a high-fidelity image or feature distribution from the encoded features.

[0067] This example uses a region focus loss function based on cosine similarity to calculate the region focus cosine loss between two feature tensors a and b, focusing on regions with low similarity through a focus mechanism.

[0068] The reconstruction loss function is the region focus cosine loss function, and the calculation formula is as follows:

[0069]

[0070] In the formula, These are lists of feature tensors output by the encoder and decoder, respectively. , ; These represent the encoder and decoder at the 1st and 2nd respectively. Layer The values ​​of each feature element; The cosine similarity function is used; α is the focus weight index. This represents the dynamic threshold within the corresponding tensor. This represents the total number of feature levels involved in the calculation. Indicates the first The total number of feature elements contained in the layer feature tensor; Furthermore, to ensure the effective operation of the memory addressing mechanism based on semantic category constraints, this example introduces a memory entropy loss. This loss is based on the attention weights of the memory matrix. The loss of memory entropy was calculated. The definition is as follows:

[0071] In the formula, Indicates the number of input images; For the first The memory matrix and attention weight vector corresponding to each input sample; This represents the normalized attention weights. The specific formula is shown below.

[0072]

[0073] in, , .

[0074] The purpose of this loss is to minimize the entropy of the attention weights, thereby forcing the model to make sparse selections during addressing. It constrains the model to match feature queries with a single or very few most relevant memory prototypes, avoiding the averaging of query features with multiple unrelated prototypes, ensuring the semantic purity of feature reconstruction, and is an important guarantee for the stable operation of the memory matrix in anomaly detection.

[0075] This example uses a multi-task weighted joint loss function to train the model end-to-end, aiming to simultaneously optimize character reconstruction accuracy and memory addressing sparsity. Total Loss It consists of two core parts: reconstruction loss and memory entropy loss Let the reconstruction loss weight be... The memory entropy loss weight is ,satisfy .

[0076]

[0077] This ratio is an optimal value obtained through experimental verification. Experiments show that when the reconstruction loss weight is below 0.7, the model's anomaly detection accuracy decreases; when the memory entropy loss weight is below 0.1, the sparsity constraint effect is not significant, and cross-class confusion is not effectively suppressed. 0.8 / 0.2 achieved the best F1 score in multiple tests. However, this ratio is not the only possible value and can be adjusted within a certain range according to specific application scenarios. Reconstruction Loss The primary task of model training is to ensure that the decoder can reconstruct normal sample features highly consistent with the encoder's original features from the prototype features retrieved from the memory matrix. This loss directly determines the model's reconstruction accuracy of normal samples, thus affecting the significance of the reconstruction error of abnormal samples, and is the core guarantee of anomaly detection performance. Memory entropy loss This is an auxiliary regularization term that encourages the model to sparsify attention weights when retrieving memory prototypes, avoiding the model averaging prototype features from multiple different categories and thus reducing the risk of cross-category semantic confusion. This loss does not directly participate in optimizing reconstruction accuracy but serves as a constraint guiding the learning direction of the memory matrix. Therefore, reconstruction loss should be dominant, with entropy loss as an auxiliary constraint; the weight ratio of the two should reflect the design principle of "reconstruction as the primary focus and entropy constraint as secondary." It should be noted that the values ​​of 0.8 and 0.2 are experimentally verified optimal values, but not the only fixed values. In practical applications, those skilled in the art can adjust the values ​​based on specific character datasets, the number of character categories, image quality, etc., while maintaining... Adjust the weighting coefficients appropriately within the range, for example... or The basic technical effects of this invention can still be achieved. All weighted joint training methods that primarily use reconstruction loss and secondarily use entropy loss fall within the scope of protection of this invention.

[0078] S4. Compare the original features with the reconstructed features, calculate the cosine similarity between the two, and generate an anomaly detection result to indicate the missing character position based on the cosine similarity.

[0079] In this embodiment, comparing the original features with the reconstructed features, calculating the cosine similarity between the two, and generating an anomaly detection result indicating the location of missing characters based on the cosine similarity, further includes: Extract the self-attention matrix of the last layer of the decoder and calculate the attention entropy at each pixel position; The output features of the encoder and decoder at multiple levels are extracted, and the cosine similarity of each level is calculated to generate a multi-scale anomaly heatmap. The attention entropy is used as a confidence weight and fused with the multi-scale anomaly heatmap to obtain a fused anomaly map.

[0080] In this embodiment, the specific method of fusion is as follows: in, , The encoder and decoder are respectively the first Layer output features, For cosine similarity, This is the normalized attention entropy. For upsampling operation, The number of feature layers, This is the final anomaly graph.

[0081] In this embodiment, the method further includes a post-processing step: Based on the F1 score maximization strategy, multiple candidate thresholds are traversed, and the precision and recall corresponding to each threshold are calculated. The threshold with the highest F1 score is selected as the optimal classification threshold. If there are no positive samples in the current sample, the maximum value in the fusion anomaly map is used as the optimal classification threshold. When the pixel value in the fused anomaly map is greater than or equal to the optimal threshold, it is determined to be an abnormal region; otherwise, it is determined to be a normal region, and a binary anomaly mask is generated.

[0082] For example, in the inference part, to address the background noise interference caused by relying solely on feature similarity, this example proposes a multi-scale feature fusion strategy guided by attention entropy. The process is as follows: calculate the cosine similarity of the encoder and decoder feature tensors output from the same image and generate a multi-scale anomaly heatmap; then extract the self-attention matrix of the last layer of the decoder. Calculate the position of each pixel attention entropy Finally, the multi-scale anomaly heatmap and attention entropy are fused together.

[0083]

[0084]

[0085] In the formula, , The encoder and decoder are respectively the first Layer output features, For cosine similarity, This is the normalized attention entropy; For upsampling operations, bilinear interpolation is used. The number of feature layers; This is the final anomaly map. By introducing attention entropy, the system can distinguish subtle differences caused by background texture. Although the background textures differ, the model's attention remains stable. This fusion method is equivalent to adding a "confidence filter" to the heatmap, significantly reducing the false detection rate in complex industrial backgrounds.

[0086] The post-processing section is responsible for refining and finalizing the anomaly detection results output by the model. This example uses maximizing the F1 score to determine the optimal classification threshold, where the classification refers to two categories: NG and OK. First, the input ground truth mask and predicted scores are flattened and converted into a one-dimensional array for unified calculation. Then, the system performs a full negative sample check on the ground truth mask to determine if all samples in the current sample are negative (i.e., no positive examples exist). If positive samples are present, the system calculates the precision and recall corresponding to each threshold using precision-recall curves, and further calculates the F1 score based on their relationship to measure the model's overall performance under different thresholds. By comparing the F1 scores corresponding to all thresholds, the system automatically selects the threshold with the highest F1 score as the optimal threshold for subsequent anomaly detection. When there are no positive samples in the detection results, the system automatically uses the maximum predicted score as the threshold, thus ensuring stable judgment capability even in special cases.

[0087] Input preprocessing:

[0088]

[0089] Decision-making logic:

[0090] F1 score calculation:

[0091] Precision P:

[0092] Recall rate R:

[0093] in, This represents the actual label mask. Indicates the anomaly graph score. and These represent the transformation of two two-dimensional matrices into one-dimensional arrays. Represents a series of thresholds For each of these thresholds Each corresponds to a set of precision rates and recall rate . These represent the threshold values ​​respectively. The number of true positives, false positives, and false negatives is calculated. The F1 score is calculated by iterating through all thresholds, and the threshold with the highest F1 score is selected as the optimal threshold.

[0094] After obtaining the optimal threshold Then, this method iterates through and calculates for each pixel in the anomaly map. The system first compares the predicted anomaly score map with the calculated optimal threshold pixel by pixel. When the pixel score is greater than or equal to the threshold, the pixel is determined to be an anomaly region; otherwise, it is determined to be a normal region. In this way, accurate segmentation and labeling of anomaly regions can be achieved at the pixel level.

[0095] This process can be formally represented as:

[0096] In the formula, The coordinates in the anomaly graph are... pixel score, This represents the threshold corresponding to the maximum F1 score. This is the generated binary anomaly mask.

[0097] In summary, this invention provides a character missing detection method based on semantic category constraints, aiming to solve the problems of cross-category semantic confusion and missed detection of minor missing characters caused by similar character shapes in industrial scenarios. The method mainly includes the following steps: First, the input image is preprocessed and then fed into the target detection model, simultaneously acquiring the precise location and semantic category label of the character, and standardizing the character cropping. During the model training phase, unsupervised learning is performed using only normal character samples to construct a memory enhancement bottleneck layer based on semantic category constraints. By utilizing the real category labels, the memory matrix is ​​logically divided into multiple semantically independent feature subspaces, storing the standard prototype features of each character category respectively. During the inference phase, the model uses the category labels predicted by the classification branch to generate a sparse category mask, forcing the memory module to perform controlled addressing only within the memory subspace of the corresponding category. This mechanism ensures that even if the input character has severe missing parts (such as a broken "8"), the model is still restricted to using the complete prototype of that category ("8") for reconstruction, thereby generating significant feature differences in the missing region and completely avoiding missed detections caused by incorrect matching to similar characters (such as "6"). Finally, the system generates a multi-scale anomaly heatmap based on feature reconstruction differences and attention entropy, achieving high-precision localization and quantitative evaluation of missing characters. This invention possesses strong anti-interference capabilities and semantic robustness, making it particularly suitable for high-precision industrial character printing quality inspection.

[0098] Example 2 Please see Figure 7 An optical character missing detection system based on anomaly detection, characterized in that the system comprises: The preprocessing module is used to preprocess the input image to obtain a preprocessed image; The detection and localization module is used to input the preprocessed image into the target detection model, obtain the position information and category label of each character, and crop out a single character image based on the position information; An anomaly detection module includes an encoder, a memory-enhanced bottleneck layer, and a decoder connected in sequence. The memory-enhanced bottleneck layer contains a memory matrix, which is logically divided into multiple semantically independent feature subspaces, each corresponding to a character category. The anomaly detection module receives a single character image and its corresponding category label. The encoder encodes the input character image and outputs the original features. The memory-enhanced bottleneck layer generates a category constraint mask based on the input category label, limiting the retrieval of prototype features only from the feature subspace corresponding to the category label in the memory matrix. The retrieved prototype features are then weighted and summed to output reconstructed features. An anomaly detection module is used to compare the original features with the reconstructed features, calculate the cosine similarity between the two, and generate anomaly detection results to indicate the location of missing characters based on the cosine similarity.

[0099] In this embodiment, in order to better utilize the method described in one of the embodiments, this application proposes an optical character missing detection system based on anomaly detection. Each module corresponds to each step of the above method, and its specific principle has been described above and will not be repeated here.

[0100] Example 3 A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in one of the embodiments.

[0101] In this embodiment, to better operate and use the method described in one of the embodiments, the above method is stored in a computer-readable storage medium, and the above method is implemented using a processor. It should be noted that the principle and effect of each step have been described above and will not be elaborated further here.

[0102] The above description is only a part of the embodiments of the present invention and does not limit the scope of protection of the present invention. Any equivalent device or equivalent process transformation made based on the content of the present invention specification and drawings, or direct or indirect application in other related technical fields, are similarly included within the patent protection scope of the present invention.

Claims

1. A method for detecting missing optical characters based on anomaly detection, characterized in that, The method includes: The input image is preprocessed to obtain a preprocessed image; The preprocessed image is input into the target detection model to obtain the position information and category label of each character, and a single character image is cropped according to the position information; The single character image and its corresponding category label are input into the anomaly detection model. The anomaly detection model includes an encoder, a memory enhancement bottleneck layer, and a decoder connected in sequence. The memory enhancement bottleneck layer contains a memory matrix, which is logically divided into multiple semantically independent feature subspaces, each corresponding to a character category. The encoder encodes the input character image and outputs the original features. The memory enhancement bottleneck layer generates a category constraint mask based on the input category label, limiting the retrieval of prototype features only from the feature subspace corresponding to the category label in the memory matrix, and performs a weighted summation of the retrieved prototype features to output the reconstructed features. The original features and the reconstructed features are compared, and the cosine similarity between them is calculated. An anomaly detection result is generated based on the cosine similarity to indicate the location of missing characters.

2. The optical character missing detection method based on anomaly detection as described in claim 1, characterized in that, The memory matrix is ​​logically divided into multiple semantically independent feature subspaces in the following ways: Based on the number N of character categories, the memory matrix will be... The system is divided into N feature subspaces, each with a dimension of D. Each subspace stores the prototype features of normal samples corresponding to the character category.

3. The optical character missing detection method based on anomaly detection as described in claim 1, characterized in that, The category constraint mask is generated as follows: When the input category label is L, in the memory matrix, the attention weight position corresponding to the Lth feature subspace is set to a queryable state, and the attention weight positions corresponding to all other non-L category feature subspaces are set to negative infinity, so that the model can only retrieve prototype features from the Lth feature subspace.

4. The optical character missing detection method based on anomaly detection as described in claim 1, characterized in that, The training phase of the anomaly detection model includes: Only normal samples with complete character structures are used as training data; Freeze all parameters of the encoder and update only the parameters of the memory enhancement bottleneck layer and the decoder; End-to-end training is performed using a weighted joint loss of the reconstruction loss function and the memory entropy loss function.

5. The optical character missing detection method based on anomaly detection as described in claim 4, characterized in that, The reconstruction loss function is the region focus cosine loss function, which is used to measure the similarity between the output features of the encoder and the decoder; The memory entropy loss function is used to constrain the attention weights of the memory enhancement bottleneck layer so that they satisfy the sparse distribution characteristics.

6. The optical character missing detection method based on anomaly detection as described in claim 1, characterized in that, Comparing the original features with the reconstructed features, calculating the cosine similarity between them, and generating an anomaly detection result indicating the location of missing characters based on the cosine similarity, further includes: Extract the self-attention matrix of the last layer of the decoder and calculate the attention entropy at each pixel position; The output features of the encoder and decoder at multiple levels are extracted, and the cosine similarity of each level is calculated to generate a multi-scale anomaly heatmap. The attention entropy is used as a confidence weight and fused with the multi-scale anomaly heatmap to obtain a fused anomaly map.

7. The optical character missing detection method based on anomaly detection as described in claim 6, characterized in that, The specific method of fusion is as follows: in, , The encoder and decoder are respectively the first Layer output features, For cosine similarity, This is the normalized attention entropy. For upsampling operation, The number of feature layers, This is the final anomaly graph.

8. The optical character missing detection method based on anomaly detection as described in claim 6, characterized in that, The method also includes a post-processing step: Based on the F1 score maximization strategy, multiple candidate thresholds are traversed, and the precision and recall corresponding to each threshold are calculated. The threshold with the maximum F1 score is selected as the optimal classification threshold. If there are no positive samples in the current sample, the maximum value in the fusion anomaly graph is used as the optimal classification threshold. When the pixel value in the fused anomaly map is greater than or equal to the optimal threshold, it is determined to be an abnormal region; otherwise, it is determined to be a normal region, and a binary anomaly mask is generated.

9. An optical character missing detection system based on anomaly detection, characterized in that, The system includes: The preprocessing module is used to preprocess the input image to obtain a preprocessed image; The detection and localization module is used to input the preprocessed image into the target detection model, obtain the position information and category label of each character, and crop out a single character image based on the position information; An anomaly detection module includes an encoder, a memory-enhanced bottleneck layer, and a decoder connected in sequence. The memory-enhanced bottleneck layer contains a memory matrix, which is logically divided into multiple semantically independent feature subspaces, each corresponding to a character category. The anomaly detection module receives a single character image and its corresponding category label. The encoder encodes the input character image and outputs the original features. The memory-enhanced bottleneck layer generates a category constraint mask based on the input category label, limiting the retrieval of prototype features only from the feature subspace corresponding to the category label in the memory matrix. The retrieved prototype features are then weighted and summed to output reconstructed features. An anomaly detection module is used to compare the original features with the reconstructed features, calculate the cosine similarity between the two, and generate anomaly detection results to indicate the location of missing characters based on the cosine similarity.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the method of any one of claims 1 to 8.

Images