Industrial character detection and recognition method and device based on YOLO framework and medium

Abstract

The invention belongs to the technical field of character recognition, and provides an industrial character detection and recognition method and device based on a YOLO framework and a medium. The method comprises the following steps: a feature extraction module processes an industrial scene image to obtain a multi-level feature map, and a detection head predicts the spatial position of a single character based on the multi-level feature map to obtain a character rotation frame of the single character; the segmentation head carries out up-sampling on all hierarchical feature maps to the same spatial resolution and carries out splicing on a channel dimension to obtain fusion features; performing high-dimensional mapping on the fused features to obtain a high-dimensional mapping feature map; the identification head generates a sampling grid according to the affine matrix, and a sampling function aligns the high-dimensional mapping feature map to a unified output scale based on the sampling grid to obtain an aligned high-dimensional mapping feature map; and performing sequence modeling and attention decoding processing on the aligned high-dimensional mapping feature map to obtain a predicted character sequence. According to the method, a multi-branch weak supervision training mechanism is utilized, and the recognition capability of the model on unknown characters and new combinations is enhanced.

Description

Technical Field

[0001] This invention relates to the field of character recognition technology, and more specifically, to an industrial character detection and recognition method, apparatus, and medium based on the YOLO framework. Background Technology

[0002] Currently, mainstream technologies in the field of character recognition are mostly based on end-to-end whole string recognition methods, that is, combining convolutional neural networks with sequence modeling to directly output the complete string. These methods can achieve good recognition accuracy in standard scenarios, but they have the following problems and shortcomings in industrial scenarios: Insufficient robustness: In industrial environments, characters are often subject to interference factors such as blurring, occlusion, oil stains, and deformation. Overall string recognition is prone to failure due to local errors, resulting in poor fault tolerance.

[0003] Poor generalization: Existing methods rely on a large number of labeled string samples for training, and are often designed for specific fixed string combinations. When characters or new combinations that have not appeared in the training set appear in real-world scenarios, the recognition accuracy drops significantly.

[0004] The data bottleneck is prominent: the workload of single-character annotation is enormous, and industrial enterprises usually find it difficult to provide large-scale single-character annotation data; while most public datasets are for natural scene text recognition, and the effect of directly transferring them to industrial character recognition is limited.

[0005] Limited training modes: Existing methods lack effective weak supervision mechanisms for utilizing public datasets, making it difficult to establish effective collaboration between a small amount of industrial annotations and a large amount of non-industrial data, resulting in high model training costs and limited effectiveness.

[0006] Therefore, improving the accuracy of industrial character detection and recognition is a technical problem that urgently needs to be solved. Summary of the Invention

[0007] To address the shortcomings and deficiencies of the existing technologies, this invention provides an industrial character detection and recognition method, apparatus, and medium based on the YOLO framework.

[0008] This invention provides an industrial character detection and recognition method based on the YOLO framework, comprising the following steps: Based on the gray coverage optimization mechanism, data augmentation processing is performed on industrial character sample data, and the augmented industrial character sample data is used to perform multi-stage training on the detection head, segmentation head and recognition head. The feature extraction module processes industrial scene images to obtain multi-level feature maps. The detection head predicts the spatial position of a single character based on the multi-level feature maps to obtain the character rotation box of the single character. The segmentation head upsamples all hierarchical feature maps to the same spatial resolution and concatenates them along the channel dimension to obtain fused features; the fused features are then mapped in a high dimension to obtain a high-dimensional mapped feature map. The recognition head generates a sampling grid based on the affine matrix. The sampling function aligns the high-dimensional mapping feature map to a uniform output scale based on the sampling grid to obtain an aligned high-dimensional mapping feature map. Sequence modeling and attention decoding are performed on the aligned high-dimensional mapping feature map to obtain the predicted character sequence. The affine matrix is ​​constructed based on the width, height, and rotation angle obtained after normalizing and scaling the character rotation box.

[0009] The present invention also provides an industrial character detection and recognition device based on the YOLO framework, including a training module, a feature extraction module, a segmentation head, and a recognition head; The training module performs data augmentation processing on industrial character sample data based on a gray coverage optimization mechanism, and uses the augmented industrial character sample data to perform multi-stage training on the detection head, segmentation head and recognition head. The feature extraction module processes the industrial scene image to obtain a multi-level feature map. The detection head predicts the spatial position of a single character based on the multi-level feature map to obtain a character rotation box for the single character. The segmentation head upsamples all hierarchical feature maps to the same spatial resolution and concatenates them along the channel dimension to obtain fused features; the fused features are then mapped in a high dimension to obtain a high-dimensional mapped feature map. The recognition head generates a sampling grid based on the affine matrix. The sampling function aligns the high-dimensional mapping feature map to a uniform output scale based on the sampling grid to obtain an aligned high-dimensional mapping feature map. Sequence modeling and attention decoding are performed on the aligned high-dimensional mapping feature map to obtain the predicted character sequence. The affine matrix is ​​constructed based on the width, height, and rotation angle obtained after normalizing and scaling the character rotation box.

[0010] This invention also provides a medium having computer instructions embodied thereon, the instructions being executable by one or more processors to perform the method as described in any of the preceding claims.

[0011] To address the shortcomings and deficiencies of the existing technologies, this invention proposes an industrial single-character detection method based on multi-branch weakly supervised training, which has the following advantages and beneficial effects: (1) Single character level detection to improve robustness: By introducing a single character positioning mechanism in the detection branch, it is possible to recognize characters one by one in complex industrial environments, effectively avoiding the failure of overall recognition due to local errors.

[0012] (2) Weak supervision utilizes public data to alleviate data shortage: Design a three-branch joint structure of detection, segmentation and recognition, and introduce a weak supervision training strategy so that large-scale public string datasets can be fully utilized without a large number of single character annotations, thereby reducing annotation costs.

[0013] (3) Multi-stage training to improve convergence and generalization: By freezing and unfreezing different branches in stages, the network is gradually optimized to ensure training stability and convergence effect, while improving the model’s recognition performance on unknown characters and non-fixed combinations.

[0014] (4) Outstanding industrial application value: Under limited data conditions, a highly robust and versatile industrial character detection model can be constructed, which is suitable for various practical industrial application scenarios such as metal surface engraving, electronic component coding, and packaging date code. Attached Figure Description

[0015] Figure 1 This is a flowchart illustrating an industrial character detection and recognition method based on the YOLO framework disclosed in an embodiment of the present invention. Figure 2 This is a schematic diagram of the improved YOLO structure disclosed in an embodiment of the present invention; Figure 3 This is a schematic diagram of the training process disclosed in an embodiment of the present invention; Figure 4 This is a schematic diagram of the structure of an industrial character detection and recognition device based on the YOLO framework disclosed in an embodiment of the present invention. Detailed Implementation

[0016] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments.

[0017] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0018] To facilitate understanding of the embodiments of the present invention, further explanations and descriptions will be provided below with reference to the accompanying drawings and specific embodiments. These embodiments do not constitute a limitation on the embodiments of the present invention.

[0019] like Figure 1 As shown in the figure, this invention discloses an industrial character detection and recognition method 100 based on the YOLO framework, which includes the following steps: 100. Based on the gray coverage optimization mechanism, data augmentation processing is performed on industrial character sample data, and the augmented industrial character sample data is used to perform multi-stage training on the detection head, segmentation head and recognition head. In this step, single-character samples are relatively scarce in industrial scenarios, often requiring data augmentation to improve the robustness of the model. However, in conventional random cropping augmentation operations, individual character regions may be partially truncated, causing the generated samples to lose their complete semantics and become unreadable. These invalid samples interfere with model training and reduce recognition accuracy. To address this issue, this invention introduces a gray masking optimization mechanism into the data augmentation strategy. Specifically, when a character is partially missing during cropping, the truncated character is not directly retained; instead, a fixed gray value is used to fill and cover the area. This maintains the cropping diversity of the overall image while avoiding the negative impact of missing characters on the recognition branch.

[0020] Using the data that has undergone the augmentation process described above, the three core components of the model (detection head, segmentation head, and recognition head) are trained. For example... Figure 3 As shown, the training in this invention does not refer to one-time joint training, but rather a multi-stage training strategy to solve the technical problem of instability in end-to-end training, which will be explained in detail later.

[0021] 200. The feature extraction module processes the industrial scene image to obtain a multi-level feature map. The detection head predicts the spatial position of a single character based on the multi-level feature map and obtains the character rotation box of the single character. In this step, the feature extraction module is the backbone of the YOLO framework, responsible for extracting features with rich semantic information and different scales from the input industrial scene image. The output is a multi-level feature map, denoted as . Low-level feature maps have high resolution and rich details, while high-level feature maps have low resolution and strong semantic information.

[0022] The YOLO-based detection head receives and fuses the aforementioned multi-level feature maps to simultaneously utilize both detailed and semantic information. Its core task is to perform single-character-level object detection, predicting the precise position and orientation of each character, with the output being a rotated bounding box, denoted as a quintuple. It is not just a horizontal rectangle.

[0023] It is understood that the feature extraction module and the detection head are the original YOLO module, and in this invention, the YOLO module preferably uses YOLOv8. For example... Figure 2As shown, this invention designs three parallel branches—a detection head, a segmentation head, and a recognition head—based on traditional target detection networks (such as YOLOv8). Through a weak supervision mechanism, it effectively combines publicly available string recognition data with a small amount of industrial character annotation data, thereby achieving robust detection and recognition of single characters.

[0024] 300. The segmentation head upsamples all hierarchical feature maps to the same spatial resolution and concatenates them along the channel dimension to obtain fused features; the fused features are then mapped in a high dimension to obtain a high-dimensional mapped feature map. In this step, the segmentation head is used to prepare high-quality features for the recognition head. Specifically, the segmentation head receives multi-level feature maps from the backbone network: ;in, Let be the number of channels in the i-th layer feature. Let be the height and width of the feature in the i-th layer.

[0025] Upsampling unifies all feature maps to the highest resolution (e.g., ...). (scale) .

[0026] Then, the data is spliced ​​along the channel dimension, which allows for the fusion of details and semantic information at different scales, resulting in rich fusion features: .

[0027] Next, the features after fusion Performing a 1x1 convolution operation, also known as high-dimensional mapping, maps the number of channels to a new dimension. Generate a high-dimensional mapping feature map: .

[0028] As input features for subsequent recognition heads, they contain finely aligned and enhanced character features.

[0029] 400. The recognition head generates a sampling grid based on the affine matrix. The sampling function aligns the high-dimensional mapping feature map to a uniform output scale based on the sampling grid to obtain an aligned high-dimensional mapping feature map. Sequence modeling and attention decoding are performed on the aligned high-dimensional mapping feature map to obtain the predicted character sequence. The affine matrix is ​​constructed based on the width, height, and rotation angle obtained after normalizing and scaling the character rotation box.

[0030] In this step, the recognition head needs to process each detected character region, and its input includes a high-dimensional mapped feature map from the segmentation head. and character rotation box from the detection head .

[0031] First, rotate the frame Normalization, scaling transformation, and orientation consistency adjustment are performed as follows: The rotating frame of the text area is represented by a quintuple: ,in, Indicates the center coordinates of the text box. They are width and height, respectively. The angle is the rotation angle.

[0032] Since the text boxes use normalized coordinates, they need to be scaled to the current feature map scale: .in, These represent the width and height of the feature map, respectively.

[0033] To ensure consistent character orientation, when Then, swap the width and height and adjust the angle to obtain the parameters. :

[0034] Then, the text box region is mapped to a fixed-size sampling grid through an affine transformation. Specifically, based on parameters... Constructing an affine matrix ,

[0035] Based on the above affine matrix Generate a sampling grid G, and map the high-dimensional feature map using the grid_sample (sampling function) operation. Alignment to a uniform output scale, i.e., mapping feature maps from high dimensions. The text describes a process of precisely cropping and affine transforming character feature blocks into aligned, uniformly sized blocks. .in, For the number of channels, This is the predefined character sequence length. This step ensures that the features fed into the recognition head are standardized regardless of how the original characters are tilted or vary in size.

[0036] Next, the aligned feature blocks (i.e., aligned high-dimensional mapping feature maps) captures the contextual relationships between characters through sequence modeling (such as bidirectional LSTM), specifically: ROI features are expanded along the sequence direction (feature dimensions are swapped): ; Input bidirectional LSTM: .in, Indicates splicing, For the hidden layer dimension.

[0037] Then, an attention decoding mechanism automatically focuses on the relevant parts of the features, gradually outputting the predicted probability distribution of each character, ultimately forming a complete character sequence recognition result. Specifically: Generate character sequences using an attention-based predictor. For each time step... Attention weight Defined as: ;in, For sequence features, This refers to the decoder state from the previous step.

[0038] Context vector: ; The character prediction probability is: ; The final output is: .in, For character set size, For the maximum sequence length, each It is a probability distribution over a character set.

[0039] This invention effectively solves key technical challenges in industrial character recognition. Addressing the issue of poor robustness in overall recognition methods, it employs a strategy of separating single-character detection and recognition, avoiding recognition failures due to partial occlusion or damage, and significantly improving stability in complex environments such as blurry or oily conditions. To address the problems of poor generalization and data scarcity, it utilizes a multi-branch weakly supervised training mechanism, fully integrating publicly available datasets with a small amount of labeled industrial data, reducing reliance on large amounts of precise annotations and enhancing the model's ability to recognize unknown characters and new combinations. Simultaneously, the gray-cover optimization data augmentation method effectively avoids interference from invalid samples, further improving training efficiency and model reliability. This method achieves high-precision and highly robust industrial character recognition under limited data conditions, possessing significant engineering application value.

[0040] As an example, the segmentation head uses the binary cross-entropy (BCE) loss function; the recognition head uses the cross-entropy loss function.

[0041] In this embodiment, the goal of the network in the segmentation head is to perform binary classification (character / non-character) on each pixel of the input image. Therefore, the Binary Cross-Entropy (BCE) loss function is used as the supervision signal.

[0042] Prediction results The mask probability map predicted by the segmentation head, the value of each pixel This indicates the probability of belonging to a "character region". Real label. Similarly, its pixels It can be 0 (representing the background) or 1 (representing a character area).

[0043] For a single pixel, BCE is defined as: .

[0044] For the entire feature map, sum the results and then divide by the total number of pixels: .

[0045] Correspondingly, the segmentation head also utilizes reduceBlock (dimensionality reduction module) to compress the number of channels of the fused features to... : .

[0046] The dimensionality-reduced mapping feature map obtained after dimensionality reduction Above, a multi-scale dilated convolutional structure is used for pixel-level prediction, and the final output is the segmentation result, i.e., the binary segmentation mask: .

[0047] For the recognition head, cross-entropy loss is used in the loss calculation. Meanwhile, to avoid the padding character ([PAD]) affecting the loss, this item is excluded from the calculation. ;in, This is an indicator function; when the label is [PAD], this item is not included in the calculation. The predicted output for the nth sample at time step t is a probability distribution over all character categories; This represents the true character label for the nth sample at time step t, with its value coming from a predefined character set; the special marker [PAD] indicates that the position is a padding character; N is the number of sample sequences contained in a training batch.

[0048] As an example, the data augmentation processing of industrial character sample data based on the gray coverage optimization mechanism includes: During the random data augmentation process, if the cropping operation results in partial loss of a character region, the character sample is deemed invalid. Using predefined fixed grayscale values, the missing regions of the invalid character samples are filled and covered to generate effective augmented samples that maintain image diversity while avoiding interference from incomplete characters during training.

[0049] In this embodiment, data augmentation (such as random cropping) is a common technique in deep learning to improve the generalization ability of models. However, in character recognition tasks, the integrity and readability of characters are fundamental to recognition. Conventional random cropping generates new samples indiscriminately, easily resulting in characters being cut off and producing semantically incomplete characters. These invalid samples are fed into the network for training, and the model attempts to learn from these incomplete or even incorrect labels. Not only does it fail to learn effective features, but it also interferes with the training process, introduces noise, and reduces the model's final recognition accuracy.

[0050] To address this, the present invention incorporates a judgment logic. After each random cropping operation, a detection mechanism is executed to determine whether the cropping resulted in the partial loss of any character. This detection mechanism, for example, involves checking the intersection ratio between the character bounding box and the cropped region. Once it is confirmed that a character has become incomplete, the sample generated by that augmentation is determined to be an invalid sample and will not be directly used for training.

[0051] For samples deemed invalid, the missing character region is located. Then, this region is filled with pixels using a pre-set fixed grayscale value (e.g., 128), completely covering the original incomplete character portion. Thus, through grayscale overlay, harmful and misleading information from the missing characters in the image can be thoroughly removed, preventing them from negatively interfering with the model in downstream recognition tasks. It should be noted that fixed grayscale overlay can simulate the uniform, informationless background commonly found in industrial scenarios, allowing the model to focus more on learning complete, readable character features during training.

[0052] After the above judgment and overlay processing, the originally invalid samples were transformed into valid augmented samples. The random cropping augmentation operation itself was still performed, and the image composition and background context changed, preserving the data distribution diversity advantage brought by the augmentation technique. At the same time, the post-processing of gray overlay eliminated the possible side effects of augmentation (incomplete characters), ensuring that every sample fed into the model was clean and learnable.

[0053] Therefore, the mechanism described above in this invention can balance the sometimes conflicting goals of increasing data diversity and ensuring sample quality. It is a refined data augmentation strategy tailored to the characteristics of character recognition tasks, which can ultimately improve model robustness and accuracy.

[0054] As an example, the cropping operation results in partial loss of character regions, including: Calculate the ratio of the visible area of ​​the cropped character to the area of ​​the original character. If the ratio is lower than the dynamic threshold, it is determined to be partially missing. The dynamic threshold is dynamically adjusted according to the current character category and size. Specifically, for characters with complex structures or small sizes, a first threshold is used to strictly protect the integrity of the character; for characters with simple structures or large sizes, a second threshold is used to retain more data and enhance diversity; the first threshold is greater than the second threshold.

[0055] In this embodiment, firstly, the ratio of the visible area of ​​the cropped character to the area of ​​the original character is calculated. This ratio reflects the degree of integrity of the character after cropping. If this ratio is lower than a preset dynamic threshold, it is determined that the character region is partially missing, and the current augmented sample is considered invalid.

[0056] The dynamic threshold is not a fixed value, but is dynamically adjusted based on the characteristics of the current character. Specifically, a higher first threshold is used for characters with complex structures (such as those containing multiple strokes or rich details) or small characters. This setting aims to strictly protect the structural integrity and readability of characters, avoid the loss of key features due to slight cropping, and ensure the semantic effectiveness of the augmented samples.

[0057] For characters with simple structures (such as those with few strokes and regular shapes) or larger sizes, a lower second threshold is used. This setting allows for the retention of more cropped and transformed samples while ensuring that the characters are basically recognizable, thereby improving the diversity and coverage of data augmentation.

[0058] This dynamic decision-making mechanism effectively balances the contradiction between character integrity protection and augmented diversity preservation. It avoids the interference of invalid samples on model training and makes full use of the potential of pruning and augmentation to improve data diversity, thereby enhancing the robustness and generalization ability of the model in complex industrial scenarios.

[0059] As an example, the dynamic threshold is dynamically adjusted based on the current character's category and size, and also includes: Analyze the distribution and scarcity of industrial character sample data, and determine the strength and historical effects of augmentation strategies; Based on the distribution and scarcity, intensity and historical performance, the optimization coefficients are determined, and the first threshold and the second threshold are optimized into the third threshold and the fourth threshold, respectively, using the optimization coefficients. The optimized third and fourth thresholds are used as the dynamic thresholds.

[0060] In this embodiment, the dynamic threshold adjustment mechanism, based on initial settings (first threshold, second threshold) based on character category and size, further introduces feedback factors from data distribution and augmentation strategies to achieve more refined adaptive adjustment. Specifically: First, the distribution and scarcity of industrial character sample data are continuously analyzed. This process includes statistically analyzing the frequency of different character categories in the sample set, identifying low-frequency characters or scarce categories, and assessing the balance of the data distribution. Simultaneously, the strength and historical effectiveness of the augmentation strategy are tracked, including recording the effective generation rate of each character category, the proportion of invalid samples, and strength parameters such as the trimming ratio range used in the current batch during historical augmentation operations.

[0061] Next, based on the above analysis results, an optimization coefficient is calculated. This coefficient is a quantitative value that integrates the data scarcity and the feedback on the augmentation effect. For example, for character categories with low frequency and scarce samples, the optimization coefficient tends to lower the threshold to allow for the generation of more diverse augmented samples and alleviate the problem of insufficient data; conversely, for high-frequency characters, the optimization coefficient may maintain or increase the threshold to avoid over-augmentation. At the same time, if historical data shows that the augmented samples of a certain type of character are highly effective, the threshold can be appropriately relaxed (optimization coefficient < 1); if the proportion of invalid samples is high, the threshold is tightened (optimization coefficient > 1) to improve the quality of augmentation.

[0062] Then, this optimization coefficient is used to scale and optimize the initially set first and second thresholds, resulting in the third and fourth thresholds, respectively. The calculation formulas are, for example: Third threshold = First threshold * Optimization coefficient, Fourth threshold = Second threshold * Optimization coefficient. Through this calculation, the original fixed threshold strategy is optimized into a dynamic threshold that can respond to data state and augmentation effects.

[0063] Finally, the optimized third and fourth thresholds are used as the final dynamic thresholds for determining whether a character is partially missing. This mechanism intelligently balances the two objectives of "character integrity protection" and "data diversity enhancement," considering not only the static attributes of characters (category, size) but also the dynamic distribution characteristics of the dataset and historical augmentation performance, thereby improving the effectiveness of data augmentation and the robustness of model training at a global level. As an example, the multi-stage training of the detection head, segmentation head, and recognition head using augmented industrial character sample data includes: Phase 1: Freeze the OCR pre-trained weights in the recognition head, and perform weakly supervised optimization by combining the segmentation results output by the segmentation head with the partial recognition output generated by the recognition head; At this stage, the pre-trained OCR model weights in the recognition head are frozen and do not participate in gradient updates. The training focuses on the unfrozen portions of the segmentation and recognition heads. The network uses the augmented data for forward propagation, with the segmentation head outputting the segmentation results of the character regions (segmentation mask), and the recognition head generating partial recognition outputs based on its current capabilities. These outputs together serve as weak supervision signals, guiding the network in its initial learning.

[0064] The goal at this stage is not to pursue extremely high recognition accuracy, but to enable the network, especially the segmentation head, to learn to capture the basic shape and positional features of characters, while adapting the recognition head to the visual patterns of industrial characters.

[0065] Phase Two: Unfreeze the OCR pre-training weights of the recognition head and fine-tune the segmentation head and recognition head together with a small learning rate to enhance the feature synergy capability of the segmentation head and recognition head. Building upon Phase 1, this phase unfreezes the OCR pre-trained weights of the recognition head, allowing all its parameters to participate in training. A small learning rate is used throughout the training process to jointly fine-tune the segmentation and recognition heads. This small learning rate is designed to avoid destroying valuable general features in the pre-trained weights and to ensure the stability of the training process. The core objective of this phase is to promote feature synergy and alignment between the segmentation and recognition heads.

[0066] Through joint optimization, the feature maps generated by the segmentation head can better serve the recognition task, while the feedback from the recognition head also helps the segmentation head generate more accurate character regions that are conducive to recognition, thereby significantly enhancing the collaborative ability between the two branches.

[0067] In Phase 3, the parameters of the segmentation head and recognition head trained in Phase 2 are fixed, and only the detection head is trained to achieve stable convergence with the help of the learned semantic and shape features. In the later stage of training, the entire network is unfrozen, and the detection head, segmentation head and recognition head are globally optimized in an end-to-end manner.

[0068] This stage first fixes the parameters of the segmentation and recognition heads after sufficient training in Stage 2. Then, the detection head is trained centrally using augmented datasets. At this point, the fixed segmentation and recognition heads provide the detection head with rich prior knowledge, namely, the learned character semantics and shape features. Guided by this powerful auxiliary information, the detection head can more accurately learn the spatial location and boundaries of characters, thus achieving stable and rapid convergence. After the detection head has been trained to a certain extent, the training enters its final stage: all parameters of the entire network (including the detection, segmentation, and recognition heads) are unfrozen, and the model is finally optimized using an end-to-end global fine-tuning strategy.

[0069] This step aims to eliminate inter-module bias that may result from phased training, further optimize information flow, enable the three branches to reach the best collaborative state, and ultimately comprehensively improve the detection and recognition performance of the model.

[0070] like Figure 4 As shown, this embodiment of the invention also provides an industrial character detection and recognition device 200 based on the YOLO framework, including a training module 201, a feature extraction module 202, a segmentation head 203, and a recognition head 204; The training module 201 performs data augmentation processing on industrial character sample data based on the gray coverage optimization mechanism, and uses the augmented industrial character sample data to perform multi-stage training on the detection head, segmentation head and recognition head. The feature extraction module 202 processes the industrial scene image to obtain a multi-level feature map. The detection head 203 predicts the spatial position of a single character based on the multi-level feature map to obtain a character rotation box of the single character. The segmentation head 204 upsamples all hierarchical feature maps to the same spatial resolution and concatenates them along the channel dimension to obtain fused features; the fused features are then mapped in a high dimension to obtain a high-dimensional mapped feature map. The recognition head generates a sampling grid based on the affine matrix. The sampling function aligns the high-dimensional mapping feature map to a uniform output scale based on the sampling grid to obtain an aligned high-dimensional mapping feature map. Sequence modeling and attention decoding are performed on the aligned high-dimensional mapping feature map to obtain the predicted character sequence. The affine matrix is ​​constructed based on the width, height, and rotation angle obtained after normalizing and scaling the character rotation box.

[0071] As an example, the segmentation head uses a binary cross-entropy loss function; the recognition head uses a cross-entropy loss function.

[0072] As an example, the training module specifically: During the random data augmentation process, if the cropping operation results in partial loss of a character region, the character sample is deemed invalid. Using predefined fixed grayscale values, the missing regions of the invalid character samples are filled and covered to generate effective augmented samples that maintain image diversity while avoiding interference from incomplete characters during training.

[0073] As an example, data augmentation processing of industrial character sample data based on the gray coverage optimization mechanism includes: During the random data augmentation process, if the cropping operation results in partial loss of a character region, the character sample is deemed invalid. Using predefined fixed grayscale values, the missing regions of the invalid character samples are filled and covered to generate effective augmented samples that maintain image diversity while avoiding interference from incomplete characters during training.

[0074] As an example, the training module specifically: Calculate the ratio of the visible area of ​​the cropped character to the area of ​​the original character. If the ratio is lower than the dynamic threshold, it is determined to be partially missing. The dynamic threshold is dynamically adjusted according to the current character category and size. Specifically, for characters with complex structures or small sizes, a first threshold is used to strictly protect the integrity of the character; for characters with simple structures or large sizes, a second threshold is used to retain more data and enhance diversity; the first threshold is greater than the second threshold.

[0075] As an example, the training module specifically: Analyze the distribution and scarcity of industrial character sample data, and determine the strength and historical effects of augmentation strategies; Based on the distribution and scarcity, intensity and historical performance, the optimization coefficients are determined, and the first threshold and the second threshold are optimized into the third threshold and the fourth threshold, respectively, using the optimization coefficients. The optimized third and fourth thresholds are used as the dynamic thresholds.

[0076] As an example, the training module specifically: Phase 1: Freeze the OCR pre-trained weights in the recognition head, and perform weakly supervised optimization by combining the segmentation results output by the segmentation head with the partial recognition output generated by the recognition head; Phase Two: Unfreeze the OCR pre-training weights of the recognition head and fine-tune the segmentation head and recognition head together with a small learning rate to enhance the feature synergy capability of the segmentation head and recognition head. In Phase 3, the parameters of the segmentation head and recognition head trained in Phase 2 are fixed, and only the detection head is trained to achieve stable convergence with the help of the learned semantic and shape features. In the later stage of training, the entire network is unfrozen, and the detection head, segmentation head and recognition head are globally optimized in an end-to-end manner.

[0077] This invention also provides a medium having computer instructions embodied thereon, the instructions being executable by one or more processors to perform the method as described in any of the preceding claims.

[0078] Although the invention has been specifically shown and described with reference to preferred embodiments, those skilled in the art will understand that various modifications in form and detail may be made without departing from the spirit and scope of the invention. Accordingly, the disclosed invention should be considered merely illustrative and limited only by the scope specified in the appended claims.

Claims

1. An industrial character detection and recognition method based on the YOLO framework, characterized in that, Includes the following steps: Based on the gray coverage optimization mechanism, data augmentation processing is performed on industrial character sample data, and the augmented industrial character sample data is used to perform multi-stage training on the detection head, segmentation head and recognition head. The feature extraction module processes industrial scene images to obtain multi-level feature maps. The detection head predicts the spatial position of a single character based on the multi-level feature maps to obtain the character rotation box of the single character. The segmentation head upsamples all hierarchical feature maps to the same spatial resolution and concatenates them along the channel dimension to obtain fused features; High-dimensional mapping is performed on the fused features to obtain a high-dimensional mapped feature map; The recognition head generates a sampling grid based on the affine matrix, and the sampling function aligns the high-dimensional mapped feature map to a uniform output scale based on the sampling grid to obtain an aligned high-dimensional mapped feature map. Sequence modeling and attention decoding are performed on the aligned high-dimensional mapping feature map to obtain the predicted character sequence; the affine matrix is ​​constructed based on the width, height, and rotation angle obtained after normalizing and scaling the character rotation box.

2. The industrial character detection and recognition method based on the YOLO framework according to claim 1, characterized in that: The segmentation head uses a binary cross-entropy loss function; the recognition head uses a cross-entropy loss function.

3. The industrial character detection and recognition method based on the YOLO framework according to claim 1, characterized in that: Data augmentation processing of industrial character sample data is performed based on a gray coverage optimization mechanism, including: During the random data augmentation process, if the cropping operation results in partial loss of a character region, the character sample is deemed invalid. Using predefined fixed grayscale values, the missing regions of the invalid character samples are filled and covered to generate effective augmented samples that maintain image diversity while avoiding interference from incomplete characters during training.

4. The industrial character detection and recognition method based on the YOLO framework according to claim 3, characterized in that: The cropping operation resulted in partial loss of character regions, including: Calculate the ratio of the visible area of ​​the cropped character to the area of ​​the original character. If the ratio is lower than the dynamic threshold, it is determined to be partially missing. The dynamic threshold is dynamically adjusted according to the current character category and size. Specifically, for characters with complex structures or small sizes, a first threshold is used to strictly protect the integrity of the character; for characters with simple structures or large sizes, a second threshold is used to retain more data and enhance diversity; the first threshold is greater than the second threshold.

5. The industrial character detection and recognition method based on the YOLO framework according to claim 4, characterized in that: The dynamic threshold is dynamically adjusted based on the current character's category and size, and also includes: Analyze the distribution and scarcity of industrial character sample data, and determine the strength and historical effects of augmentation strategies; Based on the distribution and scarcity, intensity and historical performance, the optimization coefficients are determined, and the first threshold and the second threshold are optimized into the third threshold and the fourth threshold, respectively, using the optimization coefficients. The optimized third and fourth thresholds are used as the dynamic thresholds.

6. The industrial character detection and recognition method based on the YOLO framework according to claim 1, characterized in that: Multi-stage training was performed on the detection head, segmentation head, and recognition head using augmented industrial character sample data, including: Phase 1: Freeze the OCR pre-trained weights in the recognition head, and perform weakly supervised optimization by combining the segmentation results output by the segmentation head with the partial recognition output generated by the recognition head; Phase Two: Unfreeze the OCR pre-training weights of the recognition head and fine-tune the segmentation head and recognition head together with a small learning rate to enhance the feature synergy capability of the segmentation head and recognition head. In Phase 3, the parameters of the segmentation head and recognition head trained in Phase 2 are fixed, and only the detection head is trained to achieve stable convergence with the help of the learned semantic and shape features. In the later stage of training, the entire network is unfrozen, and the detection head, segmentation head and recognition head are globally optimized in an end-to-end manner.

7. An industrial character detection and recognition device based on the YOLO framework, characterized in that, It includes a training module, a feature extraction module, a segmentation head, and a recognition head; The training module performs data augmentation processing on industrial character sample data based on a gray coverage optimization mechanism, and uses the augmented industrial character sample data to perform multi-stage training on the detection head, segmentation head and recognition head. The feature extraction module processes the industrial scene image to obtain a multi-level feature map. The detection head predicts the spatial position of a single character based on the multi-level feature map to obtain a character rotation box for the single character. The segmentation head upsamples all hierarchical feature maps to the same spatial resolution and stitches them together along the channel dimension to obtain fused features; High-dimensional mapping is performed on the fused features to obtain a high-dimensional mapped feature map; The recognition head generates a sampling grid based on the affine matrix, and the sampling function aligns the high-dimensional mapping feature map to a uniform output scale based on the sampling grid to obtain an aligned high-dimensional mapping feature map. Sequence modeling and attention decoding are performed on the aligned high-dimensional mapping feature map to obtain the predicted character sequence; the affine matrix is ​​constructed based on the width, height, and rotation angle obtained after normalizing and scaling the character rotation box.

8. The industrial character detection and recognition device based on the YOLO framework according to claim 7, characterized in that: The segmentation head uses a binary cross-entropy loss function; the recognition head uses a cross-entropy loss function.

9. The industrial character detection and recognition device based on the YOLO framework according to claim 7, characterized in that: The training module specifically includes: During the random data augmentation process, if the cropping operation results in partial loss of a character region, the character sample is deemed invalid. Using predefined fixed grayscale values, the missing regions of the invalid character samples are filled and covered to generate effective augmented samples that maintain image diversity while avoiding interference from incomplete characters during training.

10. A medium having computer instructions embodied thereon, characterized in that: The instructions can be executed by one or more processors to perform the method for implementing any one of claims 1-6.

Images