Character detection method based on improved YOLO11 open world scene

Abstract

The invention discloses a character detection method in an open world scene based on improved YOLO11, which relates to the field of computer vision processing and comprises the following steps of: constructing a character detection data set in the open world scene; the method comprises the following steps: constructing a character detection model in an open world scene based on YOLO11, designing a multi-scale efficient convolution module, focusing on multi-scale feature extraction and fusion by using a plurality of convolution kernels with different sizes, and enhancing the perception ability of the improved model for detecting different-scale information in development world scene characters while keeping the calculation efficiency; a YOLO11 feature pyramid pooling module is improved into FocalModulation, the module utilizes a focus attention mechanism to enhance the feature extraction capability of a model for a specific area, and local and global features are combined to improve the text detection precision of the model in an open world scene, reduce the calculation complexity of the model and improve the reasoning speed of the model; by adopting CIoU as a bounding box regression loss function for improving YOLO11, the character detection precision in an open world scene can be improved, and good detection performance is kept for character information targets in different scales and complex backgrounds.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision processing and is a text detection method in an open world scene based on an improved YOLO11. Background Art

[0002] Existing text detection technologies are mainly used in specific closed scenarios, such as document scanning and billboard recognition. These methods usually rely on a large amount of annotated data and predefined font libraries, and are difficult to cope with the diverse and dynamically changing text environments in the open world. Text detection in open-world scenarios faces many challenges, including but not limited to different languages ​​and character sets, multiple font styles, complex background interference, multi-scale text, and real-time detection requirements. When dealing with open-world scenarios, traditional deep learning text detection methods often have problems such as low detection accuracy, weak generalization ability, and insufficient adaptability to new fonts and new languages. In addition, the computational efficiency of these methods in highly complex environments is also difficult to meet the requirements of real-time applications, resulting in high false detection rates and missed detection rates, which limits their promotion and application in practical applications. Summary of the invention

[0003] The purpose of the present invention is to solve the problems of poor adaptability, low detection accuracy and insufficient generalization ability of existing text detection technologies in open world scenarios. A text detection method in open world scenarios based on improved YOLO11 is proposed, and a multi-scale feature extraction module is introduced. By fusing feature information at different levels, the model's detection ability for texts of various sizes is enhanced, and the recognition accuracy for small texts and large-area texts is improved. The method of the present invention achieves high accuracy, high generalization ability and real-time text detection effects in open world scenarios.

[0004] The technical solution adopted by the present invention is:

[0005] A text detection method in an open world scene based on improved YOLO11 is specifically performed in the following steps:

[0006] Step 1: Collect data in the open world scene, use the LabelImg tool to annotate it, and create a text detection dataset in the open world scene;

[0007] Step 2: Build and improve the YOLO11 open-world text detection model, design an efficient convolution module (Efficient Multi Scale Module) to improve the YOLO11 trunk structure and neck structure. The multi-scale efficient convolution module uses multiple convolution kernels of different sizes (such as 1x1, 3x3, 5x5, 7x7) to focus on multi-scale feature extraction and fusion, and improves the YOLO11 feature pyramid pooling module to FocalModulation. This module uses the focus mechanism to enhance the model's feature extraction ability for specific areas, combines local and global features, and improves the loss function to CIoU to improve the accuracy of text detection in open-world scenarios;

[0008] Step 3: Input the training set image data in the open world scene text detection dataset into the improved YOLO11 network model in step 2 for training, save the model parameters of the improved YOLO11 model with the highest accuracy on the validation set during the training process, and name its file best.pt;

[0009] Step 4: Input the image to be detected containing text targets in the open world scene into the improved YOLO11 model, load the optimal weight file best.pt in step 3 into the detection model for inference detection, and obtain whether the target in the image to be detected contains text information.

[0010] Furthermore, step 1 specifically includes:

[0011] Step 1.1: Collect the ICDAR text detection dataset in the open world scene, a total of 462 images

[0012] Step 1.2: Use LabelImg to label the data set and generate the corresponding xml file. The label category is text, and convert it into the corresponding txt file format, dividing it into training set: validation set: test set according to 8:1:1.

[0013] Furthermore, step 2 specifically includes:

[0014] Step 2.1: Design an efficient convolution module (Efficient Multi Scale Module) to improve the YOLO11 backbone structure and neck structure. The multi-scale efficient convolution module focuses on multi-scale feature extraction and fusion by using multiple convolution kernels of different sizes (such as 1x1, 3x3, 5x5, 7x7). When the efficient convolution module is initialized, the input channel channel is divided into multiple groups of channels, and the number of channels in each group is channel / groups. The formula is shown in (1):

[0015]

[0016] The input tensor is X, where B is the batch size, C is the number of channels, H and W are the height and width respectively, G is the number of groups, and kernels defines different convolution kernel sizes, which are 1x1, 3x3, 5x5, and 7x7, respectively. The corresponding model obtains different receptive fields, allowing the module to extract features at different scales and improve the perception of multi-scale image information, as shown in formula (2). In formula (2), each group g∈{1,....,G}, with different convolution kernels K g application,

[0017]

[0018] Convolution kernels of different sizes are used to perform corresponding convolution operations on each group to achieve parallel multi-scale feature extraction. The output of the multi-scale convolution is rearranged again to output a B×C×H×W shape. After a 1×1 convolution, channel fusion operations are performed to integrate features of different scales and enhance the richness of feature representation, as shown in formula (3):

[0019] Y fused =Conv 1×1 (Concat([Y 1 ,....,Y G ])) (3)

[0020] Step 2.2: Add the focus mechanism to enhance the model's ability to extract features for specific areas. By aggregating local and global features, the feature pyramid pooling module FocalModulation is added to the ninth layer of the YOLO11 backbone network to optimize the network structure, reduce computational complexity, and improve reasoning speed, maintaining good detection performance for targets of different scales and complex backgrounds.

[0021] Step 2.3: Improve the bounding box regression loss function of the text detection model in the YOLO11 open family scene using CIoU_Loss, which not only considers the overlapping area, but also the center point distance and aspect ratio. By introducing the center point distance, CIoU can better optimize the position of the bounding box. By considering the aspect ratio, CIoU helps to more accurately locate the target shape. The CIoU_Loss formula is shown in (4):

[0022]

[0023] IoU: Intersection over Union, which measures the ratio of the overlapping area to the joint area, ρ(b,b g ): Euclidean distance between the center points of the two boxes, c: diagonal length of the minimum enclosing area containing the two boxes, α: weight coefficient used to balance the aspect ratio υ: consistency measure of the aspect ratio;

[0024] Furthermore, step 3 specifically includes:

[0025] Input the text detection dataset in the open world scenario constructed in step 1 into the improved YOLO11 text detection model in step 2 for training. Set the network model training parameters, the number of training rounds is 300 rounds, the batch-size is 32, and SGD is used as the optimizer. After the training is completed, save the optimal weight of the text detection model effect in the open world scenario and name it best.pt;

[0026] Furthermore, step 4 specifically includes:

[0027] In step 4, the image to be detected containing text information is input into the improved YOLO11 model, and the optimal weight file best.pt in step 3 is loaded into the detection model for inference detection to obtain whether the target in the image to be detected contains text information.

[0028] Beneficial effects:

[0029] The present invention proposes a method for detecting text in an open world scenario based on an improved YOLO11, constructs a text detection model in an open world scenario based on YOLO11, and designs an Efficient MultiScale Module. The Efficient MultiScale Module focuses on multi-scale feature extraction and fusion by using multiple convolution kernels of different sizes (such as 1x1, 3x3, 5x5, 7x7), and improves the YOLO11 feature pyramid pooling module to FocalModulation. The module uses a focal attention mechanism to enhance the model's feature extraction capability for a specific area, and combines local and global features. The improved YOLO11 model training obtains a recall rate and mAP value of 92% and 91.2%, respectively, which are 1% and 0.9% higher than YOLO11, respectively. The method of the present invention achieves high accuracy, high generalization capability and real-time text detection effects in an open world scenario. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flowchart of a text detection method in an open world scenario based on improved YOLO11 in the present invention;

[0031] Figure 2 This is a network model architecture diagram of a text detection method in an open world scenario based on an improved YOLO11 in the present invention;

[0032] Figure 3 This is a principle structure diagram of an efficient multi-scale convolution module (Efficient Multi Scale Module) in a text detection method in an open world scenario based on an improved YOLO11 in the present invention;

[0033] Figure 4 This is a principle structure diagram of a feature pyramid module (FocalModulation) with a focus mechanism based on an improved YOLO11 open world scene text detection method of the present invention;

[0034] Figure 5 This is a PR graph of model training in a text detection method in an open world scenario based on improved YOLO11 in the present invention;

[0035] Figure 6 A recall graph of model training in a text detection method in an open world scenario based on improved YOLO11 in the present invention;

[0036] Figure 7 This is a rendering of a text detection method in an open world scenario based on improved YOLO11 in the present invention;

[0037] Figure 8 This is the effect diagram of the original YOLO11 text detection method in the open world scene;

[0038] Fig. 9 This is a rendering of a text detection method in an open world scenario based on improved YOLO11 in the present invention;

[0039] Fig.10 This is the original YOLO11 detection effect of power workers wearing helmets; DETAILED DESCRIPTION

[0040] In order to make the purpose, features and advantages of the present invention clearer, the following is a clear and complete description of a text detection method in an open world scene based on an improved YOLO11 in conjunction with the accompanying drawings and specific embodiments. In addition, the specific embodiments described in the present invention are only used to explain the present invention and are not used to limit the present invention.

[0041] The purpose of the present invention is to solve the problems of poor adaptability, low detection accuracy and insufficient generalization ability of existing text detection technology in open world scenarios. A text detection method in open world scenarios based on improved YOLO11 is proposed, and a multi-scale feature extraction module is introduced. By fusing feature information at different levels, the model's detection ability for texts of various sizes is enhanced, and the recognition accuracy for small texts and large-area texts is improved. The method of the present invention achieves high accuracy, high generalization ability and real-time text detection effects in open world scenarios. Specifically, Figures 1 to 10 As shown, a text detection method in an open world scene based on improved YOLO11 includes the following steps:

[0042] Step 1: Collect data in the open world scene, use the LabelImg tool to annotate it, and create a text detection dataset in the open world scene;

[0043] Collect ICDAR, a text detection dataset in open world scenes, with a total of 462 pictures. Use LabelImg to label the dataset and generate the corresponding xml file. The label category is text, which is converted into the corresponding txt file format and divided into training set: validation set: test set according to 8:1:1.

[0044] Step 2: Build and improve the YOLO11 open-world text detection model, design an efficient convolution module (Efficient Multi Scale Module) to improve the YOLO11 trunk structure and neck structure. The multi-scale efficient convolution module uses multiple convolution kernels of different sizes (such as 1x1, 3x3, 5x5, 7x7) to focus on multi-scale feature extraction and fusion, and improves the YOLO11 feature pyramid pooling module to FocalModulation. This module uses the focus mechanism to enhance the model's feature extraction ability for specific areas, combines local and global features, and improves the loss function to CIoU to improve the accuracy of text detection in open-world scenarios;

[0045] The efficient convolution module (Efficient Multi Scale Module) is designed to improve the YOLO11 backbone structure and neck structure. The multi-scale efficient convolution module focuses on multi-scale feature extraction and fusion by using multiple convolution kernels of different sizes (such as 1x1, 3x3, 5x5, 7x7). When the efficient convolution module is initialized, the input channel channel is divided into multiple groups of channels, and the number of channels in each group is channel / groups. The formula is shown in (1):

[0046]

[0047] The input tensor is X, where B is the batch size, C is the number of channels, H and W are the height and width respectively, G is the number of groups, and kernels defines different convolution kernel sizes, which are 1x1, 3x3, 5x5, and 7x7, respectively. The corresponding model obtains different receptive fields, allowing the module to extract features at different scales and improve the perception of multi-scale image information, as shown in formula (2). In formula (2), each group g∈{1,....,G}, with different convolution kernels K g application,

[0048]

[0049] Convolution kernels of different sizes are used to perform corresponding convolution operations on each group to achieve parallel multi-scale feature extraction. The output of the multi-scale convolution is rearranged again to output a B×C×H×W shape. After a 1×1 convolution, channel fusion operations are performed to integrate features of different scales and enhance the richness of feature representation, as shown in formula (3):

[0050] Y fused =Conv 1×1 (Concat([Y 1 ,....,Y G ])) (3)

[0051] Add the focus mechanism to enhance the model's ability to extract features from specific areas. By aggregating local and global features, the feature pyramid pooling module FocalModulation is added to the ninth layer of the YOLO11 backbone network to optimize the network structure, reduce computational complexity, and improve reasoning speed, maintaining good detection performance for targets of different scales and complex backgrounds.

[0052] Improved YOLO11 Open World scene text detection model bounding box regression loss function uses CIoU_Loss, which not only considers the overlapping area, but also the center point distance and aspect ratio. By introducing the center point distance, CIoU can better optimize the position of the bounding box. By considering the aspect ratio, CIoU helps to more accurately locate the target shape. The CIoU_Loss formula is shown in (4):

[0053]

[0054] IoU: Intersection over Union, which measures the ratio of the overlapping area to the joint area, ρ(b,b g ): Euclidean distance between the center points of the two boxes, c: diagonal length of the minimum enclosing area containing the two boxes, α: weight coefficient used to balance the aspect ratio υ: consistency measure of the aspect ratio;

[0055] Step 3: Input the training set image data in the open world scene text detection dataset into the improved YOLO11 network model in step 2 for training, save the model parameters of the improved YOLO11 model with the highest accuracy on the validation set during the training process, and name its file best.pt;

[0056] Input the text detection dataset in the open world scenario constructed in step 1 into the improved YOLO11 text detection model in step 2 for training. Set the network model training parameters, the number of training rounds is 300 rounds, batch-size is 32, and SGD is used as the optimizer. The detailed configuration of the model training environment is shown in Table 1. After the training is completed, save the optimal weight of the text detection model effect in the open world scenario and name it best.pt;

[0057] The experimental environment of the text detection method in an open world scene based on the improved YOLO11 is Ubuntu20.04, PyTorch2.2.2 version, the graphics card is NVIDIA GeForceRTX3090, and the specific configuration is shown in Table 1. The network is trained in GPU mode, the network model training parameters are set, the number of training rounds is 300 rounds, the batch-size is 32, and SGD is used as the optimizer. After the training is completed, the optimal weight of the helmet wearing detection model effect is saved and named best.pt.

[0058] Table 1 Experimental environment configuration table

[0059]

[0060] Step 4: Input the image to be detected containing text targets in the open world scene into the improved YOLO11 model, load the optimal weight file best.pt in step 3 into the detection model for inference detection, and obtain whether the target in the image to be detected contains text information.

[0061] In step 4, input the image to be detected containing text information into the improved YOLO11 model, load the optimal weight file best.pt in step 3 into the detection model for inference detection, and obtain whether the target in the image to be detected contains text information. Input the target image to be detected, load the optimal weight best.pt in the open world scene for image inference, and finally display the detection category and detection accuracy in the output image. The detection accuracy value in the detection result is proportional to the detection effect. In order to verify the accuracy improvement of a text detection method based on the improved YOLO11 in the open world scene, Recall (recall rate) and mAP (average precision) are used as evaluation indicators.

[0062] A text detection method based on the improved YOLO11 in the open world scene generates Recall and Precision graphs after training for 300 rounds, as shown in the figure below. Figure 5 , Figure 6As shown in the figure, the PR graph represents mAP, and the Recall graph represents the recall rate value. Table 2 is a comparison table of model performance. Under the same experimental environment and parameters, the recall value of the algorithm in this paper reached 92%, which was 1% higher than that of YOLO11, and the average precision mAP value reached 91.2%, which was 0.9% higher than that of YOLO11. This proves that the text detection method proposed in this paper based on the improved YOLO11 in open world scenarios is feasible.

[0063] Table 2 Performance comparison table

[0064]

[0065] The algorithm proposed in this paper improves the accuracy of helmet wearing detection in a small sample power operation scene data set. It introduces a multi-scale feature extraction module and enhances the model's detection ability for text of various sizes by fusing feature information at different levels, improving the recognition accuracy of small text and large-area text. The method of the present invention achieves high accuracy, high generalization ability and real-time text detection effect in open world scenes. It solves the problems of poor adaptability, low detection accuracy and insufficient generalization ability of existing text detection technology in open world scenes. Figure 7 , Fig. 9 They are the detection effect diagrams obtained by the algorithm proposed in this paper. Figure 8 , Fig.10 This is the detection effect diagram of the original YOLO11. By comparison, it can be seen that the original YOLO11 has low text detection accuracy in open world scenes, while the algorithm proposed in this paper has high detection accuracy. The present invention proposes a text detection method in open world scenes based on the improved YOLO11, which solves the problems of poor adaptability, low detection accuracy and insufficient generalization ability of existing text detection technologies in open world scenes.

[0066] The above embodiment is an implementation mode of the present invention, but the implementation mode of the present invention is not limited thereto. Any modification, replacement and improvement made by technicians in this field without departing from the principle and spirit of the present invention are included in the protection scope of the present invention.

Claims

1. A text detection method in an open world scene based on improved YOLO11, characterized by: The specific steps are as follows: Step 1: Collect data in the open world scene, use the LabelImg tool to annotate it, and create a text detection dataset in the open world scene; Step 2: Build and improve the YOLO11 open-world text detection model, design an efficient convolution module to improve the YOLO11 backbone and neck structures, and use multiple convolution kernels of different sizes to focus on multi-scale feature extraction and fusion. The YOLO11 feature pyramid pooling module is improved to FocalModulation, which uses the focus mechanism to enhance the model's feature extraction capability for specific areas. It combines local and global features and improves the loss function to CIoU to improve the text detection accuracy in open-world scenarios. Step 3: Input the training set image data in the open world scene text detection dataset into the improved YOLO11 network model in step 2 for training, save the model parameters of the improved YOLO11 model with the highest accuracy on the validation set during the training process, and name its file best.pt; Step 4: Input the image to be detected containing text targets in the open world scene into the improved YOLO11 model, load the optimal weight file best.pt in step 3 into the detection model for inference detection, and obtain whether the target in the image to be detected contains text information.

2. According to claim 1, a method for detecting text in an open world scene based on improved YOLO11 is characterized in that: The step 1 specifically comprises the following steps: Step 1.1: Collect the ICDAR dataset for text detection in open world scenes, a total of 462 images; Step 1.2: Use LabelImg to label the data set and generate the corresponding xml file. The label category is text, and convert it into the corresponding txt file format, dividing it into training set: validation set: test set according to 8:1:

1.

3. The method for detecting text in an open world scenario based on improved YOLO11 according to claim 2, characterized in that: The step 2 specifically includes the following steps: Step 2.1: Design an efficient convolution module to improve the YOLO11 backbone and neck structures. The multi-scale efficient convolution module focuses on multi-scale feature extraction and fusion by using multiple convolution kernels of different sizes. When the efficient convolution module is initialized, the input channel channel is divided into multiple groups of channels, and the number of channels in each group is channel / groups. The formula is shown in (1): The input tensor is X, where B is the batch size, C is the number of channels, H and W are the height and width respectively, G is the number of groups, and kernels defines different convolution kernel sizes, which are 1x1, 3x3, 5x5, and 7x7, respectively. The corresponding model obtains different receptive fields, allowing the module to extract features at different scales and improve the perception of multi-scale image information, as shown in formula (2). In formula (2), each group g∈{1,....,G}, with different convolution kernels K g application, Yg=Conv Kg (Xgroup[......,g]) (2) Convolution kernels of different sizes are used to perform corresponding convolution operations on each group to achieve parallel multi-scale feature extraction. The output of the multi-scale convolution is rearranged again to output a B×C×H×W shape. After a 1×1 convolution, channel fusion operations are performed to integrate features of different scales and enhance the richness of feature representation, as shown in formula (3): AND fused =Conv 1×1 (Concat([Y1,....,Y G ])) (3) Step 2.2: Add the focus mechanism to enhance the model's ability to extract features for specific areas, by aggregating local and global features through the feature pyramid pooling module FocalModulation to the ninth layer of the YOLO11 backbone network; Step 2.3: Improve the bounding box regression loss function of the text detection model in the YOLO11 open family scene using CIoU_Loss, introduce the center point distance, consider the aspect ratio, CIoU more accurately locates the target shape CIoU_Loss, the formula is shown in (4): IoU: Intersection over Union, which measures the ratio of the overlapping area to the joint area, ρ(b,b g ): Euclidean distance between the center points of the two boxes, c: diagonal length of the minimum enclosing area containing the two boxes, α: weight coefficient used to balance the aspect ratio υ: consistency measure of the aspect ratio.

4. The method for detecting text in an open world scenario based on improved YOLO11 according to claim 3, characterized in that: The step 3 specifically comprises the following steps: Input the text detection dataset in the open world scenario constructed in step 1 into the improved YOLO11 text detection model in step 2 for training. Set the network model training parameters, the number of training rounds is 300 rounds, the batch-size is 32, and SGD is used as the optimizer. After the training is completed, save the optimal weight of the text detection model effect in the open world scenario and name it best.pt.

5. The method for detecting text in an open world scenario based on improved YOLO11 according to claim 4, characterized in that: The step 4 specifically comprises the following steps: In step 4, the image to be detected containing text information is input into the improved YOLO11 model, and the optimal weight file best.pt in step 3 is loaded into the detection model for inference detection to obtain whether the target in the image to be detected contains text information.

Images