Real-time illegal flag detection method based on YOLOV7 target detection algorithm

Abstract

This invention relates to a real-time illegal flag detection method based on the YOLOv7 object detection algorithm, comprising the following steps: collecting illegal flag images by writing a web crawler, determining the features for illegal flag detection, labeling the collected images using an image annotation tool to create an illegal flag dataset, and simultaneously collecting normal flags with similar features to the illegal flags to create a confused feature flag dataset; optimizing the model algorithm by improving the YOLOv7 object detection algorithm based on focus and global object detection distillation to obtain an adaptive Siamese distillation YOLOv7-tiny object detection algorithm model; differing from existing technologies, this invention improves the illegal flag recognition capability by creating a confused feature flag dataset, and improves the YOLOv7 object detection algorithm based on focus and global object detection distillation to obtain a Siamese distillation YOLOv7-tiny object detection algorithm model, thereby improving detection accuracy and achieving real-time detection.

Description

Technical Field

[0001] This invention relates to the fields of computer vision technology and edge intelligent computing technology, and in particular to a real-time illegal flag detection method based on the YOLOv7 target detection algorithm. Background Technology

[0002] In today's era, internet technology is advancing rapidly, and the speed and reach of information dissemination are astonishing. Image data, as one of the main sources of information, is ubiquitous in our lives. Smartphones, computers, and large-screen displays provide us with a wide variety of video and image information, enriching our lives. However, while technology brings convenience, it also presents numerous challenges, among which cybersecurity has always been a hot topic of research. Some images containing illegal content exist online, and if they are not detected and filtered, their astonishing speed and reach will have extremely negative consequences. As flags are highly sensitive data, real-time detection is particularly important.

[0003] Currently, the screening and filtering of illegal images mainly focuses on the server side, that is, collecting image data transmitted over the network to the cloud, and then using certain methods to detect and identify it. For example, major video and social media platforms use manual review and user-reported methods for monitoring.

[0004] These methods have the following drawbacks:

[0005] 1. Manual inspection methods have very low accuracy and efficiency, a very high false detection rate, and require a lot of human resources.

[0006] 2. Cloud-based detection relies too heavily on the network, resulting in significant latency and an inability to perform real-time detection. Furthermore, cloud-based detection methods also pose network security risks and are susceptible to attacks.

[0007] Deep learning technology has developed rapidly, and object detection technology, as a fundamental task of computer vision, has been widely used in people's production and life, such as security monitoring, autonomous driving, and intelligent manufacturing. Due to the limitation of computing power, edge computing requires object detection models to have fewer model parameters and faster detection speed to achieve real-time detection. YOLO series object detection algorithms are widely used in edge intelligence scenarios because of their faster detection speed.

[0008] Further information related to the above technical solutions can be found in the following documents:

[0009] Patent CN114092743B discloses a method, apparatus, storage medium, and device for compliance detection of sensitive images, belonging to the field of image processing technology. The method is used in a compliance detection network, which includes a target detection network and at least one detection branch, each detection branch corresponding to a compliance detection strategy. The method includes: acquiring a sensitive image to be detected; using the target detection network to identify sensitive regions and sensitive categories from the sensitive image, wherein the sensitive regions contain sensitive targets, and the sensitive categories are the categories to which the sensitive targets belong; selecting a target detection branch corresponding to the sensitive category from the at least one detection branch; and using the compliance detection strategy of the target detection branch to detect whether the sensitive image is compliant.

[0010] In the process of realizing this invention, the inventors discovered the following problems in the prior art:

[0011] In existing technologies, there is no dataset of flags with obfuscated features, the ability to identify illegal flags is poor, the detection accuracy of YOLO series object detection algorithms is poor, and the slow inference speed of the models makes it impossible to achieve real-time detection on edge devices. Summary of the Invention

[0012] In view of the above problems, this application provides a real-time illegal flag detection method based on the YOLOv7 target detection algorithm, which solves the problems in the prior art, such as the lack of a dataset of flags with obfuscated features, poor identification ability of illegal flags, and the inability to achieve real-time detection on edge devices due to slow model inference speed.

[0013] To achieve the above objectives, in a first aspect, the inventors provide a real-time illegal flag detection method based on the YOLOv7 target detection algorithm, comprising the following steps:

[0014] Dataset creation

[0015] By writing a web crawler to collect images of illegal flags, determining the features for detecting illegal flags, and using image annotation tools to annotate the collected images, an illegal flag dataset is created. At the same time, normal flags with similar features to illegal flags are collected to create a confused feature flag dataset. The illegal flag dataset and the confused feature flag dataset are sampled and recombined according to a specific ratio to create training and validation sets.

[0016] Optimize the model algorithm

[0017] Based on focus and global target detection distillation, the YOLOV7 target detection algorithm is improved to obtain the adaptive twin distillation YOLOV7-tiny target detection algorithm model;

[0018] Model training

[0019] The YOLOV7-tiny target detection algorithm model was trained using the prepared dataset to obtain a preliminary target flag detection model, adaptive twin distillation YOLOV7-tiny.

[0020] Model optimization

[0021] The initial target flag detection model was optimized through experiments and various strategies. At the same time, the robustness of the target flag detection model in real-world application scenarios was improved to obtain an illegal flag detection model for deployment. The training and optimization of the illegal flag detection model were both completed on the server side.

[0022] Model Deployment

[0023] The illegal flag detection model used for deployment is quantified and deployed onto the AI ​​edge computing device development board;

[0024] Video detection

[0025] The video input to the development board is processed by extracting frames. The extracted frames are then input into the deployed illegal flag detection model for detection. Images containing illegal flags are filtered or alerted to ensure the compliance of the video scene.

[0026] Unlike existing technologies, the above technical solution improves the ability to identify illegal flags by creating a dataset of obfuscated flag features. It improves the YOLOv7 target detection algorithm by distilling based on focus and global target detection, obtaining an adaptive Siamese distillation YOLOv7-tiny target detection algorithm model to improve detection accuracy. The model is optimized through a large number of experiments and various strategies to achieve the highest accuracy. The training and optimization of the model are completed on the server side because it has higher computing power and faster training speed, enabling real-time detection.

[0027] As one embodiment of the present invention, in the step of creating the dataset, when the collected images are labeled using an image annotation tool, the features of the illegal flags and their corresponding logos are selected at the same time. Then, the image annotation tool is used to annotate the illegal flag targets and their corresponding logos in the images to create an illegal flag dataset.

[0028] Thus, the analysis found that selecting both the characteristics of the illegal flag and its corresponding logo can achieve better results, and proposed a "flag + logo" strategy for marking illegal flags.

[0029] As one embodiment of the present invention, in the step of creating the dataset, feature comparison is performed on various types of illegal flags and flags of various political parties around the world, and flag images with similar features to various illegal flags are collected to create a dataset of flags with confusing features.

[0030] Thus, obfuscating the feature dataset can significantly improve the detection model's ability to identify features of illegal flags.

[0031] As one embodiment of the present invention, in the step of optimizing the model algorithm, YOLOV7 is used as the teacher model, and the feature maps output by the backbone network and the feature maps output by the head of the YOLOV7 model are simultaneously subjected to focal global distillation. In the early stage of model training, the feature distillation based on the backbone network can accelerate the convergence of the student model. In the later stage of model training, the features output by the head of the teacher model can better guide the student model to achieve higher detection accuracy. At the same time, the output features of the backbone network and the head of the YOLOV7-tiny object detection algorithm model are distilled, and adaptive weights are designed for the two parts of features.

[0032] Thus, the model detection accuracy achieved by performing focal global distillation on the feature maps output from the backbone network and the head output of the YOLOV7 model is significantly better than the effect of distillation based solely on head features. The adaptive Siamese distillation YOLOV7-tiny object detection algorithm model can significantly improve the detection accuracy of illegal flags without affecting the model size or inference speed, and also has a faster convergence speed. At the same time, it also brings significant performance improvement on the public object detection dataset Pascal VOC, demonstrating the versatility of our proposed method.

[0033] In one embodiment of the present invention, in the model training step, firstly, the YOLOv7 object detection algorithm model is trained using the prepared dataset to obtain a teacher model, and then the YOLOv7-tiny object detection algorithm model is trained using adaptive twin distillation to obtain a preliminary target flag detection model, adaptive twin distillation YOLOv7-tiny.

[0034] This can effectively improve the detection accuracy of illegal flags.

[0035] As one embodiment of the present invention, in the model training step, if it is found that the target area containing the flag is cropped, the stitching process is repeated until the enhanced image meets the requirement of not damaging the flag area.

[0036] Therefore, YOLOv7's built-in Mosic data augmentation method, which randomly combines and crops images, can cause some loss and damage to the target flag region. This is particularly damaging to the limited sample size of illegal flag data, severely impacting the model's final detection performance. To address this, we propose Boundary Constraints-Mosaic (BC-Mosaic), a Mosaic data augmentation method suitable for small samples. During Mosic data augmentation, if a target region containing a flag is found to be cropped, the image is re-stitched until the augmented image meets the requirement of not damaging the flag region. This fully utilizes limited dataset resources, allowing the model to learn more features. Experiments show that using BC-Mosaic for data augmentation improves the flag detection model's accuracy by 2 percentage points. Furthermore, experiments were conducted on a small-sample public dataset. Two hundred images from each class were randomly selected from the Pascal VOC object detection dataset for training. The model trained using the BC-Mosaic data augmentation method showed significantly higher detection accuracy than the traditional Mosaic augmentation method.

[0037] In one embodiment of the present invention, the proportion of color-enhanced data decreases cosinely with the number of training iterations of the model through progressive annealing color enhancement.

[0038] Thus, through numerous experiments, it was found that flag detection has very high requirements for color features. Therefore, it is unreasonable to use color enhancement as a data augmentation method throughout the entire training process. This invention proposes a progressive annealing color enhancement method, in which the proportion of color enhancement decreases cosinely with the number of training iterations of the model. This method does not affect the convergence speed of the model and can improve the accuracy of the model.

[0039] As one embodiment of the present invention, in the model optimization step, the following strategy is used to optimize the preliminary illegal flag detection model:

[0040] An adaptive mapping strategy is employed for random mapping. A flag image is randomly selected, and the flag target area is cropped. A background image is randomly selected as the mapping background. The cropped image is randomly pasted onto the background image, with each image being randomly pasted 3-5 times. This ensures that the mapping position does not overlap with other target areas. A dynamic mapping method is used instead of static mapping before inputting the images into the network for training. During each iteration of the training process, the mapping is performed in accordance with the base image. Figure 1 Second-rate;

[0041] The confidence-level data retraining mechanism uses the original trained model results to infer and detect compliance flag dataset images. False detection images are graded according to their confidence scores to create a model performance enhancement dataset. The created dataset is then fed into the network for retraining.

[0042] The model is optimized using a quantitative perception training strategy.

[0043] Thus, through experimental comparison and verification, dynamic textures performed far better than static textures, and the higher the randomness, the better the effect. The model inferred and detected more than 3 million normal and compliant images, covering various scenes, and the model has a high generalization ability. Experiments revealed that when the trained model was deployed to terminal devices, the model accuracy dropped significantly due to quantization issues. To solve this problem, we abandoned the training-then-quantization approach and adopted a quantization-aware training strategy to optimize the model, which solved the problem of significantly reduced model deployment accuracy.

[0044] As one embodiment of the present invention, in the model deployment step, the illegal flag detection model is deployed to the AI ​​edge computing device development board, which can detect illegal flag images in videos in real time in offline or local area network conditions.

[0045] In this way, illegal flag images in videos can be detected in real time and accurately, whether offline or on a local area network.

[0046] In one embodiment of the present invention, in the video detection step, the video input to the development board is extracted frame by frame and then fed into the trained illegal flag detection model for detection.

[0047] Thus, in the video detection stage, due to the advantages of the detection accuracy and inference speed of the adaptive twin distillation YOLOv7-tiny, the input video is sampled frame by frame and then fed into the trained detection model for detection. Compared with other models that have to set frame sampling intervals due to the limitation of inference speed, frame-by-frame detection has better results, higher accuracy, and ensures real-time detection.

[0048] The above description of the invention is merely an overview of the technical solution of this application. In order to enable those skilled in the art to better understand the technical solution of this application and to implement it based on the description and drawings, and to make the above-mentioned objectives and other objectives, features and advantages of this application easier to understand, the following description is provided in conjunction with the specific embodiments and drawings of this application. Attached Figure Description

[0049] The accompanying drawings are only used to illustrate the principles, implementation methods, applications, features, and effects of specific embodiments of this application and other related content, and should not be considered as limitations on this application.

[0050] In the accompanying drawings of the instruction manual:

[0051] Figure 1 This is a system architecture diagram of a real-time illegal flag detection based on the YOLOv7 target detection algorithm according to an embodiment of this application;

[0052] Figure 2 This is a flowchart illustrating a real-time illegal flag detection method based on the YOLOv7 target detection algorithm according to an embodiment of this application.

[0053] Figure 3 This is an example diagram of an AI edge computing device development board according to one embodiment of this application;

[0054] Figure 4 A diagram illustrating the annotation process for a dataset according to one embodiment of this application;

[0055] Figure 5 This is a schematic diagram of the training phase of the twin distillation YOLOV7-tiny target detection algorithm model according to an embodiment of this application;

[0056] Figure 6 This is a schematic diagram of the detection stage of the twin distillation YOLOV7-tiny target detection algorithm model according to an embodiment of this application;

[0057] Figure 7 This is a detection effect diagram of one embodiment of this application. Detailed Implementation

[0058] To illustrate the possible application scenarios, technical principles, implementable specific solutions, and achievable objectives and effects of this application in detail, the following description, in conjunction with the listed specific embodiments and accompanying drawings, provides a detailed explanation. The embodiments described herein are merely illustrative of the technical solutions of this application and are therefore intended to limit the scope of protection of this application.

[0059] In this document, the term "embodiment" means that a specific feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The term "embodiment" appearing in various places throughout the specification does not necessarily refer to the same embodiment, nor does it specifically limit its independence or connection with other embodiments. In principle, in this application, as long as there are no technical contradictions or conflicts, the technical features mentioned in each embodiment can be combined in any way to form corresponding implementable technical solutions.

[0060] Unless otherwise defined, the technical terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the use of related terms herein is merely for the purpose of describing particular embodiments and is not intended to limit this application.

[0061] In the description of this application, the term "and / or" is used to describe the logical relationship between objects, indicating that three relationships can exist. For example, A and / or B means: A exists, B exists, and A and B exist simultaneously. Additionally, the character " / " in this document generally indicates that the preceding and following objects have an "or" logical relationship.

[0062] In this application, terms such as “first” and “second” are used only to distinguish one entity or operation from another, and do not necessarily require or imply any actual quantity, hierarchy or order relationship between these entities or operations.

[0063] Unless otherwise specified, the use of terms such as “comprising,” “including,” “having,” or other similar expressions in this application is intended to cover non-exclusive inclusion, which does not exclude the presence of additional elements in a process, method, or product that includes the stated elements, such that a process, method, or product that includes a list of elements may include not only those defined elements but also other elements not expressly listed, or elements inherent to such a process, method, or product.

[0064] Similar to the understanding in the Examination Guidelines, in this application, expressions such as "greater than," "less than," and "exceeding" are understood to exclude the stated number; expressions such as "above," "below," and "within" are understood to include the stated number. Furthermore, in the description of the embodiments in this application, "multiple" means two or more (including two), and similar expressions related to "multiple" are also understood in this way, such as "multiple groups" and "multiple times," unless otherwise explicitly specified.

[0065] In existing technologies, there is no dataset of flags with obfuscated features, the ability to identify illegal flags is poor, the detection accuracy of YOLO series object detection algorithms is poor, and the slow inference speed of the models makes it impossible to achieve real-time detection on edge devices.

[0066] The applicant's research found that by creating a dataset of flags with obfuscated features, the ability to identify illegal flags can be improved. By improving the YOLOV7 target detection algorithm through distillation based on focus and global target detection, an adaptive twin distillation YOLOV7-tiny target detection algorithm model is obtained, thereby improving detection accuracy.

[0067] The real-time illegal flag detection method based on the YOLOv7 target detection algorithm described in this embodiment can be applied to various technical fields of illegal flag detection in images.

[0068] According to some embodiments of this application, please refer to Figures 1 to 7 This embodiment relates to a real-time illegal flag detection method based on the YOLOv7 target detection algorithm, including the following steps:

[0069] Dataset creation

[0070] By writing a web crawler to collect images of illegal flags, determining the features for detecting illegal flags, and using image annotation tools to annotate the collected images, an illegal flag dataset is created. At the same time, normal flags with similar features to illegal flags are collected to create a confused feature flag dataset. The illegal flag dataset and the confused feature flag dataset are sampled and recombined according to a specific ratio to create training and validation sets.

[0071] This embodiment involved extensive experimental investigation and analysis, revealing that simultaneously selecting the features of the illegal flags and their corresponding logos achieves better results. Therefore, a "flag + logo" illegal flag labeling strategy was proposed. The image labeling tool labelimg was used to label the collected images with both flags and logos, creating an illegal flag dataset. The labeling process is shown in the attached figure. Figure 4 As shown, the illegal flags can be replaced and modified, and flags of various countries and political parties around the world can be marked. Inspired by the ability of humans to improve their discrimination ability by learning similar confusing features, we not only created a dataset of illegal flags, but also conducted a feature comparison study on various illegal flags and flags of various countries and political parties around the world. We specifically collected flag images with similar features to various illegal flags to create a dataset of confusing feature flags. Finally, the two datasets were sampled and recombined in a 7:3 ratio to create training and validation sets.

[0072] Optimize the model algorithm

[0073] Based on focus and global target detection distillation, the YOLOV7 target detection algorithm is improved to obtain the adaptive twin distillation YOLOV7-tiny target detection algorithm model;

[0074] This embodiment applies the concept of focal global distillation to the YOLOv7 object detection algorithm and conducts exploratory research. It finds that by using YOLOv7 as the teacher model and simultaneously performing focal global distillation on both the feature maps output from the backbone network and the feature maps output from the head of the YOLOv7-tiny model, the model detection accuracy is significantly better than distillation based solely on head features. Furthermore, in the early stages of model training, distillation based only on backbone network features accelerates the convergence of the student model. In the later stages of model training, the features output from the teacher model's head can better guide the student model to achieve higher detection accuracy. Based on this, we propose the Adaptive Siamese Distillation-YOLOv7-tiny object detection method, abbreviated as ASD-YOLOv7-tiny object detection algorithm. This method simultaneously distills the output features from both the backbone network and the head of the YOLOv7-tiny model and designs adaptive weights for these two feature parts. The structure diagram is attached. Figure 5 For the training phase of the model, we use YOLOv7 as the teacher model and YOLOv7-tiny as the student model. We perform distillation at both locations simultaneously. The feature loss function output by the backbone network is:

[0075] L1 = F(ft1,fs1) (1)

[0076] The feature loss function for detecting the head output is:

[0077] L2 = F(ft2,fs2) (2)

[0078] Where ft1 and fs1 represent the output feature maps of the backbone networks of the teacher model and the student model, respectively, ft2 and fs2 represent the output feature maps of the detection heads of the teacher model and the student model, respectively, and F is the focus global distillation algorithm proposed in the prior art.

[0079] Let the loss function of the original YOLOv7 model be L3. Then, the final loss function of our proposed ASD-YOLOV7-tiny is:

[0080] loss = w1*L1 + w2*L2 + L3 (3)

[0081] Where w1 and w2 are adaptive weights, w1 decreases from 1 to 0.1 as the number of training iterations increases, and w2 increases from 0 to 0.9 as the number of training iterations increases, with a step size of 0.01. The model is built based on the deep learning framework PyTorch.

[0082] Model training

[0083] The YOLOV7-tiny target detection algorithm model was trained using the prepared dataset to obtain a preliminary target flag detection model, adaptive twin distillation YOLOV7-tiny.

[0084] We first trained a teacher model using YOLOv7 on a pre-trained dataset, and then distilled the YOLOv7-tiny model to obtain the initial target flag detection model. The input image size was 640x640, the batch size was 32, and the model was trained for 400 epochs on a Tesla V100 GPU to obtain the optimal training model.

[0085] Model optimization

[0086] The initial target flag detection model and the illegal flag detection model were optimized through experiments and various strategies. At the same time, the robustness of the target flag detection model and the illegal flag detection model in real-world application scenarios were improved to obtain an illegal flag detection model for deployment. The training and optimization of the illegal flag detection model were both completed on the server side.

[0087] This embodiment adjusts the model parameters through numerous experiments and improves the model to achieve the highest accuracy in flag detection.

[0088] 1. During our experiments, we discovered that YOLOv7's built-in Mosic data augmentation method, which randomly combines and crops images, can cause some loss and damage to the target flag region. This is particularly damaging to the limited sample size of illegal flag data, severely impacting the model's final detection performance. Therefore, we propose Boundary Constraints-Mosaic (BC-Mosaic), a Mosaic data augmentation method suitable for small samples. During Mosic data augmentation, if a target region containing a flag is found to be cropped, the image is re-stitched until the augmented image meets the requirement of not damaging the flag region. This fully utilizes limited dataset resources, allowing the model to learn more features. Experiments show that using BC-Mosaic for data augmentation improves the flag detection model's accuracy by 2 percentage points. Furthermore, we conducted experiments on a small-sample public dataset. We randomly selected one class from the Pascal VOC object detection dataset, selecting 200 images from each class for training. The model trained using the BC-Mosaic data augmentation method showed significantly higher detection accuracy than the traditional Mosaic augmentation method.

[0089] 2. To address the issue of limited data volume, we employed a 4+1 adaptive mapping strategy for random mapping, building upon conventional data augmentation. The four random elements are: randomly selecting a flag image to crop the flag target area, randomly selecting a background image as the mapping background, and randomly pasting the cropped image onto the background image. Each image is randomly pasted 3-5 times to ensure the mapping position does not overlap with other target areas. The one dynamic element is: instead of statically pasting the images before inputting them into the network for training, we use a dynamic mapping method, performing random mapping in each iteration of the training process. Figure 1 Experimental comparisons have shown that dynamic textures are far superior to static textures, and the higher the randomness, the better the effect.

[0090] 3. Extensive experiments revealed that flag detection has very high requirements for color features. Therefore, using color enhancement data augmentation throughout the entire training process is unreasonable. This invention proposes a progressive annealing color enhancement method. Similar to cosine annealing, during the first 100 epochs of model training, the probability of images being selected for color transformation data augmentation is gradually reduced. From the 101st epoch onwards, color transformation data augmentation is no longer used, ensuring the model's accuracy in flag color recognition. The proportion of color enhancement decreases cosinely with the number of training iterations, which neither affects the model's convergence speed nor hinders its accuracy.

[0091] 4. We used the results of the original trained model to infer and detect compliant flag dataset images. We then classified the false positive images according to their confidence scores to create a model performance enhancement dataset. We then put the created dataset into the network for retraining. Using this method, the model inferred and detected more than 3 million normal and compliant images, covering various scenarios, and the model has a high generalization ability.

[0092] 5. Experiments revealed that when the trained model was deployed to terminal devices, the model accuracy decreased significantly due to quantization issues. To address this problem, we abandoned the pre-training and post-quantization approach and adopted a quantization-aware training strategy to optimize the model, thus resolving the issue of significantly reduced model deployment accuracy.

[0093] Model Deployment

[0094] The illegal flag detection model used for deployment is quantified and deployed onto the AI ​​edge computing device development board;

[0095] The training results are deployed to the NVIDIA Jetson Nano development board, an edge computing terminal device (see attached). Figure 3 To improve system performance, we used C++ to complete the deployment process.

[0096] Video detection

[0097] The video input to the development board is processed by extracting frames. The extracted frames are then input into the deployed illegal flag detection model for detection. Images containing illegal flags are filtered or alerted to ensure the compliance of the video scene.

[0098] The video input from the development board is processed by frame extraction. These extracted frames are then fed into a trained model for detection. Images containing illegal flags are filtered out, and a safe, playable video is output. Due to the extremely high inference speed of YOLOv7, we perform frame-by-frame detection to ensure the compliance of each frame. Final system test results show that, in offline mode, the flag detection system described in this invention can perform real-time and accurate detection of input videos.

[0099] Unlike existing technologies, the above technical solution improves the ability to identify illegal flags by creating a dataset of obfuscated flag features. It improves the YOLOv7 target detection algorithm by distilling based on focus and global target detection, obtaining an adaptive Siamese distillation YOLOv7-tiny target detection algorithm model to improve detection accuracy. The model is optimized through a large number of experiments and various strategies to achieve the highest accuracy. The training and optimization of the model are completed on the server side because it has higher computing power and faster training speed, enabling real-time detection.

[0100] According to some embodiments of this application, optionally, in the step of creating the dataset, when using an image annotation tool to annotate the collected images, the features of the illegal flags and their corresponding logos are selected simultaneously, and then the image annotation tool is used to annotate the illegal flag targets and their corresponding logos appearing in the images to create an illegal flag dataset.

[0101] Thus, the analysis found that selecting both the characteristics of the illegal flag and its corresponding logo can achieve better results, and proposed a "flag + logo" strategy for marking illegal flags.

[0102] According to some embodiments of this application, optionally, in the step of creating the dataset, feature comparison is performed on various types of illegal flags and flags of various political parties around the world, and flag images with similar features to various types of illegal flags are collected to create a dataset of flags with confusing features.

[0103] Thus, obfuscating the feature dataset can significantly improve the detection model's ability to identify features of illegal flags.

[0104] According to some embodiments of this application, optionally, in the step of optimizing the model algorithm, YOLOV7 is used as the teacher model, and the feature maps output by the backbone network and the feature maps output by the head of the YOLOV7 model are simultaneously subjected to focal global distillation. In the early stage of model training, the feature distillation based on the backbone network can accelerate the convergence of the student model. In the later stage of model training, the features output by the head of the teacher model can better guide the student model to achieve higher detection accuracy. At the same time, the output features of the backbone network and the head of the YOLOV7-tiny object detection algorithm model are distilled, and adaptive weights are designed for the two parts of features.

[0105] Thus, the model detection accuracy achieved by performing focal global distillation on the feature maps output from the backbone network and the head output of the YOLOV7 model is significantly better than the effect of distillation based solely on head features. The adaptive Siamese distillation YOLOV7-tiny object detection algorithm model can significantly improve the detection accuracy of illegal flags without affecting the model size or inference speed, and also has a faster convergence speed. At the same time, it also brings significant performance improvement on the public object detection dataset Pascal VOC, demonstrating the versatility of our proposed method.

[0106] According to some embodiments of this application, optionally, firstly, a teacher model is obtained by training the YOLOv7 object detection algorithm model using a pre-prepared dataset; then, the YOLOv7-tiny object detection algorithm model is trained using adaptive Siamese distillation to obtain a preliminary target flag detection model, adaptive Siamese distillation YOLOv7-tiny. In the model training step, the teacher model is first obtained by training the Siamese distillation YOLOv7-tiny object detection algorithm model using a pre-prepared dataset; then, the Siamese distillation YOLOv7-tiny object detection algorithm model is trained using distillation to obtain a preliminary target flag detection model.

[0107] This can effectively improve the detection accuracy of illegal flags.

[0108] According to some embodiments of this application, optionally, during the model training step, if it is found that the target area containing the flag is cropped, the stitching process is re-performed until the enhanced image meets the requirement of not damaging the flag area.

[0109] Therefore, YOLOv7's built-in Mosic data augmentation method, which randomly combines and crops images, can cause some loss and damage to the target flag region. This is particularly damaging to the limited sample size of illegal flag data, severely impacting the model's final detection performance. To address this, we propose Boundary Constraints-Mosaic (BC-Mosaic), a Mosaic data augmentation method suitable for small samples. During Mosic data augmentation, if a target region containing a flag is found to be cropped, the image is re-stitched until the augmented image meets the requirement of not damaging the flag region. This fully utilizes limited dataset resources, allowing the model to learn more features. Experiments show that using BC-Mosaic for data augmentation improves the flag detection model's accuracy by 2 percentage points. Furthermore, experiments were conducted on a small-sample public dataset. Two hundred images from each class were randomly selected from the Pascal VOC object detection dataset for training. The model trained using the BC-Mosaic data augmentation method showed significantly higher detection accuracy than the traditional Mosaic augmentation method.

[0110] According to some embodiments of this application, optionally, the proportion of color-enhanced data decreases cosinely with the number of training iterations of the model through progressive annealing.

[0111] Thus, through numerous experiments, it was found that flag detection has very high requirements for color features. Therefore, it is unreasonable to use color enhancement as a data augmentation method throughout the entire training process. This invention proposes a progressive annealing color enhancement method, in which the proportion of color enhancement decreases cosinely with the number of training iterations of the model. This method does not affect the convergence speed of the model and can improve the accuracy of the model.

[0112] According to some embodiments of this application, optionally, in the model optimization step, the following strategies are used to optimize the initial illegal flag detection model:

[0113] An adaptive mapping strategy is employed for random mapping. A flag image is randomly selected, and the flag target area is cropped. A background image is randomly selected as the mapping background. The cropped image is randomly pasted onto the background image, with each image being randomly pasted 3-5 times. This ensures that the mapping position does not overlap with other target areas. A dynamic mapping method is used instead of static mapping before inputting the images into the network for training. During each iteration of the training process, the mapping is performed in accordance with the base image. Figure 1 Second-rate;

[0114] The confidence-level data retraining mechanism uses the original trained model results to infer and detect compliance flag dataset images. False detection images are graded according to their confidence scores to create a model performance enhancement dataset. The created dataset is then fed into the network for retraining.

[0115] The model is optimized using a quantitative perception training strategy.

[0116] Thus, through experimental comparison and verification, dynamic textures performed far better than static textures, and the higher the randomness, the better the effect. The model inferred and detected more than 3 million normal and compliant images, covering various scenes, and the model has a high generalization ability. Experiments revealed that when the trained model was deployed to terminal devices, the model accuracy dropped significantly due to quantization issues. To solve this problem, we abandoned the training-then-quantization approach and adopted a quantization-aware training strategy to optimize the model, which solved the problem of significantly reduced model deployment accuracy.

[0117] According to some embodiments of this application, optionally, in the model deployment step, the illegal flag detection model is deployed to an AI edge computing device development board, which can detect illegal flag images in videos in real time, either offline or in a local area network state.

[0118] In this way, illegal flag images in videos can be detected in real time and accurately, whether offline or on a local area network.

[0119] According to some embodiments of this application, optionally, in the video detection step, the video input to the development board is extracted frame by frame and then sent to the trained illegal flag detection model for detection.

[0120] Thus, in the video detection stage, due to the advantages of YOLOv7's detection accuracy and inference speed, the input video is sampled frame by frame and then fed into the trained detection model for detection. Compared with other models that have to set frame sampling intervals due to the limitation of inference speed, frame-by-frame detection has better results, higher accuracy, and ensures real-time detection.

[0121] Those skilled in the art will understand that the above embodiments can be provided as methods, apparatus, or computer program products. These embodiments may take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. All or part of the steps in the methods involved in the above embodiments can be implemented by a program instructing related hardware, and the program can be stored in a computer-readable storage medium for executing all or part of the steps described in the methods of the above embodiments. The computer device includes, but is not limited to: personal computers, servers, general-purpose computers, special-purpose computers, network devices, embedded devices, programmable devices, smart mobile terminals, smart home devices, wearable smart devices, in-vehicle smart devices, etc.; the storage medium includes, but is not limited to: RAM, ROM, magnetic disks, magnetic tapes, optical disks, flash memory, USB flash drives, portable hard drives, memory cards, memory sticks, network server storage, network cloud storage, etc.

[0122] The above embodiments are described with reference to flowchart illustrations and / or block diagrams of the methods, apparatus (systems), and computer program products according to the embodiments. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a computer device to produce a machine, such that the instructions, which execute via the processor of the computer device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0123] These computer program instructions may also be stored in a computer device-readable storage medium that can direct a computer device to operate in a particular manner, such that the instructions stored in the computer device-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0124] These computer program instructions can also be loaded onto a computer device, causing a series of operational steps to be performed on the computer device to produce a computer-implemented process, thereby providing instructions that execute on the computer device for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0125] It should be noted that although the above embodiments have been described herein, this does not limit the scope of patent protection of the present invention. Therefore, any changes and modifications made to the embodiments described herein based on the innovative concept of the present invention, or equivalent structural or procedural transformations made using the content of the present invention's specification and drawings, directly or indirectly applying the above technical solutions to other related technical fields, are all included within the scope of patent protection of the present invention.

Claims

1. A real-time illegal flag detection method based on the YOLOv7 target detection algorithm, characterized in that, Includes the following steps: Dataset creation By writing a web crawler to collect images of illegal flags, determining the features for detecting illegal flags, and using image annotation tools to annotate the collected images, an illegal flag dataset is created. At the same time, normal flags with similar features to illegal flags are collected to create a confused feature flag dataset. The illegal flag dataset and the confused feature flag dataset are sampled and recombined according to a specific ratio to create training and validation sets. Optimize the model algorithm Based on focus and global target detection distillation, the YOLOV7 target detection algorithm is improved to obtain the adaptive twin distillation YOLOV7-tiny target detection algorithm model; Model training The YOLOV7-tiny target detection algorithm model was trained by twin distillation using the prepared dataset to obtain a preliminary target flag detection model adaptive twin distillation YOLOV7-tiny. Model optimization The initial target flag detection model was optimized through experiments and various strategies. At the same time, the robustness of the target flag detection model in real-world application scenarios was improved to obtain an illegal flag detection model for deployment. The training and optimization of the illegal flag detection model were both completed on the server side. In the model optimization step, the following strategies are used to optimize the initial illegal flag detection model: An adaptive mapping strategy is adopted for random mapping. A flag image is randomly selected and cropped for the flag target area. A background image is randomly selected as the mapping background. The cropped image is randomly pasted onto the background image. Each image is randomly pasted 3-5 times to ensure that the mapping position does not overlap with other target areas. A dynamic mapping method is adopted instead of static mapping and then inputting it into the network for training. In each iteration of the training process, a random mapping is performed once. The confidence-level data retraining mechanism uses the original trained model results to infer and detect compliance flag dataset images. False detection images are graded according to their confidence scores to create a model performance enhancement dataset. The created dataset is then fed into the network for retraining. The model is optimized using a quantitative perception training strategy; Model Deployment The illegal flag detection model used for deployment is quantified and deployed onto the AI ​​edge computing device development board; Video detection The video input to the development board is processed by extracting frames. The extracted frames are then input into the deployed illegal flag detection model for detection. Images containing illegal flags are filtered or alerted to ensure the compliance of the video scene.

2. The real-time illegal flag detection method based on the YOLOv7 target detection algorithm according to claim 1, characterized in that, In the process of creating the dataset, when annotating the collected images using an image annotation tool, the features of the illegal flags and their corresponding logos are selected. Then, the image annotation tool is used to annotate the illegal flag targets and their corresponding logos in the images to create the illegal flag dataset.

3. The real-time illegal flag detection method based on the YOLOv7 target detection algorithm according to claim 2, characterized in that, In the process of creating the dataset, features of various illegal flags and flags of political parties from different countries around the world are compared. Flag images with similar features to various illegal flags are collected to create a dataset of flags with confusing features.

4. The real-time illegal flag detection method based on the YOLOv7 target detection algorithm according to claim 1, characterized in that, In the optimization step of the model algorithm, YOLOV7 is used as the teacher model, and the feature maps output by the backbone network and the head of the YOLOV7 model are subjected to focal global distillation. In the early stage of model training, the feature distillation based on the backbone network can accelerate the convergence of the student model. In the later stage of model training, the features output by the head of the teacher model can better guide the student model to achieve higher detection accuracy. At the same time, the output features of the backbone network and the head of the YOLOV7-tiny object detection algorithm model are distilled, and adaptive weights are designed for the two parts of features.

5. The real-time illegal flag detection method based on the YOLOv7 target detection algorithm according to claim 1, characterized in that, In the model training process, firstly, the YOLOv7 model is trained using the prepared dataset to obtain the teacher model. Then, the YOLOv7-tiny target detection algorithm model is trained using adaptive twin distillation to obtain the preliminary target flag detection model, adaptive twin distillation YOLOv7-tiny.

6. The real-time illegal flag detection method based on the YOLOv7 target detection algorithm according to claim 5, characterized in that, During the model training process, if it is found that the target area containing the flag has been cropped, the image is re-stitched until the enhanced image meets the requirement of not damaging the flag area.

7. The real-time illegal flag detection method based on the YOLOv7 target detection algorithm according to claim 6, characterized in that, The color enhancement method using progressive annealing results in a cosine-dependent decrease in the proportion of color-enhanced data as the model is trained.

8. The real-time illegal flag detection method based on the YOLOv7 target detection algorithm according to claim 1, characterized in that, In the model deployment step, the illegal flag detection model is deployed to the AI ​​edge computing device development board, which can detect illegal flag images in videos in real time, either offline or in a local area network.

9. The real-time illegal flag detection method based on the YOLOv7 target detection algorithm according to claim 1, characterized in that, In the video detection step, the video input from the development board is extracted frame by frame and then fed into the trained illegal flag detection model for detection.

Images