Station logo detection and model training method, device, equipment and storage medium

Abstract

The present disclosure provides a method, apparatus, device and storage medium for detecting and training a station logo model, and relates to the field of artificial intelligence technology, specifically to technical fields such as target detection, computer vision, and deep learning. The station logo detection method includes: performing self-supervised training on an initial pre-trained model to obtain a final pre-trained model; using the final pre-trained model to determine the correct position information and correct category information of the station logo in an image sample; based on the correct position information, intercepting a station logo image at the position of the station logo in the image sample; constructing a pseudo image based on a background image and the station logo image, and obtaining the true position information of the station logo image on the pseudo image; and training the initial station logo detection model based on the pseudo image, the true position information and the correct category information to obtain a final station logo detection model. The present disclosure can improve the effect of station logo detection.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, specifically to technical fields such as target detection, computer vision, and deep learning, and in particular to a method, device, equipment, and storage medium for station logo detection and model training. Background Art

[0002] With the continuous development of technology, image recognition technology has gradually matured and is widely used. Among them, station logo detection technology is a type of image recognition technology that can identify various types of station logos in images, such as trademarks, logos, emblems, and logos.

[0003] There are some station logo detection solutions in the relevant technology, but the effect needs to be improved. Summary of the Invention

[0004] The present disclosure provides a station logo detection and model training method, apparatus, device and storage medium.

[0005] According to one aspect of the present disclosure, a method for training a station logo detection model is provided, comprising: performing self-supervised training on an initial pre-trained model to obtain a final pre-trained model; using the final pre-trained model to determine the correct position information and correct category information of the station logo in an image sample; based on the correct position information, intercepting a station logo image at the position of the station logo in the image sample; constructing a pseudo image based on a background image and the station logo image, and obtaining the real position information of the station logo image on the pseudo image; and training the initial station logo detection model based on the pseudo image, the real position information and the correct category information to obtain a final station logo detection model.

[0006] According to another aspect of the present disclosure, a method for detecting a station logo is provided, comprising: acquiring an image to be processed; performing station logo detection processing on the image to be processed using a final station logo detection model to determine a detection result of the station logo in the image to be processed; wherein the final station logo detection model is trained using a method as described in any one of the above embodiments.

[0007] According to another aspect of the present disclosure, a training device for a station logo detection model is provided, comprising: a first training module for performing self-supervised training on an initial pre-trained model to obtain a final pre-trained model; a determination module for determining correct position information and correct category information of the station logo in an image sample using the final pre-trained model; a capture module for capturing a station logo image at the position of the station logo in the image sample based on the correct position information; a construction module for constructing a pseudo image based on a background image and the station logo image, and obtaining real position information of the station logo image on the pseudo image; and a second training module for training the initial station logo detection model based on the pseudo image, the real position information and the correct category information to obtain a final station logo detection model.

[0008] According to another aspect of the present disclosure, a station logo detection device is provided, comprising: an acquisition module for acquiring an image to be processed; a detection module for performing station logo detection processing on the image to be processed using a final station logo detection model to determine a detection result of the station logo in the image to be processed; wherein the final station logo detection model is trained using a method as described in any one of the above embodiments.

[0009] According to another aspect of the present disclosure, an electronic device is provided, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any one of the methods described in any one of the above aspects.

[0010] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to execute any one of the methods according to any one of the above aspects.

[0011] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program, wherein when the computer program is executed by a processor, the computer program implements the method according to any one of the above aspects.

[0012] According to the technical solution disclosed in the present invention, the station logo detection effect can be improved.

[0013] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0015] Figure 1 is a schematic diagram according to a first embodiment of the present disclosure;

[0016] Figure 2 is a schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0017] Figure 3 is a schematic diagram of the overall architecture of a method for training a station logo detection model according to an embodiment of the present disclosure;

[0018] Figure 4 is a schematic diagram of the training and testing process of the final pre-trained model provided according to an embodiment of the present disclosure;

[0019] Figure 5 is a schematic diagram according to a second embodiment of the present disclosure;

[0020] Figure 6 Schematic diagram of the process of obtaining self-supervisory position information according to an embodiment of the present disclosure;

[0021] Figure 7 is a schematic diagram of a pseudo image acquisition process provided according to an embodiment of the present disclosure;

[0022] Figure 8 is a schematic diagram according to a third embodiment of the present disclosure;

[0023] Figure 9 is a schematic diagram according to a fourth embodiment of the present disclosure;

[0024] Figure 10 is a schematic diagram according to a fifth embodiment of the present disclosure;

[0025] Figure 11 Schematic diagram of an electronic device used to implement the station logo detection method or the training method of the station logo detection model according to the embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0027] Station logo detection technology involves processing images to identify station logos within them. A station logo can be represented by a logo, such as a trademark, logotype, emblem, or logo. Specifically, a station logo can be a station logo, such as XX TV Station or XX Satellite TV.

[0028] When performing station logo detection on an image, a station logo detection model can be used. The image is input into the station logo detection model, and the output is the detection result of the station logo. The detection result includes, for example, the location information of the station logo (represented by a 2D detection box) and category information (such as TV station logo, traffic sign, etc.).

[0029] The station logo detection model is a deep neural network model obtained through training.

[0030] In the related art, a station logo detection model can be trained and used to detect station logos. When training a station logo detection model, a large amount of manually annotated data is required for image samples, for example, the location information and category information of various logos in the image samples. Since there are many categories of logos and the targets are small, the annotation cost is high. In general, training is performed directly based on image samples and their annotated data. For example, the image samples are input into the model to be trained to obtain the predicted value. The loss function is constructed by comparing the manually annotated true value with the predicted value, and the model parameters are adjusted using the loss function. The effect of the station logo detection model obtained based on this method also needs to be improved.

[0031] In order to improve the detection effect of station logos, the present disclosure provides the following embodiments.

[0032] Figure 1 is a schematic diagram according to the first embodiment of the present disclosure. This embodiment provides a method for training a station logo detection model, the method comprising:

[0033] 101. Perform self-supervised training on the initial pre-trained model to obtain the final pre-trained model.

[0034] 102. Use the final pre-trained model to determine correct position information and correct category information of the station logo in the image sample.

[0035] 103. Based on the correct position information, capture the station logo image at the position where the station logo is located in the image sample.

[0036] 104. Construct a pseudo image based on the background image and the station logo image, and obtain real position information of the station logo image on the pseudo image.

[0037] 105. Train an initial station logo detection model based on the pseudo image, the real position information, and the correct category information to obtain a final station logo detection model.

[0038] Among them, the initial pre-training model is an existing model with good effect, which can be selected as needed. Specifically, it can be a Region Convolutional Neural Network (RCNN) series model, such as RCNN, Fast RCNN, Cascade RCNN, etc.

[0039] The RCNN series model is a two-stage station logo detection model with the advantages of high precision and good effect.

[0040] RCNN is a technology for object detection based on algorithms such as Convolutional Neural Network (CNN), linear regression, and Support Vector Machine (SVM). RCNN requires feature extraction, classification, and regression for each region proposal.

[0041] Fast RCNN is an improvement based on RCNN. It can extract features of the entire image. Different proposed regions can share image features, which effectively improves the training speed.

[0042] CascadeRCNN extends the structure of FastRCNN and includes multiple detection heads. The output of the previous detection head is used as the input of the next detection head, which effectively improves the training accuracy.

[0043] Self-supervised training, also known as self-supervised learning, aims to improve the model's feature extraction capabilities by designing auxiliary tasks to mine the data's own representational features as supervisory signals for unlabeled data.

[0044] Since self-supervised training does not require manually labeled data, the labeling cost can be reduced.

[0045] After obtaining the final pre-trained model through self-supervised training, the final pre-trained model can be used to detect the station logo in the image sample.

[0046] The final pre-trained model can be used to obtain the location information and category information of the station logo. The location information is, for example, a detection box. For the sake of distinction, the location information and category information obtained by the final pre-trained model can be referred to as correct location information and correct category information.

[0047] After obtaining the correct position information of the station logo, the image sample can be intercepted based on the correct position information to obtain a partial image at the location of the station logo. This partial image can be called a station logo image.

[0048] The background image is an image of a specific scene collected in advance. For example, for a news scene, a video of the news scene may be subjected to frame extraction processing to obtain an image, which is used as the background image.

[0049] Furthermore, the background image can include background images of multiple scenes, for example, the background image of the above-mentioned news scene, and the entertainment video can also be frame-extracted to obtain the background image of the entertainment scene, so that the background image can include the background image of the news scene and the background image of the entertainment scene, etc.

[0050] After obtaining the background image and the station logo image, a pseudo image can be constructed based on the background image and the station logo image. Specifically, the station logo image can be pasted onto the background image. Since the station logo image is obtained by intercepting the location where the station logo is located, the station logo image contains the station logo. The background image may not contain the station logo, but after pasting the station logo image onto the background image, a background image containing the station logo can be obtained. The background image containing the station logo can be called a pseudo image. Specifically, pasting the station logo image onto the background image means replacing the pixel values ​​at the corresponding position on the background image with the pixel values ​​of the station logo image.

[0051] The position information of the station logo image on the pseudo image can be called real position information. The position information can be determined randomly. For example, the position of the station logo image is randomly determined, and the station logo image is pasted to the random position of the background image to obtain a pseudo image.

[0052] After obtaining the pseudo image, the pseudo image, the real position information, and the correct category information can be used to train an initial logo detection model to obtain a final logo detection model. The final logo detection model will be used for logo detection in the inference phase.

[0053] The station logo detection model is usually smaller in scale than the above-mentioned pre-trained model. Due to its smaller scale, the speed of station logo detection can be accelerated and the processing efficiency can be improved.

[0054] Specifically, the pseudo image can be input into the station logo detection model, and the predicted information of the station logo, the real position information and the correct category information can be output as the real information. A loss function is constructed based on the real information and the predicted information, and the parameters of the station logo detection model are adjusted using the loss function until the preset conditions are met (such as reaching the preset number of iterations) to obtain the final station logo detection model.

[0055] Among them, the station logo detection model (initial station logo detection model and final station logo detection model) can be a yolo (you only look once) series model, specifically a PaddlePaddle yolo (pp-yoloe) model.

[0056] The Yolo model is a one-stage logo detection model that can improve detection efficiency.

[0057] pp-yoloe is a target detection model based on yolo, which can detect small targets quickly and well.

[0058] In this embodiment, the station logo detection model is trained based on pseudo images, real position information and correct category information, rather than directly training based on image samples and manually annotated data. On the one hand, the real position information and correct category information are obtained based on the final pre-trained model, and do not require manual annotation, which can reduce the annotation cost. On the other hand, the final pre-trained model is adopted, that is, another model (pre-trained model) is introduced in addition to the station logo detection model. Compared with the method of directly training the station logo detection model, the good characteristics of the final pre-trained model can be utilized to improve the performance of the station logo detection model. In addition, due to the variety of station logo categories, the above-mentioned method without manual annotation can significantly reduce the amount of annotation, which is suitable for station logo detection scenarios; due to the small size of the station logo, the correct position information and correct category information are obtained through the final pre-trained model, and subsequent training is carried out. By utilizing the performance of the final pre-trained model, it is also beneficial to improve the detection effect of the station logo; and by constructing pseudo images, the performance of the station logo detection model can be enhanced, which is beneficial to the detection of station logos of various categories. The final pre-trained model is obtained through self-supervised training. Since self-supervised training does not require manual labeling of data, the labeling cost can be reduced. By constructing pseudo images based on background images and station logo images, and training the station logo detection model based on the pseudo images, pseudo images of different scenes can be constructed, thereby improving the scalability of the station logo detection model.

[0059] In order to better understand the present disclosure, the application scenarios of the embodiments of the present disclosure are described.

[0060] Figure 2 This is a schematic diagram of an application scenario provided by an embodiment of the present disclosure. Figure 2 As shown, the electronic device 201 can train a station logo detection model based on pre-collected image samples and background images, and the obtained station logo detection model can be used in the station logo detection process in the inference stage. The electronic device 201 can be a user terminal, such as a personal computer (Personal Computer), a laptop computer, a mobile device (such as a mobile phone), etc. Alternatively, the electronic device 201 can also be a server, and the server can be a cloud server or a local server. Alternatively, the user can also send the image sample and background image to the server through the user terminal, and the server will perform model training and then return the obtained station logo detection model to the user terminal, and perform station logo detection in the inference stage through the user terminal.

[0061] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0062] Take the cascade RCNN model as an example and the pp-yoloe model as the pre-training model. Figure 3 As shown, the training process of the overall station logo detection model includes: self-supervised training of the initial cascade RCNN model to obtain the final cascade RCNN model; using the final cascade RCNN model to perform station logo detection on image samples to obtain the regional information of the station logo (such as detection frame information); based on the regional information, the image samples are intercepted to obtain the station logo image; based on the station logo image and the background image, a pseudo image is constructed, and the pseudo image is supervised training to obtain the pp-yoloe model.

[0063] For self-supervised training and testing, such as Figure 4 As shown, the image sample can be segmented to obtain the mask information (mask) of the station logo in the image sample, and the self-supervised position information is obtained based on the mask information. The initial cascade RCNN model is self-supervised trained according to the self-supervised position information to obtain the final cascade RCNN model, and then the final cascade RCNN model is used to detect the image sample to obtain the regional information of the station logo. The above-mentioned image segmentation processing can specifically use the TokenCut algorithm for image segmentation processing. The TokenCut algorithm is a station logo segmentation algorithm that specifically uses the features learned by the self-supervised Transformer (Vision Transformer) and uses graph segmentation (graph cut) to detect and segment salient station logos in images and videos.

[0064] In combination with the above application scenarios, the present disclosure also provides a training method for a station logo detection model.

[0065] Figure 5 is a schematic diagram according to a second embodiment of the present disclosure. This embodiment provides a method for training a station logo detection model, the method comprising:

[0066] 501. Based on the image sample, obtain self-supervised position information of the station logo in the image sample.

[0067] The image sample may be subjected to image segmentation processing to obtain mask information of the station logo in the image sample; and the self-supervised position information of the station logo may be obtained based on the mask information.

[0068] Among them, such as Figure 6As shown, the TokenCut algorithm can be used to perform image segmentation processing on the image sample to obtain mask information (mask) 601 of the logo in the image sample.

[0069] After obtaining the mask information 601 of the station logo, an outer bounding box 602 of the mask information may be calculated, and the outer bounding box 602 may be used as the self-supervisory position information of the station logo.

[0070] In this embodiment, by determining the mask information of the station logo through image segmentation processing, relatively accurate mask information can be obtained, and self-supervised position information can be obtained based on the mask information, which can improve the accuracy of the self-supervised position information and thus improve the accuracy of the model.

[0071] In addition, since the station logo is usually the foreground, the TokenCut algorithm can effectively detect the foreground. Therefore, based on the TokenCut algorithm, accurate mask information of the station logo can be obtained, thereby improving the model accuracy and station logo detection accuracy.

[0072] 502. Use an initial pre-trained model to perform prediction processing on the image sample to obtain predicted position information of the station logo.

[0073] The input of the initial pre-trained model is an image sample, and the output includes the predicted position information of the station logo in the image sample.

[0074] 503. Construct a self-supervised loss function based on the self-supervised position information of the station logo and the predicted position information of the station logo.

[0075] Among them, the loss function is constructed based on the true value and the predicted value. In the self-supervised scenario, the above-mentioned self-supervised position information is used as the true value, and the predicted position information obtained by the initial pre-training model is used as the predicted value, so that the corresponding loss function can be constructed. This loss function can be called a self-supervised loss function.

[0076] In this embodiment, a self-supervised loss function is constructed based on the self-supervised position information and the predicted position information. Since the self-supervised position information is obtained based on image samples, no manual labeling is required, which can reduce the labeling cost and save resource overhead.

[0077] Furthermore, there are multiple predicted position information, and constructing the self-supervised loss function based on the predicted position information and the self-supervised position information may include: calculating the overlap (Intersection over Union, IoU) between each predicted position information and the self-supervised position information; among the multiple predicted position information, selecting the predicted position information whose overlap is greater than a preset threshold; and constructing the self-supervised loss function based on the selected predicted position information and the self-supervised position information.

[0078] For a certain predicted position information A and self-supervised position information B, the calculation formula of the IoU between A and B is: IoU = (A∩B) / (A∪B); where A∩B is the area of ​​the intersection area of ​​A and B, and A∪B is the area of ​​the union area of ​​A and B.

[0079] The IoU value ranges from 0 to 1. A larger value indicates a higher degree of overlap between the predicted position (predicted box) and the true position (true box), and a more accurate detection result. When IoU equals 1, it means that the predicted box completely covers the true box.

[0080] The above preset threshold can be selected as 0.01. Therefore, the predicted position information and self-supervised position information with IoU greater than 0.01 are selected to construct the loss function.

[0081] The mask information generated by the TokenCut algorithm may miss some pixels of the station logo. Using such a mask as the ground truth for subsequent training will be detrimental to model performance because the ground truth is imperfect. The loss function constructed in this way can be called the DropLoss function. That is, during training, if the overlap between the predicted position information and the self-supervised position information (the ground truth) is less than 0.01, the loss value of the predicted position information is ignored, encouraging exploration of different image regions.

[0082] 504. Based on the self-supervised loss function, train the initial pre-training model to obtain a final pre-training model.

[0083] After obtaining the self-supervised loss function, the back propagation (BP) algorithm can be used to adjust the model parameters based on the self-supervised loss function until the preset conditions (such as the preset number of iterations) are met, resulting in the final pre-trained model. Taking the cascade RCNN model as an example, since the pre-trained model includes a backbone network (CNN network) and a head network, the backbone network parameters and the head network parameters are adjusted.

[0084] In this embodiment, the self-supervised loss function is constructed based on the selected predicted position information and the self-supervised position information, and part of the predicted position information can be ignored during training, thereby improving the effect of the final pre-trained model.

[0085] 505. Use the final pre-trained model to determine correct position information and correct category information of the station logo in the image sample.

[0086] After obtaining the final pre-trained model, the image sample can be input into the final pre-trained model for processing. The output of the final pre-trained model includes the location information of the station logo within the image sample. To distinguish it from the predicted location information during the training process, the location information obtained by the final pre-trained model can be referred to as correct location information. In addition to the aforementioned location information, the output of the final pre-trained model also includes the category information of the station logo, which can be referred to as correct category information.

[0087] Since the final pre-training model has a better effect, the correct position information that can be obtained is also better.

[0088] 506. Based on the correct position information, capture the station logo image at the position where the station logo is located in the image sample.

[0089] The correct position information is usually a detection frame. In the image sample, an image of the area where the detection frame is located can be captured as a station logo image, and the station logo image contains the station logo.

[0090] 507 : Construct a pseudo image based on the background image and the station logo image, and obtain real position information of the station logo image on the pseudo image.

[0091] The background image may include background images of various scenes, such as background images of entertainment scenes, background images of news scenes, etc. Taking a news scene as an example, frame extraction processing can be performed in a news video to obtain a background image of the news scene.

[0092] Specifically, the method may include: determining a random position within a preset area of ​​the background image, wherein the size of the random position is consistent with the size of the station logo image; and replacing the pixel values ​​of the random position in the background image with the pixel values ​​of the station logo image to obtain the pseudo image. This process can be referred to as pasting the station logo image onto the background image to obtain the pseudo image.

[0093] The preset area can be selected as needed, for example, the long side of the station logo image can be controlled within a set range of the background image, such as a range of 0.008-0.16.

[0094] For example, Figure 7 As shown, after determining a random position on the background image 701 , the station logo image 702 is pasted on the random position to obtain a pseudo image 703 .

[0095] In addition, if the logo image is too large, it can be scaled down first and then pasted onto the background image. For example, if the long side of the logo image is larger than a preset value, such as 360 pixels, the entire logo image can be scaled down by half.

[0096] In addition, the station logo can be of multiple categories, and the corresponding station logo images can be of multiple categories. Each background image can have a random number of station logo images within a preset range attached to it. For example, if the preset range is 1 to 5, then a number (such as 3) can be randomly determined within the range (1 to 5). Then, the random number of station logo images can be selected from multiple categories, such as selecting 3 station logo images, and then attaching the 3 station logo images to the same background image. Multiple station logo images on the same background image do not intersect with each other.

[0097] In addition, each logo image can be pasted onto a preset number (eg, 50) of background images.

[0098] In this embodiment, a pseudo image is constructed based on a random position, and then a station logo detection model is trained based on the pseudo image. This can improve the robustness of the station logo detection model and enhance the model detection effect.

[0099] 508. Based on the pseudo image, the real position information, and the correct category information, train an initial station logo detection model to obtain a final station logo detection model.

[0100] Based on the above example, the real position information is the above-mentioned random information. Training the station logo detection model may specifically include: using the initial station logo detection model to process the pseudo image to obtain the output information of the initial station logo detection model; constructing a supervised loss function based on the random position, correct category information and the output information; based on the supervised loss function, training the initial station logo detection model to obtain the final station logo detection model.

[0101] For the initial logo detection model, a supervised training method can be used, and the corresponding loss function can be called a supervised loss function.

[0102] After obtaining the pseudo image, the pseudo image can be input into the initial station logo detection model for processing to obtain output information. The output information can specifically include: predicted position information and predicted category information of the station logo in the pseudo image.

[0103] The supervised loss function L can be constructed based on the position loss function L1 and the category loss function L2, such as L=L1+L2.

[0104] The loss function is constructed based on the true value and the predicted value. For the position loss function L1, the predicted value is the predicted position information obtained by the initial logo detection model, and the true value is the random position corresponding to the logo image. For the category loss function L2, the predicted value is the predicted category information obtained by the initial logo detection model, and the true value can be the category information obtained by the final pre-trained model.

[0105] Furthermore, in order to solve the long-tail problem between categories, the equalization loss function (EqualizationLoss) can be specifically used for the category loss function L2. The equalization loss function is a loss function used to solve the imbalance between classes. Its tail class solves the problem of long-tail rare categories by simply ignoring the influence of the gradient of the head class on it.

[0106] Through EqualizationLoss, the sample weights can be dynamically adjusted to balance the number of samples in different categories, thereby optimizing and improving the accuracy of the model.

[0107] The trained station logo detection model can be a pp-yoloe model, which includes a backbone network, a neck network and a head network. For the neck network, the initial width and depth (the width and depth corresponding to the CNN layer) can be changed. After testing, the width and depth corresponding to the neck network in the final model structure can be set to 1 / 3 of the initial one, which can significantly improve the processing speed.

[0108] In this embodiment, since both the random position and the correct category information can be obtained based on image samples, manual labeling is not required, which can reduce the labeling cost; the scale of the station logo detection model (such as pp-yoloe) is usually smaller than the scale of the pre-trained model (such as cascadeRCNN), which can provide users with a smaller final station logo detection model, reduce user resource overhead, and improve reasoning efficiency.

[0109] Overall, cascadeRCNN can be selected as the pre-training model, pp-yoloe can be selected as the station logo detection model, and TokenCut algorithm can be selected for image segmentation. These models and algorithms have better performance because the final station logo detection model has better effect, which can reduce the false detection rate and missed detection rate, and also improve the processing speed.

[0110] The final logo detection model can be obtained through the above method, and the final logo detection model can be used for logo detection in the inference stage.

[0111] Figure 8 FIG3 is a schematic diagram according to a third embodiment of the present disclosure. This embodiment provides a station logo detection method, the method comprising:

[0112] 801. Obtain an image to be processed.

[0113] 802. Use the final station logo detection model to perform station logo detection processing on the image to be processed to determine a detection result of the station logo in the image to be processed.

[0114] The final station logo detection model can be obtained using the method shown in the above embodiment.

[0115] The station logo is, for example, a station logo, and the detection result of the station logo includes, for example, location information of the station logo and category information of the station logo.

[0116] The final station logo detection model is specifically a pp-yoloe model.

[0117] In this embodiment, since the final logo detection model is used, the final logo detection model is a model with better effect, so the logo detection effect can be improved.

[0118] The station logo detection technology can be applied to the following scenarios:

[0119] Resource Management: Media organizations often need to manage a large number of media resources, including images, videos, and audio. By identifying and classifying the logos within these resources, they can better manage and categorize resources, improving their utilization efficiency and reusability.

[0120] Automated editing: During the video editing process, operations such as clipping, merging, and transitioning are required. By recognizing the station logos in the video, video editing can be automated to quickly generate video materials that meet the requirements.

[0121] Media Analysis: Media organizations need to analyze and research various media content, including public opinion analysis, market analysis, and user behavior analysis. By identifying and analyzing station logos within media content, relevant research and analysis can be better conducted, improving the accuracy and efficiency of analysis results.

[0122] Advertising placement: Media organizations often need to place advertisements. By identifying station logos in video content, they can better determine the timing and location of advertisement placement, thereby improving the effectiveness and returns of advertising.

[0123] Copyright protection: Media organizations need to protect their copyright and intellectual property rights. By identifying and analyzing station logos in media content, they can better detect infringements and protect their copyright and interests.

[0124] To sum up, station logo recognition has broad application prospects in various scenarios, and can help relevant institutions improve work efficiency, reduce costs, improve service quality and protect their own rights and interests.

[0125] Figure 9 This is a schematic diagram according to the fourth embodiment of the present disclosure. This embodiment provides a training device for a station logo detection model. The device 900 includes: a first training module 901, a determination module 902, a capture module 903, a construction module 904 and a second training module 905.

[0126] The first training module 901 is used to perform self-supervised training on the initial pre-training model to obtain the final pre-training model; the determination module 902 is used to use the final pre-training model to determine the correct position information and correct category information of the station logo in the image sample; the interception module 903 is used to intercept the station logo image at the position of the station logo in the image sample based on the correct position information; the construction module 904 is used to construct a pseudo image based on the background image and the station logo image, and obtain the real position information of the station logo image on the pseudo image; the second training module 905 is used to train the initial station logo detection model based on the pseudo image, the real position information and the correct category information to obtain the final station logo detection model.

[0127] In this embodiment, the station logo detection model is trained based on pseudo images, real position information and correct category information, rather than directly training based on image samples and manually annotated data. On the one hand, the real position information and correct category information are obtained based on the final pre-trained model, and do not require manual annotation, which can reduce the annotation cost. On the other hand, the final pre-trained model is adopted, that is, another model (pre-trained model) is introduced in addition to the station logo detection model. Compared with the method of directly training the station logo detection model, the good characteristics of the final pre-trained model can be utilized to improve the performance of the station logo detection model. In addition, due to the variety of station logo categories, the above-mentioned method without manual annotation can significantly reduce the amount of annotation, which is suitable for station logo detection scenarios; due to the small size of the station logo, the correct position information and correct category information are obtained through the final pre-trained model, and subsequent training is carried out. By utilizing the performance of the final pre-trained model, it is also beneficial to improve the detection effect of the station logo; and by constructing pseudo images, the performance of the station logo detection model can be enhanced, which is beneficial to the detection of station logos of various categories. The final pre-trained model is obtained through self-supervised training. Since self-supervised training does not require manual labeling of data, the labeling cost can be reduced. By constructing pseudo images based on background images and station logo images, and training the station logo detection model based on the pseudo images, pseudo images of different scenes can be constructed, thereby improving the scalability of the station logo detection model.

[0128] In some embodiments, the first training module 901 is further used to: use the initial pre-training model to perform prediction processing on the image sample to obtain the predicted position information of the station logo; obtain the self-supervised position information of the station logo based on the image sample; construct a self-supervised loss function based on the predicted position information and the self-supervised position information; train the initial pre-training model based on the self-supervised loss function to obtain the final pre-training model.

[0129] In this embodiment, a self-supervised loss function is constructed based on the self-supervised position information and the predicted position information. Since the self-supervised position information is obtained based on image samples, no manual labeling is required, which can reduce the labeling cost and save resource overhead.

[0130] In some embodiments, the first training module 901 is further configured to: perform image segmentation processing on the image sample to obtain mask information of the station logo; and obtain self-supervised position information of the station logo based on the mask information.

[0131] In this embodiment, by determining the mask information of the station logo through image segmentation processing, relatively accurate mask information can be obtained, and self-supervised position information can be obtained based on the mask information, which can improve the accuracy of the self-supervised position information and thus improve the accuracy of the model.

[0132] In some embodiments, there are multiple predicted position information, and the first training module 901 is further used to: calculate the overlap between each predicted position information and the self-supervised position information; select the predicted position information whose overlap is greater than a preset threshold among the multiple predicted position information; and construct the self-supervised loss function based on the selected predicted position information and the self-supervised position information.

[0133] In this embodiment, the self-supervised loss function is constructed based on the selected predicted position information and the self-supervised position information, and part of the predicted position information can be ignored during training, thereby improving the effect of the final pre-trained model.

[0134] In some embodiments, the construction module 904 is further used to: determine a random position within a preset area of ​​the background image, the size of the random position being consistent with the size of the station logo image; and use the pixel value of the station logo image to replace the pixel value of the random position in the background image to obtain the pseudo image.

[0135] In this embodiment, a pseudo image is constructed based on a random position, and then a station logo detection model is trained based on the pseudo image. This can improve the robustness of the station logo detection model and enhance the model detection effect.

[0136] In some embodiments, the real position information is the random position, and the second training module 905 is further used to: use the initial station logo detection model to process the pseudo image to obtain the output information of the initial station logo detection model; construct a supervised loss function based on the random position, the correct category information and the output information; and train the initial station logo detection model based on the supervised loss function to obtain the final station logo detection model.

[0137] In this embodiment, since both the random position and the correct category information can be obtained based on image samples, manual labeling is not required, which can reduce the labeling cost; the scale of the station logo detection model (such as pp-yoloe) is usually smaller than the scale of the pre-trained model (such as cascadeRCNN), which can provide users with a smaller final station logo detection model, reduce user resource overhead, and improve reasoning efficiency.

[0138] Figure 10 is a schematic diagram according to the fifth embodiment of the present disclosure. This embodiment provides a station logo detection device, the device 1000 including: an acquisition module 1001 and a detection module 1002 .

[0139] The acquisition module 1001 is used to acquire an image to be processed; the detection module 1002 is used to use the final logo detection model to perform logo detection processing on the image to be processed to determine the detection result of the logo in the image to be processed.

[0140] The final station logo detection model can be obtained using the method shown in the above embodiment.

[0141] The station logo is, for example, a station logo, and the detection result of the station logo includes, for example, location information of the station logo and category information of the station logo.

[0142] The final station logo detection model is specifically a pp-yoloe model.

[0143] In this embodiment, since the final logo detection model is used, the final logo detection model is a model with better effect, so the logo detection effect can be improved.

[0144] It can be understood that in the embodiments of the present disclosure, the same or similar contents in different embodiments can be referenced to each other.

[0145] It can be understood that the terms “first”, “second”, etc. in the embodiments of the present disclosure are only used for distinction and do not indicate the degree of importance, time sequence, etc.

[0146] In the technical solutions disclosed herein, the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0147] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0148] Figure 11A schematic block diagram of an example electronic device 1100 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, servers, blade servers, mainframe computers, and other suitable computers. The electronic device 1100 can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0149] like Figure 11 As shown, the electronic device 1100 includes a computing unit 1101, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1102 or a computer program loaded from a storage unit 1108 into a random access memory (RAM) 1103. Various programs and data required for the operation of the electronic device 1100 can also be stored in the RAM 1103. The computing unit 1101, the ROM 1102, and the RAM 1103 are connected to each other via a bus 1104. An input / output (I / O) interface 1105 is also connected to the bus 1104.

[0150] Multiple components in the electronic device 1100 are connected to the I / O interface 1105, including an input unit 1106, such as a keyboard, a mouse, etc.; an output unit 1107, such as various types of displays, speakers, etc.; a storage unit 1108, such as a magnetic disk, an optical disk, etc.; and a communication unit 1109, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1109 allows the electronic device 1100 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0151] The computing unit 1101 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1101 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1101 performs the various methods and processes described above, such as a station logo detection method or a training method for a station logo detection model. For example, in some embodiments, the station logo detection method or the training method for a station logo detection model may be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 1108. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1100 via the ROM 1102 and / or the communication unit 1109. When the computer program is loaded into the RAM 1103 and executed by the computing unit 1101, one or more steps of the station logo detection method or the training method for the station logo detection model described above may be performed. Alternatively, in other embodiments, the computing unit 1101 may be configured to execute the station logo detection method or the training method of the station logo detection model in any other appropriate manner (for example, by means of firmware).

[0152] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0153] The program code for implementing the methods of the present disclosure can be written in any combination of one or more programming languages. Such program code can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable load balancing device, so that when the program code is executed by the processor or controller, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The program code can be executed entirely on the machine, partially on the machine, as a standalone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0154] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0155] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0156] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0157] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0158] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not a limitation herein.

[0159] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A method for training a station logo detection model, comprising: Perform self-supervised training on the initial pre-trained model to obtain the final pre-trained model; Using the final pre-trained model, determining correct position information and correct category information of the station logo in the image sample; Based on the correct position information, intercepting the station logo image at the position where the station logo is located in the image sample; constructing a pseudo image based on the background image and the station logo image, and obtaining true position information of the station logo image on the pseudo image; Training an initial station logo detection model based on the pseudo image, the real position information, and the correct category information to obtain a final station logo detection model; The self-supervised training of the initial pre-trained model to obtain the final pre-trained model includes: Using the initial pre-trained model, performing prediction processing on the image sample to obtain predicted position information of the station logo; Based on the image sample, obtaining self-supervised position information of the station logo; constructing a self-supervised loss function based on the predicted position information and the self-supervised position information; Training the initial pre-trained model based on the self-supervised loss function to obtain the final pre-trained model; The predicted position information is multiple, and constructing a self-supervised loss function based on the predicted position information and the self-supervised position information includes: Calculating the overlap between each predicted position information and the self-supervised position information; Selecting, from the plurality of predicted position information, predicted position information with the overlap greater than a preset threshold; The self-supervised loss function is constructed based on the selected predicted position information and the self-supervised position information.

2. The method according to claim 1, wherein The obtaining, based on the image sample, the self-supervised position information of the station logo includes: performing image segmentation processing on the image sample to obtain mask information of the station logo; The self-supervised position information of the station logo is obtained based on the mask information.

3. The method according to any one of claims 1 to 2, wherein: The constructing of a pseudo image based on the background image and the station logo image includes: Determining a random position within a preset area of ​​the background image, wherein the size of the random position is consistent with the size of the station logo image; The pixel values ​​of the station logo image are used to replace the pixel values ​​at the random positions in the background image to obtain the pseudo image.

4. The method according to claim 3, wherein: The real position information is the random position, and the training of an initial station logo detection model based on the pseudo image, the real position information, and the correct category information to obtain a final station logo detection model includes: Processing the pseudo image using the initial station logo detection model to obtain output information of the initial station logo detection model; Constructing a supervised loss function based on the random position, the correct category information and the output information; Based on the supervised loss function, the initial station logo detection model is trained to obtain the final station logo detection model.

5. A method for detecting a station logo, comprising: Get the image to be processed; Using the final station logo detection model, performing station logo detection processing on the image to be processed to determine a detection result of the station logo in the image to be processed; The final logo detection model is trained using the method according to any one of claims 1 to 4.

6. A training device for a station logo detection model, comprising: The first training module is used to perform self-supervised training on the initial pre-trained model to obtain the final pre-trained model; a determination module, configured to determine correct position information and correct category information of the station logo in the image sample using the final pre-trained model; a capture module, configured to capture the station logo image at the location of the station logo in the image sample based on the correct position information; A construction module, configured to construct a pseudo image based on a background image and the station logo image, and obtain real position information of the station logo image on the pseudo image; A second training module is configured to train an initial station logo detection model based on the pseudo image, the real position information, and the correct category information to obtain a final station logo detection model; The first training module is further configured to: Using the initial pre-trained model, performing prediction processing on the image sample to obtain predicted position information of the station logo; Based on the image sample, obtaining self-supervised position information of the station logo; constructing a self-supervised loss function based on the predicted position information and the self-supervised position information; Training the initial pre-trained model based on the self-supervised loss function to obtain the final pre-trained model; The predicted position information is multiple, and the first training module is further used to: Calculating the overlap between each predicted position information and the self-supervised position information; Selecting, from the plurality of predicted position information, predicted position information with the overlap greater than a preset threshold; The self-supervised loss function is constructed based on the selected predicted position information and the self-supervised position information.

7. The device according to claim 6, wherein The first training module is further configured to: performing image segmentation processing on the image sample to obtain mask information of the station logo; The self-supervised position information of the station logo is obtained based on the mask information.

8. The device according to any one of claims 6 to 7, wherein: The construction module is further configured to: Determining a random position within a preset area of ​​the background image, wherein the size of the random position is consistent with the size of the station logo image; The pixel values ​​of the station logo image are used to replace the pixel values ​​at the random positions in the background image to obtain the pseudo image.

9. The device according to claim 8, wherein The real position information is the random position, and the second training module is further used to: Processing the pseudo image using the initial station logo detection model to obtain output information of the initial station logo detection model; Constructing a supervised loss function based on the random position, the correct category information and the output information; Based on the supervised loss function, the initial station logo detection model is trained to obtain the final station logo detection model.

10. A station logo detection device, comprising: An acquisition module, used for acquiring an image to be processed; a detection module, configured to perform logo detection processing on the image to be processed using the final logo detection model to determine a detection result of the logo in the image to be processed; The final logo detection model is trained using the method according to any one of claims 1 to 4.

11. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 5.

12. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 5.

13. A computer program product comprising a computer program, which, when executed by a processor, implements the method according to any one of claims 1 to 5.

Images