Method for station logo detection and related apparatus

Abstract

The application provides a station mark detection method and related equipment. First, the local part of the station mark is detected, and a candidate combination frame is determined according to a suspected frame pair in the result that meets a screening condition. The suspected frame pair includes a station mark pattern detection frame and a station mark character detection frame. The candidate combination frame is a circumscribed rectangle of the station mark pattern detection frame and the station mark character detection frame. The serialized features of the candidate combination frame are extracted and classified to determine whether the candidate combination frame is a station mark combination. Finally, the sub-frame and the parent frame in the station mark are output. The station mark pattern detection frame and the station mark character detection frame are the sub-frames, and the candidate combination frame is the parent frame. By using the topological relationship (correlation relationship) between the local structure of the station mark, the detection of the local part (sub-frame) and the whole (parent frame) of the station mark can be realized end to end, the situation of missing detection of the local part of the station mark is reduced, and the efficiency and accuracy of station mark detection and identification are improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of communication, and more particularly, to a station logo detection method and related device thereof. BACKGROUND

[0002] A station logo is a special logo used to represent a TV station, a TV channel, a radio station or a digitized traditional media. The station logo can be in the form of a pattern, artistic text or a combination of pattern and text, and can include the name of a TV station, a channel number and a channel name. Currently, many station logos are composed of a pattern and / or text, and can be presented in various ways.

[0003] With the increasing popularity of intelligent devices and the growing demand for cultural and entertainment, video portals need to constantly introduce new video resources to meet the needs of users. Video platforms need to detect and identify station logos in a large amount of video content.

[0004] There are two commonly used station logo detection schemes, namely, an independent detection scheme and an overall detection scheme. However, whether the overall detection scheme or the independent detection scheme is used, local detection is prone to be missed, resulting in low accuracy and efficiency of station logo detection, and failing to effectively identify the station logo. SUMMARY

[0005] The present application provides a station logo detection method and related device thereof. By using the correlation between the layout of text and pattern in a station logo, the local part of the station logo is first detected, and a candidate combination box is determined according to a suspected box pair in the detection result that meets the screening condition. One suspected box pair includes a station logo pattern detection box and a station logo text detection box, and the candidate combination box is an external tangent rectangle of the station logo pattern detection box and the station logo text detection box. The serialized features of the candidate combination box are extracted and classified, and it is determined whether the candidate combination box is a station logo combination. Finally, the sub-box and the parent box in the station logo are output, the station logo pattern detection box and the station logo text detection box are the sub-box, and the candidate combination box is the parent box. By using the topological relationship (correlation) between the local structure of the station logo, the present application can realize the detection of the local part (sub-box) and the overall part (parent box) of the station logo end to end, reduce the local detection of the station logo, and improve the efficiency and accuracy of station logo detection and identification.

[0006] In a first aspect, a method for station logo detection is provided. The method comprises: obtaining a plurality of video frames of a video to be detected; detecting a first video frame in the plurality of video frames using a station logo detection model to obtain information of at least one detection box pair included in a station logo in the first video frame, wherein one detection box pair includes a station logo pattern detection box and a station logo text detection box, and the first video frame is any one of the plurality of video frames; when a first detection box pair satisfies a first condition, determining a candidate combination box according to the first detection box pair, the first detection box pair includes a first station logo pattern detection box and a first station logo text detection box, the candidate combination box includes the first station logo pattern detection box and the first station logo text detection box, and the first detection box pair is any one of the at least one detection box pair; extracting a serialized feature of the candidate combination box; inputting the serialized feature into a station logo combination classification model to determine whether the candidate combination box is a station logo combination, the station logo combination classification model being used to determine whether a candidate combination box is a correct station logo; and determining a sub-box and a parent box in a station logo in a video frame according to whether the candidate combination box is a station logo combination and the information of the first detection box pair, the first station logo pattern detection box and the first station logo text detection box being the sub-box, and the candidate combination box being the parent box.

[0007] The method for station logo detection provided in the first aspect can reduce the case of local missing detection of a station logo, achieve the effect of complete covering of a station logo in a watermark removal scene, present a detection result in the form of a sub-box and a parent box, contain more structural information, provide more diversified processing modes for subsequent steps, and improve the efficiency and accuracy of station logo recognition and detection. Moreover, based on local independent detection and overall combination box classification of a station logo, not only is the performance of station logo text detection optimized, but also label conflicts among positive samples in a training stage caused by mixing of samples are avoided, the model can be trained using overall and local samples of a station logo at the same time, and strong robustness and generalization are achieved.

[0008] Illustratively, the data source of the video to be detected can be data provided by a CP or UGC.

[0009] Illustratively, a station logo can only consist of a pattern. Alternatively, a station logo consists of a pattern and text, but the layout or relative position of the pattern and the text is different.

[0010] In a possible implementation manner of the first aspect, the candidate combination box is a circumscribed rectangle of the first station logo pattern detection box and the first station logo text detection box.

[0011] Illustratively, the first station logo pattern detection box and the first station logo text detection box are both rectangles.

[0012] In a possible implementation manner of the first aspect, the information of the first logo pattern detection box includes: position coordinates of the first logo pattern detection box, classification confidence of the first logo pattern detection box, and a category to which the first logo pattern detection box belongs; the information of the first logo character detection box includes: position coordinates of the first logo character detection box, classification confidence of the first logo character detection box, and a category to which the first logo character detection box belongs; wherein the classification confidence of the first logo pattern detection box is used to indicate a probability that content included in the first logo pattern detection box is a pattern, and the category to which the first logo pattern detection box belongs is used to indicate a category or a source of the content included in the first logo pattern detection box.

[0013] For example, the category to which a detection box belongs can be Huawei video, Huawei music, Tencent video, or iQIYI video, and the like.

[0014] In a possible implementation manner of the first aspect, the position coordinates of the first logo pattern detection box are (x, y, w, h), wherein x represents a horizontal coordinate of a center point of the first logo pattern detection box, y represents a vertical coordinate of the center point of the first logo pattern detection box, w represents a width of the first logo pattern detection box, and h represents a height of the first logo pattern detection box.

[0015] For example, the coordinates of the circumscribed rectangle of the first logo pattern detection box and the first logo character detection box can be determined according to the coordinates of the first logo pattern detection box and the coordinates of the first logo character detection box.

[0016] In a possible implementation manner of the first aspect, the first condition includes: the classification confidence of the first logo pattern detection box is greater than or equal to a first threshold value, the classification confidence of the first logo character detection box is greater than or equal to a second threshold value, the category to which the first logo pattern detection box belongs is the same as the category to which the first logo character detection box belongs, and a ratio of a sum of areas of the first logo pattern detection box and the first logo character detection box to an area of a circumscribed rectangle is greater than or equal to a third threshold value.

[0017] In a possible implementation manner of the first aspect, the serialized features of the candidate combined box include: horizontal serialized features and vertical serialized features of the candidate combined box.

[0018] Exemplarily, the serialized features of the candidate combination frame are extracted, including: first, extracting the feature map of the candidate combination frame at the position corresponding to the backbone network, and using a pooling technology (such as roi pooling) or the like to unify the feature scale of the feature map, and then inputting the feature map after the scale unification into the serialized branch network, while the horizontal and vertical serialized features of the feature map are extracted. In the process of extracting the horizontal and vertical serialized features of the feature map, a neural network such as RNN or LSTM can be used.

[0019] In a possible implementation of the first aspect, the station mark detection model is trained by station mark pattern boundary box labels and station mark character boundary box labels.

[0020] Exemplarily, the station mark detection network is trained by pattern boundary box labels and character boundary box labels in the independent station mark.

[0021] In a possible implementation of the first aspect, the serialized features are input into the station mark combination classification model to determine whether the candidate combination frame is a station mark combination, including: inputting the serialized features g(x) of the candidate combination frame into a fully connected network for classification, and if the classification result of the candidate combination frame is True, it is proved that the candidate combination frame belongs to the same station mark, that is, it is proved that the candidate combination frame belongs to the station mark combination. If the classification result of the candidate combination frame is false, it is proved that the candidate combination frame does not belong to the same station mark.

[0022] In a possible implementation of the first aspect, the method further includes: outputting a station mark detection result, the station mark detection result including: information of a sub-frame and information of a parent frame. The information of the parent frame includes: position coordinates of the parent frame, classification confidence of the parent frame and a category to which the parent frame belongs. The information of the sub-frame includes: position coordinates of the sub-frame and classification confidence of the sub-frame.

[0023] In a possible implementation of the first aspect, the method further includes: watermarking the parent frame.

[0024] In a possible implementation of the first aspect, the method further includes: sending the video after adding the watermark to a user.

[0025] In a possible implementation of the first aspect, if a station mark is composed of only a pattern or a character, the output result of the station mark detection network only includes one detection frame. In this case, the detection result including the position coordinates, confidence and category of the detection frame (sub-frame or parent frame) can be directly output.

[0026] In a second aspect, a station mark detection apparatus is provided, which comprises units for performing the steps in the method according to the first aspect or any possible implementation of the first aspect.

[0027] In a third aspect, a station mark detection apparatus is provided, which comprises at least one processor and a memory coupled to the processor, and the memory stores program instructions which, when executed by the processor, perform the method according to the first aspect or any possible implementation of the first aspect.

[0028] In a fourth aspect, a station mark detection apparatus is provided, which comprises at least one processor and an interface circuit, and the at least one processor is configured to perform the method according to the first aspect or any possible implementation of the first aspect.

[0029] In a fifth aspect, an electronic device or a server is provided, which comprises the station mark detection apparatus according to the second aspect, the third aspect or the fourth aspect.

[0030] In a sixth aspect, a computer program product is provided, which comprises a computer program configured to perform the method according to the first aspect or any possible implementation of the first aspect when executed by a processor.

[0031] In a seventh aspect, a computer-readable storage medium is provided, which stores a computer program configured to perform the method according to the first aspect or any possible implementation of the first aspect when executed by a processor.

[0032] In an eighth aspect, a chip is provided, which comprises a processor configured to invoke and run a computer program from a memory, so that a communication device installed with the chip performs the method according to the first aspect or any possible implementation of the first aspect.

[0033] The station mark detection method and related device provided in the present application can reduce the situation of local missing detection of station marks, achieve the effect of complete covering of station marks in the watermark removal scene, present the detection result in the form of sub-mother frame, contain more structural information, provide more diversified processing methods for subsequent steps, and improve the efficiency and accuracy of station mark identification and detection. Moreover, based on the local independent detection of station marks and the overall combined frame classification, not only is the performance of station mark character detection optimized, but also the label conflict between positive samples in the training stage caused by the mixing of samples is avoided, the model can be trained simultaneously by using overall and local samples of station marks, and strong robustness and generalization are achieved. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a schematic structural diagram of a station logo provided by the present application.

[0035] Figure 2 is a schematic flow chart of a station logo detection method provided by the prior art.

[0036] Figure 3 is a schematic flow chart of another station logo detection method provided by the prior art.

[0037] Figure 4 is a schematic flow chart of still another station logo detection method provided by the prior art.

[0038] Figure 5 is a schematic diagram of an application scenario suitable for embodiments of the present application provided by the present application.

[0039] Figure 6 is a schematic diagram of a communication system architecture suitable for embodiments of the present application provided by the present application.

[0040] Figure 7 is a schematic flow chart of a station logo detection method provided by the present application.

[0041] Figure 8 is a schematic diagram of a video to be detected provided by the present application.

[0042] Figure 9 is a schematic diagram of video frame extraction of a video to be detected provided by the present application.

[0043] Figure 10 is a schematic diagram of the relative position relationship of the detection box box1 and the detection box box2 and the circumscribed rectangle in the coordinate axis provided by the present application.

[0044] Figure 11 is a schematic diagram of extracting serialized features of candidate combination boxes and classifying the serialized features provided by the present application.

[0045] Figure 12 is a schematic diagram of jointly training a neural network model provided by the present application.

[0046] Figure 13 is a schematic diagram of a station logo detection result provided by the present application.

[0047] Figure 14 is a schematic diagram of de-watermarking a station logo provided by the present application.

[0048] Figure 15 is a schematic block diagram of a communication structure provided by an embodiment of the present application.

[0049] Figure 16 is a schematic block diagram of an example communication device structure provided in an embodiment of the present application. DETAILED DESCRIPTION

[0050] The technical solutions in the present application will be described below with reference to the drawings.

[0051] In the description of the embodiments of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" herein is only a description of the association relationship of the associated objects, which means that there can be three relationships, for example, A and / or B, which can mean that A exists alone, A and B exist together, and B exists alone. In addition, in the description of the embodiments of the present application, "multiple" means two or more than two.

[0052] Hereinafter, the terms "first" and "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first" and "second" can explicitly or implicitly include one or more features. In the description of the embodiments, unless otherwise specified, the meaning of "multiple" is two or more than two.

[0053] In addition, various aspects or features of the present application can be implemented as a method, apparatus or article of manufacture using standard programming and / or engineering techniques. The term "article of manufacture" used in the present application encompasses computer programs accessible from any computer-readable device, carrier or medium. For example, the computer-readable medium can include, but is not limited to: magnetic storage devices (for example, hard disk, floppy disk or magnetic tape, etc.), optical discs (for example, compact disc (CD), digital versatile disc (DVD), etc.), smart cards and flash memory devices (for example, erasable programmable read-only memory (EPROM), card, stick or key drive, etc.). In addition, the various storage media described herein can represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" can include, but is not limited to, wireless channels and various other media capable of storing, containing and / or carrying instructions and / or data.

[0054] The technical solutions of the embodiments of the present application can be applied to various communication systems, for example, a Global System of Mobile communication (GSM) system, a Code Division Multiple Access (CDMA) system, a Wideband Code Division Multiple Access (WCDMA) system, a General Packet Radio Service (GPRS), a Long Term Evolution (LTE) system, an LTE Frequency Division Duplex (FDD) system, an LTE Time Division Duplex (TDD), a Universal Mobile Telecommunication System (UMTS), a Worldwide Interoperability for Microwave Access (WiMAX) communication system, a 5th Generation (5G) system or a New Radio (NR), and the like.

[0055] In addition, various aspects or features of the disclosure can be realized as a method, apparatus, or article of manufacture using standard programming and / or engineering techniques. The term "article of manufacture" as used herein is intended to encompass a computer program accessible from any computer-readable device, carrier, or media. For example, computer-readable media can include but are not limited to magnetic storage devices (e.g., hard disk, floppy disk, magnetic strips, etc.), optical disks (e.g., compact disk (CD), digital versatile disk (DVD), etc.), smart cards, and flash memory devices (e.g., EPROM, card, stick, or key drive, etc.). Additionally, various storage media described herein can represent one or more devices and / or other machine-readable media for storing information. The term "machine-readable medium" can include, without being limited to, wireless channels and various other media capable of storing, containing, and / or carrying instruction and / or data.

[0056] A station logo is a special logo used to represent a TV station, a TV channel, a radio station or a digitized traditional media. It can be in the form of a pattern, artistic text or a combination of both, and its content can include the name of a TV station, a channel number and a channel name.

[0057] With the comprehensive popularity of intelligent devices and the growing demand for cultural and entertainment, video portals need to constantly introduce new video resources to meet the needs of users. Video platforms need to detect and identify the station logo in the massive video content for the following purposes: 1, resource network audit and copyright protection; 2, video editing, watermark removal when creating, and the like.

[0058] At present, many station logos are composed of patterns and texts, and there are various presentation modes, Figure 1 The station logo shown in the schematic diagram provided by the embodiment of the application. As shown in a figure in Figure 1 The station logo of "Huawei Video" shown in a figure and a figure in b and c is composed of patterns and texts. Figure 1 The station logo shown in a figure and a figure in b and c is composed of patterns and texts. Figure 1 In the station logo shown in a figure and a figure in b and c, the layout or relative position of the patterns and texts is different.

[0059] At present, there are two commonly used station logo detection schemes, namely, independent detection and overall detection.

[0060] When independent detection is used, a convolutional neural network (CNN) is used to identify the station logo. Since the CNN structure itself has weak recognition ability for texts, direct detection of texts often has poor effect, and the process of text detection and pattern detection is relatively independent, lacking the association of the topological relationship between the texts and patterns in the station logo, which cannot guarantee the simultaneous detection of the texts and patterns in the station logo, resulting in inaccurate detection results.

[0061] When overall detection is used, since the training samples of the neural network lack samples in which patterns or texts appear alone, the neural network model loses the ability to detect patterns or texts alone, resulting in inaccurate detection results.

[0062] At present, a method for detecting a station logo is provided in the related art, and the steps of the method are shown in Figure 2 a figure, including S101 to S105.

[0063] S101, inputting a video.

[0064] S102, detecting a station logo switching signal.

[0065] S103, obtaining a station logo region of a preset video frame.

[0066] S104, performing text recognition on the region image.

[0067] S105, outputting station logo information.

[0068] Figure 2The method shown first detects a station mark switching signal, then collects preset video frame images with station mark information output by a set top box, acquires a region of the station mark in the preset video frame images, and then performs text recognition on the image in the station mark region to acquire the station mark.

[0069] The method needs to preset video frames for the image to be detected, uses strong prior information, and has a very limited use scenario. In the detection process, an additional hardware device needs to be used, which is not friendly to the device. Moreover, the text recognition in the station mark is based on a text sample library, and the detection effect is poor when the text has slight disturbance. The station mark pattern is not detected and recognized, and finally the station mark information output only includes text information and does not include pattern information, resulting in inaccurate station mark detection and ineffective identification of the station mark.

[0070] The related art also provides a station mark detection method, and the steps of the method are shown as Figure 3 S201 to S206.

[0071] S201, input a video.

[0072] S202, extract a key frame of the video.

[0073] S203, divide the video into M grids.

[0074] S204, locate the station mark.

[0075] S205, classify the station mark.

[0076] S206, output a station mark detection result.

[0077] Figure 3 The method shown in the scheme first pre-processes a video data file to obtain a large data set only marking a station mark category (for example, Huawei video, iQiyi video, Tencent video, etc.) and a small data set only marking a station mark position. Then, a station mark positioning network and a station mark classification network are trained. Key frames are extracted from an input video to be detected, the extracted key frames are pre-processed, and the images obtained after the pre-processing are input into the trained station mark positioning network to obtain the predicted station mark region of the images. The predicted station mark region of the images is input into the trained station mark classification network to obtain the station mark position and the station mark category of the images.

[0078] This method treats the logo and text as a whole during detection, lacking local detection capabilities. Furthermore, the localization and classification neural networks are trained separately, making the training process cumbersome. Before detection, the video frames need to be segmented into an M-grid, and each segment needs to be input into the subsequent network for separate detection, increasing the overall time consumption. The training of the neural network model and the prediction of video frames both require the use of two network models (logo localization network and logo classification network), increasing training complexity and overall time consumption. This results in inaccurate logo detection and an inability to effectively identify the logo.

[0079] The related technology also provides a method for detecting logos, the steps of which are as follows: Figure 4 As shown, this includes S301 to S307.

[0080] S301, Input video.

[0081] S302, video frame capture.

[0082] S303, Data Preprocessing.

[0083] S304, Taiwan logo testing.

[0084] S305, Threshold judgment for classification output value.

[0085] S306, Area output value threshold judgment.

[0086] S307 outputs the logo detection results.

[0087] Figure 4 The method shown first extracts frames from the input video, preprocesses the extracted video frames, and then feeds the preprocessed video frames into the trained network model. It then determines whether the classification output value is greater than a threshold. If not, the sample is considered a problem sample and is not classified. If so, it continues to determine whether the area output value is greater than a threshold. If not, the candidate box is considered an interference sample and is not output; otherwise, the prediction result is output.

[0088] In this method, the information of text and pattern in the logo is extracted by mixing features during the detection process, which is prone to local missed detection. The topological relationship between the logo pattern and text is not utilized during the detection process, resulting in inaccurate logo detection, low efficiency, and inability to effectively identify the logo.

[0089] In view of this, this application provides a method for logo detection. By utilizing the correlation between the text and graphic layout of the logo, it first detects a local part of the logo. Based on the suspected bounding box pairs that meet the screening criteria in the detection results, candidate composite boxes are identified. These candidate composite boxes are the bounding rectangles of the logo graphic detection box and the logo text detection box. Serialized features are extracted from the candidate composite boxes and classified to determine whether the candidate composite boxes are logo combinations. Finally, the child frames and parent frames in the logo are output, with the logo graphic detection box and the logo text detection box being the child frames, and the candidate composite boxes being the parent frames. This application, by utilizing the topological relationships (correlation relationships) between the local structures of the logo, can achieve end-to-end detection of both local (child frames) and overall (parent frames) parts of the logo, reducing the possibility of missed detections of local parts and improving the efficiency and accuracy of logo detection and recognition.

[0090] The following examples illustrate the method for detecting Taiwan logos provided in this application.

[0091] Figure 5 The diagram illustrates an application scenario applicable to an embodiment of this application.

[0092] like Figure 5 As shown in Figure a, video platforms require watermark removal for station logos during video editing and secondary creation. The platform needs to detect station logos, trademarks, and other similar elements in video frames or images and then cover the detected logos with watermarks. For example... Figure 5 As shown in Figure b, video platforms need to accurately identify the station logo in the video during the review of video resources for network access and copyright protection, in order to determine the source of the video copyright and protect the copyright.

[0093] Understandable, Figure 5 The application scenarios shown do not constitute a specific limitation on the application scenarios of the embodiments of this application. The embodiments of this application can also be applied to other scenarios that require logo recognition and detection, and the embodiments of this application are not limited thereto.

[0094] Figure 6 The diagram shown is a schematic representation of a communication system architecture applicable to an embodiment of this application.

[0095] like Figure 6As shown, the content provider (CP) and user generated content (UGC) are the main sources of image and video data. The video key frame extraction module processes the video data stream in real time, the storage system stores the extracted video key frames, and the inference platform provides powerful graphics processing unit (GPU) and central processing unit (CPU) processing capabilities and efficient algorithm service deployment. The video key frames are subjected to station logo identification and detection using the method provided in the present application, and the station logo identification and detection are stored in the storage system and finally distributed to users through the storage system.

[0096] It can be understood that, Figure 6 The system architecture shown does not constitute a specific limitation on the communication system architecture of the embodiments of the present application. In other embodiments of the present application, the communication system architecture of the embodiments can include more or fewer units or modules than shown, or combine certain units, or split certain units, or different component units. The units shown can be implemented in hardware, software, or a combination of software and hardware. The embodiments of the present application are not limited here.

[0097] The method for station logo detection provided in the present application will be described below in conjunction with specific examples.

[0098] Figure 7 As shown is a schematic flowchart of an example of the method 400 for station logo detection provided in the present application. The method 400 can be applied in Figure 6 The system shown.

[0099] As Figure 7 shown, the method 400 includes S410 to S470. In embodiments of the present application, the execution subject of the method 400 can be a device for station logo detection. For example, it can be a server or a chip, and the embodiments of the present application are not limited here.

[0100] S410, input the video to be detected into the detection system.

[0101] In S410, the data source of the video to be detected can be data provided by the CP or UGC.

[0102] For example, Figure 8 As shown is a schematic diagram of an example of the video to be detected. In Figure 8 the example shown, the station logo in the video to be detected is "Huawei Video", and the station logo category is "Huawei_video".

[0103] S420, extract the video frames of the video to be detected.

[0104] In S420, the video frames (or can be called key frames) of the video to be detected can be extracted by a video frame (or can be called key frame) extraction algorithm. In the extraction of the video frames of the video to be detected, a plurality of video frames in the video stream can be randomly extracted, or a plurality of video frames can be extracted according to certain rules, for example, odd-numbered or even-numbered video frames, and the like. The embodiments of the present application do not limit this. In combination with the example in S410, the video frames extracted from the video to be detected can be as shown in FIG. 2. Figure 8 Figure 9

[0105] Optionally, the extracted video frames can be saved in a storage system.

[0106] In S430, a station logo detection network is used to detect the station logo in each of the extracted video frames, to obtain one or more detection boxes.

[0107] The processing procedure for one video frame will be described below. The processing procedures for other video frames are the same.

[0108] In the embodiments of the present application, the station logo detection network is trained by independent station logo pattern boundary box labels and text boundary box labels. The boundary box label includes the coordinates (x, y, w, h) of the boundary box (or can be called detection box) and the belonging category. The pattern and text boundary box category labels of the same station logo are the same. In the coordinates (x, y, w, h) of the boundary box, x and y represent the coordinates of the center point of the boundary box, and w and h represent the width and height of the boundary box, respectively. The station logo detection network outputs the classification confidence, the belonging category and the coordinate value of each detection box detected in the same video frame. The belonging category of the detection box can be understood as the category or source of the content (pattern or text) in the detection box. For example, the belonging category of a detection box can be Huawei video, Huawei music, Tencent video or iQiyi video, and the like. The classification confidence of the detection box can be understood as the probability that the content in the detection box is a pattern or text.

[0109] Suppose that, for a video frame, the output result of the station logo detection network is that the video frame includes two detection boxes, which are box1 and box2.

[0110] The output result list is:

[0111] [[“box_id”:1,”class”:”Huawei_video”,”prob”:0.88,”location”:[202,20,28,29]],

[0112] ​​["box_id": 2, "class": "Huawei_video", "prob": 0.78, "location": [248, 20, 62, 23]].

[0113] In the result list, "class" represents the class to which the detection box belongs. The "class" of box1 and box2 is "Huawei_video", which means that box1 and box2 belong to the same class.

[0114] "prob" represents the classification confidence of the detection box.

[0115] "Location" represents the coordinate value of the detection box. Among them, the position coordinates of box1 are [202, 20, 28, 29], 202 and 20 represent the coordinates of the center point of box1, and 28 and 29 represent the width and height of box1 respectively. The position coordinates of box2 are [248, 20, 62, 23], 248 and 20 represent the coordinates of the center point of box2, and 62 and 23 represent the width and height of box2 respectively. Both box1 and box2 are rectangular boxes. In the position coordinate relationship of box1 and box2, the coordinate origin of the coordinate axis is located at the upper left corner of the video frame.

[0116] It should also be understood that in the embodiments of the present application, if a station logo is only composed of a pattern or text, for example, the scene shown in Figure a of Figure 1 In this case, in S430, the output result of the station logo detection network only includes the information of one detection box, and the information of the detection box includes: the classification confidence corresponding to the detection box, the class to which the detection box belongs, and the coordinate value of the detection box.

[0117] S440, determine whether any two detection boxes of each video frame are candidate combination boxes.

[0118] In S440, for any two detection boxes (detection box pairs) detected for each video frame, it is determined whether the two detection boxes are candidate combination boxes according to the following judgment conditions.

[0119] Condition a: the classification confidence of the two detection boxes is greater than or equal to a preset threshold;

[0120] Condition b: the classes to which the two detection boxes belong are the same;

[0121] Condition c: the ratio of the sum of the areas of the two detection boxes to the area of the circumscribed rectangle of the two detection boxes is greater than or equal to a preset threshold;

[0122] If the conditions a to c are satisfied, the two detected boxes (a suspected box pair) are determined as candidate combination boxes. And the coordinates of the circumscribed rectangle of the two candidate combination boxes are determined as the coordinates of the candidate combination boxes.

[0123] The area of the detected box can be calculated by the following formula:

[0124]

[0125]

[0126] S i = w i × h i (3)

[0127] In the formulas (1) to (3), and represent the coordinate value of the upper left corner of the detected box i, and represent the coordinate value of the right lower corner of the detected box i. w i represents the width of the detected box i, h i represents the height of the detected box i. S i represents the area of the detected box i.

[0128] The area of the circumscribed rectangle of the two detected boxes can be calculated by the following formula:

[0129]

[0130]

[0131] S = W × H (6)

[0132] In the formulas (4) to (6), W represents the width of the circumscribed rectangle, H represents the height of the circumscribed rectangle, and S represents the area of the circumscribed rectangle.

[0133] The ratio of the sum of the areas of the two detected boxes to the area of the circumscribed rectangle of the two detected boxes can be calculated by formula (7):

[0134]

[0135] In formula (7), R represents the ratio of the sum of the areas of the two detected boxes to the area of the circumscribed rectangle of the two detected boxes.

[0136] In combination with the above example:

[0137] [[“box_id”:1,”class”:”Huawei_video”,”prob”:0.88,”location”:[202,20,28,29]],

[0138] ["box_id":2,"class":"Huawei_video","prob":0.78,"location":[248,20,62,23]].

[0139] For condition a: Assuming the preset threshold is 0.3, the classification confidence of box1 is 0.88, the classification confidence of box2 is 0.78, and box1 and box2 satisfy condition a.

[0140] For condition b: box1 and box2 belong to the same category, both being "Huawei_video", box1 and box2 satisfy condition b.

[0141] For condition c: Assume the preset threshold is 0.4.

[0142] The coordinates of box1 are [202, 20, 28, 29], where 202 and 20 represent the coordinates of the center point of box1, and 28 and 29 represent the width and height of box1, respectively. The coordinates of box2 are [248, 20, 62, 23], where 248 and 20 represent the coordinates of the center point of box2, and 62 and 23 represent the width and height of box2, respectively. Both box1 and box2 are rectangles.

[0143] The area of ​​box1 is 28 × 29 = 812, and the area of ​​box2 is 62 × 23 = 1426. The sum of the areas of box1 and box2 is 2238.

[0144] Based on the center coordinates of box1 and box2, as well as their width and height, we can calculate... The value of is calculated using formulas (4), (5) and (6). The coordinates of the circumscribed rectangle are: [234, 20, 91, 29]. 234 and 20 represent the coordinates of the center point of the circumscribed rectangle, and 91 and 29 represent the width and height of the circumscribed rectangle, respectively. The area of ​​the circumscribed rectangle is 2639.

[0145] The sum of the areas of box1 and box2 has an area ratio of 0.84 to the area of ​​their circumscribed rectangle, which is greater than the threshold of 0.4. Therefore, box1 and box2 satisfy condition c.

[0146] For example, Figure 10 The diagram shows the relative positions of box1, box2, and the circumscribed rectangle on the coordinate axes.

[0147] Optionally, in the embodiments of this application, box1 and box2 can be referred to as sub-frames, and the circumscribed rectangle can be referred to as the mother frame.

[0148] S450, extract the serialization features of the candidate combo boxes.

[0149] In S450, feature maps of candidate bounding boxes at their corresponding positions in the backbone network are first extracted. Pooling techniques with a unified output scale (e.g., ROI pooling) are then used to unify the feature map's scale. The scale-unified feature map is then input into the serialization branch network, simultaneously extracting horizontal and vertical serialization features from the feature map. During the extraction of horizontal and vertical serialization features, recurrent neural networks (RNNs) or long short-term memory (LSTMs) can be used. This embodiment of the application is not limited to these methods.

[0150] For example, in S450, after extracting the feature maps of the candidate combo boxes at the corresponding positions in the backbone network, the feature maps can first be subjected to max pooling in the vertical and horizontal directions, and then encoded in the horizontal and vertical directions using neural networks such as RNN or LSTM, thereby extracting the horizontal and vertical serialized features of the feature maps. This embodiment of the application is not limited herein.

[0151] Based on the above example, the coordinates of the candidate combo box (i.e., the circumscribed rectangle) are [234, 20, 91, 29]. Features at the corresponding positions of the candidate combo box are extracted from the feature map F(x) output by the backbone network of the detection model. Features output by the backbone network. Figure 1 Generally, downsampling by a factor of n is required, and the candidate bounding boxes also need to be downsampled by a corresponding factor and rounded down. After coordinate transformation, the candidate bounding box coordinates are [234 / n, 20 / n, 91 / n, 29 / n]. The local feature map F(x) is obtained from this. x∈box The feature map F(x) is pooled using ROI pooling. x∈box After scaling, the input is processed into the serialization branch, and horizontal and vertical serialization features g(x) are extracted simultaneously.

[0152] S460 classifies the serialized features of candidate bounding boxes in a fully connected network and determines whether the candidate bounding boxes belong to the logo combination based on the classification results.

[0153] In S460, the serialized feature g(x) of the candidate combination box is sent to the full connection network for classification. If the classification result of the candidate combination box is True, it is proved that the candidate combination box belongs to the same station mark, i.e., it is proved that the candidate combination box belongs to the station mark combination. If the classification result of the candidate combination box is false, it is proved that the candidate combination box does not belong to the same station mark, i.e., it is proved that the candidate combination box does not belong to the station mark combination. The negative samples in the training of the full connection network (i.e., the station mark combination classification model) are composed of the detected boxes screened out by S440.

[0154] Optionally, as another possible implementation, the negative samples in the training of the full connection network (i.e., the station mark combination classification model) can also be artificially collected and labeled station mark combination classification dataset samples, which is not limited in the embodiments of the present application.

[0155] Figure 11 As shown in the figure, an example of extracting the serialized feature of the candidate combination box and classifying the serialized feature is shown. Figure 11 As shown in the figure, after the feature scale of the feature map is unified by using the roi pooling pooling technology, the serialized feature branch network is input, and the horizontal serialized feature and the vertical serialized feature of the feature map are extracted, and then the serialized feature of the candidate combination box is sent to the full connection network for classification to obtain the classification result.

[0156] Optionally, in the embodiments of the present application, the candidate serialized feature extraction network (model) in S450 and the station mark classification network (model) in S460 can be jointly trained with the station mark detection network (model) in S430. Figure 12 As shown in the figure, an example of joint training is shown.

[0157] In S470, the child box and the parent box of the station mark are obtained according to the prediction result of the station mark detection network and the station mark combination classification result.

[0158] In combination with the above example, the position coordinates of box1 are [202, 20, 28, 29], and the position coordinates of box2 are [248, 20, 62, 23], and the coordinates of the circumscribed rectangle are [234, 20, 91, 29]. Among them, box1 and box2 are two child boxes of the station mark, and the circumscribed rectangle is the parent box of the station mark. The categories of the child box and the parent box are both “Huawei_video”. The final detection result is shown in the figure. Figure 13 As shown in the figure, the station mark region in the video to be detected includes two child boxes and one parent box.

[0159] Exemplarily, the parameter code of the final station mark recognition and detection result can be as follows:

[0160]

[0161]

[0162] For example, in the embodiment of the present application, after the parent frame and the child frame in the station logo of the video frame are identified, the watermark is added to the position of the parent frame, that is, the video is completed to remove the watermark, and the inference platform saves the result of removing the watermark in the storage system and subsequently distributes it to the user. Figure 14 The station logo in the video is removed.

[0163] It should also be understood that if a station logo is only composed of patterns or characters, the output result of the station logo detection network in S430 only includes one detection frame. In this case, S440 to S470 can not be executed, and the detection result including the position coordinates, confidence and belonging category of the detection frame (child frame or parent frame) is directly output after S430.

[0164] The station logo detection method provided by the present application can reduce the local missing detection of the station logo, achieve the effect of complete covering of the station logo in the watermark removal scene, and present the detection result in the form of child and parent frames, contain more structural information, provide more diversified processing methods for subsequent steps, and improve the efficiency and accuracy of station logo identification and detection. Moreover, the station logo local independent detection and overall combined frame classification not only optimizes the station logo character detection performance, but also avoids the label conflict between positive samples in the training stage caused by the mixing of samples, can train the model by using the overall and local samples of the station logo at the same time, and has strong robustness and generalization.

[0165] It should be understood that the above is only to help those skilled in the art better understand the embodiments of the present application, and is not intended to limit the scope of the embodiments of the present application. Those skilled in the art can obviously make various equivalent modifications or changes according to the above examples given, for example, some steps in the above method 400 can not be necessary, or some steps can be newly added, etc. Or the combination of any two or any more embodiments. The modified, changed or combined scheme also falls within the scope of the embodiments of the present application.

[0166] It should also be understood that the parameter codes in the above various embodiments are only exemplary and should not limit the format of the parameter codes carrying information in the embodiments of the present application. In other embodiments of the present application, the format of the parameter codes carrying information in the embodiments of the present application can also be other formats, etc. The embodiments of the present application are not limited here.

[0167] It should also be understood that the division of the ways, cases, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features in various ways, categories, cases and embodiments can be combined without contradiction.

[0168] It should also be understood that various numerical numbers involved in the embodiments of the present application are only for the convenience of differentiation, and are not used to limit the scope of the embodiments of the present application. The size of the serial number of the above processes does not mean the order of execution, and the execution order of the processes should be determined according to their functions and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0169] It should also be understood that the above description of the embodiments of the present application focuses on the differences between the various embodiments, and the same or similar parts not mentioned can be referred to each other, and for the sake of brevity, will not be repeated here.

[0170] It should also be understood that in the embodiments of the present application, "predefined" can be realized by pre-storing corresponding codes, tables or other ways that can be used to indicate related information in the device, and the specific implementation of the present application is not limited.

[0171] The above describes the embodiments of the method for station mark detection provided by the embodiments of the present application, and the following describes the related device provided by the embodiments of the present application. Figures 1-14 The embodiments of the method for station mark detection provided by the embodiments of the present application are described, and the following describes the related device provided by the embodiments of the present application.

[0172] The embodiments can divide the device (such as server or chip, etc.) for station mark detection into functional modules according to the above method. For example, each function can be divided into a functional module, or two or more functions can be integrated into a processing module. The integrated module can be realized in the form of hardware. It should be noted that the division of the modules in the embodiments is illustrative, and is only a logical function division. Actual implementation can have another division manner.

[0173] It should be noted that the related content of each step involved in the above method embodiments can be cited to the function description of the corresponding functional module, and will not be repeated here.

[0174] The device for station mark detection provided by the embodiments of the present application is used to execute the above method for station mark detection, and thus can achieve the same effect as the above implementation method. In the case of integrated unit, the device for station mark detection can include a processing module, a storage module and a communication module. The processing module can be used to control and manage the actions of the device for station mark detection. For example, it can be used to support the device for station mark detection to execute the steps executed by the processing unit. The storage module can be used to support the storage of program codes and data, etc. The communication module can be used to support the communication between the device for station mark detection and other devices.

[0175] The processing module can be a processor or a controller. It can implement or execute the various exemplary logical blocks, modules, and circuits described in connection with the disclosure. The processor can also be a combination of implementing computing functions, such as a combination of one or more microprocessors, a combination of digital signal processing (DSP) and microprocessors, and the like. The storage module can be a memory. The communication module can be a device for interacting with other electronic devices, such as a radio frequency circuit, a Bluetooth chip, a Wi-Fi chip, and the like.

[0176] Exemplarily, Figure 15 A hardware structure schematic diagram of a communication device 500 provided by the present application is shown. The communication device 500 can be the device for station detection described above. As shown in the figure, Figure 15 The communication device 500 can include a processor 510, an external memory interface 520, an internal memory 521, a universal serial bus (USB) interface 530, a charge management module 540, a power management module 541, a battery 542, an antenna 1, an antenna 2, a wireless communication module 550, and the like.

[0177] It can be understood that the structure shown in the embodiments of the present application does not constitute a specific limitation on the communication device 500. In other embodiments of the present application, the communication device 500 can include more or fewer components than the figure, or combine certain components, or split certain components, or different component arrangements. The components shown in the figure can be implemented in hardware, software, or a combination of software and hardware.

[0178] The processor 510 can include one or more processing units. For example: the processor 510 can include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), and the like. Different processing units can be independent components, or can be integrated in one or more processors. In some embodiments, the communication device 500 can also include one or more processors 510. Among them, the controller can generate operation control signals according to instruction operation codes and timing signals to complete the control of fetching instructions and executing instructions.

[0179] In some embodiments, the processor 510 can include one or more interfaces. The interfaces can include an inter-integrated circuit (I2C) interface, an integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a USB interface, etc. The USB interface 530 is an interface conforming to the USB standard specification, and can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 530 can be used to connect a charger to charge the communication device 500, and can also be used to transmit data between the communication device 500 and a peripheral device.

[0180] It can be understood that the interface connection relationship between the modules shown in the embodiments of the present application is only illustrative and does not constitute a structural limitation of the communication device 500. In some other embodiments of the present application, the communication device 500 can also use different interface connection methods or a combination of multiple interface connection methods.

[0181] The wireless communication function of the communication device 500 can be realized by the antenna 1, the antenna 2, and the wireless communication module 550, etc.

[0182] The wireless communication module 550 can provide a wireless communication solution including Wi-Fi (including Wi-Fi sensing and Wi-Fi AP), Bluetooth (BT), a wireless data transmission module (for example, 433MHz, 868MHz, 515MHz), etc. applied to the communication device 500. The wireless communication module 550 can be one or more devices integrating at least one communication processing module. The wireless communication module 550 receives electromagnetic waves via the antenna 1 or the antenna 2 (or the antenna 1 and the antenna 2), filters and frequency-modulates the electromagnetic wave signals, and sends the processed signals to the processor 510. The wireless communication module 550 can also receive signals to be sent from the processor 510, frequency-modulate them, amplify them, and radiate them as electromagnetic waves via the antenna 1 or the antenna 2.

[0183] The external storage interface 520 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the communication device 500. The external memory card communicates with the processor 510 through the external storage interface 520 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0184] Internal memory 521 can be used to store one or more computer programs, which include instructions. Processor 510 can execute the instructions stored in internal memory 521, thereby causing communication device 500 to perform the logo detection method provided in some embodiments of this application, as well as various applications and data processing. Internal memory 521 may include a code storage area and a data storage area. The code storage area may store the operating system. The data storage area may store data created during the use of communication device 500. In addition, internal memory 521 may include high-speed random access memory, and may also include non-volatile memory, such as one or more disk storage components, flash memory components, universal flash storage (UFS), etc. In some embodiments, processor 510 can execute instructions stored in internal memory 521 and / or instructions stored in memory disposed in processor 510, thereby causing communication device 500 to perform the logo detection method provided in embodiments of this application, as well as other applications and data processing.

[0185] It should be understood that the specific process by which the communication device 500 performs the above-mentioned steps is described in the preceding text. Figure 7 For the sake of brevity, the relevant descriptions of the execution steps shown are omitted here.

[0186] Figure 16 This illustration shows a schematic block diagram of another example of a communication device 600 provided in an embodiment of this application. Each module or unit in this communication device 600 is used to perform the actions or processing procedures described in the various embodiments of the method 400 described above, such as... Figure 16 As shown, the communication device 600 may include a processing unit 610 and a communication unit 620. Optionally, the communication device 600 may also include a storage unit 630.

[0187] It should be understood that the specific process by which each unit in the communication device 600 performs the above-mentioned corresponding steps is described in the preceding text. Figure 7 The relevant descriptions of the execution steps described in the document are omitted here for the sake of brevity.

[0188] Optionally, the communication unit 620 can include a receiving unit (module) and a sending unit (module) for performing the steps of receiving information and sending information in the aforementioned various methods 400. The storage unit 630 is used to store instructions executed by the processing unit 610 and the communication unit 620. The processing unit 610, the communication unit 620 and the storage unit 630 are communicatively connected, the storage unit 630 stores instructions, the processing unit 610 is used to execute the instructions stored by the storage unit, and the communication unit 620 is used to perform specific signal transceiving under the driving of the processing unit 610.

[0189] It should be understood that the communication unit 620 can be a transceiver, an input / output interface or an interface circuit, etc., which can be implemented by, for example, a wireless communication module 550 in the embodiment shown in FIG. 5. Figure 15 The storage unit can be a memory, which can be implemented by, for example, an external memory interface 520 and an internal memory 521 in the embodiment shown in FIG. 5. Figure 15 The processing unit 610 can be implemented by a processor 510 in the embodiment shown in FIG. 5, or can be implemented by the processor 510, as well as the external memory interface 520 and the internal memory 521. Figure 15 The processing unit 610 can be implemented by a processor 510 in the embodiment shown in FIG. 5, or can be implemented by the processor 510, as well as the external memory interface 520 and the internal memory 521.

[0190] It should also be understood that the division of the units in the above apparatus is only a logical functional division, and all or part of the units can be integrated into a physical entity or physically separated when actually implemented. The units in the apparatus can all be implemented in the form of software invoked by a processing element; or all be implemented in the form of hardware; or part of the units are implemented in the form of software invoked by a processing element, and part of the units are implemented in the form of hardware. For example, each unit can be a separately established processing element, or can be integrated into a chip of the apparatus, and in addition, can be stored in the form of a program in a memory, and the function of the unit is invoked and executed by a processing element of the apparatus. The processing element can also be referred to as a processor, which can be an integrated circuit with signal processing capability. In the implementation process, each step of the above method or each unit can be implemented by an integrated logic circuit of hardware in the processing element or in the form of software invoked by the processing element. In one example, the units in any of the above apparatuses can be one or more integrated circuits configured to implement the above method, such as one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), or a combination of at least two of the integrated circuit forms. For another example, when the units in the apparatus can be implemented in the form of a program invoked by a processing element, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processor that can invoke a program. For another example, the units can be integrated together to implement in the form of a system on a chip (SOC).

[0191] The embodiment of the present application further provides a computer readable storage medium for storing computer program codes, the computer program codes comprising instructions for executing any of the station mark detection methods provided by the embodiments of the present application. The readable medium can be a read-only memory (ROM) or a random access memory (RAM), and the embodiments of the present application do not limit this.

[0192] The present application further provides a computer program product comprising instructions which, when executed by a processor, cause an apparatus or device for station mark detection to perform corresponding operations corresponding to the above method.

[0193] The embodiment of the present application further provides a chip in a communication device, which comprises a processing unit, for example, a processor, and a communication unit, for example, an input / output interface, a pin or a circuit, etc. The processing unit can execute computer instructions to enable the communication device to perform any station mark detection method provided by the embodiment of the present application.

[0194] Optionally, the computer instructions are stored in a storage unit.

[0195] Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc., and the storage unit can also be a storage unit outside the chip in the terminal, such as a ROM or other types of static storage devices that can store static information and instructions, a random RAM, etc. The processor mentioned in any of the above can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for executing programs to control the transmission method of the feedback information. The processing unit and the storage unit can be decoupled and arranged on different physical devices, and are connected through wired or wireless means to realize the respective functions of the processing unit and the storage unit, so as to support the system chip to realize various functions in the above embodiments. Alternatively, the processing unit and the storage unit can also be coupled on the same device.

[0196] The communication device, the computer readable storage medium, the computer program product or the chip provided by the embodiment can be used to execute the corresponding method provided above, and thus the beneficial effects achieved can refer to the beneficial effects of the corresponding method provided above, which will not be described here again.

[0197] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a ROM, a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically EPROM (EEPROM), or a flash memory. The volatile memory can be a RAM used as an external cache. There are many different types of RAM, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synch link DRAM (SLDRAM), and direct rambus RAM (DRRAM).

[0198] In the present application, various objects such as messages / information / devices / network elements / systems / devices / actions / operations / processes / concepts, etc. that can occur in the present application are named. It can be understood that these specific names do not constitute a limitation on the related objects, and the assigned names can be changed according to the scene, context or usage habits, etc. The technical meaning of the technical terms in the present application should be mainly determined according to the function and technical effect embodied / executed in the technical scheme.

[0199] In various embodiments of the present application, the terms and / or descriptions of different embodiments are consistent and can be mutually referred to if there is no special description and logical conflict. The technical features in different embodiments can be combined to form new embodiments according to their inherent logical relationship.

[0200] Those of ordinary skill in the art can realize that the units and algorithm steps of the examples described in combination with the embodiments disclosed herein can be realized in electronic hardware, or a combination of computer software and electronic hardware. Whether the functions are performed in hardware or software depends on the specific application and design constraints of the technical scheme. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of the present application.

[0201] The method in the embodiments of the present application can be implemented by software, hardware, firmware or any combination thereof, in whole or in part. When implemented by software, it can be implemented in the form of a computer program product, in whole or in part. The computer program product includes one or more computer programs or instructions. When the computer programs or instructions are loaded and executed on a computer, the processes or functions described in the embodiments of the present application are performed, in whole or in part. The computer can be a general purpose computer, a special purpose computer, a computer network, or other programmable apparatus. The computer programs or instructions can be stored in or transmitted by a computer readable storage medium. The computer readable storage medium can be any available medium or a data storage device integrated with one or more available media, accessible by a computer.

[0202] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described system, device and unit can refer to the corresponding processes in the foregoing method embodiments, which will not be repeated here.

[0203] In several embodiments provided in the present application, it should be understood that the disclosed system, device and method can be implemented by other ways. For example, the above-described device embodiments are only schematic, for example, the division of the units is only a logical function division, and actual implementation can have another division manner, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the coupling or direct coupling or communication connection between the units shown or discussed can be indirect coupling or communication connection through some interface, device or unit, and can be electrical, mechanical or other forms.

[0204] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, that is, they can be located in one place, or can be distributed on a plurality of network units. Part or all of the units can be selected according to actual needs to achieve the purpose of the embodiment scheme.

[0205] In addition, each functional unit in each embodiment of the present application can be integrated in one processing unit, or each unit can exist physically, or two or more units can be integrated in one unit.

[0206] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a readable storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned readable storage medium includes various media that can store program codes, such as U disk, mobile hard disk, ROM, RAM, magnetic disk or optical disk, etc.

[0207] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art can easily think of changes or replacements within the technical scope disclosed in the present application, which should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A method of station logo detection, characterized by, The method comprises: obtaining a plurality of video frames of a video to be detected; detecting a first video frame in the plurality of video frames by using a station logo detection model to obtain information of a station logo comprising at least one detected frame pair in the first video frame, wherein one detected frame pair comprises a station logo pattern detected frame and a station logo text detected frame, and the first video frame is any one of the plurality of video frames; when a first detected frame pair satisfies a first condition, determining a candidate combination frame according to the first detected frame pair, the first detected frame pair comprising a first station logo pattern detected frame and a first station logo text detected frame, and the candidate combination frame comprising the first station logo pattern detected frame and the first station logo text detected frame, and the first detected frame pair being any one of the at least one detected frame pair; extracting a serialized feature of the candidate combination frame; inputting the serialized feature into a station logo combination classification model to determine whether the candidate combination frame is a station logo combination, the station logo combination classification model being used to determine whether a candidate combination frame is a correct station logo; determining a sub-frame and a parent frame in a station logo in a video frame according to whether the candidate combination frame is a station logo combination and information of the first detected frame pair, the first station logo pattern detected frame and the first station logo text detected frame being the sub-frame, and the candidate combination frame being the parent frame; the serialized feature of the candidate combination frame comprising a horizontal serialized feature and a vertical serialized feature of the candidate combination frame, and the extracting the serialized feature of the candidate combination frame comprising: extracting a feature map of the candidate combination frame at a position corresponding to a backbone network, uniformly processing a feature scale of the feature map by using a pooling technology, inputting the feature map with the uniform scale into a serialized branch network, and extracting the horizontal serialized feature and the vertical serialized feature.

2. The method of claim 1, wherein: the candidate combination frame is a circumscribed rectangle of the first station logo pattern detected frame and the first station logo text detected frame.

3. The method of claim 1 or 2, wherein: the information of the first station logo pattern detected frame comprises position coordinates of the first station logo pattern detected frame, a classification confidence of the first station logo pattern detected frame, and a category to which the first station logo pattern detected frame belongs; the information of the first station logo text detected frame comprises position coordinates of the first station logo text detected frame, a classification confidence of the first station logo text detected frame, and a category to which the first station logo text detected frame belongs; wherein the classification confidence of the first station logo pattern detected frame is used to indicate a probability that content comprised by the first station logo pattern detected frame is a pattern, and the category to which the first station logo pattern detected frame belongs is used to indicate a category or source of the content comprised by the first station logo pattern detected frame.

4. The method of claim 3, wherein: The position coordinates of the first logo pattern detection frame are (x, y, w, h), wherein x represents the horizontal coordinate of the center point of the first logo pattern detection frame, y represents the vertical coordinate of the center point of the first logo pattern detection frame, w represents the width of the first logo pattern detection frame, and h represents the height of the first logo pattern detection frame.

5. The method of claim 3, wherein, The first condition comprises: The classification confidence of the first logo pattern detection frame is greater than or equal to a first threshold value, the classification confidence of the first logo character detection frame is greater than or equal to a second threshold value, the category to which the first logo pattern detection frame belongs is the same as the category to which the first logo character detection frame belongs, and the ratio of the sum of the areas of the first logo pattern detection frame and the first logo character detection frame to the area of a circumscribed rectangle is greater than or equal to a third threshold value, wherein the circumscribed rectangle is a circumscribed rectangle of the first logo pattern detection frame and the first logo character detection frame.

6. The method of claim 1, 2, 4 or 5, wherein The logo detection model is trained by logo pattern bounding box labels and logo character bounding box labels.

7. The method of claim 1, 2, 4, or 5, wherein, The method further comprises: outputting a logo detection result, wherein the logo detection result comprises information of the sub-frames and information of the parent frame; The information of the parent frame comprises position coordinates of the parent frame, classification confidence of the parent frame, and a category to which the parent frame belongs. The information of the sub-frames comprises position coordinates of the sub-frames and classification confidence of the sub-frames.

8. The method of claim 1, 2, 4, or 5, wherein, The method further comprises: watermarking the parent frame.

9. The method of claim 8, wherein, The method further comprises: sending the video with the added watermark to a user.

10. An apparatus for station logo detection, the apparatus comprising: The apparatus comprises: a processor, a memory, and a computer program stored on the memory, which, when executed by the processor, causes the apparatus to perform the following steps: obtaining a plurality of video frames of a video to be detected; detecting a first video frame in the plurality of video frames by using a logo detection model to obtain information of at least one detection frame pair comprising a logo in the first video frame, wherein one detection frame pair comprises one logo pattern detection frame and one logo character detection frame, and the first video frame is any one of the plurality of video frames; when a first detection frame pair satisfies a first condition, determining a candidate combination frame according to the first detection frame pair, wherein the first detection frame pair comprises a first logo pattern detection frame and a first logo character detection frame, the candidate combination frame comprises the first logo pattern detection frame and the first logo character detection frame, and the first detection frame pair is any one of the at least one detection frame pair; extracting a serialized feature of the candidate combination frame; inputting the serialized feature into a logo combination classification model to determine whether the candidate combination frame is a logo combination, wherein the logo combination classification model is used to determine whether a candidate combination frame is a correct logo; and outputting a logo detection result, wherein the logo detection result comprises information of the sub-frames and information of the parent frame. determine a sub-frame and a parent-frame in a logo in a video frame according to whether the candidate combination frame is a logo combination frame and information of the first detected frame pair, the first logo pattern detected frame and the first logo character detected frame being the sub-frame, and the candidate combination frame being the parent-frame; the serialization features of the candidate combination frame include horizontal serialization features and vertical serialization features of the candidate combination frame, and the extracting the serialization features of the candidate combination frame includes: extracting a feature map of the candidate combination frame at a corresponding position of the backbone network, performing uniformity of feature scales on the feature map by using a pooling technology, inputting the feature map after the scale uniformity into a serialization branch network, and extracting the horizontal serialization features and the vertical serialization features.

11. The apparatus of claim 10, wherein the candidate combination frame is a circumscribed rectangle of the first logo pattern detected frame and the first logo character detected frame.

12. The apparatus of claim 10 or 11, wherein the information of the first logo pattern detected frame includes position coordinates of the first logo pattern detected frame, a classification confidence of the first logo pattern detected frame, and a category to which the first logo pattern detected frame belongs; the information of the first logo character detected frame includes position coordinates of the first logo character detected frame, a classification confidence of the first logo character detected frame, and a category to which the first logo character detected frame belongs; wherein the classification confidence of the first logo pattern detected frame is used to indicate a probability that content included in the first logo pattern detected frame is a pattern, and the category to which the first logo pattern detected frame belongs is used to indicate a category or a source of the content included in the first logo pattern detected frame.

13. The apparatus of claim 12, wherein the position coordinates of the first logo pattern detected frame are (x, y, w, h), wherein x represents a horizontal coordinate of a center point of the first logo pattern detected frame, y represents a vertical coordinate of the center point of the first logo pattern detected frame, w represents a width of the first logo pattern detected frame, and h represents a height of the first logo pattern detected frame.

14. The apparatus of claim 12, wherein, the first condition includes: the classification confidence of the first logo pattern detected frame is greater than or equal to a first threshold value, the classification confidence of the first logo character detected frame is greater than or equal to a second threshold value, the category to which the first logo pattern detected frame belongs is the same as the category to which the first logo character detected frame belongs, and a ratio of a sum of areas of the first logo pattern detected frame and the first logo character detected frame to an area of a circumscribed rectangle of the first logo pattern detected frame and the first logo character detected frame is greater than or equal to a third threshold value.

15. The apparatus of claim 10, 11, 13, or 14, wherein the logo detection model is trained by a logo pattern bounding box label and a logo character bounding box label.

16. The apparatus of claim 10, 11, 13, or 14, wherein, the apparatus further performs the following steps: outputting a logo detection result, the logo detection result including information of the sub-frame and information of the parent-frame. The information of the parent frame includes position coordinates of the parent frame, classification confidence of the parent frame, and a category to which the parent frame belongs. The information of the sub-frame includes position coordinates of the sub-frame, and classification confidence of the sub-frame.

17. The apparatus of claim 10, 11, 13, or 14, wherein, The device further performs the following steps: Watermarking the video on the parent frame.

18. The apparatus of claim 17, wherein, The device further performs the following steps: Sending the video with the added watermark to a user.

19. A computer-readable storage medium, characterized in that, The computer readable storage medium stores a computer program, and the computer program includes program instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 9.

20. A chip, characterized by The computer readable storage medium stores a computer program, and the computer program includes program instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 9. The computer readable storage medium stores a computer program, and the computer program includes program instructions which, when executed by a processor, cause the processor to perform the method according to any one of claims 1 to 9.

Images